gpsd/www/replacing-nmea.adoc

= Towards a Better GPS Protocol
Eric S. Raymond <esr@thyrsus.com.>
1.7, 27 January 2020
:date: 25 January 2021
:keywords: gps, gpsd, NMEA
:manmanual: GPSD Documentation
:mansource: The GPSD Project
:robots: index,follow
:sectlinks:
:toc: macro
:type: article
:webfonts!:

include::inc-menu.adoc[]

== Abstract

The NMEA 0183 protocol used by GPS units might best be described as
"layer upon layer of cruft".  In this paper, we examine the problems
and consider a cleaner design.

== What's Wrong with NMEA 0183, and Why Fix It?

The protocol used by GPS devices to report to computers is a small
subset of NMEA 0183. NMEA stands for "National Marine Electronics
Association", and the features GPSes use for reporting
time/position/velocity information are a small part of a protocol
originally designed for communication between parts of complex marine
navigation systems. Thus the full protocol includes support for depth
sounders, LORAN, and many other things irrelevant to a modern GPS.

The lowest level of NMEA 0183 is quite sensibly designed. The protocol
consists of sentences, each led by a dollar sign and an identifying text
tag, followed by multiple comma-separated textual fields, ended by an
asterisk, a checksum, and LF/CR. This is a simple, clean format with
good extensibility, easy to parse and generate. It is well adapted to
its job, which is to pass small amounts of numeric and status
information. The textual format makes it easy to log NMEA sessions, edit
them, and play them back -- a substantial advantage in developing
talker and parser software.

Unfortunately, the good news ends there. The design of the upper layers
of NMEA 0183 is patchy, kludgy, and replete with the kind of errors that
arise from growth by accretion rather than forethought. Here are some of
the more obvious problems:

* NMEA timestamps usually (e.g in GPBWC, GPBWR, GPGBS, GPGGA, GPGLL,
GPGXA, GPTRF, GPZTG) report time-of-day only. The exceptions (GPRMC,
GPTRF, GPZDA) report only two digits of year and no century. Time
precision is unspecified, usually to the second though some devices
report a fractional decimal part to millisecond precision.
* It is not possible to get a time/position/velocity report in a single
sentence. Some sentences (GPRMC) report time and 2D position and
velocity, some (GPGGA) report time and 3D position, some (GPVTG) report
velocity only. As a result, the API for a protocol client is complicated
by the necessity of maintaining separate age indications for 2D
position, 3D position, and velocity,
* NMEA sentences have at least three kinds of validity indicators --
mode (GPGSA only), status (GPGLL, GPGGA), and the Active/Void field
(GPRMC, GPGLL). And that's before we get into the FAA extensions in late
revisions of the protocol. Interpreting these status bits is a black art
involving knowledge of undocumented and often vendor-specific quirks.
* There is no standard way of indicating that part of a
time/position/velocity report is invalid (e.g. because the device does
not have a fix of the required quality). Many devices simply report 0
for invalid fields, which doesn't sound bad unless you're near the
zero-zero point in the Bay of Benin -- or at sea level. It is also
not generally possible to distinguish between information a GPS is not
yet reporting but will return on a good fix (such as altitude) from
information it will never report (such as, on many units, local magnetic
variation).
* As least one messy bit in NMEA 0183 was an adaptation to machines with
only small amounts of expensive RAM: the fact that satellite status may
show up in a sequence of as many as three sentences to be processed
serially. On modern machines, RAM buffers are cheap. It makes more sense
to ship a single long sentence and decrease code complexity in the
receiver by not requiring a stateful parser.
* Position accuracy estimates are not easy to compute from NMEA reports.
Reporting a measurement without giving its 95% confidence interval is
bad practice.
* For modern GPS devices, even the small piece of NMEA directly
concerned with GPS capabilities is seriously over-complex. Whereas older
GPS devices included elaborate facilities for waypoint tracking and
navigational computation, newer ones are designed under the assumption
that they are connected to a general-purpose computer that is more
powerful and flexible at these things; thus, the GPS only needs to be a
time/position/velocity oracle.
* NMEA 0183 is in general very loosely specified and poorly documented.
Its problems are compounded by the fact that it is a proprietary
specification, jealously guarded by IP lawyers.

As a result of these problems, implementing NMEA 0183 talker software is
far more complex than need be, and the protocol tends to introduce
latencies that vary in an unpredictable way. The latter is a serious
problem for aviation and other high-precision GPS applications, and
probably provided a technical reason that one major GPS vendor (Garmin)
dropped NMEA 0183 support entirely in 2004 in favor of a tighter binary
protocol (we refrain from speculating on other less creditable motives
for this move).

== How To Do Better

The critique above immediately suggests several ways to improve a
protocol for GPS reports;

* Keep the low-level syntax, because it's not broken. It has all the
advantages of textual protocols. Going to a more tightly-packed binary
format might look attractive at first glance, but the gain in
information would be marginal at best. Textual formats already use 7 out
of 8 bits per byte and encode variable-length numeric fields more
efficiently than binary; also they avoid endianness issues.
* Add to the syntax standard ways of indicating that either (a) the GPS
cannot now ship valid data for the field, or (b) the GPS will _never_
ship data for this field.
* Include a full timestamp, to millisecond precision or better, with
every sentence. Every timestamp should be in the same standard form and
should include a full date with century.
* Report the uncertainty corresponding to a 95% confidence interval on a
standard normal distribution for each measurement field.
* Design the protocol around a single core sentence that reports
time/position/velocity and the uncertainties in same.
* Make it an objective of the design for an informal specification to
fit on a single page.

== Informal specification: SGPS

Here, then, is a proposed informal specification for SGPS, Simple GPS
Protocol.

The protocol consists of sentences, each led by a dollar sign and an
identifying sentence tag, followed by multiple comma-separated textual
fields, ended by an asterisk, a CRC32 checksum, and LF/CR. Sentences are
at most 255 characters long, counting the trailing CR/LF.

A field that is empty indicates that the talker does not have valid
information for this field but promises to report it in the future. A
field consisting of a question mark (?) indicates that the talker does
not expect to ever ship valid information for this field.

The first field of every SGPS report sentence is a full timestamp in the
format of the
https://web.archive.org/web/20150919174330/https://www.w3.org/TR/1998/NOTE-datetime-19980827[W3C
profile of ISO 8601], with the timezone required to be Zulu (UTC) time.

=== GPTPV

The core sentence of SGPS has the following layout:

[arabic]
. The sentence tag is GPTPV, standing for Time/Position/Velocity.
. The first field is the required timestamp.
. The second field is the uncertainty of the timestamp in (fractional)
seconds.
. The third field is signed latitude in degrees. Encoding must be
decimal degrees, not degree/minute/second.
. The fourth field is signed longitude in degrees. Encoding must be
decimal degrees, not degree/minute/second.
. The fifth field is horizontal uncertainty in meters (95% confidence).
. The sixth field is altitude in meters.
. The seventh field is vertical uncertainty in meters (95% confidence).
. The eighth field is speed over ground in meters per second.
. The ninth field is speed-over-ground uncertainty in meters per second
(95% confidence).
. The tenth field is course over ground in degrees from true north.
. The eleventh field is uncertainty of course over ground in degrees
(95% confidence).
. The twelfth field is climb/sink in meters per second.
. The thirteenth field is uncertainty of climb/sink in meters per second
(95% confidence).
. The fourteenth field is an FAA mode indicator.

These fourteen fields completely describe the position and velocity of
an object and the associated uncertainties. The FAA mode field is added
to satisfy a U.S. regulator's requirement.

Here is an example:

....
$GPTPV,2005-02-11T04:40:51.231Z,?,49.45,-123.12,2.3,70.1,52.0,01.0,02.1,23.1,0.6,,,8,A*31

     2005-02-11T04:40:51.231Z,  Time (Feb 11 04:40:51 UTC 2005)
     ?,                         Timestamp uncertainty will never be reported
     49.45,                     Latitude (- sign indicates latitude south)
     -123.12,                   Longitude (- sign indicates longitude west)
     2.3,                       Meters of horizontal uncertainty of position
     70.1,                      Altitude, meters above sea level
     52,                        Uncertainty of altitude
     0.01,                      Speed, meters/sec
     0.02,                      Speed uncertainty
     23.1,                      Course over ground relative to true North
     0.6,                       Course uncertainty in degrees.
     ,                          Climb/sink not reported
     ,                          Climb/sink uncertainty not reported
     A                          FAA mode indicator A (Auto).
     31                         Checksum.
....

=== GPSVU

A second sentence describes GPS satellite status.

[arabic]
. The sentence tag is GPSVU, standing for Satellite View Update.
. The first field is the required timestamp.
. The second field is a count of satellites.
. The remainder of the sentence fields are groups of four, one for each
predicted position of a visible satellite. Each group has the following
four elements:
[arabic]
.. The PRN or satellite ID.
.. Elevation in degrees
.. Azimuth, degrees
.. Signal-to-noise ratio in decibels. If this satellite was used in the
last fix, suffix this field with a '!'.

Here is an example:

....
$GPSVU,2005-02-11T04:40:51.231Z,11,03,03,111,00,04,15,270,00,06,01,010,00,13,06,292,00,14,25,170,00,16,57,208,39!,18,67,296,40!,19,40,246,00,22,42,067,42,24,14,311,43!,27,05,244,00*40
....

=== GPVID

A third sentence identifies the device. It is GPVID for Version ID, and
the fields are as follows:

[arabic]
. The sentence tag is GPVID, standing for Vendor ID.
. The first field is the required timestamp.
. The second field is the SGPS revision level.
. The third field is the vendor name.
. The fourth field is the device name or model number.
. The fifth field is a chipset designation.
. The sixth field may be empty or a subtype ID, typically a firmware
revision level.

All fields must consist of US-ASCII text not containing commas. The
total length of the sentence must not exceed the old NMEA maximum of 82.

Here is an example:

....
$GPVID,2006-11-17T12:29:37Z,1.0,Haicom,H204S,SiRF-II,231.00.00*5C
....

=== GPGSP

With the addition of a fourth sentence, $GPSGP, transition to the new
protocol would be easy. It would have two forms:

$GPGSP,1: directs the receiver to emit GPPVT and GPSVU only, if it is
not already doing so.

$GPGSP,0: directs the receiver to return to NMEA-classic mode, if it is
capable of doing so.

Example:

....
$GPGSP,1*4E
....

An SGPS-conformant receiver is required to respond with
$GPSGP,timestamp,x,y where x is 1 or 0 reflecting the command, and y is
1 or 0 reporting its new mode.

Other listeners can distinguish GPGSP responses from requests by
checking whether field 1 contains an IS8601 timestamp; an easy way to
check this is to look for the trailing Z.

=== Other considerations

Finally, SGPS-compliant receivers are required to respond to the
requests $GPPVT, $GPVSU, $GPVID, and $GPGSP (without arguments) with the
corresponding report based on most recent available data.

== Could This Be Adopted?

Astute readers will already have noted that the SGPS sentences might be
sold as a minor extension to NMEA 0183. first supplementing and
eventually obsolescing the half-dozen or so sentences emitted by most
modern GPSes.

The only fields reported in the SGPS set that cannot be trivially
derived from data already computed for NMEA reports are (a) Climb/sink,
and (b) GPSTPV uncertainty fields. None of these should be difficult to
derive.

== Revision History

|===
|Version |Date| Author | Comments

|1.7 |25 January 2020 |gem | Convert from DocBook to AsciiDoc

|1.6 |5 January 2016 |esr
|Corrected timezone in example
W3 datetime document has disapperard, point to Internet Archive

|1.5 |25 February 2009 |esr
|Fixed GPSGP so the response isn't identical to the send,
avoiding problems on multidrop lines.  Added SGPS revision
field to $GPVID. Went back to requiring checksums, because
you just know it was going to bite someday otherwise.
Changed sentence length limit to 255.

|1.4 |21 November 2006 |esr
|Fixed timestamp to Zulu time.  Specified signed latitude.

|1.3 |16 November 2006 |esr
|Added GPSVID and GPSGP, changed to mandate ISO8601 dates.

|1.2 |25 April 2005 |esr
|Specify UTC.  Fix time-uncertainty units.  Vertical course
angle changed to climb/sink rate.

|1.1 |11 February 2005 |esr
|Corrected SGPS example, thanks to Kevin Niehage for the bug report.

|1.0 |04 January 2005 |esr |Initial draft.
|===

== COPYING

This file is Copyright 2004 by the GPSD project +
This file is Copyright 2004 by Eric S. Raymond +
SPDX-License-Identifier: BSD-2-clause