Saturday, February 14, 2015

NMEA 2000 Standard Committee

I think a group picture of the NMEA 2000 Standards Committee deserves a place on the Wall of Shame.  NMEA 2000, aka N2K, has so much potential, and has helped the marine industry in many ways.  But it's implementation falls well short of it's real potential.  It's great for wiring up a 25' center console boat with one of this and one of that.  But try building up a bigger system with lots of devices, many of which are redundant, and it very quickly comes unglued.  It's no wonder that N2K is not part of any IMO approved device.

One of the biggest problems is that N2K is a secret.  To get a copy of the spec costs $5000 or more, so nobody other than a manufacturer will ever see it.  And even if you cough up to $5000 to buy the spec, it comes with a stipulation that you keep it secret.  The result is that none of "us" - not boat owners, not repair techs, and not electronics installers - really know how it's all supposed to work.  So when something isn't behaving properly, there is no way for us to know what's right and what's wrong, and hence no way to chase down whoever's product is misbehaving.

Instead, we get the run around.  Has anyone else ever experienced this?  You are having a problem, and might even be using an analyzer like Actisense's NMEA Reader, or Maretron's N2KAnalyze (both great products) and have an idea where the problem might be.  So you call tech support for vendor A, describe the problem and tell them how you think their product might be misbehaving.  Vendor A gives you some mumbo jump about how things are supposed to work and tells you that Vendor B's product isn't doing things correctly.  Not knowing any better, you believe them and now call Vendor B, explaining to them what you were told by Vendor A.  Vendor B now gives you some different mumbo jumbo and explains how Vendor A is doing things wrong and that they are the problem.  Now you are in the dark here, so have no way of calling bull-shit on either of them.  And these two companies won't pick up the phone to the other company to sort out the issue, but instead leave you as monkey in the middle.  Sound familiar to anyone?

If purchasing a spec includes other stipulations, one should be that you are not allowed to tell a customer that another vendor's product is broken, but rather be required to call that vendor and work it out.  If these guys want to sell more product, it has to work.  By way of example, my boat started out 100% N2K.  I have subsequently removed about 30% of the N2K devices, and the replacement communications is via 0183.  I did NOT want to do that, but it has proven to be the only way to make it work in a reliable and predictable way.

The next issue is NMEA 2000 Certification.  I read over and over again about how all troubles will go away if you only use NMEA 2000 Certified devices.  These are devices that have voluntarily subjected their products to a battery of tests that demonstrate proper adherence to the N2K standard.  Well, I can tell you in no uncertain terms that NMEA 2000 Certification does not indicate that a product complies with the standard, and there is one very large and glaring example.  Instancing.

[Editor's note, Dec 12, 2015.  Please note that in this post I am talking about NMEA 2000 Device Instance Numbers, not Data Instance Numbers.  I was not clear and I believe this has led to a lot of confusion.  Here is an article explaining the difference, so please take a look]

[Editor's note, Dec 12 2015.  Please also note that the following discussion of Device Instancing is heavily used by Maretron, and not used at all by many other vendors.  Please read all the way through to understand NMEA 2000 NAMEs and how they provide an alternate, and more commonly used way distinguish and select devices.  But keep in mind that these two different approaches and the conflicting support in device certification is exactly the point of this post.]

Instancing is a way to distinguish between otherwise similar devices.  It's a number that you assign to devices, kind of like taking a ticket at a deli counter.  The picture below shows a real life example of where this comes into play.  The network in the pictures shows a chart plotter, a sat compass, a GPS, and a rate compass.

Example N2K network with redundant sensors

The sat compass and GPS both provide position, speed, and course information, but the sat compass is the superior instrument providing more accurate info than the GPS.  Similarly, the sat compass and the rate compass both provide heading and rate of turn info, and once again the sat compass is the superior instrument yielding more accurate info.

In the absence of instance numbers, the chart plotter would have no way to distinguish between the two duplicate sources of data.  It might pick randomly, or even worse it might use both, mixing data from different sources.

Instancing solves this problem in two steps.  The first step is that every device producing data, typically referred to as a Source (the sat compass, GPS, and rate compass in this example) must have a way to change its instance number.  There is a standard message that can be sent across the N2K bus to change it, and many devices support this, though it's not required.  But if a device doesn't support that method, it must provide some other way.

The second part of the solution is that any device making use of data on the bus, sometimes called a Listener, needs to have a way to pick which device it wants to listen to.  More sophisticated devices can be given a prioritized list of devices, and they will always use the highest priority device that is available.

Bringing this back to N2K Certification, I have encountered numerous devices that are certified and that do not provide programmable instance number.  These are certified as standard compliant devices, yet they blatantly violate the spec.  And if you try to use these devices in anything other than the most simplistic network, or more accurately a network where there is never more than one such device, it doesn't work.  This is a HUGE loop hole in the certification process, and a huge short fall in vendors implementation of N2K devices.

Devices where the Instance number cannot be changed or doesn't work properly:
  • ICOM M506 N2K
  • ICOM M506 N2K-AIS
  • Simrad HS70 sat compass
  • Simrad AT10
  • Simrad AT10HD
  • Simrad NAIS400 AIS transceiver
  • Furuno NavPilot 700 computer (FAP7002)
Looking at the other side of the problem, it gets even more complicated.  I actually don't know whether devices are required to be capable of source selection.  If I could read the spec I could tell you, but I can't read the spec, and if I could I wouldn't be allowed to tell you.  What I do know is that there are a variety of ways a device COULD do source selection, they all seem to be valid and compliant, but are often times incompatible.  Here are two examples.

Every N2K device has a "Name", and that name is guaranteed to be unique.  A device's name is composed of a number of different things similar to a first, middle, and last name in our US culture.  An N2K name is a combination of the device type, manufacturer, instance number (there it is again) and a unique number assigned by the manufacturer.  Worth noting is that every device has a unique name even if they all have the same instance number.  So every device on the planet could have instance number zero and you would still be able to differentiate all of them.

When doing source selection, some vendors use the unique name and are not dependent of programmable instance numbers.  But other vendors depend on programmable instance numbers and use it as the primary way to perform source selection.  Both approaches are legitimate, but it results in two very different views on the importance if programmable instance numbers.

One guy at NMEA told me that a network should be completely usable and operable even if all instance numbers or zero.  That says that one cannot depend solely on instance numbers for source selection.  And a number of vendors I have spoken with about their non-programmable instance number point out that it's not needed.  Well, maybe it's not needed by them.

Maretron, a prominent vendor and member of the NMEA 2000 Standards Committee has build a product line that depends 100% on programmable instance numbers to do source selection.  If you can't program a devices instance number, then you can't tell it apart from another device sending similar data.  Maretron defends their position saying that programmable instance numbers are mandatory in the standard and that other vendors needs to implement it.

Both are right, but the result is that building larger, redundant networks is very problematic, and we the customer are caught in the middle.  It also means that we the customer aren't buying as much product as we would if we could reliably build redundant systems using N2K.

All this comes back to the N2K Standards Committee.  If they can enforce secrecy of the specification, then surely they can enforce compliance with the spec.  And at a minimum, certification should actually reflect compliance with the spec.  Then it would be a mark that we could seek out to buy with confidence.  But right now, NMEA 2000 Certified doesn't mean much of anything.

14 comments:

  1. Sorry for your troubles Peter but thank you for sharing your experience. it sounds as though many folks are betting their life navigating with inferior or flat out non functional equipment. Sad.

    ReplyDelete
  2. Very instructive!

    THANKS!
    Norm

    Quiet Company
    Great Harbour GH47

    ReplyDelete
  3. NMEA sounds just as dysfunctional and illogical as our present Congress :).

    ReplyDelete
  4. Sorry, but I think you have Instances pretty mixed up. They're important if you want to use two or more of the same device type simultaneously, but not for selecting one device of several redundant ones to use.

    For example, if you have N2K engine sensors on two engines, one is typically Instance 0 and the other Instance 1. But nearly every display on my quite mixed up N2K networks -- including Simrad and Maretron -- can easily select between redundant GPS, Heading, and other devices regardless of the Instance numbers.

    Best,

    Ben Ellison, Panbo.com

    ReplyDelete
  5. Hi Ben,

    True, and not true. In many ways the problem is Maretron who have elected to use instancing as their means to select between devices of the same type. With their display products, two GPSs or two heading sensors or two rudder indicators MUST have different instance numbers if you want to spell out which to use. This can happen if you want to display both devices' data at the same time, and it can happen if you want to stipulate which of the two should be used for a single display. Alternately, you can just let them pick for you (there is an "Any" option), but you wouldn't do that where your two heading sensors are of significantly different quality as in the case of a sat compass and a rate compass.

    They have good arguments for doing things the way they do, and it is a legitimate approach. Not the only approach, as demonstrated by pretty much all other vendors, but a valid one. The down side is that their approach depends on Instance numbers being supported and programmable in all devices. They further argue (I think correctly) that the standard requires this support in all devices, so depending on it is reasonable. But as we all know, in practice, support for programmable Device Instance Numbers is inconsistent at best.

    NMEA says you should identify devices based on their "NAME" which is an amalgamation if fields in the product identifier including the manufacturer's ID, device name, device type device function, device instance number, and a unique serial number. By virtue of the unique serial number, this NAME is unique for every device, guaranteed.

    I've seen other vendors who use the full name to identify and select devices, and use the Serial number to differentiate between otherwise identical devices. With these products an HS70 sat compass and SSC200 rate compass are distinguishable by their name alone - no need to worry about device instance numbers and serial numbers. But not Maretron. They require that those two devices have different Device Instance Numbers to distinguish them. Different names like "HS70" and "SSC200" don't matter and are ignored, as are the serial numbers. And therein lies the problem.

    This wouldn't be an issue if Maretron used the whole NAME field to select data sources, but they don't. I have bent Rich Gauer's ear on this, but don't expect it will change their view.

    This also wouldn't be an issue if devices actually complied with the standard and supported programmable Device Instance Numbers. But they commonly don't.

    My gripe with NMEA is that they tout the importance of buying Certified devices and that all our compatibility worries will melt away if we do. Well, I did, yet there is this gaping hole. We have a big vendor, Maretron, who have one of the richest sets of N2K products out there (and most of it quite good) who depend on devices being standard-compliant to work with their display products. And at the same time we have a compliance certification process that does not test for compliance in this critical area. And we the customer's are left as Monkey In The Middle.

    I think it's things like this that frustrate the hell out of owners and installers, give N2K a bad name, and inhibit sales.

    ReplyDelete
  6. Ben, I just re-read my blog post and see that I have learned more (that's always a good thing) since writing it. In particular I now realize that it's really only Maretron who are 100% dependent on Device Instance numbers (and Data Instance Numbers too) to differentiate between data sources. Other products can differentiate even if all devices have instance number zero.

    Although I prefer the way other people do it, I do think Maretron's approach is legitimate, and I do think that a certified product should actually comply with the spec in all ways, including full support for Instancing.

    ReplyDelete
  7. Peter, If you have two compasses on your network and don't want to see one on your Maretron display, just go to Configuration/Device Selection and turn off the compass you don't want to use. It takes seconds and does not involve Instances.

    ReplyDelete
  8. Thanks for the reminder on that. I don't have a DSM250 anymore and had forgotten about that capability.

    But it's only suitable if you want the DSM250 to completely ignore a particular device, right? And that capability doesn't exist in N2KView which is what I'm using now. Unless I'm totally missing something, Instancing is the only way to differentiate between two devices sending the same PGNs with that product.

    Back on Panbo we were talking about which devices transmit True Heading, and I'd be interested to know which ones you have found. My sampling is admittedly limited, but I have observed the following:

    - Simrad RC42 rate compass - Sends Vessel Heading PGN in Magnetic format only.

    - Maretron SSC200 rate compass - Sends Vessel Heading PGN in magnetic format only.

    - Furuno tells me the PG700 is magnetic only, but I haven't tried it myself.

    - Hemisphere V102/Simrad HS70 Sat Compass. Send Vessel Heading PGN in True format.

    I could see how some of the combination GPS/Heading sensors could send True heading since they know or could figure out both magnetic heading and Magnetic variation.

    ReplyDelete
  9. Peter, I believe that most MFDs and N2K instruments will compute True Heading if they see Magnetic Heading and Variation on the network. And typically they output the True Heading value as well as display it (though sometimes output is an option you have to invoke).

    I'm far away from my boat now to double check this, but I'm nearly positive that my Simrad network with only an RC42 heading device has True Heading flowing through it. And the same used to be true on the other network when it had only an SSC200.

    Meanwhile the Airmar P200 (now 200WX) and Raymarine EVO sensors compute and output True Heading themselves, and neither is a satellite compass.

    Having a system value dependent on two sensors is not necessarily without problems and there was a time when I put everything on Gizmo in True because it seemed like some displays were correcting Magnetic for Variation when they shouldn't have, or something ;-)

    But most of the time it just works, and I think that's especially true on systems where most of the devices are from one manufacturer.

    ReplyDelete
  10. For our readers, Ben's last comment about True headings is a carry-over from a discussion about Class A AIS and its requirement for an external True Heading input. http://www.panbo.com/archives/2015/03/amec_camino_701_class_a_ais_first_with_a_nearly_complete_nmea_2000_interface.html

    For now, let me bring this back to the Blog subject which is some of the loop holes in the NMEA 2000 spec and certification that causes problems for those of us using this stuff.

    I think the Airmar 200WX is a great example of where Instancing can cause a Check-Mate for someone building a system. Say you have the Airmar weather station plus some other higher quality heading sensor. And assume you want to display both weather data and ships's heading on your Maretron DSM250 or on N2KView. But both the 200WX and the other heading sensor are transmitting Vessel Heading, so you need a way to tell the displays to show the heading value from the better device, not the 200WX.

    You can't tell the DSM250 to ignore the 200WX, because doing so would also block all the weather data. And N2KView doesn't have the "ignore device" feature at all so it's not even an option there.

    So now on the Maretron display products you are completely dependent on Device Instance Numbers to differentiate between the two heading sources. If the weather device and the heading device don't properly support instancing, then you are stuck and have to let the display device pick which heading source to use. Maybe you will get the one you want, and maybe you won't.

    Bringing this back to the original blog post, support for programmable instance numbers is mandatory in the NMEA2000 spec, and whether you agree with Maretron's approach to source selection or not, I think it's a legitimate use of features that are required to be present in all products. Yet NMEA certification does not test for this, and many certified products do NOT support programmable instance numbers.

    Given this, we the owners and/or installers of this gear are left to sort all this out via experimentation. This is the sort of thing that becomes a massive time sink for installers and causes them to lose money and/or drives up the cost of installed systems.

    ReplyDelete
  11. Peter, you can change the Device Instance of an Airmar 200WX, so the Check-Mate scenario is false.

    (You can also turn off individual PGNs coming out the 200WX, like Heading, if you don't need it.)

    Moreover, most displays don't have a problem if you just leave redundant sensor Instances as they are.

    I've used the Airmar PB200 (predecessor to the 200WX) with redundant heading, GPS, and weather sensors and what happens on Furuno, Garmin, Navico, and (recently) Raymarine displays is that can choose GPS, Heading, Baro, Air Temp, etc. each by sensor name or leave the choice to Auto mode where the display will pick the first one it sees and roll over to another if the first one stops sending PGNs. Most users would be pretty pleased I think.

    Even what Maretron is doing with redundant sensors having the same instance (that Auto mode) generally works for most users.

    I'm not saying that NMEA 2000 is perfect. It certainly isn't, and it's great that NMEA is now focusing on improving N2K with some guidance from passionate manufacturers like Maretron.

    I'm just concerned that you've gotten confused about what the issues really are and have also painted a bleaker picture of N2K than it deserves.

    ReplyDelete
  12. I don't think I'm confused at all about what the issues are. Maretron depends on Instancing to select devices, and devices do not consistently implement instancing. It's black and white. Yes, lots of times it can be worked around in one way or another, but that doesn't make the issue go away, just the symptoms. Heck, I've gotten the systems that I've built to work by working around various issues using tricks like you have described, and in some cases by replacing devices with ones that operate better than others or support additional features that solve problems.

    I'll also be the first to admit (I said this several times on this blog) that I'm much more demanding of tech products than most people, and accept that lots, perhaps most people won't care about any of this. If you are happy with any data source then N2K almost always works great. And I agree that lots of products do source selection very well, and in a way that has no dependence on Instancing.

    And if you want to build larger, more complex systems where the probability of running into these issues is higher, you just need to be prepared to fuss with things to get them to work, or to adjust your goals to what's achievable.

    For example, on N2KView I have created an instrument status page. It shows the key data coming from a wide range of sensors, and where there are two such sensors, it shows each side by side. I've found it to be a really handy way to see at a glance that everything is working, including my secondary sensors that may otherwise not be in use, and to confirm that redundant sensors are reporting the same data. Showing the rudder position from two redundant rudder indicators has been a challenge.

    My first go-around, which worked, used Simrad RF25 rudder indicators that are N2K-connected. They support programmable device instances, and with that I was able to show both side by side on N2KView.

    Next, I upgraded from N2KView V4.x to 5.x, and now my rudder indicators were gone. Discussions with Maretron revealed that they were unhappy with the Data Instance Value in the PGNs (note, the Data Instance is different from a Device Instance), and were now rejecting them. I got an update from Simrad that set the PGN Data Instance to the same value as the Device Instance as a work around, which I thought was very clever in it's simplicity. This made N2KView happy and it once again could display the rudder positions, but it subsequently turned out that Simrad's auto pilot could no longer properly select the rudder indicator, so I had to back out that change.

    Round 3 involved switching to a Furuno auto pilot, and in that case it's the AP computer that reports the rudder position. N2KView is happy with the format of the PGN and displays it correctly, but it can only display one of the two rudder indicators. Why? Because the Device Instance can't be changed in the Furuno AP computer. It's always zero. So this time the work around was to give up on trying to show both rudder indicators at the same time.

    ReplyDelete
  13. Note that most PGNs don't even have an instance ID field so the Maretron position is flawed, to say the least. Many many devices do not need an instance ID whatsoever.

    The idea is that a PGN will include an instance ID field if ONE sender can report on MULTIPLE thingies.

    So a Depth PGN 128267 does not include an instance ID, because a depth sensor will typically only contain one transducer. But a tank monitor typically can monitor multiple tanks, which is why the Fluid Level PGN 127505 does contain an Instance field (and it will produce one datagram per tank.)

    See https://github.com/canboat/canboat/blob/master/analyzer/pgn.h (search for "Instance" and the above PGN numbers.)

    ReplyDelete
  14. Kees, your comment here along with our email exchange finally caused the light bulb to go off in my head. We are talking about different "Instance" numbers. N2K has Device Instance Numbers and Data Instance Numbers. They are similar, but different. Device Instance Numbers only get exchanged at network config time or if a device specifically queries for it. A Data Instance Number is carried in every PGN for select data types. The example you describe are Data Instance Numbers, where I have been talking about Device Instance Numbers. I think this may also be why Ben Ellison and I seemed to be talking across each on this subject both here and on Panbo.com

    ReplyDelete

Make comments here