OpenPlotter

Open Up

OpenPloter is not the first "marine OS"  to assemble a variety of boating-related software into a single downloadable image:  Xinutop and Navigatrix are both x86 GNU/Linux distributions which provide OpenCPN and other free marine software pre-configured.  OpenPlotter, which has gained a lot of friends in the past year, differs from those in a number of ways.  Most obviously it is aimed at ARM systems, notably the Raspberry Pi, and rather than being simply a collection of useful software it provides sailors with an interest in DIY a platform on which to build an integrated boat electronics system using the plethora of cheap sensors that arena available.  Not just a collection of 3rd party software, OpenPlotter adds a GUI to provide unified management of many of its functions, making command-line and configuration-file oriented Linux/UNIX software like kplex more accessible to a wider range of DIY-ers. Additionally you can purchase tested hardware through the OpenPlotter web site.

Roberto Serrano, OpenPlotter's lead developer, kindly agreed to answer a few questions about the project...

What does "sailoog" (the name of your website and pseudonym you post under) mean?

I would like to tell a nice story here about some recondite indigenous language or something similar but I cannot. I just nicked it from one of our previous and unsuccessful projects, a marine blog system. Sailblog was taken so, playing with logos and letters we said Hey, the Internet is full of double “o”s (google, yahoo, moodle, joomla…) so let’s be cool!

Why did you start the openplotter project?

The financial crisis hit hard in southern Europe and took us our jobs, our homes, our boats and our dreams. I am from Catalonia and the only option was to go abroad to work so I started OpenPlotter project just as an exercise to learn English, improve my skills programming in languages like python and use some tools like github. But I soon realized that I was dealing with something with real projection and maybe a way to hit back.

We believe we should be able to equip even small boats with high quality, affordable and robust technology without having to rely on the big corporations. 

Governments and proprietary technologies no longer have more guarantees than real democracy and open-source technologies so we could say that my main motivations on this project are ethical and political but I can not speak by the others, perhaps we could even disagree but who cares? This is open, this is free, and above all this is fun.\

How many people work on putting openplotter together?

It is hard to say. A lot of people contribute to software or hardware while they are configuring the system on their boats and then disappear as the sailing season starts but there is also an army of loyal beta-testers who follow the development closely. To be more precise, we are currently a group of two people coding and three people developing and testing hardware.

Navigatrix fulfils a similar function to openplotter but on a different platform. Are you continuing to develop navigatrix, are you putting all your effort into openplotter, or do you intend to combine the two into one multi-platform project?

Despite having contributed to Navigatrix development in different ways for a long time, I can not consider myself part of the development team. Navigatrix has its own course and still a long road ahead.

I think both are complementary projects. Navigatrix is focused on running all the available nautical software on any PC computer. Its main feature is portability. You can sail on any ship with your Navigatrix laptop and you can even sail with Navigatrix installed on a USB stick in your pocket and run it on any computer on board. However OpenPlotter is focused on hardware, permanent installations and enhanced interaction with sensors.

How many people do you think are currently using openplotter / navigatrix?

It is hard to say again. Since this must be sustainable, we use free file sharing sites and torrent downloads and we do not have an accurate number of users or downloads. We have feedback from every continent and sea area, so I guess they could be hundreds or even thousands.

You sell accessories which are fully supported with openplotter and are completely transparent with the (optional) donation attached to the "sponsoring edition" SD card. It's a fantastic business model, but is openplotter a business, or is it still a hobby?

We believe in open source, transparency and fair business.  Development and coordination take a lot of time. We need to make some profit to keep working on it and this is how we try to do it:

Probably all your instruments are manufactured in Asia, but development, support and merchandising is done by third parties who obviously increase the final price. Retail shopping in the Asian market may be cheaper but it's also a traumatic experience due to the lack of product documentation and sales support. Too often you finally get a product that doesn't fit the promised specifications, doesn't work properly or takes more than a month to arrive. We try to be a filter. We do the hard work of finding reliable providers and charge a little money for doing so.

In the coming months we will open the "fake shop": a cost price online shop with all the junk and fake products that we have gathered in order to give them an opportunity for a second life in other projects. There will also be a list of providers we have found unreliable to help those who can't even afford our selected products and want to fight their own way through the Asian jungle.

If "still a hobby": What's your day job?

This model of donations/selling tested products seems to work and I can work half time on OpenPlotter. I still need to work as freelance web developer if I want to a living wage but at least now I can be selective about the clients and projects I choose.

Do you sail a boat, and if so what do you use openplotter for aboard?

My boat is gone but it is said that the best boat will be always your friends’ boat. I have seen OpenPlotter installations in all kind of boats.

What features of openplotter are the most popular with users?

The headline is always OpenCPN and its plugins. Second is the SDR AIS receiver followed by engine sensors and instrument panels.  Third is "home automation" features to monitor and control your boat while you are away.

Are openplotter users making much use of Signal K?

Not much at the moment. I think they feel that SK is just a way to receive N2K data but then they don't know what to do with these data. They know the basics but they don't have a sexy instrument panel or phone app to realize its power yet.

Galaxy Gear S2 driven by OpenPlotter and Signal K

Do you see Signal K overtaking NMEA-0183 as the predominant protocol for marine data communications between applications on IP networks, and if so when?

NMEA 0183 will be with us for a long time because it is the language which GPS and AIS talks but above all because it is the only language that OpenCPN can understand at the moment. But NMEA 0183 is not capable of managing non strict navigation data. Many types of sensors have begun to appear in our boats and homes so we need to store, manage, and show these data and another proprietary and limited protocol as N2K is not an option. There are already some SK servers on the market like OpenPlotter but I think the big change is down to SK apps developers, OpenCPN included.

What would make Signal K better?

Everything takes time, money, and monkeys. You need a lot from any two groups, and a little from the third. An increase in any one reduces the requirement for the other two. Change occurs when one of those three change.

Moe's Law, Navigatrix team.

I think SK now needs monkeys to get to a stable version asap and to improve the documentation.

You recently added MQTT support to openplotter, marrying marine data with the mainstream IoT world.  Have users reacted positively or remained uninterested?

Yes, some of them are excited about this and they are already playing with sensors but most of them do not have a clear idea about how they can set up a IoT system on their boats with OpenPlotter. They need manuals and a good documentation and we cannot offer this at the moment since our resources are limited and we have to choose between developing or documenting.

User-contributed MQTT-driven dashboard: http://forum.openmarine.net/showthread.php?tid=162

What format are you using to publish to MQTT and why not use Signal K/JSON objects?

OpenPlotter can send any kind of string including JSON objects over MQTT. It is up to the subscribers to parse and manage the string. In future OpenPlotter releases we will make building JSON objects easier, NMEA sentences… using any navigational data.

Is openplotter being used as a generic IoT device in non-marine applications?

I don't know, maybe. There are a lot of IoT home systems based on the Raspberry Pi, since OpenPlotter is too much boats oriented, maybe it would be better to use any of them.

What are the future plans for the openplotter project?

We are still under construction. The main goal now is to fully implement SK in version 0.9.0 in order to be able to deal with analogue sensors (batteries, engine, tanks...) properly.  We would also like to offer a good virtual instrument panel to round things off.

Once the goals are achieved, the next logic step is focus on hardware. Nowadays OpenPlotter is a project for “makers” and this always involves some skill with electronics or at least a big curiosity. We need to simplify the required devices and develop well documented hardware to make it easier for everyone to build a custom OpenPlotter system on board.

vyacht router: Now with Signal K

First Blood

Swedish marine electronics producer vyacht appears to be first to market with a commercial Signal K offering.  New firmware released at the end of January for the vyacht WiFi NMEA router supports Signal K.

The vyacht router is an open source (hardware and software) device which bridges between NMEA-2000, NMEA-0183 plus optionally Seatalk-1 networks and wifi (and optionally ethernet) with the base model costing just SEK1499: a little under €170 at the time of writing.

Bernd from vYacht was kind enough to answer some questions about vyacht, the router, the new firmware release and Signal K.

About vyacht

You’ve been producing the vyacht router for longer than vyacht AB has existed.  Did the project start as a hobby?

Its a very intensive hobby, although it's grown bigger and more successful than you could run outside a Swedish aktiebolag [corporation] without losing sleep.

The vyacht workbench

Do you now work full time on vyacht?

I am currently starting to look into scaling the business up and make it full time. I need people looking after sales and support and setting up distribution in various countries: the direct sales model is limiting. 

And an investor. If someone interested reads this: its not too late to talk to me. 

The vyacht router is a very mature product now and probably the most advanced, flexible and extensible on the market. The market is huge and has just started to ramp up in vyacht's space of innovating in the integration of boat networks, web, mobile and wireless. 

Do you have any other products planned?

There are at least 3 or 4 ready-to-ship products. I just haven't got around to launching them as I am focusing on the router. 

There is also new functionality in the pipeline for the router itself. It's easy to extend even on the hardware side. It's capable of a lot already but ideas and possibilities are endless just on the software side.

The vyacht router

Is the vyacht router NMEA certified, and if not, is that something potential customers should be concerned about?

Like most small companies, vyacht hasn't sought NMEA certification. The protocols are publicly well documented thanks to the work of many volunteers. vyacht works with all the software and apps out there: none of those that I am aware of is NMEA certified. They are based on the public documentation just like vyacht.

The end user doesn't need to worry. vyacht users have successfully tested the router against probably hundreds of different NMEA instruments and devices. I have seen and fixed all sorts of issues but never any that was specifically about NMEA incompatibility.
To my knowledge even Raymarine only certify their MFDs but most of their other products are not NMEA certified either.

I am very critical of propriety standards. Unlike some I don't even see a positive marketing or quality effect of using the NMEA certification.

Are all the vyacht software sources available on github?

Sometimes I am a bit behind, but I aim to have all software uploaded there reflecting the latest state.

The vyacht router is notably much cheaper than competing products from much larger manufacturers. How do you manage this?

Margin! On the suggested retail price they will have a margin of at least 200 USD, probably much more depending on their hardware.

Also my product is a bit simpler in terms of the case and terminals.

I produce in batches of 100.  I'm not sure of their batch sizes. Theirs will cost as much in production. If they produce 1000 then prices will drop dramatically. 

I run it as a very extensive hobby project and 2nd job costing my health ;-) I am not financially dependent on it - and my profit is 0. 

I started the whole project because I was so frustrated about the price level. Boat owners are the "milking cows" for nice earnings - I thought.  Now I know: hardware is expensive, unbelievably expensive and much more than the sum of its sourced parts. If you have a case and knobs and dials it becomes extremely expensive. If you run a company you need to have at least 200 - 300% margin and volume > 1k to survive. Just accounting rules around warehousing will kill you.

The new vyacht firmware release

As far as you are aware, is the vyacht router the first commercially available Signal K device?

To my knowledge, yes. The first commercial product and company behind Signal K with full user support for the Signal K components installed.

Why did you implement Signal K on vyacht? Have there been requests from customers?

There was not much interest in Signal K from the vyacht community and so far it hasn't been driving any additional interest into the router. 

I see it as an investment, experiment and as support for a great project. I like and share the vision behind Signal K, and everyone can immediately see the added user value. The instrument panel is built into the router accessible without any additional software installed anywhere: all you need is the router and a browser.

Does the new vyacht firmware implement a full Signal K server, or like the iKommunicate act as a gateway device translating NMEA and Seatalk data into Signal K deltas?

Its a full Signal K server that serves the full format and the delta of updates. 

It might be lacking some implementation details such as the full blown set of own boat data, some engine data or receiving of 3rd party Signal K data as an input. 

I await user feedback on where to invest more to close the gaps.

Have you based the server implementation on the Signal K reference servers or written your own from the specifications?

I used the node server as a reference at times but actually wrote mine from the specifications. There is a difference between the specification and the two reference servers, so nothing is set in stone as things change often in Signal K.

Last year you suggested to the Signal K team that they look at popular IoT transports for use with Signal K. Does your new firmware implement MQTT or CoAP?

No. Maybe as a gateway to Signal K if it starts to make sense.

Are there any incompatibilities or notable differences in operation or functionality between the vyacht Signal K implementation and that of the reference servers?

The node reference server is different from the Java server. The vyacht server which is written in pure C is probably slightly different from either of them. 

There are also many web servers implementations out there which are all very different from each other but which communicate with any web browser using HTML. 

I successfully tested interoperability e.g. of the instrument panel and the vyacht server.

Is conversion from NMEA protocols to Signal K one way or can the vyacht router send Signal K deltas on its NMEA output?

It would be a 2 line change in the software to enable that if someone asks for it, but Signal K is quite chatty and I'd expect performance issues over a serial interface such as an NMEA 0183 output.

Signal K is still a "work in progress". How often do you anticipate having to update the vyacht firmware to keep pace with developments?

Probably more often than most users would like. They just want to have something ready to work and not install firmware updates. 

Its still impossible for a non-developer and time consuming even for a developer to get started on this project. 

At least for components like the instrument panel my main effort is almost like that of any Linux package maintainer: pick out the stable parts and package them in a usable way. 

The real advantage of vyacht for Signal K is that vyacht makes Signal K very accessible to users by being pre-installed and supported. 

What I deliver is just there and works out of the box.

Your latest firmware works with Navionics sonarchart live. How did you implement that?

It was just a matter of experimenting and making it configurable for the end user: broadcasting standard NMEA 0183 sentences via UDP on port 2000 and making sure timestamps for GPS data were correct. 

In the end Navionics even supported me based on user demand. I probably messed up their community charts quite a bit though with all the artificially generated test data that I used to make it work. 

Signal K

What do you believe the impact of Signal K will be on marine electronics and software?

Signal K or at least the ideas behind it will completely change marine electronics and software. 

The marine hardware industry and the NMEA are light years behind any other industry, in a nice comfort zone delivering technically outdated overly expensive equipment to customers used to spending a lot of money.

This is changing already without Signal K. First of all its not driven by an industry any more but by many individual enthusiasts. It all started with us just bringing to our own boats what we felt was normal everywhere else already: accessibility and availability everywhere, communication, ease of use and affordability.

Today you can get better-than-ever racing or weather routing software almost for free and made by extremely experienced sailing pros. You had to spend some thousand $ for this just 2 years ago. 

Signal K itself is not only about networking on a boat. The web, Facebook and the iPhone refined our expectations about how things should work. And Signal K, vyacht and many others take this and refine how electronics should work on boats and maybe even more important: between boats. 

Sure, there is a way to go still, but we have come a long way already and large vendors sleep tight. It currently feels a bit like the web around 1995. All the companies, software and stores we know today were born then or after that. Almost none of the older ones are relevant to the web today.

Do you see Signal K as a potential alternative to NMEA protocols, or would an “open” marine network require a protocol better suited to constrained devices?

Signal K is not an alternative to the NMEA protocols. Signal K is a complement or amendment to NMEA. Like a NMEA 2.0 for web. This is probably why the NMEA as an association recognised Signal K. 

Signal K is designed for the web and the communication between a browser and a web service. Its unsuitable for communication with a sonar. 

Signal K is perfect for the communication between a vyacht router and an iPad or tablet but too web-centric for even powerful constrained devices.

As a commercial manufacturer, what aspects of Signal K do you think would benefit most from further development?

1. Make it easy for normal boat folks who are not software engineers to set it up and run. We need to build a wider community and attract all those not so deeply technical boat folks out there who can drive it forward.
2. Reduce the protocol's chattiness and redundancies as well as clean out and document the overall structure for a better understanding and higher level of suitability for smaller devices.
3. Signal K suffers from the same unclear semantics, data life time and data cycle issues that the NMEA 0183 protocol suffers from which NMEA 2000 partially fixes: that needs to be improved.
4. Cut ties with the NMEA and remove references to legacy NMEA protocols in Signal K.
5. Signal K needs to become (more) open.

The last points are maybe the most important ones: The decision making process in Signal K is quite closed and focused on the core development group. That isn't optimal for gaining active participation in development of a new and open standard. Open standards give every participant equal voting rights.

Also you can't really distinguish between the reference implementation and the specification. I think the reference implementation and not the community is driving the protocol specification. 

Signal K needs to decide if it wants to be a piece of software or a standard that changes the industry. Currently its the software. And in the past 20 years I have seen many good standards going down the drain by making this mistake.

I know from talking to a lot of small vendors that they are waiting to see how things will go for Signal K. Some have even implemented it but haven't released it due to concerns they share.

As long as nobody moves or they move in different directions we obviously don't get anywhere. If 100 or even just 20 software, app and hardware developers all move in the same direction, then we will change the marine software and hardware landscape quickly for the benefit of both us young marine electronics and software folks and all the boat enthusiasts out there.

iKommunicate

People in Tha Middle

Last friday a post on the Signal K google group announced a kickstarter campaign for the first commercial NMEA-to-signal-K gateway to be announced.  The iKommunicate has been prototyped by Digital Yacht, a company whose track record includes being one of the earliest to market with NMEA-to-wireless-IP converters.  iKommunicate features an NMEA-2000 interface, two NMEA-0183 interfaces and an ethernet interface.  NMEA data are converted to Signal K format which is then available for Signal K client applications to query.

Paul Sumpner is Digital Yacht's CTO.  We had talked about Signal K at the London Boat Show in January and when I read the iKommunicate announcement I mailed him to ask if he'd mind answering a few questions about it.  He very kindly agreed.....

Digital Yacht and Signal K

Paul, Digital Yacht were instrumental in popularising the use of “NMEA-0183-over-IP” with their NMEA-to-wifi bridges. What do you believe will be the most significant driver for Signal K supplanting this as data format of choice for app developers?

NMEA-0183 over IP was a good initial solution and for some people will be quite satisfactory, but for the developer there is still the issue of getting the NMEA0183 information, the cost of the spec and NMEA membership puts a lot of developers off and also they have to support a relatively specialist and old protocol.  

The driver for Signal K will be the NMEA2000 data that is not supported in NMEA0183, such as Engine data which until now has been inaccessible to all but a very few developers and once developers get their hands on this data, I think we will see some really interesting and innovative apps.

There are currently very few apps which can use Signal K data and almost none which can only use Signal K as a data source. Who do you see as the early adopters who will buy the first tranche of iKommunicates?

I think once a couple of apps that have a large user base, such as Active Captain or OpenCPN fully release with Signal K support (both have beta releases) then I think iKommunicate will be in demand. Michael Zapf the developer of the popular NMEARemote Instrument App is the first iOS developer to support SK and we will be contacting other iOS and Android App developers about Signal K and iKommunicate.

A kickstarter campaign is an unusual method for an established business like Digital Yacht to fund a new product and your $20,000 target surely can't cover development costs. Did you take this approach to mitigate risk in building a product with unknown demand, to create a pre-release buzz or for another reason?

It is very easy to look at Kickstarter and just see it as a way of getting funding for start-ups but I think any company can benefit from using this medium. If you are developing a completely new and unique product or trying to break in to a new market, then Kickstarter is a great way to promote your new initiative, gain feedback on whether there is a market for your new product/service and to get directly in contact with the early adopters in that market 

Apart from the time and effort in setting up the campaign, there are no other costs and if it turns out that no one likes your new product/service and you do not achieve funding, then you have avoided wasting a lot more time and effort developing a product/service that no one will buy. This is our first Kickstarter campaign and we would certainly use it again.

The kickstarter campaign was launched only days ago but you are more than a quarter of the way to achieving funding (I will be surprised if it's not at least “half” by the time you receive this). In the unlikely event that you don't meet your target, what would be the trigger for Digital Yacht to revisit iKommunicate in future?

You are right, as I am writing this answer, we have just broken the 50% funding target, so fingers crossed we will reach and hopefully exceed our original $20,000 target. Even if we do not reach our target, we hate wasting development time and effort, so we will probably regurgitate the work we have done in to some other new product in the future, but it would be a real shame as I think what Signal K is trying to achieve would be good for our industry.

Even at the full $299 SRP, the iKommunicate is competitively priced compared with some other NMEA-to-IP bridge products. Is it worth buying for its N2K/NMEA-0183 ethernet bridging and multiplexing capabilities alone?

Absolutely, we have really tried to make iKommunicate competitively priced and straight out of the box it will be a very powerful NMEA2000 and NMEA0183 to ethernet bridge, supplying data to many existing apps and software packages over TCP or UDP. Then as Signal K becomes more and more established, iKommunicate can simultaneously supply other devices on the network with the Signal K data.

iKommunicate Product Details

Does iKommunicate implement a complete Signal K server, and if so were compromises needed to make it run in only 1MB memory?

The differences between a Signal K server and a Signal K gateway are somewhat blurred and some Signal K developers would argue that there should be no difference between them. I don't entirely agree with this and designed iKommunicate to be primarily a gateway but also capable of being a basic Signal K server. 

My own view is that in a smaller boat, you will probably just have an iKommunicate and some Apps (Signal K "consumers") that will use the data. So in this instance, iKommunicate is the Signal K server. 

On a larger boat, you may have an iKommunicate and a Signal K server where iKommunicate is just a gateway converting the NMEA data in to Signal K but the server is taking this data, managing it and then serving it up to the apps or sending it to the cloud. 

iKommunicate has been designed to be a very efficient gateway and a low power server, capable of hosting some simple web apps, but not capable of running sophisticated apps, with charts, messaging, logging, etc. We are not running LINUX on iKommunicate, just native C code on an ARM M4 Cortex processor and currently we are not even using half of the 1MB of memory, even when receiving 300+ AIS targets. 

So rather than compromises, we feel that a number of new and different design decisions were taken with iKommunicate and I am very happy with how our "proof of concept" has performed and been received. 

I think a full blown Signal K server needs to run an OS, which is an overhead that requires a faster processor and more memory, but you get access to loads of software services and utilities, making it so much easier to do really powerful and innovative functions and features. That is why the Raspberry Pi, Beagle Board and other COTs hardware solutions have been initially used as the basis for Signal K servers.

Does iKommunicate store vessel state (gleaned from NMEA inputs) in an internal data model, or simply dynamically translate NMEA PGNs/sentences into delta format with Signal K context which are then supplied to all websocket clients? And are there any static data supported/query-able in the initial release?

iKommunicate has a Signal K namespace memory in which it retains the latest values of all the NMEA data that has been received including AIS data (other Vessels). When a NMEA sentence of PGN is received, it updates the namespace and sends out a Signal K delta file. If an HTTP: restAPI GET request comes in, then this is extracted from the namespace and output as SIgnal K HTTP.

We will have an option in the iKommunicate web config page called "Manage Own Data" or similar and this will make the iKommunicate responsible for populating all of the Signal K "Self" data. The static Own/Self data will be populated where possible automatically from the NMEA data or the user can enter the data manually like Boat Name, Dimensions, MMSI, etc.

What (if any) are the major differences between Digital Yacht's Signal K server implementation and the reference open source server?

The main differences are:

1. iKommunicate connects directly to the NMEA2000 network, rather than via a 3rd party NMEA2000 to USB gateway.
2. iKommunicate is a LAN rather than a WAN device and another server or app is required to handle the external transmission and reception of the Signal K data in to the internet.   
3. iKommunicate (at least in Version 1) does not support subscription websockets, it simply sends all "delta" files through the websocket i.e. every time a new NMEA PGN or Sentence is received it transmits a Signal K "delta" file. The idea with subscriptions is that the app asks for specific data and the server remembers what data the app is interested in and only sends this data when it changes

Is the flow of information through iKommunicate “one-way” or can data be passed back to the NMEA side, e.g. to control an autopilot?

In Version 1 of iKommunicate there is only data flowing from the NMEA networks to Signal K, apart from the minimum NMEA2000 network
interaction i.e. address claiming, heartbeat PGN, etc. We will be adding bi-directional NMEA communication though in a future firmware release, which by the way is designed to be really easy to update; either by uploading an update via its built-in web interface, downloading directly from the internet or copying it to the internal SD card.

Security is something that has been discussed more than directly implemented in Signal K. What (if any) security challenges will iKommunicate address in its initial implementation and what do you intend to address in future firmware updates?

The security of the "Internet of Things" in general is being discussed ad infinitum by many, many technology companies and organisations and still has some way to go before the procedures, tools and mechanisms are in place for a secure "IoT" world. The Signal K team could have wasted a lot of time and effort on security and it was felt that, at least for Version 1 of Signal K and iKommunicate, that the issue of security should be left with the Apps and Services that were taking Signal K data from the boat and transmitting it in to the internet. 

In reality every boat has a large and very good security "firewall", the open sea they are sailing on, and for the vast majority of Signal K users, the data will stay firmly on the boat, at least until more apps and services become available. On larger commercial vessels and super yachts, there will already be a secure and protected network onboard which Signal K can hide behind, as after all it is just normal network data. 

We are following closely how the security of the "IoT" is progressing outside of the marine world, in particular open source initiatives such as Let's Encrypt (https://letsencrypt.org/) to create free SSL certificates. As and when a clear security solution for "IoT" appears, we will aim to include it in Signal K.

Future Direction

At this stage Signal K complements rather than replaces NMEA protocols, with products like iKommunicate providing the necessary bridge. How much harder / more expensive (if at all) would it be to implement a sensor (e.g. GPS, Heading) outputting “native” Signal K (in its current form) than one which outputs NMEA-0183 or N2K (or, dare I mention, OneNet?)

Any "smart" sensor that has a microprocessor, rather than just a transducer inside, could in theory be a Signal K sensor. In component cost terms there is a bit of a step up from NMEA0183 to Signal K, but the difference between an NMEA2000 interface and a Signal K interface is very small and in fact it might actually be cheaper to do a Signal K interface. 

You would really want the Signal K sensor to use PoE (Power over Ethernet) which will add some additional component cost but it is not completely out of the question, I am not sure it will be Digital Yacht, but I am sure we will see Signal K sensors before too long.

If iKommunicate proves successful, what other Signal K products are Digital Yacht considering?

We intentionally did not include Wi-Fi in the first version of iKommunicate, as we felt that most early adopters would be pretty technical users and most likely have their own wireless network onboard already, but a wireless iKommunicate would be useful for smaller boats, so that is definitely on our future product roadmap, as is a J1939 version for direct connection to engines that do not natively use NMEA2000. As for anything else, assuming Signal K is the success we hope it will be, let us just say "watch this space" !

As well as implementing Signal K in its products, do you see Digital Yacht (and indeed other commercial organisations) taking an active role in Signal K development?

I hope so and Digital Yacht certainly intend to be an active contributor to Signal K. There are challenges for commercial companies to be involved in "Open Source" projects and I am sure, a certain level of dis-trust from the purist Open Source community members, but there are benefits for both sides and as long as commercial companies put in as much (or more) as they take out, then successful results can be achieved.

Many thanks to Paul for taking the time to share this.  As I write this, just 5 days into the kickstarter campaign and with 47 days still to go iKommunicate has reached nearly 4 times its funding target with more than 340 units spoken for and the number climbing all the time.  Seems like iKommunicate will be a real product and Signal K does have a commercial market.

Signal K revisited (part 2)

Talking 'Bout a Revolution

Signal K isn't what I was looking for last year when seeking an "NMEA protocol alternative". The problem to be solved is how to interface software with sensors and devices on a boat when those sensors and devices use a closed communications protocol whose owners want a (to many independent developers prohibitively high) fee for its use and forbid its use in free software. Wireless networks, tablets and smart phones are now commonplace on boats.  A good solution opens the gateway to even more applications, potentially revolutionising the way many people interact with their vessels.

I believed the solution was a new protocol satisfying the constraints of current NMEA implementations (compact, friendly to resource-limited devices and low-powered networks, easily implemented in low level languages) but running over Internet Protocol and free and open rather than proprietary.  I was initially disappointed that Signal K isn't that thing but I started to like it a lot more once I accepted that and appreciated it for what it is.

Communications Breakdown

Signal K addresses the same problem in a different way.  It's a way of representing the state of a vessel and a format for transfer of all or part of that state to a client application.  It isn't oriented to minimal resource on low powered devices.  Instead it is built to provide data to applications running on consumer devices over full-fat wifi or ethernet networks and is geared towards functionality rather than minimalism.

A "Signal K server" is an entity which acts as an aggregator for information inputs which can come from a variety of sources using any protocols which the Signal K server understands. It uses these data and static information which the user has manually input to create a model of the "state" of the boat and potentially other vessels around it.  In early iterations of Signal K, client applications would update themselves on boat data by regularly transferring the server's internal data model.  As Signal K progressed a method of transferring only new data was adopted (the "delta format") and a method of "subscribing" to updates of particular data.  The transport by which data are transferred is not mandated, although there is now a method of querying the available transports (over http).  A reliable transport is occasionally assumed and the de facto standard transport as used in the reference server implementation seems to be websockets.

What back-end protocols does a Signal K server understand? This is implementation dependent.  The Signal K project defines the architecture and provides a reference implementation (in java and javascript) but the architecture could be implemented by others differently.

The reference server includes providers which can interpret NMEA-0183 and the output of the canboat  analyzer, the latter enabling those with a method of reading NMEA-2000 PGNs (the Actisense NGT-1) to interpret some N2K data.

You're Not the One (I was looking for)

I don't care for monolithic central server architectures, nor of putting extra steps between data producers and data consumers but the data model I initially disliked has many practical advantages.  Although the security implementation is a little sketchy at present it offers the potential for sophisticated access control (like a directory server) allowing detailed specification of who has access to what.  It allows clients to specify how often they want to update their data rather than being passive consumers.  The reality is also that for a while yet our sensors and other marine devices are still going to be talking (a variety of) proprietary protocols rather than open ones so translation will be needed, and the Signal K server provides a straight forward pluggable architecture for that.  I'm not a fan of the (to my mind) unnecessary overhead of http / websockets but this is not a concern in the typical Signal K use case, it simplifies coding in the higher level languages that most app developers use, and the Signal K developers would argue that those transports are not required (despite them being de facto standards).

Picture This

There's currently plenty of details which aren't currently filled in with Signal K.  The most common data have standard representations but much does not.  In many cases this is just a matter of time and work which but much of this needs to be done for Signal K to realise its potential. It is frequently argued that Signal K's JSON format makes it infinitely extensible but it is not without some challenges.

A problem for current NMEA protocols is that they have no standard way of representing imaging data from RADAR and Sonar, relying instead on individual manufacturer's proprietary protocols.  Signal K doesn't yet solve this.  Representing image data in a text-based protocol generally requires encoding which makes high bandwidth data even more bloated.  One proposal to address this has been to use Signal K only to transport the URL of a binary imaging data source which would then be connected to by other means.

Signal K has had many of its rough edges smoothed in the past six months, but it is still a work in progress.

Safe From Harm

Security doesn't feature much in traditional marine data communications.  Most people are aware that that has to change as marine devices become accessible from outside the closed environment of a boat's internal wiring.  There are many aspects to "security" including authentication (proving something says what it says it is), authorisation (saying who can do or see what) and encryption (concealing things from prying eyes).  Using a Signal K server rather than devices publishing data directly to the network allows for a great deal of control over who has access to what.  This potential is not yet well realised, but the architecture should be well suited to it.

Authentication and encryption are the more glamorous aspects of "security" and have been extensively discussed on the Signal K google group but with no definitive mechanisms yet mandated.  The use of web protocols should allow a huge range of well-proven systems to be leveraged but possibly due to a reticence to mandate transports for Signal K, reliance hasn't been put on readily available security mechanisms associated with those transports. I can't help but wonder if standardising on the transports which people are currently using Signal K with and restricting the problem space might result in a more full-featured solution for the common use case rather than an unconstrained but less well defined product.

Welcome to the Machine

Signal K still has a few details to be filled in but it certainly has great potential in its primary use case: providing data from NMEA devices to mobile and web applications and enabling such applications to communicate with each other using the same, extensible format. I feel confident that it will gain considerably popularity over the next year.  It lacks compactness and the tight specification to be ideal as a replacement for NMEA protocols but that's not where it is targeted. It is a complement to those protocols.  The biggest initial driver to adoption is likely to be the ability to represent NMEA-2000 data, much of which can't be represented in the currently popular "NMEA-0183-over-IP" format.  Due to CAN interfaces (as used by NMEA-2000) being less trivial than RS-422 interfaces (as used by NMEA-018) and only part of NMEA-2000 having been reverse engineered, this is where commercial NMEA-to-Signal K gateways (whose vendors have access to the complete NMEA-2000 specification) come in as part of the architecture now embraced by the Signal K team and the NMEA alike.  The average user will need to buy such a device to link their Signal K applications to their boat network.

The big news this week is that Digital Yacht have just announced potentially the first commercial NMEA-to-Signal-K gateway: the iKommunicate   Even more exciting (well, for me anyway), Digital Yacht CTO Paul Sumpner has kindly agreed to answer some questions on the new product and Signal K.  Stay tuned!

Marine Data Transports Re-visited

Last year I wrote a post in which I extolled the virtues of UDP multicast for transmission of sensor information within a boat network.  Whilst that conclusion still stands in theory, real world practice is somewhat different when it comes to 802.11 wireless networks.

In my post on transmission of RADAR data over wireless networks, I mentioned that at layer 2 (the "datalink layer" in the OSI network model) "wifi" multicast packets (of which broadcast are a special case) are not acknowledged by the receiving end whereas unicast packets (those explicitly sent to one receiving station) are, so the sending station knows when it must retransmit a dropped packet.  Some may say "but isn't UDP always unreliable and unacknowledged?".  At the transport layer yes but the 802.11 ("wifi") frames that these packets are transported in don't care about the transport layer.   "Wifi" is far more "lossy" than transmission over copper cable, so 802.11 addresses this where it can by acknowledgements and retransmissions.  This is not possible for multicast (/broadcast) packets, but is for unicast.

In using my homebrew Raspberry Pi 2-based access point/data server I was noticing the GPS fix regularly dropping out in OpenCPN and AIS targets taking a long time to appear.  Some testing showed appallingly bad reliability of transmission using UDP broadcast.

The test was simple.  A MacBook Pro (the receiving station) was a wireless client of the Pi access point and located 2 metres from it in the saloon of my GRP boat.  The Pi was running kplex and, at that point, transmitting only GPS information from a directly attached Adafruit Ultimate GPS at a rate of 6 sentences per second.  The access point was not connected to the Internet and thus other traffic was minimal.  There are few access points visible on the Mac, and only the Pi transmitting on its chosen channel.

tcpdump was run simultaneously on the Pi and the Mac to record the number of "NMEA-0183" packets transmitted and received respectively over UDP broadcast during a 10 minute period.  Packet loss was then calculated: 96%!

The test was then repeated but with kplex transmitting packets via UDP unicast.  This time packet loss was, as expected, 0%.

It should be remembered that this was a home-brew access point solution using a low cost USB dongle and I'd be interested to hear of other people's experiences with commercial access points, especially expensive "marine" units like Navico's Wifi-1.  I note that Brookhouse Online, manufacturers of the "iMux" commercial multiplexer, state that "UDP is unsuitable for transmission of navigation data" (although I'll argue that actually they mean "UDP broadcast over wifi").

Meantime though, 96% packet loss clearly makes broadcast/multicast over wifi non-viable.  The alternatives?

• Use TCP.  Easy. Well supported
• Use UDP unicast.  Client addresses must be known in advance or some (currently non-existent) subscribe/publish protocol needs to be developed for marine data
• Use an access point which implements UDP multicast over 802.11 unicast (as discussed in the RADAR post).  Support for this is rare, as is support for NMEA-0183 data over multicast.

For most people the solution for now would seem to be to use TCP for delivering marine data over wireless networks.

Navionics SonarChart™ Live on Android

Skip to the good bit

For a quick hack to get SonarChart™ Live on android working with a home-brew or "unsupported" serial-to-wifi solution without the background, jump here.

How Deep Is Your Love?

A few months back a kplex user made me aware of a new feature in Navionics Boating: SonarChart™ Live.  This is a new feature in Navionics which enables users to generate bathymetric maps using current (hence, presumably, "Live") depth information from their own instruments.  More specifically,  data provided via selected partners' products. 

Anthony was trying to work out how to feed depth information via kplex to Navionics.  He'd done some investigation of the iOS version of Navionics and determined that the app was trying to contact fixed IP addresses and ports associated with the default set-up of the various "supported" partner products.  In subsequent analysis of the app I also noticed references to the bonjour services associated with GoFree (_nmea-0183._tcp) and Vesper (_vesper-nmea0183._tcp).

By setting up his pi to emulate the Digital Yacht Sonar Server, Anthony managed to feed depth information into Navionics from kplex but I thought that there must be a better way to achieve this so I emailed Navionics to ask how Navionics Boating users with existing serial-to-wifi solutions could realise the benefits of this exciting new feature. Not possible came the reply, and no they wouldn't reveal how their "automatic" connection worked:

We have developed at code level this features together with our partners. We do not open it externally at the moment and there's not any future plan to do it.

I asked for this to be escalated to product management as surely they'd want as many people as possible to enjoy this feature. I was promised a reply but despite several prompts over the following weeks one never came.

Echoes

A few days ago my android phone prompted me to upgrade Navionics Boating. The android version previously didn't support SonarChart™ Live but now it does, so despite my disinclination to allow the draconian new permission requirements (access to all my accounts? Make phone calls? Access to my contacts?  Why?) I upgraded.  With my Raymarine plotter bridging my seatalk-1 depth sounder's data to NMEA, converted to wifi by kplex running on my raspberry pi access point, I started experimenting.

Say Hello

First up...would advertising _nmea-0183._tcp as Navico's GoFree does tell Navionics to look for depth data in the right place?  I installed avahi (the linux bonjour implementation) and configured a service, creating a file /etc/avahi/services/kplex.service as follows:

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->

<!DOCTYPE service-group SYSTEM "avahi-service.dtd">

<service-group>

  <name replace-wildcards="yes">%h NMEA</name>

  <service>

    <type>_nmea-0183._tcp</type>

    <port>11010</port>

  </service>

</service-group>

No joy.  For good measure I also tried configuring the _vesper-nmea0183._tcp service. Still nothing.

Listen Like Thieves

Some network snooping showed the phone attempting to connect to 192.168.0.10 on port 10110.  This was the only address I could see it trying apart from an immense volume of traffic to MarkMonitor (apparently a "brand protection" company), Flurry and Amazon Web Services (presumably Navionics's servers).  Good to know how they're using your bandwidth and exploiting all those security privileges then.

My pi is set up as a router (so will move traffic from one interface to another) so I ran up a secondary interface, assigned it the address 192.168.0.10, restarted kplex (which was implementing a tcp server on all interfaces), restarted Navionics boating and Bingo!  SonarChart™ Live functionality.

My Name Is

By curious co-incidence 192.168.0.10 is the first DHCP address handed out by the Navico GoFree Wifi-1 in default configuration and would often be the address handed out to a plotter (distributing GoFree data) on a Navico network.  Further experimentation whilst attempting to emulate a Navico network revealed something bizarre.  Without the 192.168.0.10 address configured, if the SSID of the network the phone connects to contains the string "GoFree" (e.g. "GoFree-1234") and a data service on 192.168.0.10 is not available, the Navionics app throws up a screen asking the user to input the address for the source of depth information.  Port is not configurable and seems to be fixed at 10110 but inputting the pi NMEA server's address once again gave me SonarChart™ Live functionality.

This seems to be a very crude method of attempting to limit the data servers which can be used with SonarChart™ Live given that SSID is configurable and some GoFree wifi-1 users will have doubtless changed theirs from default.  Knowing less about the set-up of other supported data servers (e.g. Vexilar T-Box, Raymarine Dragonfly) I have not investigated whether default SSID plays a part in presuming them to be sources of data.

Disco 2000

So what of UDP?  For some reason Digital Yacht's products have used 2000 as a port for their wireless NMEA servers in the past.  I've had a hard time finding a manual for their SonarServer (one of the Navionics "SonarChart" supported products).  I tried UDP broadcasts from kplex on port 10110 to no avail but changing the port to 2000 once again yielded success.

Where's Your Head At?

Interestingly Navionics Boating seems to be parsing GPS data it is being sent over wifi.  Turn off location services on android and the app doesn't know your position when it fires up.  Turn on wireless data which includes GPS as well as depth sentences and your position is plotted.   This should be of particular interest to people who bought duff mobile devices which don't include an integrated GPS chipset.

Different Trains

It seems to be the case that the iOS version of Navionics may well function differently from the android one in how it determines where its data comes from.  Bonjour is well established on Apple platforms but Network Service Discovery was only added in android 4.1.  On the other hand "wifi multicast reception" is one of the (many, many) privileges Boating requests on android.  Sadly I don't have an iOS device with Navionics to play with in order to learn more.

As Navico's use of bonjour as one of the two ways they advertise their GoFree data service isn't a bad plan, I've decided to retain my avahi configuration on the pi for the time being.

The First Cut is the Deepest

So how best to get SonarChart™ Live on android to use our home-brew data server?  You could run up an interface on 192.168.0.10 but the easiest solution would appear to be one of the following.

If you prefer to access your data over TCP, change your SSID to something with the string "GoFree" in it, e.g. "GoFree-1234" and run an nmea server on port 10110.  When you open Navionics Boating it will ask you for the address of your server, tell it, and you can start using SonarCharts™ Live.

Otherwise broadcast your data over UDP port 2000.

If anyone has a better suggestion on how to achieve this with either the android or iOS versions of Navionics Boating, please do post a comment below.