The last few years have produced many new specifications and interaction patterns focused specifically on the Internet of Things (IoT) and its countless siblings such as the Internet of Everything, Industrial Internet or Web of Things. Part of my job is to track those developments and look for trends in API design which will inform our future product design.
While there is a passionate debate about how similar or different IoT applications and products are to other common API use cases, I think one key difference everyone can agree on is the increasing use of asynchronous message exchange pattern, also known as publish and subscribe. Our physical world is asynchronous: “stuff just happens.” Of course this is nothing new in the enterprise world?IBM pioneered this pattern with WebSphere MQ and it is commonly used in financial institutions. And as it happens, interacting with the physical world is a lot closer to “swiping a credit card” than the request-response patterns exemplified in HTTP. In fact, the mobile industry has long been struggling to fit “push notifications” on top of HTTP’s request-response protocol, using techniques like long polling to simulate true push notification while reducing the overhead of the mobile device constantly polling the server.
Well, here is what I have learned regarding IoT protocols: It’s a zoo out there, with lots of protocols trying to become the next HTTP. And some candidates deploy a formidable array of marketing, making it exceedingly hard to cut through the hype.
Main Contenders
My current shortlist of main contenders from a protocol perspective includes (in alphabetical order): CoAP, MQTT, and XMPP. I might add STOMP to that list, just for its simple brilliance. STOMP is a text-based messaging protocol that has recently been extended to allow for binary content. I’ve also started talking with some transportation companies and learning about their use of DDS (Data Distribution Service) from OMG (Object Management Group), which might be another candidate.
In addition, some companies have been looking at binary protocols like Google Protobuf, Apache Thrift and Apache Avro as Data Interchange Protocols. The common bond within this group of protocols is that they all use some form of Interface Description Language to describe the RPC-style interface and generate client and server code for binary message exchanges. For instance, Evernote’s Developer API is based on Apache Thrift.
But in the corner of the reigning champion, we have JSON/HTTP. Not content to see this protocol pushed into early retirement, advocates have been developing some very interesting approaches that attempt to ensure the continuing relevance of HTTP for asynchronous small messages?WebSockets and WebHooks being the most well-known. Hypercat, Simple Thing Protocol and EventedAPI represent just a small sample of the interesting approaches emerging to support async eventing and messaging with HTTP. A very interesting approach on top of WebSockets is WAMP (Web Application Messaging Protocol) since it allows both RPC and pub-sub style interactions.
Each protocol has weaknesses: MQTT and CoAP appear to be weak in security; DDS seems to be complex to scale and has version dependencies; XMPP is considered heavy-weight.
But they all have strengths too, of course: DDS has the deployments in the field to prove its relevance; XMPP supports EXI (Efficient XML Interchange) for message compression, and WebSockets is known for both efficiency and a proven track record for scalable networks; both DDS and XMPP are extremely mature and have built-in security. Whereas XMPP has proven its scalability in countless deployments like Google Chat, MQTT is the protocol of choice of Facebook Messenger.
Given the industry interest in MQTT as well as CoAP, I am confident that whatever security problems exist will be fixed as the use cases mature.
Eventual Consistency
There are other potential solutions emerging from the mobile app space which might impact this area as well. As we have built massively scalable distributed databases relying on a concept of “Eventual Consistency,” mobile app developers are realizing that connectivity cannot be taken for granted, and are increasingly relying on a local data persistence layer on the mobile device itself that provides data synchronization with the backend service. Most commonly, such a local data persistence layer is provided by mobile database clients from CouchDB and Firebase. StrongLoop recently added a similar offline synchronization feature to its Node.js distribution. Data persistence through “Eventual Consistency” has the great advantage of providing consistent app behavior in both online and offline mode?and it takes only a small step to apply the same technique to connected things and embedded devices.
So where does this leave you when having to decide which IoT API technology to choose or invest in? Well?it depends. If you can stick to using HTTP, stay with it. Look at WebSockets, WebHooks and Simple Thing Protocol for inspiration on how you can deal with async messaging. If security and scalability are essential, I would recommend looking into XMPP. If you are working on constrained devices like Arduino, or are ready for something new or just want something simple with a big ecosystem of fellow enthusiasts it is worth looking at CoAP and MQTT?but keep in mind that you will not be able to run it securely on most embedded devices. Happy hacking!
About the Author
Holger Reinhardt is responsible for developing business strategy and partnerships around the IoT, M2M, and Big Data for CA Technologies API Management business. He has an MS in Computer Science and an MBA in Entrepreneurship, and he has been programming since he was 14 years old. He is also co-founder and chief developer of launchd.io.
?
