#IoT

Microservices and the Internet of Things – First impressions

by
I must say I was sceptical when I first heard the term “microservices”. It sounded like yet another wash-rinse-repeat cycle of earlier incarnations of SOA. It appears I was wrong – this architectural pattern has some  interesting characteristics that, in my opinion, offer some real potential for event-driven, edge-processing systems (that are prevalent in the Internet of Things).
After watching Fred George’s video, I realised what he described was an event-driven, agent-based, systems’ model, rather than how many of us see SOA implementations today (often way-off the original notion of a SOA). At a conceptual level, the pattern describes a ‘Complex Adaptive’ system.  Essential principles of the architecture, however, appear teasingly elegant and simple. Few of these design principles are unique to microservices, but in combination, they make a compelling story:
Publish anything of interest – don’t wait to be asked, if your microservice thinks it has some information that might be of use to the microservices ecosystem, then publish-and-be-damned.



Amplify success & attenuate failure – microservices that publish useful information thrive, while those that left unsubscribed, wither-on-the-vine. Information subscribers determine value, and value adjusts over time/changing circumstances.



Adaptive ecosystem – versions of microservices are encouraged –may-the-best-service-win mentality introduces variety which leads to evolution.



Asynchronous & encapsulated – everything is as asynchronous as possible – microservices manage their own data independently and then share it in event messages over an asynchronous publish-subscribe bus.



Think events not entities – no grand BDUF data model, just a cloud of ever-changing event messages – more like Twitter than a DBMS. Events have a “use-by-date” that indicates the freshness of data.



Events are immutable – time-series snapshots, no updates allowed.


Designed for failure – microservices must expect problems and tell the world when they encounter one and send out “I’m alive” heart-beats.



Self-organizing & self-monitoring – a self-organizing System-of-systems’ that needs no orchestration. Health monitoring and other administration features are established through a class of microservices.



Disposable Code – microservices are very, very small (typically under 1000 lines of code). They can be developed in any language.



Ultra-rapid deployment – new microservices can be written and deployed with hours with a zero-test SDLC.

It struck me that many of these design principles could apply, in part, to a 2020 Smart Grid architecture I’m working on, and to the much boarder ‘Internet of Things’ecosystem.

The microservices pattern does seem to lend itself to the notion of highly autonomous, location-independent s/w agents that could reside at the centre, mid-point or edge of an environment. I can imagine that the fundamental simplicity of the model would help, rather than hinder, data privacy and protection by being able to include high-level system contexts, policies and protocols (e.g. encryption and redaction) applied to the event-streams. This pattern, of course, won’t be the ‘right-fit’ for all situations, but it does seem to offer interesting opportunities in:

  • Agility – very small disposable services are deployable within hours
  • Resilience – withstands service failures and supports service evolution
  • Robustness – it’s hard to break due to: simplicity, in-built failure handling and lack of centralized orchestration

It may be that the microservices pattern can only be applied to operational decision-support and behaviour profiling situations. But if that’s the case, I still see great potential in a world where many trillions of sensor-generated events will be published, consumed, filtered, aggregated, and correlated. I’m no longer a developer, but as an architect, I’m always on the look-out for patterns that could: either apply to future vendors’ products and services, or could act as a guide for in-house software development practice.

As always, I’d be keen to hear your views, examples and opinions about microservices and their potential application to the IoT. Have you come across examples of microservices pattern in an IoT context – deployed or in the labs?

I whole-heartily recommend setting aside an hour to watch the video of Fred George’s presentation on microservices:


131108 1110 Dune Fred George Recording on 2013-11-08 1106-Vimeo from Øredev Conference on Vimeo.

Post-post:
  • Another great post about microservices  – including downsides.
  • More here including “The 8 fallacies of distributed computing”.
Duke Energy are doing some interesting things in the Edge Processing space.

Here’s a video on microservices in the conext of IoT  (worth ignoring the references to Cloud/Azure):

http://www.microsoftvirtualacademy.com/training-courses/exploring-microservices-in-docker-and-microsoft-azure

I’d like to talk to anyone who’s impelmenting/ thinking about a Staged Event Driven Architecture using microservices for Edge Processing.

Phil Wills on experience of deploying microservices at The Gaurdian

The Internet of Things Means Business

by

According to a survey by the research arm of The Economist, businesses are slightly more likely to be using the Internet of Things for internal operations and processes than in external products or services. It’s important to draw a distinction between forward-facing IoT and what I call the Internet of Business Things (IoBT). The knowledge, skills and alliances it will take to instrument the business are different than outfitting consumer products with connected technology. The […]

The CIO’s Role in the Internet of Things

by

In our soon-to-be-released Digital IQ survey of over 1,400 business and technology executives, 20% of respondents say they plan to invest in sensors. We feel confident in predicting that the Internet of Things (IoT) or the Internet of Everything will finally begin to take off this year, as futurists have forecasted for years. What remains to be seen is whether or not CIOs will win their rightful place in product design planning and the development […]

A Brain for The Internet of Things

by

Grok has popped up a few times in messages and conversations in the past few weeks and seems to be worth watching for anyone interested in m2m, and other signal processing/event processing, smarts.Numenta, the latest startup from Palm creator Jeff Hawk…

2013: The year the Internet-of-Things takes-off?

by

I’ve been reading a lot about M2M/’The Internet of Things’, many pundits believe 2013 will be the year the concept finally goes mainstream – it’s been a while since its inception in the late ‘90s!

I have to say I’m among those believers, but I can see a lot of dust in the air before we get to anything that might resemble a ubiquitous eco-system for all-things-Smart.  Here are a few reasons why:
  1. Open standards for connectivity and data interchange will take a while to agree. Having said that, I don’t think Businesses and Consumers will want to see proprietary platforms (the nightmarish vision of an ‘ITunes-for-M2M’)!
  2. Object Identity standards: Everyone seems to be talking about IP v6 for this purpose, very few in the ‘new wave’ of M2M seem to be aware of the Electronic Product Code(EPC)  from GS1/EPCGlobal. I’ve not been close to EPC developments since 2004, but a lot of good thinking was done that tackled many of the issues yet to be resolved in the M2M world – not the least of which, delineation between the identity of the object and the identity of an object’s interface(s) – a debate that continues around the use of IPv6 for identity.
  3. Who will lead the M2M market? Or should it be markets? Will it be consumers leading with ‘Home & Personal’ gadgets , as Alex Hawkinson, founder/CEO of http://smartthings.com/  believes. Or could it be led by large Energy providers with their Smart Grid projects – often subsidized and encouraged by governments? Or will the Telco’s and Network equipment guys to fight back?  Many telcos (mostly outside the US) have been dabbling in this space for ten years or more – I worked with BT on an early Auto ID service back in 2003. Some Telcos have continued to invest, for example; Telefonica recently announced their proprietary ‘Smart M2M’ solution and clearly have global ambitions in M2M services. Meanwhile, the likes of Alcatel-Lucent, Ericsson, HP, Juniper Networks, Motorola Mobility, Qualcomm, Samsung and Texas Instruments are rallying around the OneM2M movement (http://www.onem2m.org). Not to forget Cisco’s long standing ambitions in this space.
  4. Who will lead Enterprise-quality data integration? This would probably include correlation, aggregation and other, signal-based & enterprise app generated data. Think multi-source, data ‘mash-up’ services. This feels to me that this could be the sweet-spot for the more ‘application-and-data-as-a-service’ focused Cloud vendors. Or could be a SI or large software vendor play (SAP or Oracle spring to mind).
  5. Which roles will the Complex Event Processing (CEP) platform vendors adopt in the M2M eco-system? How far can we expect the likes of TIBCO, Oracle and IBM to push M2M/CEP/Big Data combinations within the Enterprise?
But despite the above challenges and battles yet to be won, I do believe we will see far greater deployment of ‘Smart’ things over the next 12-24 months than at any time since those early days of Auto-ID. My bet is that both Smart Grid and consumer-led lifestyle solutions will lead the early adopters.  Don’t, however, hold your breath for pan-industry standards at anything above low-level communications protocols!

2013: The year the Internet-of-Things takes-off?

by

I’ve been reading a lot about M2M/’The Internet of Things’, many pundits believe 2013 will be the year the concept finally goes mainstream – it’s been a while since its inception in the late ‘90s!

I have to say I’m among those believers, but I can see a lot of dust in the air before we get to anything that might resemble a ubiquitous eco-system for all-things-Smart.  Here are a few reasons why:
  1. Open standards for connectivity and data interchange will take a while to agree. Having said that, I don’t think Businesses and Consumers will want to see proprietary platforms (the nightmarish vision of an ‘ITunes-for-M2M’)!
  2. Object Identity standards: Everyone seems to be talking about IP v6 for this purpose, very few in the ‘new wave’ of M2M seem to be aware of the Electronic Product Code(EPC)  from GS1/EPCGlobal. I’ve not been close to EPC developments since 2004, but a lot of good thinking was done that tackled many of the issues yet to be resolved in the M2M world – not the least of which, delineation between the identity of the object and the identity of an object’s interface(s) – a debate that continues around the use of IPv6 for identity.
  3. Who will lead the M2M market? Or should it be markets? Will it be consumers leading with ‘Home & Personal’ gadgets , as Alex Hawkinson, founder/CEO of http://smartthings.com/  believes. Or could it be led by large Energy providers with their Smart Grid projects – often subsidized and encouraged by governments? Or will the Telco’s and Network equipment guys to fight back?  Many telcos (mostly outside the US) have been dabbling in this space for ten years or more – I worked with BT on an early Auto ID service back in 2003. Some Telcos have continued to invest, for example; Telefonica recently announced their proprietary ‘Smart M2M’ solution and clearly have global ambitions in M2M services. Meanwhile, the likes of Alcatel-Lucent, Ericsson, HP, Juniper Networks, Motorola Mobility, Qualcomm, Samsung and Texas Instruments are rallying around the OneM2M movement (http://www.onem2m.org). Not to forget Cisco’s long standing ambitions in this space.
  4. Who will lead Enterprise-quality data integration? This would probably include correlation, aggregation and other, signal-based & enterprise app generated data. Think multi-source, data ‘mash-up’ services. This feels to me that this could be the sweet-spot for the more ‘application-and-data-as-a-service’ focused Cloud vendors. Or could be a SI or large software vendor play (SAP or Oracle spring to mind).
  5. Which roles will the Complex Event Processing (CEP) platform vendors adopt in the M2M eco-system? How far can we expect the likes of TIBCO, Oracle and IBM to push M2M/CEP/Big Data combinations within the Enterprise?
But despite the above challenges and battles yet to be won, I do believe we will see far greater deployment of ‘Smart’ things over the next 12-24 months than at any time since those early days of Auto-ID. My bet is that both Smart Grid and consumer-led lifestyle solutions will lead the early adopters.  Don’t, however, hold your breath for pan-industry standards at anything above low-level communications protocols!