#IoT

A Tale of Two IT Departments, or How Governance is Essential in the Hybrid Cloud and Bimodal IT Era

by

Transcript of an Open Group discussion/podcast on the role of Cloud Governance and Enterprise Architecture and how they work together in the era of increasingly fragmented IT. Listen to the podcast. Find it on iTunes. Get the mobile app for … Continue reading →

Design Patterns for Event-driven Distributed Intelligence

by
Following discussions with @seabird20  and @darachennis, among others last year. I decided to publish a rough idea of the event-driven Distributed Intelligence architecture for Smart Grid (this could apply to the broader IoT). This is loosely-based on the microservices concepts & principles described here and builds on Duke Energy’s collaboration on interoperability and distributed intelligence initiative. The purpose of this post is to generate ideas and aid the ongoing dialogues. As far as I’m aware, the additional concepts I discuss here; SEDA, microservices and distributed agent patterns, are not  called out in the Duke Energy work (although the authors might reasonably claim they’re implied). My aim, however, is to make the ‘software and event data’ conceptual architecture much more explicit in this discussion.
Having said that,  the  first diagram looks pretty physical! It serves as a simple IoT-ish context: A metering sensor talks to an edge processor on a ‘Field Area Network’ and thereafter data is relayed back to the Data Centre for further processing.

In the 2nd diagram we open up the Edge Processor and to reveal a relatively simplistic view of the software architecture. This is based on the micoservices pattern described by Fred George, and my own experience as VP Product Development/Architect at VI Agents.




I found the concept of small, autonomous agents, work very well for us at VI. Moreover, I spotteda lot of parallels with Fred George’s description of microservices:

Publish anything of interest – don’t wait to be asked, if your microservice thinks it has some information that might be of use to the microservices ecosystem, then publish-and-be-damned.



Amplify success & attenuate failure – microservices that publish useful information thrive, while those that left unsubscribed, wither-on-the-vine. Information subscribers determine value, and value adjusts over time/changing circumstances.



Adaptive ecosystem – versions of microservices are encouraged –may-the-best-service-win mentality introduces variety which leads to evolution.



Asynchronous & encapsulated – everything is as asynchronous as possible – microservices manage their own data independently and then share it in event messages over an asynchronous publish-subscribe bus.



Think events not entities – no grand BDUF data model, just a cloud of ever-changing event messages – more like Twitter than a DBMS. Events have a “use-by-date” that indicates the freshness of data.



Events are immutable – time-series snapshots, no updates allowed.


Designed for failure – microservices must expect problems and tell the world when they encounter one and send out “I’m alive” heart-beats.



Self-organizing & self-monitoring – a self-organizing System-of-systems’ that needs no orchestration. Health monitoring and other administration features are established through a class of microservices.



Rapids, Rivers & Ponds
I also particularly liked Fred’s use of the labels Rapid and Rivers to describe two seperate instances of message brokers and Ponds to describe persisted data. Again very similar to the VI SixD architecture where we collected signals on a ‘Rapids’ bus and  business event & document messages flowed over a ‘Rivers’ bus, and Event Logs & Document Queues were our  ‘Ponds’. But that was back in 2002,  I do think the microservices pattern is much more elegant and more extensible.


Staged Event Driven Architecture
The 3rd diagram overlays what I see as the SEDA Stages in our Smart Grid architecture:

Stage1: Raw signal filtering, simple aggregation& correlation, negative-event detection etc.

Stage2: Goal and role based functionality via an ‘Edge Agent’ that does complex aggregation and correlation, sense-making and alerting. These might be implemented as microservices, or at least, share many of the attributes described by Fred.


Stage3: There will  be more aggregation & correlation, routing alerting and broadcasting done here. The junction of Information Technology (IT) and Operational Technology (OT)  is probably here. These are the the software components that sit back at the Data Centre, that receive and process the event messages produced by the ‘Edge Agents’. There’s also a nod to some sort of management console and the publishing of commands and requests to the ‘Edge Processing’ domain. This diagram is very sketchy right now, with loads of components missing.   I just want to keep things very simple at this stage. 

Stage4: This is where the  ‘Big Data’ heavy-lifting of historical data and predictive analytics stuff is done.

We used  the SEDA pattern to define an Events Network that was capable of processing 4bn events per day without having to invest in massive network bandwidth and huge back end processing capabilities for the Royal Mail – I believe the U.S. postal service didn’t implement a SEDA  and  throw a huge amount of cash at the problem. I love the elegance of a series of cascading queues that gradually whittle the tsunami of raw signals in real time, down to ‘right-time’ stream or tap of business-meaningful messages.



This is the early stage thinking and part of a much bigger Smart Grid, and ultimately Smart City architecture. Please feel free to tweet about. or better still, comment below. Thx.

The Open Group Madrid 2015 – Day Two Highlights

by

By The Open Group On Tuesday, April 21, Allen Brown, President & CEO of The Open Group, began the plenary presenting highlights of the work going on in The Open Group Forums. The Open Group is approaching 500 memberships in … Continue reading →

Six Ways to Define Your Internet of Things Strategy

by

Guest post by Frank Burkitt By 2020, an estimated 50 billion devices around the globe will be connected to the Internet. Perhaps a third of them will be computers, smartphones, tablets, and TVs. The remaining two-thirds will be other kinds of “things”: sensors, actuators, and newly invented intelligent devices that monitor, control, analyze, and optimize our world. This seemingly sudden trend has been decades in the making, but is just now hitting a tipping point. […]

Organizations Must Leverage Enterprise Architecture as a Strategic Resource to Advance their Internet of Things Strategies

by

You’re probably thinking, “wow that is a bold statement”, and you would be right in thinking so. However, it doesn’t make it any less true. Over the past few months I’ve been working with Gartner IoT experts and listening and observing customers around the world in their usage of IoT technologies. One thing is clear, […]

The post Organizations Must Leverage Enterprise Architecture as a Strategic Resource to Advance their Internet of Things Strategies appeared first on Mike J Walker.

Organizations Must Leverage Enterprise Architecture as a Strategic Resource to Advance their Internet of Things Strategies

by

You’re probably thinking, “wow that is a bold statement”, and you would be right in thinking so. However, it doesn’t make it any less true. Over the past few months I’ve been working with Gartner IoT experts and listening and observing customers around the world in their usage of IoT technologies. One thing is clear, […]

The post Organizations Must Leverage Enterprise Architecture as a Strategic Resource to Advance their Internet of Things Strategies appeared first on Mike J Walker.