entarch

Evolution Architecture

by

Caoilte O’Connor is a developer at ITV, the UK’s largest Commercial Terrestrial TV Network. Here’s a short  (~2 min) clip from Domain Service Aggregators: A Structured Approach to Microservice Composition . In this clip he describes how …

Going Digital & Agile Architecture

by

Recently, I’ve been reworking & tightening up some earlier ideas and putting them in a ‘Going Digital’ context. I’m posting them on LinkedIn to reach a slightly different audience.  
Here are the posts so far:
During the process, I’ve been reading/re-reading some great related articles that discuss agile architecture and the need recognise the business as a complex adaptive system. Here are the links with some favourite quotes:
Is Agile killing EA #1?  By Charles Betz “the EA team needs to match the cadence of the Agile teams. This is a central challenge”.
Is Agile killing EA #2?  By Jason Bloomberg.
” …if some company’s EA means nothing more than a lot of paperwork that gets in the way of basic goals like working software that keeps customers happy, then we can only hope Agile drives a nail into that coffin. On the other hand, sometimes the paperwork is a good thing. Only an overly dogmatic reading of the Agile Manifesto would lead one to conclude that we don’t need no stinkin’ documentation”.
“Frameworks are cocaine for executives – they give them a huge rush, and then they move to the next framework”.
Enterprise Architecture Finally Crosses the Chasm by Jason Bloomberg including an interview with Adrian Cockcroft formally the Cloud Architect at Netflix. 
“The goal of architecture was to create the right emergent behaviours”.
“..it makes more sense for them to pay most attention to the real-world  ‘wiggliness’ of organisation: the hidden, messy and informal dynamics of everyday human interaction in which they and everyone else are continuously immersed”.
With the ‘Going Digital’ series, my aim is talk about real-world experiences and emerging techniques for doing “agile architecture’, or business change design, or whatever it gets called in the future. All I know is that it isn’t framework-centric and that many who carry the title Enterprise Architect will have trouble giving up their particular drug of choice! I’m interested to hear what others are doing; what’s working for you – and what doesn’t.
I guess like many of have lived with the ‘EA’ label, I’m tending to avoid the term, so as not to confuse what I do with the framework-centric, heavy modelling, and ‘certified’ practices stuff.
Has anyone seen a good job description? 🙂 

 Update Feb 27th, 2017: To see where these thoughts now are going, please take a look at the Found In Design un-book and  the Horses and Unicorns story.

Microservices Architecture versus SOA

by

TechTarget has published another one of my “Ask the Expert” columns.  In this one, I offer up my thoughts on the differences between a Microservices Architecture and a SOA.  In a nutshell, I think the microservices trend has moved things in the right direction, a direction that many of the SOA pundits were espousing back […]

Design Patterns for Event-driven Distributed Intelligence

by
Following discussions with @seabird20  and @darachennis, among others last year. I decided to publish a rough idea of the event-driven Distributed Intelligence architecture for Smart Grid (this could apply to the broader IoT). This is loosely-based on the microservices concepts & principles described here and builds on Duke Energy’s collaboration on interoperability and distributed intelligence initiative. The purpose of this post is to generate ideas and aid the ongoing dialogues. As far as I’m aware, the additional concepts I discuss here; SEDA, microservices and distributed agent patterns, are not  called out in the Duke Energy work (although the authors might reasonably claim they’re implied). My aim, however, is to make the ‘software and event data’ conceptual architecture much more explicit in this discussion.
Having said that,  the  first diagram looks pretty physical! It serves as a simple IoT-ish context: A metering sensor talks to an edge processor on a ‘Field Area Network’ and thereafter data is relayed back to the Data Centre for further processing.

In the 2nd diagram we open up the Edge Processor and to reveal a relatively simplistic view of the software architecture. This is based on the micoservices pattern described by Fred George, and my own experience as VP Product Development/Architect at VI Agents.




I found the concept of small, autonomous agents, work very well for us at VI. Moreover, I spotteda lot of parallels with Fred George’s description of microservices:

Publish anything of interest – don’t wait to be asked, if your microservice thinks it has some information that might be of use to the microservices ecosystem, then publish-and-be-damned.



Amplify success & attenuate failure – microservices that publish useful information thrive, while those that left unsubscribed, wither-on-the-vine. Information subscribers determine value, and value adjusts over time/changing circumstances.



Adaptive ecosystem – versions of microservices are encouraged –may-the-best-service-win mentality introduces variety which leads to evolution.



Asynchronous & encapsulated – everything is as asynchronous as possible – microservices manage their own data independently and then share it in event messages over an asynchronous publish-subscribe bus.



Think events not entities – no grand BDUF data model, just a cloud of ever-changing event messages – more like Twitter than a DBMS. Events have a “use-by-date” that indicates the freshness of data.



Events are immutable – time-series snapshots, no updates allowed.


Designed for failure – microservices must expect problems and tell the world when they encounter one and send out “I’m alive” heart-beats.



Self-organizing & self-monitoring – a self-organizing System-of-systems’ that needs no orchestration. Health monitoring and other administration features are established through a class of microservices.



Rapids, Rivers & Ponds
I also particularly liked Fred’s use of the labels Rapid and Rivers to describe two seperate instances of message brokers and Ponds to describe persisted data. Again very similar to the VI SixD architecture where we collected signals on a ‘Rapids’ bus and  business event & document messages flowed over a ‘Rivers’ bus, and Event Logs & Document Queues were our  ‘Ponds’. But that was back in 2002,  I do think the microservices pattern is much more elegant and more extensible.


Staged Event Driven Architecture
The 3rd diagram overlays what I see as the SEDA Stages in our Smart Grid architecture:

Stage1: Raw signal filtering, simple aggregation& correlation, negative-event detection etc.

Stage2: Goal and role based functionality via an ‘Edge Agent’ that does complex aggregation and correlation, sense-making and alerting. These might be implemented as microservices, or at least, share many of the attributes described by Fred.


Stage3: There will  be more aggregation & correlation, routing alerting and broadcasting done here. The junction of Information Technology (IT) and Operational Technology (OT)  is probably here. These are the the software components that sit back at the Data Centre, that receive and process the event messages produced by the ‘Edge Agents’. There’s also a nod to some sort of management console and the publishing of commands and requests to the ‘Edge Processing’ domain. This diagram is very sketchy right now, with loads of components missing.   I just want to keep things very simple at this stage. 

Stage4: This is where the  ‘Big Data’ heavy-lifting of historical data and predictive analytics stuff is done.

We used  the SEDA pattern to define an Events Network that was capable of processing 4bn events per day without having to invest in massive network bandwidth and huge back end processing capabilities for the Royal Mail – I believe the U.S. postal service didn’t implement a SEDA  and  throw a huge amount of cash at the problem. I love the elegance of a series of cascading queues that gradually whittle the tsunami of raw signals in real time, down to ‘right-time’ stream or tap of business-meaningful messages.



This is the early stage thinking and part of a much bigger Smart Grid, and ultimately Smart City architecture. Please feel free to tweet about. or better still, comment below. Thx.

Next Generation of EAs

by

Recently I was asked by Oracle’s architecture community manager about the next generation of EAs. The topic was centered around where future architects should focus their energy. Since the original article can only share a small snipet of my thoughts I’ve added the remainder below. The next generation of EA would do well to spend […]

On Complexity

by

I’m not convinced all complexity is bad. There are complex elements in our lives that enrich us. I’m reminded of the first few times I tasted Thai or Indian cuisine when my mouth was alive with a myriad of flavors coming from multiple directions. Just the curry alone consists of eleven different spices hand-tuned to […]

Why Metamodels Matter

by

In the process of evolving a business the question of scope arises in my mind often. If I don’t have a boundary to work within, I can get distracted and the project can go sideways. With business design projects that leverage enterprise architecture (EA) the frameworks often supply a metamodel. I believe it is a […]

Moore’s Law Deemed Hazardous

by

The advent of digital computers has brought a seemingly unending supply of innovations. Broadband communication to our homes and portable computers ranging from laptops to pocket-sized computers (aka, smartphones) make working anywhere 24/7 viable. This innovation also makes petabytes worth of cat videos available to most of the planet, but that is a topic for […]

Why CX Matters to EA

by

Designing the experience for customers is increasing important in a noisy, saturated world. There seems to be no end of choices for food, technology, or services that are presented to us each day. What are the key differentiators that drive us to one versus the other? Typically its cost. Many years ago on a much […]

The Relative Value of EA Frameworks

by

When it comes to frameworks supporting business planning, software development, or other activities – I’m a fan. I like organization. I regularly use GTD with Omnifocus on three devices to keep my personal chaos in order. I’m a proponent of developing explicitly work breakdown structures (WBS) for any enterprise architecture (EA) work I’m about to […]