2 years, 4 months ago

Linear Thought

Various concerns have been raised about Lt. Gen. Michael Flynn, previously described as “disruptive” by a former Pentagon official, and now the subject of heated investigation and speculation around his short-lived role in the Trump administration, his alleged links with Russia and other countries, and his alleged obsessions about various topics.

According to the Guardian, US and UK intelligence officers were also anxious about Flynn’s capacity for “linear thought”.

I guess most people will interpret this concern as “insufficient capacity”. When I searched for “linear thinking” on the internet, I found a number of pages that contrasted linear thinking with various forms of supposedly bad thinking, such as “fragmented thinking”. I also found pages that tried to divide people into two camps – the scientific “leftbrain” types who think in straight lines, and the artistic “rightbrain” types who think in circles.

However, systems thinkers might be concerned about someone at that level having too much capacity for linear thought. (As one might be concerned about someone’s capacity for gossip or deception.) In a previous post on this blog, I defended Flynn’s former boss, Gen. Stanley McChrystal (labelled an “ill-fated iconoclast” by James Kitfield) against the claim that he was not a systems thinker. (This claim was based on a remark McChrystal had made about a subsequently notorious systems dynamics diagram. I argued that McChyrstal’s remark could have been made either by someone who doesn’t get systems thinking, or at the other extreme by someone who really gets systems thinking.)

The question here is about greater or lesser capacity for various kinds of thinking, because I’m trying to avoid the fallacy (identified by @cybersal) of categorizing people as this or that type of thinker. She rightly insists on seeing systems thinking not as an all-or-nothing affair but “as a lens to be applied in a particular type of situation”.

By the way, Flynn himself has appeared on this blog before. In January 2010, using the lens of organizational intelligence, I reviewed his report on Fixing Intel. While I was sceptical about some of his recommendations, I can affirm that the report showed considerable capacity for systems (non-linear) thinking. Make of that what you will.

Links

Phillip Carter, What is Michael Flynn’s game? (Slate, 31 March 2017)

Luke Harding et al, Michael Flynn: new evidence spy chiefs had concerns about Russian ties (Guardian, 31 March 2017)

James Kitfield, Flynn’s Last Interview: Iconoclast Departs DIA With A Warning (Breaking Defense, 7 August 2014)

Stanley McChrystal, The military case for sharing knowledge (TED2014, March 2014)

Stan McChrystal, Career Curveballs: No Longer A Soldier (22 April 2014)

Greg Miller and Adam Goldman, Head of Pentagon intelligence agency forced out, officials say (Washington Post, 30 April 2014)

Related Blogposts

A Job Description for Systems Thinking (November 2009)
Making Intelligence Relevant (January 2010)
Understanding Complexity (July 2010)

3 years, 11 months ago

Going Digital & Agile Architecture

Recently, I’ve been reworking & tightening up some earlier ideas and putting them in a ‘Going Digital’ context. I’m posting them on LinkedIn to reach a slightly different audience.  
Here are the posts so far:
During the process, I’ve been reading/re-reading some great related articles that discuss agile architecture and the need recognise the business as a complex adaptive system. Here are the links with some favourite quotes:
Is Agile killing EA #1?  By Charles Betz the EA team needs to match the cadence of the Agile teams. This is a central challenge”.
Is Agile killing EA #2?  By Jason Bloomberg.
” …if some company’s EA means nothing more than a lot of paperwork that gets in the way of basic goals like working software that keeps customers happy, then we can only hope Agile drives a nail into that coffin. On the other hand, sometimes the paperwork is a good thing. Only an overly dogmatic reading of the Agile Manifesto would lead one to conclude that we don’t need no stinkin’ documentation”.
“Frameworks are cocaine for executives – they give them a huge rush, and then they move to the next framework”.
Enterprise Architecture Finally Crosses the Chasm by Jason Bloomberg including an interview with Adrian Cockcroft formally the Cloud Architect at Netflix. 
“The goal of architecture was to create the right emergent behaviours”.
“..it makes more sense for them to pay most attention to the real-world  ‘wiggliness’ of organisation: the hidden, messy and informal dynamics of everyday human interaction in which they and everyone else are continuously immersed”.
With the ‘Going Digital’ series, my aim is talk about real-world experiences and emerging techniques for doing “agile architecture’, or business change design, or whatever it gets called in the future. All I know is that it isn’t framework-centric and that many who carry the title Enterprise Architect will have trouble giving up their particular drug of choice! I’m interested to hear what others are doing; what’s working for you – and what doesn’t.
I guess like many of have lived with the ‘EA’ label, I’m tending to avoid the term, so as not to confuse what I do with the framework-centric, heavy modelling, and ‘certified’ practices stuff.
Has anyone seen a good job description? 🙂 

Update Feb 27th, 2017: To see where these thoughts now are going, please take a look at the Found In Design un-book and  the Horses and Unicorns story.

4 years, 9 months ago

EA/ST Meeting Report October 2014

Perspectives on Enterprise Architecture and Systems Thinking

Unfortunately, I missed this month’s EA/ST group meeting on Tuesday 21st October. But the presentations by @KlausØstergaard, @kvistgaard, @PhilipHellyer and @tetradian are now available.

  • Klaus Østergaard, Value in Utilizing Systemic Thinking in the Field of Enterprise Architecture and Vice Versa (DropBox link)

4 years, 9 months ago

EA/ST Meeting Report October 2014

Perspectives on Enterprise Architecture and Systems Thinking

Unfortunately, I missed this month’s EA/ST group meeting on Tuesday 21st October. But the presentations by @KlausØstergaard, @kvistgaard, @PhilipHellyer and @tetradian are now available.

  • Klaus Østergaard, Value in Utilizing Systemic Thinking in the Field of Enterprise Architecture and Vice Versa (DropBox link)

5 years, 2 months ago

Microservices and the Internet of Things – First impressions

I must say I was sceptical when I first heard the term “microservices”. It sounded like yet another wash-rinse-repeat cycle of earlier incarnations of SOA. It appears I was wrong – this architectural pattern has some  interesting characteristics that, in my opinion, offer some real potential for event-driven, edge-processing systems (that are prevalent in the Internet of Things).
After watching Fred George’s video, I realised what he described was an event-driven, agent-based, systems’ model, rather than how many of us see SOA implementations today (often way-off the original notion of a SOA). At a conceptual level, the pattern describes a ‘Complex Adaptive’ system.  Essential principles of the architecture, however, appear teasingly elegant and simple. Few of these design principles are unique to microservices, but in combination, they make a compelling story:
Publish anything of interest – don’t wait to be asked, if your microservice thinks it has some information that might be of use to the microservices ecosystem, then publish-and-be-damned.



Amplify success & attenuate failure – microservices that publish useful information thrive, while those that left unsubscribed, wither-on-the-vine. Information subscribers determine value, and value adjusts over time/changing circumstances.



Adaptive ecosystem – versions of microservices are encouraged –may-the-best-service-win mentality introduces variety which leads to evolution.



Asynchronous & encapsulated – everything is as asynchronous as possible – microservices manage their own data independently and then share it in event messages over an asynchronous publish-subscribe bus.



Think events not entities – no grand BDUF data model, just a cloud of ever-changing event messages – more like Twitter than a DBMS. Events have a “use-by-date” that indicates the freshness of data.



Events are immutable – time-series snapshots, no updates allowed.


Designed for failure – microservices must expect problems and tell the world when they encounter one and send out “I’m alive” heart-beats.



Self-organizing & self-monitoring – a self-organizing System-of-systems’ that needs no orchestration. Health monitoring and other administration features are established through a class of microservices.



Disposable Code – microservices are very, very small (typically under 1000 lines of code). They can be developed in any language.



Ultra-rapid deployment – new microservices can be written and deployed with hours with a zero-test SDLC.

It struck me that many of these design principles could apply, in part, to a 2020 Smart Grid architecture I’m working on, and to the much boarder ‘Internet of Things’ecosystem.

The microservices pattern does seem to lend itself to the notion of highly autonomous, location-independent s/w agents that could reside at the centre, mid-point or edge of an environment. I can imagine that the fundamental simplicity of the model would help, rather than hinder, data privacy and protection by being able to include high-level system contexts, policies and protocols (e.g. encryption and redaction) applied to the event-streams. This pattern, of course, won’t be the ‘right-fit’ for all situations, but it does seem to offer interesting opportunities in:

  • Agility – very small disposable services are deployable within hours
  • Resilience – withstands service failures and supports service evolution
  • Robustness – it’s hard to break due to: simplicity, in-built failure handling and lack of centralized orchestration

It may be that the microservices pattern can only be applied to operational decision-support and behaviour profiling situations. But if that’s the case, I still see great potential in a world where many trillions of sensor-generated events will be published, consumed, filtered, aggregated, and correlated. I’m no longer a developer, but as an architect, I’m always on the look-out for patterns that could: either apply to future vendors’ products and services, or could act as a guide for in-house software development practice.

As always, I’d be keen to hear your views, examples and opinions about microservices and their potential application to the IoT. Have you come across examples of microservices pattern in an IoT context – deployed or in the labs?

I whole-heartily recommend setting aside an hour to watch the video of Fred George’s presentation on microservices:


131108 1110 Dune Fred George Recording on 2013-11-08 1106-Vimeo from Øredev Conference on Vimeo.

Post-post:
  • Another great post about microservices  – including downsides.
  • More here including “The 8 fallacies of distributed computing”.
Duke Energy are doing some interesting things in the Edge Processing space.

Here’s a video on microservices in the conext of IoT  (worth ignoring the references to Cloud/Azure):

http://www.microsoftvirtualacademy.com/training-courses/exploring-microservices-in-docker-and-microsoft-azure

I’d like to talk to anyone who’s impelmenting/ thinking about a Staged Event Driven Architecture using microservices for Edge Processing.

Phil Wills on experience of deploying microservices at The Gaurdian
5 years, 2 months ago

Microservices and the Internet of Things – First impressions

I must say I was sceptical when I first heard the term “microservices”. It sounded like yet another wash-rinse-repeat cycle of earlier incarnations of SOA. It appears I was wrong – this architectural pattern has some  interesting characteristics that, in my opinion, offer some real potential for event-driven, edge-processing systems (that are prevalent in the Internet of Things).
After watching Fred George’s video, I realised what he described was an event-driven, agent-based, systems’ model, rather than how many of us see SOA implementations today (often way-off the original notion of a SOA). At a conceptual level, the pattern describes a ‘Complex Adaptive’ system.  Essential principles of the architecture, however, appear teasingly elegant and simple. Few of these design principles are unique to microservices, but in combination, they make a compelling story:
Publish anything of interest – don’t wait to be asked, if your microservice thinks it has some information that might be of use to the microservices ecosystem, then publish-and-be-damned.



Amplify success & attenuate failure – microservices that publish useful information thrive, while those that left unsubscribed, wither-on-the-vine. Information subscribers determine value, and value adjusts over time/changing circumstances.



Adaptive ecosystem – versions of microservices are encouraged –may-the-best-service-win mentality introduces variety which leads to evolution.



Asynchronous & encapsulated – everything is as asynchronous as possible – microservices manage their own data independently and then share it in event messages over an asynchronous publish-subscribe bus.



Think events not entities – no grand BDUF data model, just a cloud of ever-changing event messages – more like Twitter than a DBMS. Events have a “use-by-date” that indicates the freshness of data.



Events are immutable – time-series snapshots, no updates allowed.


Designed for failure – microservices must expect problems and tell the world when they encounter one and send out “I’m alive” heart-beats.



Self-organizing & self-monitoring – a self-organizing System-of-systems’ that needs no orchestration. Health monitoring and other administration features are established through a class of microservices.



Disposable Code – microservices are very, very small (typically under 1000 lines of code). They can be developed in any language.



Ultra-rapid deployment – new microservices can be written and deployed with hours with a zero-test SDLC.

It struck me that many of these design principles could apply, in part, to a 2020 Smart Grid architecture I’m working on, and to the much boarder ‘Internet of Things’ecosystem.

The microservices pattern does seem to lend itself to the notion of highly autonomous, location-independent s/w agents that could reside at the centre, mid-point or edge of an environment. I can imagine that the fundamental simplicity of the model would help, rather than hinder, data privacy and protection by being able to include high-level system contexts, policies and protocols (e.g. encryption and redaction) applied to the event-streams. This pattern, of course, won’t be the ‘right-fit’ for all situations, but it does seem to offer interesting opportunities in:

  • Agility – very small disposable services are deployable within hours
  • Resilience – withstands service failures and supports service evolution
  • Robustness – it’s hard to break due to: simplicity, in-built failure handling and lack of centralized orchestration

It may be that the microservices pattern can only be applied to operational decision-support and behaviour profiling situations. But if that’s the case, I still see great potential in a world where many trillions of sensor-generated events will be published, consumed, filtered, aggregated, and correlated. I’m no longer a developer, but as an architect, I’m always on the look-out for patterns that could: either apply to future vendors’ products and services, or could act as a guide for in-house software development practice.

As always, I’d be keen to hear your views, examples and opinions about microservices and their potential application to the IoT. Have you come across examples of microservices pattern in an IoT context – deployed or in the labs?

I whole-heartily recommend setting aside an hour to watch the video of Fred George’s presentation on microservices:


131108 1110 Dune Fred George Recording on 2013-11-08 1106-Vimeo from Øredev Conference on Vimeo.

Post-post:
  • Another great post about microservices  – including downsides.
  • More here including “The 8 fallacies of distributed computing”.
Duke Energy are doing some interesting things in the Edge Processing space.

Here’s a video on microservices in the conext of IoT  (worth ignoring the references to Cloud/Azure):

http://www.microsoftvirtualacademy.com/training-courses/exploring-microservices-in-docker-and-microsoft-azure

I’d like to talk to anyone who’s impelmenting/ thinking about a Staged Event Driven Architecture using microservices for Edge Processing.

Phil Wills on experience of deploying microservices at The Gaurdian
5 years, 2 months ago

Architect or Coach?

Is it just me, or are others finding the Enterprise Architect role shifting towards ‘Coach/Facilitator’? 
These days I find I’m most attracted to Tweets about workshop facilitation and business analysis techniques rather than anything discussing Enterprise Architecture: frameworks, methods and tools. 
Here’s a few links I’d recommend for those interested in the former.

5 years, 2 months ago

Architect or Coach?

Is it just me, or are others finding the Enterprise Architect role shifting towards ‘Coach/Facilitator’? 
These days I find I’m most attracted to Tweets about workshop facilitation and business analysis techniques rather than anything discussing Enterprise Architecture: frameworks, methods and tools. 
Here’s a few links I’d recommend for those interested in the former.

5 years, 5 months ago

Whole-Brained Business Analysis – New Metaphor Required

I’ve been guilty using the much debated ‘Left vs Right brain’ metaphor to explain what I believe is needed. By way of example, Alec Sharp (@alecsharp), Sally Bean  (@Cybersal), Roy Grubb  (@roygrubb) and I have been Tweeting about Concept Modeling vs Concept Mapping. Alec is keen to get Data Modelers to abstract their thinking up from physical Data Models by thinking conceptually and I have been encouraging Business Analysts to think similarly when gathering requirements. This has meant that we both find that we need to introduce a different mindset: one that encourages more creative & inclusive discussion atthe initial   discovery and play-back stage of the Requirements-Solution Design journey. I expect the Agile/XP community will declare this to be their philosophy (and nothing new) and they’re probably right. But rather than get caught-up in ‘IT-centric’ methods, I’d rather think of it as a way to better understand any requirements for change – regardless of the Software Development Life-Cycle. I’d rather see such thinking applied to all aspects of business change – people, process, practice, policy and … technology.


Tried-and-tested analytical techniques should not be abandoned, they just need to be augmented with others that, in my experience, help expand ideas and produce resilient, coherent and business-value-creating solutions.  Both side of the equation are equally important. However, I’m finding (through experiment) that the more creative techniques are more engaging – simply more fun and inclusive – and, this alone, can, in my recent experience, dramatically improve business outcomes. 

In attempts to explain the need for a more ‘whole-brained’ approach, I’ve been following the lead of the ‘Design Thinking’ community in referring to both Theory X and Theory Y from MIT Sloan and the Left-brain Right-brain metaphor. This, however, is fraught with problems due, in large part to the findings of the University of Utah who debunk such binary thinking (as I was reminded by Rob England – @theitskeptic).

So I’m in a quandary: on the one hand I find that an X-Y, Left-Right, metaphor is a simple way to convey the difference between, say, Analysis vs. Synthesis, on the other hand, however, I run the risk of aligning with outdated concepts being fundamental reconsidered by neuroscientists. 

I guess the Complexity Science community might say that I’m talking about the difference between ‘Complex Adaptive’  vs. ‘Complicated’ systems, but, again, academic debate makes coming up with a simple metaphor next to impossible.

Has anyone found an alternative metaphor for a more balanced approach to Business Analysis and Enterprise Architecture?

Importantly, I’m keen to avoid the impression that people are to be seen as fundamentally one way or another. My observation is that it is the practice of Business Analysis/Enterprise Architecture that needs to be more ‘Whole-brained’ – not the individuals per se.

To get the discussion rolling, I’d like to hear views on:
  • A good Business Analyst or Enterprise Architecture must be a balance of Left-X(Reliability – Doing-things-Right) and Right-Y (Validity – Doing-the-right-thing)
  • We’ve spent to much time of methods that attempt to industrialise EA (the TOGAF 9.0 manual runs to around 800 pages in the attempt) and BAs are too often focused on methods focus on an ‘IT solution’ rather that the Whys and Whats of the current or desired business behavior
  • We need to spend more time on developing pattern-based storytelling skills in BAs and EAs to deliver break-through changes and allow for innovation in TO-BE models.
  • Economic churn and environmental challenges warrant more Y-minded thinking (with appropriate X-controls)
  • The world can’t be fully explained or governed algorithmically (thank god!)– not while values and trust dominate the way organisations function.


 

5 years, 5 months ago

Whole-Brained Business Analysis – New Metaphor Required


I’ve been guilty using the much debated ‘Left vs Right brain’ metaphor to explain what I believe is needed. By way of example, Alec Sharp (@alecsharp), Sally Bean  (@Cybersal), Roy Grubb  (@roygrubb) and I have been Tweeting about Concept Modeling vs Concept Mapping. Alec is keen to get Data Modelers to abstract their thinking up from physical Data Models by thinking conceptually and I have been encouraging Business Analysts to think similarly when gathering requirements. This has meant that we both find that we need to introduce a different mindset: one that encourages more creative & inclusive discussion atthe initial   discovery and play-back stage of the Requirements-Solution Design journey. I expect the Agile/XP community will declare this to be their philosophy (and nothing new) and they’re probably right. But rather than get caught-up in ‘IT-centric’ methods, I’d rather think of it as a way to better understand any requirements for change – regardless of the Software Development Life-Cycle. I’d rather see such thinking applied to all aspects of business change – people, process, practice, policy and … technology.


Tried-and-tested analytical techniques should not be abandoned, they just need to be augmented with others that, in my experience, help expand ideas and produce resilient, coherent and business-value-creating solutions.  Both side of the equation are equally important. However, I’m finding (through experiment) that the more creative techniques are more engaging – simply more fun and inclusive – and, this alone, can, in my recent experience, dramatically improve business outcomes. 

In attempts to explain the need for a more ‘whole-brained’ approach, I’ve been following the lead of the ‘Design Thinking’ community in referring to both Theory X and Theory Y from MIT Sloan and the Left-brain Right-brain metaphor. This, however, is fraught with problems due, in large part to the findings of the University of Utah who debunk such binary thinking (as I was reminded by Rob England – @theitskeptic).

So I’m in a quandary: on the one hand I find that an X-Y, Left-Right, metaphor is a simple way to convey the difference between, say, Analysis vs. Synthesis, on the other hand, however, I run the risk of aligning with outdated concepts being fundamental reconsidered by neuroscientists. 

I guess the Complexity Science community might say that I’m talking about the difference between ‘Complex Adaptive’  vs. ‘Complicated’ systems, but, again, academic debate makes coming up with a simple metaphor next to impossible.

Has anyone found an alternative metaphor for a more balanced approach to Business Analysis and Enterprise Architecture?

Importantly, I’m keen to avoid the impression that people are to be seen as fundamentally one way or another. My observation is that it is the practice of Business Analysis/Enterprise Architecture that needs to be more ‘Whole-brained’ – not the individuals per se.

To get the discussion rolling, I’d like to hear views on:
  • A good Business Analyst or Enterprise Architecture must be a balance of Left-X(Reliability – Doing-things-Right) and Right-Y (Validity – Doing-the-right-thing)
  • We’ve spent to much time of methods that attempt to industrialise EA (the TOGAF 9.0 manual runs to around 800 pages in the attempt) and BAs are too often focused on methods focus on an ‘IT solution’ rather that the Whys and Whats of the current or desired business behavior
  • We need to spend more time on developing pattern-based storytelling skills in BAs and EAs to deliver break-through changes and allow for innovation in TO-BE models.
  • Economic churn and environmental challenges warrant more Y-minded thinking (with appropriate X-controls)
  • The world can’t be fully explained or governed algorithmically (thank god!)– not while values and trust dominate the way organisations function.