Microservices Unplugged

by
Given my (well-known and enduring) interest in all aspects of services, I have followed Martin Fowler’s writing on microservices. But I will admit I always found the original paper more confusing than insightful. And in my client work I have resisted the temptation to use a microservices pattern, for precisely the reason that it would more than likely confuse. So I was interested to see the book Building Microservices by Sam Newman published last month, particularly as Newman is part of the Thoughtworks stable, which presumably means it is authoritative.  

Right off the bat, Newman advises that we should “think of microservices as a specific approach for SOA in the same way that XP or Scrum are specific approaches for Agile Software development”. These analogies are very interesting because my expectation was that microservices is a pattern. So I might infer that microservices is a set of process techniques as opposed to an architectural approach. Yet in the book, Newman clearly includes some elements of concept model and architecture as well as process and organization.

Probably the biggest weakness of Newman’s book is the absence of a coherent reference model. He discusses a series of heuristics such as:

                – Microservices are small, autonomous services that work together.

                – With high cohesion.

                – Gather those things that change for the same reason, and separate those things that change for different reasons.

                – Using business boundaries.

                – How small is small? Could be written in 2 weeks; small enough and no smaller; can be managed by a small team

                – Separate entity. All communications between services are network calls. Decoupled. Minimum dependency.

                – We need to think about what our services expose, and what they should allow to be hidden. If there is too much sharing our consuming services become coupled to internal representations.

And these are mostly useful principles, but are most unlikely to guide a consistent approach. Given Martin Fowler’s involvement, I would have expected at least some good patterns. But mature SOA needs a defined concept model, like CBDI-SAE, SOA-RM etc.

This lack of clarity is fundamental. Many organizations treat operations as services, and they typically end up with thousands of services. They will be confused by the very term microservice because they already have what they perceive as small services. Perhaps the term microservice has been chosen because of the ubiquitous use of the ITIL service, which undoubtedly means monolith. Most mature SOA organizations define services and operations as a primary technique to establish cohesion guided by reference models such as SOA-RM and CBDI-SAE. Just as important is the clarity and precision around specifications or contracts and what is exposed to consumers (APIs) and providers. Yet the book contains nothing on the subject of rich service specification or contract as the primary means of achieving implementation independence and business alignment. Organizations adopting microservices will need a defined reference model. Without that there will be architecture governance by personal opinion. I wonder how Thoughtworks advise their clients in this area? 

The book contains some quite general advice on business capability based decomposition – which is a tried and tested method of defining services. However there is little or no advice on what a well-formed business capability is, it’s characteristics and governance considerations. In my experience this is an area for considerable confusion; the business capability concept is so well known, but little understood, it’s an accident waiting to happen. And as for the relationship between capability and service, there’s no model provided, and more importantly, the granularity of a business capability service is likely to be non-trivial.

Similarly there’s no guidance on types of service. Do all services contain a mix of channel, process, business and data behaviors? Which you might expect given the explicit advice to gather those things that change for the same reason, and separate those things that change for different reasons.

The one area that might be helpful is the concept that microservices are highly autonomous, and a capability cluster of services would be a unit of deployment. But sadly for Newman, this is an idea that I and my colleagues worked on and popularized nearly 20 years ago. We called it Component Based Development, and the concept was closely allied to Rich Service Specifications which enforced the component boundaries, a concept that seems to have escaped him.

Once upon a time I had expected there to be a microservice pattern. But having read the book, all I see is a set of established ideas being rehashed in a very superficial manner, under a new name; with many key concepts missing or misunderstood. Apparently there is a lack of consensus on how to do SOA well. That failure of SOA is caused by vendors’ products, issues with granularity or picking wrong places to split a system. These assessments of the problem illustrate the lack of understanding of the author and his colleagues. Yes, there have been SOA failures, but mostly they have arisen through a) a focus on integration technology not business, or b) lack of reference model, reference architecture and governance. As an aside, in mature SOA we don’t consider “places to split a system”; we define the scope of a service using well defined heuristics and techniques that vary by type of service. To claim therefore that microservices is the answer to problems with SOA is downright disingenuous. And given the amount of hype I see about the topic generated by supposedly responsible organizations such as Thoughtworks and Gartner, that’s downright irresponsible. Disappointing. 

Public Domain information on Mature SOA architecture and delivery practices

——————————————————————————————

What’s A Strategy?

by

Link to article
Recently, I was asked this simple question: What is a strategy?
According to this article a business strategy is: the result of choices executives make, on where to play and how to win, to maximize long-term value. 

Which seems reasonable enough definition, for the CEO, but doesn’t really help those who need to think strategically about other initiatives,  like, for example,  business change programmes. 

Here’s some thoughts on what a strategy is, and what it isn’t:


A strategy:
– Tells a story

– Is future focused

– Can apply to short, medium on long term futures

– Provides focus on what is important (and what isn’t)
– Is actionable

– Is a framework for decision-making

– Is holistic – considers changing environment & external events

– Examines alternatives

– Assesses risks

– Takes step back from Business As Usual – it is an abstraction from the current operation
– Can be emergent and living: will undergo revisions and course-corrections.
– Is a reference point for dealing with emergent change
– Is ‘WHY’, ‘WHAT’, ‘WHEN’ and ‘HOW’ focused. The ‘HOW’ in a strategy is a ‘Meta-HOW’:  often speaking to unknowns and unplanned (unknowable) events. The ‘WHEN’ is often more sequence focused than time-definite


– Is a choice

What a strategy isn’t:

– A detailed plan of activities

– A grand design

– A set of desired aims/objectives ( although these might be the basis)
– A mission statement

– A budget plan

– A collection of pre-determined projects

– A list of possibilities (technology-based or otherwise)

– Only applicable to ‘big’ or ‘long-term’ change (things often labelled as ‘strategic’).

Addendum:
I recommend reading Roger Martin’s wise words on the subject of Stategy.  A few quotes from his post:

“The problem with a lot of strategies is that they are full of non-choices. Probably most of us have read more than a few so-called strategies that say something like, “Our strategy is to be customer centric.” But is that really a choice?”

“I would argue that 90 percent of the strategic plans I’ve seen in my life are really more accurately described as budgets with prose”.

“If you’re into destroying innovation, two words—”prove” and “it”—will do the trick”.

What’s A Strategy?

by

Link to article
Recently, I was asked this simple question: What is a strategy?
According to this article a business strategy is: the result of choices executives make, on where to play and how to win, to maximize long-term value. 

Which seems reasonable enough definition, for the CEO, but doesn’t really help those who need to think strategically about other initiatives,  like, for example,  business change programmes. 

Here’s some thoughts on what a strategy is, and what it isn’t:


A strategy:
– Tells a story

– Is future focused

– Can apply to short, medium on long term futures

– Provides focus on what is important (and what isn’t)
– Is actionable

– Is a framework for decision-making

– Is holistic – considers changing environment & external events

– Examines alternatives

– Assesses risks

– Takes step back from Business As Usual – it is an abstraction from the current operation
– Can be emergent and living: will undergo revisions and course-corrections.
– Is a reference point for dealing with emergent change
– Is ‘WHY’, ‘WHAT’, ‘WHEN’ and ‘HOW’ focused. The ‘HOW’ in a strategy is a ‘Meta-HOW’:  often speaking to unknowns and unplanned (unknowable) events. The ‘WHEN’ is often more sequence focused than time-definite


– Is a choice

What a strategy isn’t:

– A detailed plan of activities

– A grand design

– A set of desired aims/objectives ( although these might be the basis)
– A mission statement

– A budget plan

– A collection of pre-determined projects

– A list of possibilities (technology-based or otherwise)

– Only applicable to ‘big’ or ‘long-term’ change (things often labelled as ‘strategic’).

Addendum:
I recommend reading Roger Martin’s wise words on the subject of Stategy.  A few quotes from his post:

“The problem with a lot of strategies is that they are full of non-choices. Probably most of us have read more than a few so-called strategies that say something like, “Our strategy is to be customer centric.” But is that really a choice?”

“I would argue that 90 percent of the strategic plans I’ve seen in my life are really more accurately described as budgets with prose”.

“If you’re into destroying innovation, two words—”prove” and “it”—will do the trick”.

Mastering ArchiMate

by

This is not your usual architecture book as its title suggests it is about modelling using the ArchiMate notation. The author doesn’t promote any particular tool and concentrates on the notation, which recently underwent a major revision. Whether you are new to ArchiMate or have been using it for some time I think you’ll find […]

A Wiggly Path To Transformation

by

According to Sunday Times journalist, Carly Chynoweth, “Managers must learn how to adapt so they can solve problems they haven’t faced before.”

And:

“The fundamental impression they give is that the future can be organised and managed to achieve what you set out to achieve, and as long as you do it right it will come out as planned. But the reality is much more complex. Organisations are wiggly. They don’t operate in the neat, straight-line way conventional management thinking assumes”.

“It is not possible to predict outcomes — they emerge from people’s actions. You might have plans about what you are doing, but so does everyone else.”

****
The culture and traditions of 100 year old Utility business make behavioual change hard. Power-plant Engineer’s detailed and precise planning techniques don’t work for this sort of long-term change. In our case, over 70% of the business processes will change. The main hurdle for us was the lack of certainty and predictability over 5, 10 and 20 years. It was impossible to answer the basic questions; when, how and how much? There was discomfort over sanctioning any plans without the ‘details’. This resulted in a period of ‘decision-making-block’.



This is when we introduced ‘Transition State’ planning. This borrows ideas from Complexity Theory: Specifically, the characteristics of Complex Adaptive Systems. We started with the premise that we cannot predict the long range future.  We also accepted that outcomes will emerge at various points along the way.


Much like ’Scenario Planning’, we first developed a few hypothetical ideas. These were refined in workshops with subject experts until we reached a reasonable consensus of:
  • the principles we will apply to different aspects of the transformation ahead of us,
  • an initial guess of when certain outcomes are needed,
  • a list of the main influencing events/decisions assessed as ‘Known’, ‘Unknown’ or ‘Unknowable’ now – along with an assessment of risk.

Armed with this, we plotted-out a series of ‘Way-points’ over time. We call these ‘Transition States’. Each has a few goals (expected outcomes) that we estimate to complete.  At this stage, Transition States are not pinned to a hard date.  Rather they describe the sequence of outcomes, and roughly when we think they’ll happen. Transition States are re-planning points; a time to reassess and re-estimate the next phase of work. The Transition State is an opportunity to amplify value-adding and extinguish value-detracting aspects.


After a few iterations, a more precise view emerged of the early Transition States and a traditional approach to planning and deliverables could be applied.


At first, we were quite concerned that this approach would be rejected by the culture; a plan must be very detailed and accurate before work can start. This, however, wasn’t the case, decision-makers could see the merits of a more agile and iterative approach to complex change. They liked the way ‘doable’ increments became clearer after a few iterations. They also liked the explicit opportunities for re-think and course-correction.



****
Is a Complex Adaptive approach the only way to manage large-scale, long-running, change? I’d like to hear how others approach designing and managing business transformations and whether they see the merits of Complexity Theory that we’ve seen, 

A Wiggly Path To Transformation

by

According to Sunday Times journalist, Carly Chynoweth, “Managers must learn how to adapt so they can solve problems they haven’t faced before.”

And:

“The fundamental impression they give is that the future can be organised and managed to achieve what you set out to achieve, and as long as you do it right it will come out as planned. But the reality is much more complex. Organisations are wiggly. They don’t operate in the neat, straight-line way conventional management thinking assumes”.

“It is not possible to predict outcomes — they emerge from people’s actions. You might have plans about what you are doing, but so does everyone else.”

****
The culture and traditions of 100 year old Utility business make behavioual change hard. Power-plant Engineer’s detailed and precise planning techniques don’t work for this sort of long-term change. In our case, over 70% of the business processes will change. The main hurdle for us was the lack of certainty and predictability over 5, 10 and 20 years. It was impossible to answer the basic questions; when, how and how much? There was discomfort over sanctioning any plans without the ‘details’. This resulted in a period of ‘decision-making-block’.

This is when we introduced ‘Transition State’ planning. This borrows ideas from Complexity Theory: Specifically, the characteristics of Complex Adaptive Systems. We started with the premise that we cannot predict the long range future.  We also accepted that outcomes will emerge at various points along the way.


Much like ’Scenario Planning’, we first developed a few hypothetical ideas. These were refined in workshops with subject experts until we reached a reasonable consensus of:
  • the principles we will apply to different aspects of the transformation ahead of us,
  • an initial guess of when certain outcomes are needed,
  • a list of the main influencing events/decisions assessed as ‘Known’, ‘Unknown’ or ‘Unknowable’ now – along with an assessment of risk.

Armed with this, we plotted-out a series of ‘Way-points’ over time. We call these ‘Transition States’. Each has a few goals (expected outcomes) that we estimate to complete.  At this stage, Transition States are not pinned to a hard date.  Rather they describe the sequence of outcomes, and roughly when we think they’ll happen. Transition States are re-planning points; a time to reassess and re-estimate the next phase of work. The Transition State is an opportunity to amplify value-adding and extinguish value-detracting aspects.


After a few iterations, a more precise view emerged of the early Transition States and a traditional approach to planning and deliverables could be applied.


At first, we were quite concerned that this approach would be rejected by the culture; a plan must be very detailed and accurate before work can start. This, however, wasn’t the case, decision-makers could see the merits of a more agile and iterative approach to complex change. They liked the way ‘doable’ increments became clearer after a few iterations. They also liked the explicit opportunities for re-think and course-correction.



****
Is a Complex Adaptive approach the only way to manage large-scale, long-running, change? I’d like to hear how others approach designing and managing business transformations and whether they see the merits of Complexity Theory that we’ve seen, 

Become a Certified Enterprise Architect

by

The Carnegie Mellon University Certified Enterprise Architect Program is one of QualiWare Academy‘s core certification  offerings. Currently offerings include: Denmark EA Fundmentals, 14-16 September EA Applied, 19-22 October EA Advanced, 16-18 November Norway EA Fundamentals, tba EA Applied, tba EA Advanced, tba    

ArchiMate Modeling in Practice. Top-down: Business architecture models

by

In the previous posting we explained two strategies for getting started with ArchiMate modeling: top-down and bottom-up. Over-simplifying, we suggested to use “inventory style” models (to catalog the ‘things we have / do’) and then figuring out the relations between them. There’s usually a lot of information that can be (re)used. Also, the ideal approach is using a workshop setting.