What’s the current state of mainstream enterprise-architecture? Where is it headed?
I’ve just spent the past few days in and around the Open Group’s Amsterdam ‘summit’. (I say ‘in and around’ because I was there officially for just one of the three days – the ‘Archimate Day’ on the Wednesday – but also various meetings and such with various folks either side of that.) There’ve been quite a few interesting insights – interesting for me, at least! – that I’d best put down in writing now before I forget the whole lot…
#1: The continuing slow expansion outward from IT-only
This first point is a theme that’s been becoming more and more evident over all of the years I’ve worked in enterprise-architecture, and its implications are becoming all but inescapable now to all but a few ‘IT is the sole centre of the world!’ die-hards.
The point is this: the core logic of all architecture is that everything depends on everything else. Whether we like it or not, and wherever we choose to start or to focus – such as, in TOGAF’s case, with the IT-infrastructure – ultimately we’re always working on the whole as whole. There’s no way to avoid this.
Hence, to follow TOGAF’s history, it started out with its focus almost exclusively on IT-infrastructure (versions 1-7); then realised that, to understand the drivers for those infrastructures, it needed to assess the applications and data maintained on that infrastructure (version 8). Successive versions have added assessment of business use of the data and apps, and, more recently, the technology-oriented aspects of business-strategy. But it does not and cannot stop there: that’s the whole point. By its own logic, it must continue outward until it addresses the whole shared-enterprise-in-scope – or rather, to where the ‘just-enough architecture’ boundary is reached, which in practice is around three steps larger than the organisation-in-scope.
In short, we urgently need an enterprise-architecture framework that can cover the entirety of the whole-of-enterprise space – and this point is becoming more and more clear in the TOGAF space, especially to the folks I talked with at the Open Group conference who are currently working on ‘TOGAF X’, the eventually-upcoming next version of TOGAF.
The catch is that TOGAF, in its present form, is simply not equipped to do this. It’s still an IT-architecture framework, where the focus is, ultimately always and only, on IT-infrastructure. Which, unfortunately, is not how it’s ‘sold’ these days. Which brings us to the next point…
#2: TOGAF/Archimate is still a niche-product for a vertical-market
With the possible exception of Zachman, almost all of the existing enterprise-architecture frameworks are designed around the needs of a specific vertical-market or domain: for example, FEAF for US government, or TEAF for government finance, the DoDAF / MoDAF / NAF cluster for defence-industry and procurement, eTOM / SID / Frameworx for telecoms, or SCOR for logistics.
By contrast, the common claim is that TOGAF, and/or the TOGAF/Archimate pairing, is almost unique as a true horizontal: it covers all forms of enterprise-architecture, everywhere.
The practical reality, unfortunately is that that claim just is not true. Okay, yes, we could possibly assert that it’s arguably sort-of true, for a given special-case value of ‘true’ – which, however, is not the same at all… And, combined with the inevitable expansion of EA outward from the ‘classic’ IT-only box, is where this is all starting to break down.
The notion that TOGAF/Archimate is a ‘true horizontal’ rests on two key assumptions:
- all business centres around information, and thence primarily or exclusively around computer-based IT and IT-based automation
- the IT of the business – and, therefore, the business of the business – focusses primarily on highly-repeatable transaction-based applications
Or, in SCAN terms, the kinds of use-cases that sit tight up against the left-hand edge of the SCAN frame:
To be even more specific, the apparent effective aim in TOGAF is a version of the SCAN frame in which the ‘boundary of effective (un)certainty’ is held as close as possible to the left-hand side – especially when we move down to operations – and where everything to the right of that boundary is deemed irrelevant, or even not to exist. In visual form, somewhat like this:
When we put all of these together, what we end up with in the current version of TOGAF / Archimate is what might claim to be a horizontal, but in practice is a very specific ‘horizontal’ that, for the most part, occurs only within a specific subset of a very small subset of industries – in essence, the IT-support for information-centric transaction-oriented business-contexts such as finance, insurance, banking and tax. Which is exactly why the key worked-example for Archimate is a fictional insurance-company, Archisurance – because, as a framework, it really isn’t equipped to handle anything else.
(Yes, I do know that there’s a new worked-example called ArchiMetal. Which sounds like it might actually cover more of that enterprise. But doesn’t: in fact the model is solely about transactional order-processing. Again. I rest my case?)
In other words, it’s a framework to support a ‘horizontal’ that, beyond some very specific true-horizontal use-cases such as some aspects of automated order-processing, actually looks a lot more like a vertical – or, at most, a cluster of verticals – and only a subset niche within those verticals, at that.
The key point about ‘an architecture of the enterprise’ is that, by definition, it must cover the entirety of the enterprise-context – or, in other words, the entirety of the set of challenges and concerns that we would map out on a SCAN frame.
All of it – not just an arbitrarily-selected subset of it.
(And, perhaps especially, not solely ‘the easy bit’, as implied by that second diagram just above.)
Which, in its present form, TOGAF/Archimate not only doesn’t do, but can’t do.
Which leaves a rather large gap that must somehow be filled by, uh, something else.
Whatever that ‘something else’ might be.
Now that would all actually perfectly okay, if there was explicit acknowledgement that TOGAF/Archimate was really intended only for that one purpose, and nothing else. But that isn’t what happens, is it? Instead, there are explicit claims that it covers the whole scope of ‘enterprise-architecture’, for every type of organisation, industry or context.
Which it can’t. Not really.
Okay, we can kind-of kludge it to sort-of work in other industries and contexts. But the reality is that, with TOGAF, we end up forcing the context to fit the framework (rather than fitting the framework to the context, which is how we should do it); and with Archimate, were forced to invent any number of special-case fudged-up-for-the-purpose entities that increase the complexity of the underlying metamodel almost factorially with each new item, and often break the logic of the inter-entity relationship-model. None of which is a good idea…
Which leaves us with a framework/notation set that, for any whole-of-enterprise-architecture – even in that ‘vertical-cluster’ of finance, insurance, banking and tax, let alone anything else – can only be used if it’s horribly kludged, and used in ways that are neither documented nor align with the original purpose.
Which means, in practice, that the probably only true ‘whole-of-enterprise’ framework we have available to us at present is Zachman. Which is so limited and constrained that, to be honest, it’s only slightly better than useless.
Which leaves us in an awkward position. TOGAF/Archimate claims to be fully horizontal, but isn’t: it’s IT-specific, and not really usable for anything more than that. So if we use it outside of its designed-scope, but purport to be doing enterprise architecture in the literal and necessary sense of the term, we’d actually be indulging in false-advertising – again, not a good idea. And because the scope for any ‘architecture of the enterprise’ is necessarily still expanding – as per insight #1 above – then the longer we leave it, the worse that problem gets. Tricky…
TOGAF/Archimate does work quite well as a niche framework for a subset of the enterprise-architecture needs of a subset of industry-verticals – I don’t have any doubt about that. But there are steeply-increasing doubts about how well it will fit – or can legally purport to fit – for the enterprise-architecture of just about anything else: and that was a topic that came up a lot of times in my conversations at the conference.
So there’s a real choice-point for TOGAF/Archimate here: either stick with that one cluster-vertical for which it is a fairly good fit, and stop pretending that it’s doing anything more than that; or accept the need for fundamental change, if it’s to remain relevant as an enterprise-architecture framework. There are no other options – or, at least, no other options that don’t fall foul of the law on trades-descriptions and the like.
Over to you on that one, folks?
#3: We really need to break out of the ‘BDAT stack’
The sticking-point in all of this is the wretched ‘BDAT stack’ – the so-called ‘Business Architecture’, ‘Information-System Architecture’ and ‘Technology Architecture’ concept that’s embedded right into the roots of both TOGAF and Archimate:
Now if we’re doing the ‘enterprise-architecture’ solely for a technology-infrastructure concern, the BDAT-stack is an appropriate frame to use. But that ‘if’ is absolutely crucial – and way too many people in the TOGAF / Archimate space at present still don’t seem to understand that point, or why it’s so crucial, or what its implications really are.
Remember what I said above, that for enterprise-architecture, we typically need to assess a shared-enterprise that is around three steps larger than the organisation-in-scope? Well, this is what that looks like in generic form:
In the case of the BDAT-stack, we’re actually dealing with a smaller subset of the above, that looks like this:
If we think of the technology-concern as ‘the organisation’, then the applications and data that are maintained on that technology are its ‘suppliers and customers’; and what IT-folks would typically call ‘the business’ – the broader scope again, the context for those applications and data – is the metaphoric equivalent of ‘the market’. If all we’re ultimately doing is technology-architecture, then the BDAT-stack does fit this architectural-pattern well, and does work.
But it doesn’t work if we try to run the pattern back-to-front: and that’s where the BDAT-stack breaks, because that’s exactly what way too many people try to do.
Hence, for example, the common yet utterly-mistaken claim that “TOGAF / Archimate can do every form of enterprise-architecture, because, look, it’s got Business Architecture”. The reality – and it shouldn’t take more than a few minutes’ reading to verify this, is that the so-called ‘Business Architecture’ in both TOGAF and Archimate is a means to assess ‘anything not-IT that might affect IT’ – almost nothing more than that. It doesn’t cover more than a tiny IT-oriented subset of what the business would consider to be its own business-architecture – which, from that perspective, would include financial-architecture, security-architecture, compliance-architecture, brand-architecture, product-architecture, and much, much more. And it would also include those parts of the organisation’s architecture that don’t even touch IT – such as person-to-person relations, or the parts of information-management that need to take place when the IT isn’t working. For a real business-architecture, we use the same pattern above, but where the business is ‘the organisation-in-scope’ – with ‘Business Architecture’ as the root-layer, not the top.
To explain this a slightly different way, that may be a bit clearer, consider this set of relationships – inside-in, inside-out, outside-in, outside-out, and the key ‘interaction-journey’ or ‘customer-journey’ interface between inside-out and outside-in:
In developing an enterprise-architecture, we apply inside-in to the root-level – in the BDAT-stack case, that’s Technology Architecture, exactly as per TOGAF version 8 – to assess it in its own terms; and we apply inside-out to assess its view of the ‘outside world’, starting in this case with the combined Information-Systems Architecture, the two somewhat-different views for support for applications and data. They in turn have an outside-in interaction-journey with the root-level; and an outside-out view of themselves, in their own terms.
For the BDAT-stack, this sort-of repeats at the next level out, as the relationships between Information-System Architecture and Business Architecture – but the crucial difference is that our view of those it’s already only a subset, constrained by the scope of the root-level. (In effect, it’s a bit like looking upward from the bottom of a well.)
What we don’t have in the BDAT-stack is the next level outward from Business-Architecture – which, logically, would be something like a Supply-Chain Architecture, and thence onward and outward from there. And because we don’t have the next layers outward, we can’t do an inside-out assessment – or, for that matter, an impacts-oriented outside-in and outside-out assessment – for Business Architecture, or really even for Information-System Architecture: we can’t run the pattern ‘backward’, because the next-layers-outward that we need aren’t there.
The key to enterprise-architecture in general is that pattern of relationships – not the specific instantiation of those relationships. And the BDAT-stack is only an instantiation of that pattern.
If we only want to do technology-architecture, the BDAT-stack is likely to be exactly what we need – the appropriate instantiation of that broader pattern. (That’s why TOGAF worked quite well for its stated purpose up until version 8.)
If we want to develop an architecture for almost anything other than IT-infrastructure, the BDAT-stack is likely to be exactly what we don’t need – an inappropriate instantiation of that broader pattern. (That’s why TOGAF hasn’t worked well since version 8, and why TOGAF 9 was such a missed-opportunity.)
So if we want to do enterprise-architecture in general, we need to be able to access that generic pattern, and create any instantiations of that pattern that we might need.
And that’s a huge problem for TOGAF and Archimate alike, because in both cases the BDAT-stack is hardwired into their structures right at the very root: as Phases B, C and D in the TOGAF ADM, for example, and in the fundamental anatomy of Archimate. Neither TOGAF nor Archimate can allow us to create any other instantiation of that pattern than the BDAT-stack. Which means that, right now, we’re stuck: courtesy of those arbitrary hardwired-limitations, we cannot use TOGAF or Archimate reliably or successfully for anything other than IT-infrastructure architecture.
Which, of course, for many TOGAF and Archimate users, is not a problem, because that’s all that they’re doing anyway. But it’s a huge problem for anyone doing any other aspect of enterprise-architecture – even including data- or applications-architecture, as became clear during some of the panel-discussions in the Archimate sessions at the conference.
If we’re to be able to cope with the necessarily increasing scope of our enterprise-architectures – which are inherently expanding all the time, as per Item #1 above – then we must be able to break out of the BDAT-stack. If we can’t, then TOGAF and Archimate will fade away into irrelevance: that’s the inevitable consequence of failing to face this issue.
Once again, over to you on that…
#4: Archimate bridges viewpoints
This one is much simpler, and is an ‘obvious’ point about Archimate that I ‘should’ have noticed long before now, and hadn’t. Oops… (said to me, this time, not to anyone else!)
ISO42010, aka IEEE1477 – pretty much the de-facto mainstream standard for many of the concepts in mainstream ‘enterprise’-architecture – talks about views and viewpoints.
I forget the exact wording, but in essence a viewpoint is the perspective or language or terminology of a single stakeholder or stakeholder-group – in others words, both literally and metaphorically a point of view. And a view is a description of some context from that viewpoint. Each diagram or, more colloquially, ‘model’, describes a single view from a single viewpoint. All pretty straightforward, really.
Except that the neat trick of Archimate is that it allows a single diagram to combine views from more than one viewpoint. That’s where the dreaded ‘layers’ come into the picture: an Archimate diagram can show the content from the viewpoint of a single ‘layer’, or combine the views from two or more ‘layers’, and the relationships between those viewpoint-specific views. In Archimate version 1, we could only explore the context-space in terms of the BDAT-stack (though actually not much about the ‘D’, or data); in Archimate version 2, though, we can merge in a handful of other viewpoints too – mainly ‘Motivation’ (business-drivers and suchlike), ‘Change’ (project-management concerns) and ‘Location’.
Credit where credit’s due (in this case to Marc Lankhorst and his colleagues, way back when), because that’s actually a really important point: in effect, explicitly extending ISO42010. Definitely a neat trick – and one that is probably central to understanding architecture as architecture. And especially so if and when we do finally break of the stranglehold of the BDAT-stack, and allow our architectures to explore any scope as required.
#5: The challenge of ‘meta’
The next insight that I had from the conference arises from a combination of some of the above, and some folks’ reactions to it.
Or, specifically, that quite a lot of people are very unhappy about any suggestion that we should drop the centrality of the BDAT-stack.
So unhappy, in fact, that they’d brand me somewhere between a nutcase, an apostate and a traitor. Yeah, that kind of full-on religious-type intensity – or not far off it, anyway.
Okay, to be blunt, some of that overreaction comes from ‘protecting the money’. It’s no secret that even though much of the current IT-centric ‘enterprise’-architecture may well be worse than useless in quite a lot of contexts, the bald fact is that, yes, it’s proven very much ‘a nice little earner’ – ‘nice big earner’, more accurately – for some very big consultancies for some years now: and those kinds of folks don’t kindly to any kind of apparent risk to upset that applecart. Same applies to a lot of the trainers: the BDAT-stack may be hopelessly-misleading for many real-world purposes, but it’s easy to teach – and there’s many millions of dollars a year coming in to someone from that kind of teaching-activity. So yeah, there are – to say the least – some serious vested interests in maintaining the myth that the BDAT-stack ‘is’ enterprise-architecture.
Which all of us now know, for certain, that it isn’t.
Kinda tricky, that…
However, there is a way out of this little impasse – and it’s one that I’ve alluded to above, but probably need to spell out properly here.
The reality, in fact, is that they’re not only defending the indefensible, but there’s no need for all that passionate over-defence anyway. No-one is saying that we should not or never use the BDAT-stack, or that we should somehow ban it or expunge it from all mention or suchlike. That’d be stupid, because it is useful for some purposes – exactly as described above. What we are saying is that we should end the pretence that the BDAT-stack is the only-permissible (even only-possible) frame through which to describe an enterprise-architecture – which is what TOGAF and Archimate currently try to force us to do.
What matters, in most practical applications of enterprise-architecture, is that pattern of relationships between viewpoints and views, as in the ‘organisation, supply-chain, market, enterprise’ diagram above.
The BDAT-stack is an instantiation of that pattern, as also described above.
The BDAT-stack is an instantiation, not the instantiation: y’know, that’s a rather important difference there?
An instantiation of a pattern is not the same as the pattern itself! – that’s a really simple, really fundamental, seemingly-’obvious’ point that a frankly-terrifying number of self-styled ‘enterprise-architects’ still seem unable to comprehend…
I will admit I have no idea why so many people seem to struggle with this one, but it’s painfully evident that they do…
What it comes down to is this:
– There are two key approaches to tackling any problem: start from the concrete content, or start from the abstract pattern. Ultimately, for architectures, we need both concrete and abstract: abstract to deal with variance and uncertainty (‘samenesses within uniqueness’), and concrete to bring it into fruition in the real-world here-and-now (‘unique instantiation of supposed-sameness’).
– When things are simple and straightforward, and contained within a narrow scope, we can focus primarily on content. The catch is that over time – and especially if the scope also expands over time – the content required to cover every eventuality becomes unmanageable. (If TOGAF 9 had retained the same level of detail as TOGAF 8 had used for IT-infrastructure alone, the book would have been several thousand pages long – and out of date even before it had been published.)
– When things are ambiguous and/or cover a wide-ranging scope, we must start from abstracts, and then instantiate down towards context-specific content. For example, we need to use content-agnostic metaframeworks (such as the enterprise-model above) to derive content-oriented and context-specific frameworks (such as the BDAT-stack).
Or, as I put it in yet another post, ‘The science of enterprise-architecture‘:
- where managers and solution-designers need frameworks, to give clear instructions on what to do, we need metaframeworks – frameworks to help us design context-specific frameworks
- where they need methods and methodologies, we need metamethods and metamethodologies – the tools and techniques to identify and define those context-specific methods and methodologies
- where they need schemas and models and metamodels for data, we need metametamodels and metametametamodels – the tools and techniques that can create any context-specific metamodel for any required scope or level
- where they need toolsets with distinct notations to tackle distinct business-questions, we need tools that can cover every scope at every level of detail across the entire toolset spectrum
Note that a metaframework is just another framework, in exactly the same way that a metamodel is just another type of model. It’s not the framework itself that is ‘meta-’: it’s how we use the framework that makes it ‘meta-’. We don’t really need ‘yet another framework’: what we need is clarity about using frameworks with ‘meta-ness’, using frameworks in a ‘meta-’ way to create context-specific, concrete-oriented frameworks.
What we must not do – must never do – is pretend that a single instantiation of a metamodel or metaframework ‘is’ the sum-total of the whole space. It’s only an instantiation – not the whole thing.
The practical challenge, though, is that many people seem to have difficulty working with abstracts – and, in particular, the ‘translations’ from concrete to abstract, and (perhaps even more) from abstract to concrete. Likewise, down at the Simple level (in SCAN terms), most people seem to need simple step-by-step instructions – which, almost by definition, they won’t get directly from a metaframework. (Indirectly, yes, but not directly – and it seems to be that hop-skip-and-jump mental juggling-act of indirection between ‘meta-’ and concrete that’s the real source of the problem here.)
Seems to me that the way out is that although actual implementation will always need to focus on context, our architecture-training – the materials we use within the discipline itself – should focus on the meta-level, with patterns, metaframeworks, metamethodologies, metametamodels and the like; and, as-close-as-we-can-to-step-by-step instructions on how to adapt and instantiate those ‘meta-’ tools. That’s what’s currently missing from TOGAF and its equivalents: they’re still content-oriented, not meta-oriented.
Working with recursive, fractal, iterative, emergent ‘meta-’ is challenging: no doubt about that. But the blunt fact is that we cannot and must not claim to be ‘architects’ unless we do work primarily at that ‘meta-’ level – and there really is no way round that point. Concrete-level frameworks such as the BDAT-stack are useful, especially at the solution-architecture level: but if we can’t work with the ‘meta-’ level, able to create any context-appropriate context-specific equivalents of the BDAT-stack, then we’re likely to prove worse than useless when we try to cope with the near-infinite scopes and stakeholder-concerns at a true whole-of-enterprise scope and scale. And since that latter is where enterprise-architecture is headed – by definition – then this is a challenge that, collectively, we do need to address.
Which brings me to the last insight from that conference…
#6: The real role for Open Group in EA?
(What follows is not necessarily about Open Group as such: it applies just as much – if not more, in some cases – to all of its ‘competitors’ such as OMG [Object Management Group]. We also, and crucially, need to draw distinctions at times between Open-Group-the-organisation and Open-Group-the-membership – because the respective roles and responsibilities are often significantly different.)
Open Group – both organisation and membership – has done enormously-valuable work in enterprise-architecture. Yet it’s now reaching somewhat of a crossroads, where decisions and actions need to be taken about its ongoing role in EA.
The fundamental points here are that the Open Group is an IT-standards body, with an emphasis on ‘boundaryless information-flow‘.
Open-Group-the-organisation also has considerable professional expertise in guiding consortia, both within and beyond what we might call ‘the IT-industry’.
The membership-model for Open Group – with its fee-structure on a per-organisation basis, not per-person – inherently skews the membership towards larger organisations rather than smaller ones: a membership-fee of a few thousand dollars a year is high-value ‘small-change’ to a corporation the size of Microsoft or CapGemini, but prohibitively-expensive for all but the most committed of sole-proprietor consultants.
In its early stages, TOGAF was almost exclusively focussed on IT (and, more specifically, on management of large-scale IT-infrastructures) – which was a good fit for the business-needs of the large IT-consultancies and suchlike that, both then and now, form the backbone of the Open Group membership.
Over the years, the scope of enterprise-architecture has expanded, to the point where it has already shifted emphasis from IT-only to a much stronger focus on business and, increasingly, the shared-enterprise as a whole. Increasingly, other disciplines from far ‘outside’ of IT, such as psychology, anthropology, marketing, all need to be included as ‘equal citizens’ in the architecture.
As an IT-standards body, the Open Group as a whole is not equipped or well-suited to that kind of scope – especially as IT necessarily ceases to be ‘the centre of everything’, and instead merely becomes one thread amongst an infinity of ‘everything and nothing is ‘the centre’, all at the same time’.
The member-organisations of Open Group – and perhaps especially the larger and more influential members – all have a strong business-emphasis on IT in general, and often large-scale IT in particular: that’s where their bread-and-butter is, and much if not most of their business-history, too. Hence it should be no surprise that, as can be seen all often, they will – in fact probably must – resist any risk of the dethroning of IT as ‘the centre’ of what they would call ‘enterprise’-architecture. The current limitations of TOGAF 9 came about as a direct consequence of active resistance against dethroning the centrality of the BDAT-stack – despite the fact that it was already evident, way back at TOGAF 9′s release in 2009, that the scope of enterprise-architecture was already far broader than the BDAT-stack would allow.
The blunt fact is that the Open Group – perhaps particularly in terms of its membership – is no longer appropriate as a custodian for the future development of enterprise-architecture. Although IT will always play a key part in EA – to quote Andrew McAfee, “it’s not not about the technology” – it’s also about much more than just the technology: and a body whose natural tendency is to always bring everything back to IT rapidly becomes more of a hindrance than a help.
Right now there is no better alternative to Open Group: its ‘competitors’ such as OMG are no better for this, and sometimes arguably worse. Yet the fact remains that an IT-standards body is not the right place for EA to be – not any longer.
Which leaves us, as a profession, at a bit of an impasse. And I don’t have any easy answers for this one: all I do understand is that the questions do need to be asked, and asked, and asked, until we can arrive at a better resolution than this.
Open-Group-the-organisation could probably help a lot in this, given their skills and experience in building the kind of vastly-multidisciplinary consortium that would be needed for a true whole-of-enterprise EA to flourish. Yet it probably also is time that EA should bid farewell to Open-Group-the-membership – or at least, to insist on a much more equal playing-field for all parties involved.
There is, however, another really important role that Open Group as a whole could also play: and that’s around ‘boundaryless information-flow’ for architecture itself. There’ve been several attempts over the years, but right now, there is no usable cross-vendor standard for information-exchange for architecture-models and repositories – and we urgently need one.
The catch is that it’ll need to cover a much broader scope than, say, the current efforts towards interchange of Archimate models: in fact it has to be able to cover any scope, any required content – yet do so in a manner that’s fully-consistent, fully-interoperable, fully self-adaptive and self-describing yet also fully-secure, regardless of what we need to throw at it. Which means we’d need to start right out at the ‘meta-’ level, with fully-consistent distinctions between model, view, viewpoint and much more.
It’s a huge challenge – yet it’s also the kind of challenge that Open Group has excelled at, time after time, for so many different industries now. So could we perhaps ‘eat our own dogfood’ here, and at last apply those same skills to our own industry’s now-urgent needs?