David Sprott

Agile Business Modeling – The Core Heuristic?

by

How many times have I heard that the real problem with Agile is getting to the start line? There has to be some definition up front, but Agile methods don’t really help. Perhaps it’s a little secret for many organizations that they feel they must do more specification work up front because it makes it easier to control the Sprints. Oh dear!

To get to this starting gate we need to model the agile business in an Agile manner (YES!). Further we do not want to undertake complete or detailed business architecture (NO!!). We don’t have time, and anyway the core of the innovation and architecture should be done in the Agile Delivery project. But before we can fire up Agile projects we need to determine the scope and charter. If we use conventional scoping methods we may well deliver great functionality very quickly, but we probably won’t, unless we are very lucky, have delivered agile business capabilities that map to the business dynamics and can evolve along with the business.

Here’s a technique that may help.

In the first image below I show a functional decomposition for complaints management which I have clustered into “candidate capabilities” labelled 1, 2 and 3, process management, customer relationships and analysis respectively. This usefully shows that capabilities can be varying levels of abstraction; there’s absolutely no necessity to have elegant models!  The table below the decomposition shows various criteria I used to help me decide on the possible clusters. As you will see there’s variation in strategic classification; the partitioning – which may be key for deployment, some could be centralized others local; and the need for implementation independence and so on.

This analysis certainly helps me present some choices. But aside from the independence and scalability criteria and possibly standardization criteria, I feel I have not fully exhausted the analysis of the need for business agility. In the table below I develop this a little further. First I make an assessment of the potential requirement for future change in each function. I call this Agility Potential (AP) on a 1=Low and 5=High scale [1]. Not surprisingly Analysis and Skills are the capabilities that will probably be subject to considerable volatility. Second I look at the dependencies between the functions; note you have to read this as each row dependency upon a column. And low and behold, Skills and Analysis, and Analysis and Follow-up have high dependencies. This causes me to reconsider my initial cut of capability boundaries. I feel that Skills needs to be very close to Analysis as the investigatory function. And Follow-up should be similarly very close to Analysis. And what’s more these three functions score most highly on the AP scale. I feel Follow-up could easily be collapsed into Analysis, and a name change to Investigation would be perfect. I think a little more deeply about Skills. The degree to which the outcomes of Investigation need to be fed into Skills on a dynamic basis will vary depending on the type of business. If this was a safety critical business, I might recommend consolidating Skills and Investigation and renaming it Knowledge Management. But this really would depend on the business sector specific needs. 
To recap, what I have done here is developed a sharper understanding of the capabilities, and I have attributed them with governance criteria (in the first table) – I know what I must have delivered, and I am communicating some really important information to the delivery team, without constraining them at all on the implementation and delivery method. Also I now know the dependencies between the capabilities, and we can very quickly resolve the services that will be required and the inter project dependencies. And it didn’t take me very long at all.

More on Agile Business Modeling 

[1] I first outlined the idea of Agility Potential in the CBDI Journal April, 2010. Let me know if you would like a copy.

Agile Business Modeling – The Core Heuristic?

by

How many times have I heard that the real problem with Agile is getting to the start line? There has to be some definition up front, but Agile methods don’t really help. Perhaps it’s a little secret for many organizations that they feel they must do more specification work up front because it makes it easier to control the Sprints. Oh dear!

To get to this starting gate we need to model the agile business in an Agile manner (YES!). Further we do not want to undertake complete or detailed business architecture (NO!!). We don’t have time, and anyway the core of the innovation and architecture should be done in the Agile Delivery project. But before we can fire up Agile projects we need to determine the scope and charter. If we use conventional scoping methods we may well deliver great functionality very quickly, but we probably won’t, unless we are very lucky, have delivered agile business capabilities that map to the business dynamics and can evolve along with the business.

Here’s a technique that may help.

In the first image below I show a functional decomposition for complaints management which I have clustered into “candidate capabilities” labelled 1, 2 and 3, process management, customer relationships and analysis respectively. This usefully shows that capabilities can be varying levels of abstraction; there’s absolutely no necessity to have elegant models!  The table below the decomposition shows various criteria I used to help me decide on the possible clusters. As you will see there’s variation in strategic classification; the partitioning – which may be key for deployment, some could be centralized others local; and the need for implementation independence and so on.

This analysis certainly helps me present some choices. But aside from the independence and scalability criteria and possibly standardization criteria, I feel I have not fully exhausted the analysis of the need for business agility. In the table below I develop this a little further. First I make an assessment of the potential requirement for future change in each function. I call this Agility Potential (AP) on a 1=Low and 5=High scale [1]. Not surprisingly Analysis and Skills are the capabilities that will probably be subject to considerable volatility. Second I look at the dependencies between the functions; note you have to read this as each row dependency upon a column. And low and behold, Skills and Analysis, and Analysis and Follow-up have high dependencies. This causes me to reconsider my initial cut of capability boundaries. I feel that Skills needs to be very close to Analysis as the investigatory function. And Follow-up should be similarly very close to Analysis. And what’s more these three functions score most highly on the AP scale. I feel Follow-up could easily be collapsed into Analysis, and a name change to Investigation would be perfect. I think a little more deeply about Skills. The degree to which the outcomes of Investigation need to be fed into Skills on a dynamic basis will vary depending on the type of business. If this was a safety critical business, I might recommend consolidating Skills and Investigation and renaming it Knowledge Management. But this really would depend on the business sector specific needs. 
To recap, what I have done here is developed a sharper understanding of the capabilities, and I have attributed them with governance criteria (in the first table) – I know what I must have delivered, and I am communicating some really important information to the delivery team, without constraining them at all on the implementation and delivery method. Also I now know the dependencies between the capabilities, and we can very quickly resolve the services that will be required and the inter project dependencies. And it didn’t take me very long at all.

More on Agile Business Modeling 

[1] I first outlined the idea of Agility Potential in the CBDI Journal April, 2010. Let me know if you would like a copy.

James Martin – A personal reflection

by
James Martin, technologist, methodologist, entrepreneur and philanthropist died Monday 24 June 2013 aged 79.

I first came across James Martin in the early 1970s. I went to a lecture he gave in London, and he captivated an audience of about 100 people for 2 hours on the topic of real time systems design.  Later on I attended his famous seminars in London and Johannesburg. It was extraordinary how he held huge audiences for multi day sessions, with minimal audience interaction as he drove through the thousands of slides, delivered on two overhead projectors. In those days he was the consummate technology seer and he filled a need in the days before industry analysts.

Yet he was so much more than just a showman. I bought his book on real time system design in 1971 and this was my bible. And down the years I relied on his books in data modeling, database design and  information engineering; they were detailed and useful to a practitioner.

I joined James Martin Associates (JMA) in 1986. I think I was employee number 30 and I had the privilege of working in what must have been one of the most extraordinary and innovative companies at that time. Even when I meet ex JMA colleagues today, we always recall how it was such a great place to be, where everyone was on the leading edge and contributing to the overall development of information engineering. We were taking his ideas and turning them into practical method and tools for many of the world’s largest companies and government agencies.

We didn’t see that much of James Martin. He would attend our annual JAM (sic) session, and if he was in town he might drop in, but that was rare. But he did get feedback. I was deeply involved at one stage, together with Richard Veryard and Mike Mills in developing ideas for Rapid Application Development (RAD) that we took to customer projects and of course appeared later in the James Martin book.

The core of his thinking was the idea that model driven systems were the future and at JMA and subsequently Texas Instruments Software (TI) we proved this by delivering the IEF based on James’ ideas, that became the leading mainframe and client server development tool in the early 1990s. Of course we knew even then that this tool was limited to a very narrow set of patterns, and history has taught us that there is a need for a much broader range of patterns, and varying levels of abstraction and intervention. And even as early as the mid-1990s we were working on componentization and service interfaces because we understood the monolithic architecture, however commercially successful for a short few years, was in reality a simplistic first attempt. Today the term CASE tool is widely disparaged. Yet I believe that James’ original vision will be realized, although the method of realization will be radically different, and by strange coincidence I blogged on this topic very recently. 

James Martin provided inspiration for technologists by identifying big ideas, but he went further by detailing the ideas in his books and teaching and “having the courage of his convictions” by investing in start-up businesses. Which of course made him very wealthy. I would be proud to say that in my work and in everything that CBDI has developed, we have been true to some of the core principles that we hammered out nearly thirty years ago including model driven (including meta model based) and structured with maximum automation. And these are equally applicable to today’s Agile, fast moving world. Of course we (Everware-CBDI) have added and evangelized the  principles of component and service oriented, but the original vision is intact. 
James Martin was an idealist. In several of his works he developed his thinking for a utopian society where technology and automation are used for the greater good in education, health and creating a better world. Sadly I myself came to see some of his works as out of touch with reality. He failed to see the shoddy reality of how politics and politicians are incapable of leveraging technology to deliver better models for society, how the giant Internet companies are creating worldwide networks based on old fashioned capitalistic principles while espousing such things as “do no evil” and practicing tax avoidance, and governments are increasingly using technology to track the every move of citizens without understanding how to govern the use of that data. 
I believe we will remember James Martin, but not necessarily for his big ideas like the early prediction of the Internet or his philanthropy. Rather we will come in time to reflect on his guidance that technology should be leveraged to improve society. Every time we destroy tens of thousands of jobs by introducing new technologies, we should be using the power of technology in education and resource mobilization to ensure that vast numbers of our young people do not remain out of work, or that older people can continue to contribute to society beyond conventional retirement age. 

James Martin – A personal reflection

by
James Martin, technologist, methodologist, entrepreneur and philanthropist died Monday 24 June 2013 aged 79.

I first came across James Martin in the early 1970s. I went to a lecture he gave in London, and he captivated an audience of about 100 people for 2 hours on the topic of real time systems design.  Later on I attended his famous seminars in London and Johannesburg. It was extraordinary how he held huge audiences for multi day sessions, with minimal audience interaction as he drove through the thousands of slides, delivered on two overhead projectors. In those days he was the consummate technology seer and he filled a need in the days before industry analysts.

Yet he was so much more than just a showman. I bought his book on real time system design in 1971 and this was my bible. And down the years I relied on his books in data modeling, database design and  information engineering; they were detailed and useful to a practitioner.

I joined James Martin Associates (JMA) in 1986. I think I was employee number 30 and I had the privilege of working in what must have been one of the most extraordinary and innovative companies at that time. Even when I meet ex JMA colleagues today, we always recall how it was such a great place to be, where everyone was on the leading edge and contributing to the overall development of information engineering. We were taking his ideas and turning them into practical method and tools for many of the world’s largest companies and government agencies.

We didn’t see that much of James Martin. He would attend our annual JAM (sic) session, and if he was in town he might drop in, but that was rare. But he did get feedback. I was deeply involved at one stage, together with Richard Veryard and Mike Mills in developing ideas for Rapid Application Development (RAD) that we took to customer projects and of course appeared later in the James Martin book.

The core of his thinking was the idea that model driven systems were the future and at JMA and subsequently Texas Instruments Software (TI) we proved this by delivering the IEF based on James’ ideas, that became the leading mainframe and client server development tool in the early 1990s. Of course we knew even then that this tool was limited to a very narrow set of patterns, and history has taught us that there is a need for a much broader range of patterns, and varying levels of abstraction and intervention. And even as early as the mid-1990s we were working on componentization and service interfaces because we understood the monolithic architecture, however commercially successful for a short few years, was in reality a simplistic first attempt. Today the term CASE tool is widely disparaged. Yet I believe that James’ original vision will be realized, although the method of realization will be radically different, and by strange coincidence I blogged on this topic very recently. 

James Martin provided inspiration for technologists by identifying big ideas, but he went further by detailing the ideas in his books and teaching and “having the courage of his convictions” by investing in start-up businesses. Which of course made him very wealthy. I would be proud to say that in my work and in everything that CBDI has developed, we have been true to some of the core principles that we hammered out nearly thirty years ago including model driven (including meta model based) and structured with maximum automation. And these are equally applicable to today’s Agile, fast moving world. Of course we (Everware-CBDI) have added and evangelized the  principles of component and service oriented, but the original vision is intact. 
James Martin was an idealist. In several of his works he developed his thinking for a utopian society where technology and automation are used for the greater good in education, health and creating a better world. Sadly I myself came to see some of his works as out of touch with reality. He failed to see the shoddy reality of how politics and politicians are incapable of leveraging technology to deliver better models for society, how the giant Internet companies are creating worldwide networks based on old fashioned capitalistic principles while espousing such things as “do no evil” and practicing tax avoidance, and governments are increasingly using technology to track the every move of citizens without understanding how to govern the use of that data. 
I believe we will remember James Martin, but not necessarily for his big ideas like the early prediction of the Internet or his philanthropy. Rather we will come in time to reflect on his guidance that technology should be leveraged to improve society. Every time we destroy tens of thousands of jobs by introducing new technologies, we should be using the power of technology in education and resource mobilization to ensure that vast numbers of our young people do not remain out of work, or that older people can continue to contribute to society beyond conventional retirement age. 

Smart Agile Delivery

by

I was interested to read the recent McKinsey report on disruptive technologies. McKinsey identifies twelve potentially economically disruptive technologies including the Mobile Internet, Automation of Knowledge work, Internet of Things, Advanced Robotics, Next generation genomics and so on. The report also calls out general purpose technologies as ones that propel steep growth trajectories (think Steam or Internet) – that can be applied across economies and leveraged in many more specific disruptive technologies. Not surprisingly they don’t include software development in either list. The closest they come is with the automation of knowledge work, but this is restricted to artificial intelligence, machine learning and natural interfaces like voice recognition that automate many knowledge worker tasks that have long been regarded as impossible or impracticable for machines to perform.

Is this omission something we should be concerned about I wonder? While software development “practices” have been developing very rapidly with the adoption of Agile methods, it is a reasonable conclusion that software development “technologies ” are not undergoing dramatic changes that might qualify as disruptive. Yes there’s lots going on; in fact there’s a profusion of new languages, frameworks and databases, many open source initiatives, that are progressively specializing development technology. In addition there are significant advances in life cycle management and test technologies. But there isn’t any indication that these new technologies will have high economic impact in terms of dramatic improvement in productivity or quality. Or have a significant impact on the vast economic problem inherent in the worlds legacy systems. Rather there’s a huge proliferation of development diversity and some might say complexity.

Don’t get me wrong, I am not looking for a problem to solve. It’s clear that while smaller Agile projects are fine for tightly targeted problems, most organizations have struggled to scale Agile to larger projects and or enterprise class projects. The increase in dependencies and complexities become overwhelming and the probability of failure increases proportionately.

What’s needed, by larger projects is not process automation, but automation of the deliverable that allows the project to manage the dependencies at the model AND deliverable level. This raises the level of abstraction and can deliver dramatic productivity and quality improvements. As it happens there is a technology that can do this, but strangely it seems to be something that many people have already consigned to the trash heap of “been there done that”. I’m talking about Model Driven Development (MDD). There are many reasons why MDD has not succeeded in gaining widespread acceptance. It is actually extremely complex and requires considerable investment to establish. And in fairness it has been promoted primarily as a deliverable transformation and code generation tool. And many people will say, Oh NO!!! That’s just reinventing Case Tools all over again and we don’t want to go there.

But before we consign this technology to the trashcan of yesterday’s technologies, we need to take a hard look at what you can do if:
A. you have leaf node detail models in the asset repository that are tightly bound to execution deliverables.
B. you use best practice modern architecture with all functionality delivered as service bearing capabilities that minimize dependencies.
C. you can automate to a significant extent the population of the repository with harvested knowledge about legacy applications at that same leaf node level of detail.
D. you can run large scale, full life projects with full iteration of business, architecture, design and development models. (note here this doesn’t mean fully integrated and transformed models, we have gotten a lot cleverer over the years.)
In an Agile context this allows you to iterate functionality at extremely low cost, both in delivery and evolution life cycle stages. In fact experience shows it transforms the development project into an evolutionary approach in which you can really architect and build what you know and evolve to the optimal solution.

Model driven as a concept has been around a long time. Most developers (tell me they) don’t like model driven because it won’t handle complexities; because it diminishes developers’ jobs to be more mundane; that it produces poor code and so on and so forth. But the McKinsey report speaks to the inexorable progress of technology and the inevitability that as technology changes peoples jobs change or disappear. You are either on the train or under it.

Right now Agile MDD is probably only justifiable for the very large, complex projects. But as the case studies start showing higher success rates, with dramatic increases in productivity and quality, and the level of up-front investment is reduced as the capability is productized, we can expect to see the MDD project footprint to expand dramatically. Again the McKinsey report is incredibly bullish on the economic outlook for technology, and information technology in particular as the key general purpose enabling technology, and it’s clear that Agile processes alone are inadequate to support the ever increasing demand.

Being disciplined is for school kids; it’s time we got smart about how we deliver complex services and systems at scale.

McKinsey & Company: Disruptive technologies: Advances that will transform life, business, and the global economy. “Not every emerging technology will alter the business or social landscape—but some truly do have the potential to disrupt the status quo, alter the way people live and work, and rearrange value pools.”

Smart Agile Delivery

by

I was interested to read the recent McKinsey report on disruptive technologies. McKinsey identifies twelve potentially economically disruptive technologies including the Mobile Internet, Automation of Knowledge work, Internet of Things, Advanced Robotics, Next generation genomics and so on. The report also calls out general purpose technologies as ones that propel steep growth trajectories (think Steam or Internet) – that can be applied across economies and leveraged in many more specific disruptive technologies. Not surprisingly they don’t include software development in either list. The closest they come is with the automation of knowledge work, but this is restricted to artificial intelligence, machine learning and natural interfaces like voice recognition that automate many knowledge worker tasks that have long been regarded as impossible or impracticable for machines to perform.

Is this omission something we should be concerned about I wonder? While software development “practices” have been developing very rapidly with the adoption of Agile methods, it is a reasonable conclusion that software development “technologies ” are not undergoing dramatic changes that might qualify as disruptive. Yes there’s lots going on; in fact there’s a profusion of new languages, frameworks and databases, many open source initiatives, that are progressively specializing development technology. In addition there are significant advances in life cycle management and test technologies. But there isn’t any indication that these new technologies will have high economic impact in terms of dramatic improvement in productivity or quality. Or have a significant impact on the vast economic problem inherent in the worlds legacy systems. Rather there’s a huge proliferation of development diversity and some might say complexity.

Don’t get me wrong, I am not looking for a problem to solve. It’s clear that while smaller Agile projects are fine for tightly targeted problems, most organizations have struggled to scale Agile to larger projects and or enterprise class projects. The increase in dependencies and complexities become overwhelming and the probability of failure increases proportionately.

What’s needed, by larger projects is not process automation, but automation of the deliverable that allows the project to manage the dependencies at the model AND deliverable level. This raises the level of abstraction and can deliver dramatic productivity and quality improvements. As it happens there is a technology that can do this, but strangely it seems to be something that many people have already consigned to the trash heap of “been there done that”. I’m talking about Model Driven Development (MDD). There are many reasons why MDD has not succeeded in gaining widespread acceptance. It is actually extremely complex and requires considerable investment to establish. And in fairness it has been promoted primarily as a deliverable transformation and code generation tool. And many people will say, Oh NO!!! That’s just reinventing Case Tools all over again and we don’t want to go there.

But before we consign this technology to the trashcan of yesterday’s technologies, we need to take a hard look at what you can do if:
A. you have leaf node detail models in the asset repository that are tightly bound to execution deliverables.
B. you use best practice modern architecture with all functionality delivered as service bearing capabilities that minimize dependencies.
C. you can automate to a significant extent the population of the repository with harvested knowledge about legacy applications at that same leaf node level of detail.
D. you can run large scale, full life projects with full iteration of business, architecture, design and development models. (note here this doesn’t mean fully integrated and transformed models, we have gotten a lot cleverer over the years.)
In an Agile context this allows you to iterate functionality at extremely low cost, both in delivery and evolution life cycle stages. In fact experience shows it transforms the development project into an evolutionary approach in which you can really architect and build what you know and evolve to the optimal solution.

Model driven as a concept has been around a long time. Most developers (tell me they) don’t like model driven because it won’t handle complexities; because it diminishes developers’ jobs to be more mundane; that it produces poor code and so on and so forth. But the McKinsey report speaks to the inexorable progress of technology and the inevitability that as technology changes peoples jobs change or disappear. You are either on the train or under it.

Right now Agile MDD is probably only justifiable for the very large, complex projects. But as the case studies start showing higher success rates, with dramatic increases in productivity and quality, and the level of up-front investment is reduced as the capability is productized, we can expect to see the MDD project footprint to expand dramatically. Again the McKinsey report is incredibly bullish on the economic outlook for technology, and information technology in particular as the key general purpose enabling technology, and it’s clear that Agile processes alone are inadequate to support the ever increasing demand.

Being disciplined is for school kids; it’s time we got smart about how we deliver complex services and systems at scale.

McKinsey & Company: Disruptive technologies: Advances that will transform life, business, and the global economy. “Not every emerging technology will alter the business or social landscape—but some truly do have the potential to disrupt the status quo, alter the way people live and work, and rearrange value pools.”

Back to the Future with The Theory of Constraints

by

In so many situations today I find business people are much more savvy with IT than they used to be only 10 years ago. And while this is a fantastic advance, the result is they are MUCH more likely to dictate the solution right from the outset. I marvel at how very senior business executives are now so conversant with the specifics of application architecture, particular packages they wish to use and Cloud deployment architecture. But of course this level of direction frequently facilitates rapid action, but without full and thorough understanding of the business issues.

We know that business people should be focused on the inherent business architecture, surfacing the opportunities for common concepts and business services, business platforms, product lines and channels; identifying where standardization and differentiation is appropriate, and where partitions are relevant, all in context with the business and market  model. Get this level of architecture right and you have the chance of delivering an agile business. Get it wrong and you are in instant legacy territory!

With this interesting problem in mind I browsed my bookshelves and came across Eli Goldratt and the Theory of Constraints (ToC). There was an AH HA moment! I first came across Goldratt nearly 30 years ago; I went to one of his lectures in London and have read many of his books, but I haven’t used the ideas for a good while.

The starting point is to develop a Current Reality Tree, initially a list and then a dependency hierarchy of Undesirable Effects (UDEs) (see redacted example below). This is used to determine the root problem and then to develop a Future Reality Tree with Desired Effects (DEs). And the techniques naturally guide the user to focus on the HOW, and separate out the WHAT. In the process I insert an Ishikawa (Fishbone) diagram between the CRT and the FRT. It’s really useful in separating Domain based issues into (people, process, technology . . . ) clusters and also teasing out the root problem.

I would be interested to hear from others whether they are using ToC, or indeed if there are other techniques that do a similar job.

.  

Shared Vocabulary for Business Innovation and Modernization

by

Do you remember when computers were hard to use? In fact it’s just nine years since a GM press release asserted that if they developed technology like Microsoft, we would all be driving cars that for no reason at all, would crash twice a day, shut down and refuse to restart. Since then Apple has showed Microsoft the way, and we all use smart phones, tablets and PCs that are genuinely easy to use and remarkably resilient.

Because of this great leap forward in personal device usability the smart phone user on the proverbial Clapham Omnibus might reasonably expect that enterprise systems should be similarly easy to use and resilient. Unless of course she was a customer of the Royal Bank of Scotland (RBS), in which case she will have painful memories of last year’s high profile failure caused by the core banking system crash which corrupted tens of millions of accounts.

Once upon a time banks in general were regarded as leaders in the use of information technology. Yet last year several high profile systems failures signalled that banking systems, far from being leading edge, are in rapid decline. Banks aren’t the only culprits. Along with the banks, insurance companies, retailers and others are starting to offer their customers smart phone apps, notwithstanding that behind the scenes their enterprise systems are frequently held together with sticky tape and sealing wax.

The reason many enterprise systems are in such a poor state is commonly because there are three parties involved in managing the enterprise systems that have widely divergent goals and objectives. The line-of-business manager typically views the systems as support to the business process and a cost to be managed. The IT Architect views the enterprise systems as a set of capabilities that must be progressively modernized to support business innovation. The IT Project Manager is focused on delivering projects to time and cost.

These views are of course diametrically opposed. And under cost and time pressure the Architect is frequently the lower ranking player. In consequence the immediate needs of the business overrule longer term objectives of modernization, reduced complexity, flexibility and even cost of ownership.

The real issue is that the three parties do not have a shared view of the business problem. The line-of-business manager’s business process view does not correlate at all to the delivery project. The Architect should be the evangelist for business innovation and modernization but he or she is too easily squeezed in the cost and time discussion. And the Project Manager typically does not share the detailed technical project view with the line of business manager, and argues for a solution specific architecture that reduces project risk. The result is the existing enterprise systems get more complex and slower to respond to change. And the IT industry has been doing exactly this for as long as anyone can remember!

It’s extraordinary, but with all our high tech knowledge and skills we don’t have a vocabulary to articulate the business problem in a way that allows effective communications between the participants. Many IT organizations have embraced services as a way to organize systems capabilities more effectively. These might be Web Services or APIs or referred to collectively as Service Oriented Architecture (SOA). But, even if these software services are architected to align with business perspective, they are always managed as a technical matter, defined and managed by the IT organization.

Yet line-of-business managers do understand services as a business concept; virtually every business product today has a service component to it. The global service provider industry has formed around this idea, and in the UK today service industries account for 77 per cent of the economy. So while IT and business share the common underlying concept, at the practical level there is no meeting of minds.

In order to create a better bridge between business and IT we need to work with both the “how” and the “what” the business is, and we can do this by complementing business processes with business services. Business services are a very natural way to talk about “what” the business does today and tomorrow, while business processes focus on the “how”. Because you don’t reinvent an industry by just analyzing business processes, you also need to evolve and innovate with improved and new business services.

A good example of a service oriented business is Amazon.com Inc. They are well known as a service provider because they have constructed the Amazon enterprise as a set ofbusiness services which are offered to various external parties – enabling suppliers to sell second hand books or electronic goods on the Amazon platform; or providing data storage and Cloud computing services to other enterprises. The Amazon business services combine the compute and the business service integrating the commercial contracts, business processes, people, physical assets as well as the service interfaces that enable computer to computer or computer to device communications.
 

Using a common business and IT concept permits sensible analysis of whether a service is just a unit of cost, or what the strategic value is now and in the future, and what it adds to the business value chain. Given so many line-of-business managers are thoroughly familiar with the very high technology in their smart phones and other devices, it really is time for IT to treat the business as a mature partner and for the line-of-business manager to take real responsibility for the business service as a whole product.

Increasingly we see a convergence of IT and business organizations. The business service concept is an essential piece of vocabulary to focus on a business innovation and get everyone singing off the same hymn sheet to potentially huge advantage of the business. Just look at the Amazon example!

———————–
We will be running a workshop that explores these ideas in London in April in conjunction with the IASA UK Summit. If you can’t make the London event, (for geographic of schedule reasons) talk to me about how we can accommodate.

Framework for Service Oriented Ecosystem

by

I note interesting debates about the need for a next generation EA framework. However I am disappointed by the less than radical nature of debate that, at least I, have observed. I submit a good place to start is with the fundamental nature of business and how it is evolving and to consider what the enterprise of the future looks like. There are many indicators that we are entering a new phase of IT exploitation that will represent a real paradigm shift. Paul Krugman suggests IT is at last becoming significant, enabling a technology revolution to rival previous technology revolutions. Krugman cites driverless cars as an example of the technology moving into the physical world that has the potential to power growth. I will also instance a wave of disruptive technology delivering high bandwith always on connectivity for billions of workers and consumers, mobility, BYOD, social networks, big data and next generation analytics, robotics and Cloud. And the widespread adoption of Agile methods is also highly significant.

This stream of disruptive technologies is having a major impact on enterprises and the way they work. A Gartner report released this week predicts that by 2017, 25 per cent of enterprises will have enterprise app stores where workers can browse and download apps to their computers and mobile devices. I think that prediction will turn out to be conservative. It’s striking that many if not most enterprises are already being run as a continuous stream of initiatives, driven by business competitive pressures which in many cases are triggered by the disruptive technologies mentioned. And strategic innovation is typically being delivered in Agile projects which will increasingly combine business and IT expertise in defining the architecture and requirements.

But this is still a conventional view, doing what we do today, faster, better cheaper. What’s more importantly is to look at how the technology will enable profound change that spans existing enterprise boundaries. Consider Krugman’s Driverless Cars. This revolution is set to change the shape of personal transport in the relatively near term and will involve capabilities such as telematics, insurance, road tolling, mapping, navigation, vehicle recognition, which span car manufacturers, the financial industry, local or state government, emergency services and so on. This is a new ecosystem in the making which will require near real time, collaborative services spanning multiple business sectors.

Is this driverless cars ecosystem an isolated revolution? I don’t think so; consider smart shopping which is already taking off like a rocket with showrooming, or the extension of mobile devices to sector specific applications such as drug testing, health monitoring. I could go on. The future is going to look like many, many ecosystems, rapidly evolving usually not in the control of a single enterprise.

So returning to the question about a next generation EA framework, we might put a few stakes in the ground:
1. The pace of change is increasing so fast that conventional approaches (frameworks) for modelling will be left behind.
2. Ecosystem architecture should be primarily about identifying how an enterprise leverages an ecosystem by providing capabilities and their business services that collaborate and evolve along with the wider landscape.
3. The future is “business service” oriented. The application is dead. Business Service Implementation would be a better term.
4. The Capability and Service architecture will be a strategic business asset.
5. Capabilities as highly independent units of business function will be the way the business is organized.
6. The primary task of enterprise architects will be to develop the Capability and Service architectures as part of the business design.
7. Enterprise architects will probably be renamed Capability and Business Service Architects and report to the CMO.
8. The framework scope must span the entire Agile life cycle. Architecture is no longer a top down precursor to delivery, it must be an evolving set of deliverables and inherently implementable. The framework therefore needs to support concurrent development of business requirements, ecosystem, service and solution architecture, modernization, plus service and solution specification and delivery.

What’s needed is a new framework that recognizes the enterprise itself is a series of overlapping business ecosystems that are in turn part of a series of ecosystems that transcend the scope of the enterprise itself. A new framework should be focused on the capabilities and their inter-connections and manage the development of the business ecosystem(s) to the advantage of the enterprise.

While Capability is a widely used concept, notwithstanding some significant divergence of definition, the missing link is the realization of the Capability. In our work we use the Business Service concept – which delivers the capability in a context free manner. It’s extraordinary that our business vocabulary doesn’t include the formal Business Service concept in the same way that we are able to talk unequivocally about Business Process and know we will be understood.

The core model underlying the framework for future business needs to be service oriented, but it’s essential that the model is fully integrated with business concerns, and enables an implementable architecture in a way that current EA models manifestly do not. The new framework is also highly supporting of Agile methods in the entire life cycle being lightweight, twin track, narrow scope based on the Capability and Business Service, and contract based dependencies.
We will be running a workshop that explores these ideas in London in April in conjunction with the IASA UK Summit. If you can’t make the London event, (for geographic of schedule reasons) talk to me about how we can accommodate.

Paul Krugman: We Are On The Brink Of A Technology Revolution That WillTransform Our Economy


Understanding Business Services 2

by

In December 2006 I blogged on the topic of Explaining SOA to the Business Audience. It started out “I note resurgent interest in LegoTM blocks as a metaphor for explaining to the business audience the value of SOA. My advice is don’t treat the business audience as dummies!” The blog goes on to explain business services using the Laundry metaphor, and how business people get the concept because they understand “services”.

However, while my explanation was and remains perfectly OK, I will be the first to admit that I have moved on. The basic service model works perfectly, but in today’s fast moving, business innovating world, we need new vocabulary that is even more compelling, that goes beyond SOA and transactional efficiency.  

In their book Competing for the Future [1], Gary Hamel and C. K. Prahalad advise that traditional business responses to market and competitive pressure such as reengineering, downsizing and outsourcing are inadequate and insufficient. The outcome of this activity is typically just keeping one step ahead of declining margins and profits of yesterday’s business. Instead senior management need to get off the treadmill of restructuring and reengineering and instead reinvent their industry, imagining and creating their future.

What I didn’t say in 2006 was that you don’t reinvent an industry by analyzing business processes! The business process is “how” the enterprise works. Instead we need to be looking at “what” the business is – business services, the external, composite offering that enables core capabilities to be used in many different contexts. We need to elevate the concept of Business Service to the level of business offering and business product that externalizes the enterprise capability. I suggest simple definitions as follows:

Business Service: A service provided by an enterprise to its ecosystem of customers, suppliers or partners that provides one or more capabilities that facilitate a discrete business outcome according to a contract.  Example: Amazon EC2 
Business Service Operation: An execution of one or more capabilities provided by an enterprise to its ecosystem of customers, suppliers or partners according to a service contract. Example: Data load under Amazon EC2.

In Table below I have summarized some of the Hamel Prahalad strategies and shown how these are implemented as Business Services.

Hamel and Prahalad go on to pose the question, “Why did it take US automakers 40 years to decode the principles of lean manufacturing pioneered by Toyota?” Answer – because those principles challenged the core assumptions of US auto executives.

I suggest we need to establish a business centric perspective of Business Service that is as closely linked to business offering implementation as it is to the internal SOA. This will cause us to challenge some of our core principles and assumptions. It’s NOT about LegoTM, it’s about business services and business agility.
[1] Gary Hamel and C. K. Prahalad , Competing for the Future, published by Harvard Business School Publishing, Reprint 1996

Agile Architecture

by

The English language is well known for its subtlety. Sometimes it’s a delight, but on other occasions it can be very frustrating. If I use the term Gothic Architecture you will immediately understand I am describing a style of architecture that flourished in medieval times. And if like me you are interested in ecclesiastical architecture you will know that this style was used in many of the great cathedrals and churches across Europe, which were distinctive because of key architectural patterns that enabled great increases in height and internal light of the buildings without increasing the size of supporting pillars.
Now if I use the term Agile Architecture, what am I referring to? In today’s Agile world I would hazard a guess that most readers will think I am referring to the architecture techniques and tasks undertaken in the context of an Agile software development project, not the collection of patterns and practices that enable agile business systems. That is, an architecture that enables agility.

This potential for miscommunication is a core issue for enterprises. There is ample evidence that Agile Architecture is a primary contributor to business agility, yet we do not have a well understood architecture management system that integrates with Agile methods.
 

Let’s use an example readers may be familiar with. Amazon CEO Jeff Bezos famously[1]issued an edict that laid down some key architecture principles to Amazon development teams that I will summarize as:

·       All teams will henceforth expose their data and functionality through service interfaces.

·       Teams must communicate with each other through these interfaces. There will be no other form of interprocess communication allowed.

·       It doesn’t matter what technology they use.

·       All service interfaces, without exception, must be designed from the ground up to be externalizable.

·       No exceptions.

What Bezos did here was to lay down key business and technology architecture principles that you might reasonably conclude were central to the extraordinary level of business agility that we have seen demonstrated by Amazon.com, Inc. That widely circulated edict contained the foundations of the Amazon reference architecture.  

In the October 2004 CBDI Journal[2] we commented, “Two of the most successful and enduring dotcom start-ups, Amazon and eBay, now expose their core applications as Web Services. In doing so they have created a new class of platform that could have a profound impact on end-user organizations and IT vendors alike.”

And so the reference architecture became the enabler of growth and agility for the Amazon business, not we understand[3] as a grand plan, but through natural technological evolution. The services formed the platform that allowed the extraordinary expansion of the Amazon business that I would be certain not even Jeff Bezos imagined, back then in 2004. That is real business agility, and it was delivered by smart architecture backed up by clear policies and realized by agile processes.

Although Amazon has clearly evolved in pursuit of solutions to specific business opportunities and challenges, it’s also clear they have established a de facto architecture and architecture management system that guides the work of the many product delivery teams and ensures consistency of approach where it’s required. Let’s consider how an enterprise might establish a similar agile architecture management system.

A reference architecture articulates primary principles that are typically central to an entire enterprise. Principles should be focused on establishing the product and solution independent environment in which agility can be delivered and maintained, so they would be stable over time. We might refer to reference architecture as a Level 1 architecture perspective (L1) that exists purely as a set of models and guidelines.

Larger enterprises should explore the business value potential of platform based architecture as a mechanism to deliver cross enterprise consistency of core reference architecture behaviors and to enable closer integration with the wider ecosystem including customers, suppliers, end consumers etc. This is an extended management services platform which encapsulates the technology infrastructure and enables rapid delivery of business services.
The platform architecture defines common services that manage business delivery including security, life cycle management, change management, release management and operations, as well as catalogs, eCommerce, B2B, regulatory control and risk management, standardizing these key capabilities and reducing the footprint of business domain services. The platform will also manage important behaviors that deliver on specific business goals such as scalability and availability. For example, Amazon services are usually very fine grained, specifically to reduce the scope of each service in order to facilitate narrow focus SLAs and maximize scalability by reducing individual service complexity. We might refer to platform architecture as a Level 2 architecture perspective, engineered to be relatively stable in support of  large numbers of business services and consumers, but also engineered to evolve and respond rapidly to business and technology change. Not all enterprises will see business value in making their platform and business services available to their ecosystem, but some will.
Enterprises clearly vary considerably in their make up in terms of geographic and organizational, product and process standardization and differentiation, but typically there will be considerable potential for an inventory of shared assets that leverage agile architecture to support business agility. The assets may include:

·       Common services, frameworks and components that are designed to deliver common behaviors to all parts of the enterprise. For example core services that establish genuinely enterprise wide services such as Customer, Ticket, eCommerce etc; services that deliver business value by standardizing common business services and processes.

·       Configurable services, frameworks and components that are designed to provide common behaviors but are engineered to be customizable in local situations to accommodate many aspects of localization ranging from the simple – taxation, geography etc, to the complex – variant ordering patterns, variations in event and process sequence dictated by local de facto business practices. Configurable services may provide business value simply by providing reusable components, or they may establish a common core of business process and information that establishes common reporting and regulatory control in a local context, or both. Configurable services may also be an important time to market strategy for service providers who customize their services for each client or customer group.

·       Information architecture and services. Establishing a coherent approach to information is commonly a major issue for large enterprises and this architecture level defines an integrated approach for structured and unstructured (big) data, transactional and reference, enterprise reporting and regulatory control and so on.

Common and Configurable assets together with the Information Architecture might form a Level 3 architecture perspective and be widely applicable across a large, distributed enterprise.  

We then have two further levels which are closely related, Family Architecture and Product Line Architecture. Whilst many architects chose to view Family and Product Line as synonyms, I recommend that they are kept separate. A Family architecture is a domain framework that is much more specialized that L3 assets that would be applicable on a broader basis. The Family architecture establishes core business (domain) services and possibly other artifacts specific to the domain, where the domain is likely to be a subject area or a cluster of major types. For example Customer, Supply Chain, Manufacturing, Risk etc. Families are also commonly acquired products.
In contrast Product Line architecture is what it says – it’s the architecture for a product offering. The product is an offering that has direct relationship to end customer revenue and usually continuity of purpose over multiple releases. Although from a narrow technical perspective the Product and Family architectures might be similar, the way a product is managed must mirror the business product life cycle. Family architectures may therefore be engineered for stability, whereas, depending on the industry sector, product line architectures may be engineered for maximum agility and minimum response time.  

Finally we have the Solution architecture level, the architecture specific to solution project delivery, where the focus is on feature architecture and integrating solution architecture with the Level 1 to 5 architecture perspectives. It’s important to note that where product line architecture is used, then this may subsume the Solution architecture.
These six architecture levels provide us with a nomenclature for agile architecture that will be central to managing agility into the delivered product/solution. The architecture perspective guides the structure of programs and projects and the incorporation of architecture and reuse goals into delivery charters. The architecture also provides traceability and governance over realization of core architecture principles.
The question of how Agile Architecture integrates with Agile delivery is likely to prove contentious because architecture introduces a form of direction that contradicts Agile concepts. Yet the lessons from Amazon are insightful. The most senior business management need to be fully engaged and actively leading the development of architectural direction. Further in large enterprises customer project demand needs to be managed and aligned with business strategy and architectural direction.

There’s no reason why these Demand and Definition processes shouldn’t adopt Agile concepts, notably cross functional teams, time boxes and backlogs. The outcomes should be excellent visibility and traceability of key strategies and policies that provide real clarity of purpose for projects, that will increase the probability of success. In a typical large enterprise use of existing (or well understood) organizational concepts, adjusted to use aspects of Agile methods as discussed, will meet less organizational resistance. For example:  

1.     Architecture Review Board (ARB) or equivalent, a cross functional team (senior representatives of business, product management, architecture and delivery), that provide direction and funding to all architecture development.

2.     Design Authority (DA), also a cross functional team (domain specific expert level representatives of business, product management, architecture and delivery), that transform raw customer demand stream into project charters and manage the portfolio view. It is the DA that takes responsibility for aggregating and decomposing customer and strategic demand, chartering Common, Product Line and Family architecture, typically as integral elements of delivery projects, which can demonstrate business value.

3.     Investigatory architecture projects – short duration projects that validate assumptions prior to chartering composite architecture/delivery projects. Sometimes carried out as part of a Definition Phase activity concurrent with outline requirements and knowledge discovery. Using patterns as a mechanism to increase consistency of architecture decisions and communicate them to delivery projects at sensible level of detail that is useful to delivery teams.  Recommend includes delivery team members as appropriate.

Note this is a recursive model, and the process may executed at enterprise and program level.

You may ask where Enterprise Architecture is in this. The answer is that enterprise architecture is a role and responsibility that must coordinate and govern all levels of architecture. Enterprise Architects are most likely to be assigned to a specific architecture perspective level. The notion of, “one architecture to rule them all” really doesn’t exist.

  

Each enterprise should develop its own architecture management approach, and integrate this into an end to end architecture, delivery and governance process. The term Agile Architecture should be used to describe and deliver architecture that facilitates the agile business by compliance with reference, platform and other architectures that facilitate evolution, customization and plug and play. Faster cycle time and quality outcomes are then a function of both the reusable patterns and parts available for assembly and the Agile delivery process.  

In medieval times the builders of the Gothic cathedrals didn’t start their designs from scratch. But equally they didn’t have finely detailed (ivory tower) plans – the technology didn’t exist to support that. Master builders moved from city to city bringing their proven architecture in their heads, often together with experienced craftsmen, to new projects. Craftsmen and master builders together tried out new designs and gradually evolved core patterns such as the flying buttress, which became standard components in cathedrals across Europe. Sometimes the great buildings fell down during construction and the builders had to adapt the architecture and try again. They were truly early adopters of Agile methods as they combined architecture and build in what clearly was from time to time an empirical delivery approach, but they also had their equivalent of a reference architecture and patterns that enabled systematic reuse of proven designs. Of course their delivery cycle time was a little longer than today’s Agile project!
Talk to Everware-CBDIabout the Agile Enterprise Workshop. This is currently available as an in-house, intensive workshop. Public scheduled classes will hopefully follow next year.

[1] Amazon and eBay Web Services, The New Enterprise Applications? By Lawrence Wilkes, CBDI Journal October 2004

[2] Inadvertently published by Steve Yegge, 2011, in a comparison of Google and Amazon practices. http://upalc.com/google-amazon.php

[3] Werner Vogels, 2006, SOA creates order out of chaos @ Amazon, Rich Seeley, Search SOA