Aggregated enterprise architecture wisdom
Change is intrinsic to the IT industry and its customers. And most of us, whether in business or government, are deeply focused on enabling business agility, whether through technology change, product innovation, software architecture, agile practices,…
A few years ago I commented [in Beware the New Silos, ref 1 below] that in a complicated world we cope by specialization – and across the industry in general and in individual enterprises specifically we have created silos of our primary disciplines. The widely used frameworks and methods illustrate that common practice of discipline centricity. We don’t have to look too far for examples such as Enterprise Architecture (TOGAF), Governance (Cobit), SOA (separately by OASIS, OMG, Open Group), Agile methods (many and various), Business Process Management (BPM) and IT Service Management (ITIL). All of these disciplines, whether de jure or de facto standards, are all very narrowly focused with minimal treatment of how the disciplines might work together.
A good example is how Agile methods are tightly focused on application development and architecture is assumed to be an integral part of the Agile delivery project. Yet the reality in all enterprises is that significant aspects of architecture must be consistent at the domain or enterprise level. Another good example is how the three standards bodies OASIS, OMG and the Open Group were so divergent in their treatment of SOA, they commissioned a report [ref 2 below] to explain how these inherently duplicating standards and specifications relate to each other.
The level of adoption by enterprises or service providers of all these and similar practice frameworks and standards is of course highly variable. However it must be said that the very existence of the discipline based materials will frequently have some considerable influence on how enterprises organize themselves.
The thinking IT or business professional might also like to question just how up to date these frameworks are, and how they support today’s business goals, which for most of us have changed dramatically over the past few years. We might also speculate whether the education and certification ecosystems that feed off some discipline based frameworks may discourage rapid evolution. A good example is how TOGAF maintains the core architecture style as application centric and treats SOA as a style extension. This is really quite extraordinary because by now everyone knows and many accept the digital business is going to be inherently service oriented. In practice of course the TOGAF specifications are so extensive that making fundamental changes may be very difficult, but it demonstrates neatly how legacy capabilities become “part of the furniture”, not just in frameworks but also in delivered systems and services.
In effect what we did was to establish a service and solution delivery value chain that commences with the raw customer demand and finishes with the delivery and integration of some useful business capability, but crucially with a much improved balance between immediate solution needs and longer term strategic goals. And it’s this balance that many enterprises find extremely difficult to achieve.
The core problem is that disciplines are vertically integrated; set up to optimize the discipline at varying levels of abstraction. In contrast the value chain approach optimizes across disciplines in pursuit of overall value chain outcomes. And this is only achieved by value chain activities that encourage effective collaborations between multiple capabilities and disciplines.
In the beginning we (Everware-CBDI) had a framework – Service Architecture & Engineering (SAE). The name makes it clear this is a forward engineering approach, and whilst it was very strongly business driven, it would be fair to say that the business modeling components were less well worked than the architecture, design and delivery. And as described we have very naturally, as part of the process of supporting large enterprise initiatives, expanded the framework capabilities and embraced an inherently Agile approach.
The principles underlying the framework now include:
– business capability based modularity
– pervasive service architecture
– continuous modernization
– service evolution not (one time) delivery
– model driven architecture and development
– everything is inherently agile – iterative, evolving, and narrowly focused on specific business capability delivery.
So to answer my own question, we clearly need a new framework. But it’s a very different practice framework to the ones that we are are accustomed to.
In our natural evolutionary process we recognized that the original (SAE) framework was merely one component of a much broader body of knowledge and practices. The new framework is goal driven, not discipline driven and incorporates the entire value chain of capabilities. But the capabilities are not standalone, they are effectively services that are executed in a way that supports the overall business goals of the enterprise, domain or program. We refer to this as Service Oriented Application Modernization (SOAM).
Interestingly this is not a monolithic framework. It’s vital to treat the framework as a set of capabilities with defined services and dependencies. Some might say, “eating our own dog food”. In this way we don’t make the same mistake as legacy frameworks such as TOGAF, that are very difficult to keep current.
Finally what happens to the existing discipline based frameworks and standards? Of course they can be used in conjunction with the SOAM framework. But we do need to be careful not to just inherit monolithic capabilities. Increasingly we find it necessary to do this very selectively in order to use capabilities that fit and support the value chain. Perhaps in time the various disciplines will recognize the monolithic nature of their practices, and decompose and modernize into more goal oriented components. Meanwhile, in SOAM we have demonstrated that it is possible to reinvent the wheel.
Some SOAM Components:
Agile Service Factory
Agile Modernization
Conceptual (Agile) Business Modeling
SOA Reference Framework
Ref 1: Beware the New Silos
A few years ago I commented [in Beware the New Silos, ref 1 below] that in a complicated world we cope by specialization – and across the industry in general and in individual enterprises specifically we have created silos of our primary disciplines. The widely used frameworks and methods illustrate that common practice of discipline centricity. We don’t have to look too far for examples such as Enterprise Architecture (TOGAF), Governance (Cobit), SOA (separately by OASIS, OMG, Open Group), Agile methods (many and various), Business Process Management (BPM) and IT Service Management (ITIL). All of these disciplines, whether de jure or de facto standards, are all very narrowly focused with minimal treatment of how the disciplines might work together.
A good example is how Agile methods are tightly focused on application development and architecture is assumed to be an integral part of the Agile delivery project. Yet the reality in all enterprises is that significant aspects of architecture must be consistent at the domain or enterprise level. Another good example is how the three standards bodies OASIS, OMG and the Open Group were so divergent in their treatment of SOA, they commissioned a report [ref 2 below] to explain how these inherently duplicating standards and specifications relate to each other.
The level of adoption by enterprises or service providers of all these and similar practice frameworks and standards is of course highly variable. However it must be said that the very existence of the discipline based materials will frequently have some considerable influence on how enterprises organize themselves.
The thinking IT or business professional might also like to question just how up to date these frameworks are, and how they support today’s business goals, which for most of us have changed dramatically over the past few years. We might also speculate whether the education and certification ecosystems that feed off some discipline based frameworks may discourage rapid evolution. A good example is how TOGAF maintains the core architecture style as application centric and treats SOA as a style extension. This is really quite extraordinary because by now everyone knows and many accept the digital business is going to be inherently service oriented. In practice of course the TOGAF specifications are so extensive that making fundamental changes may be very difficult, but it demonstrates neatly how legacy capabilities become “part of the furniture”, not just in frameworks but also in delivered systems and services.
In effect what we did was to establish a service and solution delivery value chain that commences with the raw customer demand and finishes with the delivery and integration of some useful business capability, but crucially with a much improved balance between immediate solution needs and longer term strategic goals. And it’s this balance that many enterprises find extremely difficult to achieve.
The core problem is that disciplines are vertically integrated; set up to optimize the discipline at varying levels of abstraction. In contrast the value chain approach optimizes across disciplines in pursuit of overall value chain outcomes. And this is only achieved by value chain activities that encourage effective collaborations between multiple capabilities and disciplines.
In the beginning we (Everware-CBDI) had a framework – Service Architecture & Engineering (SAE). The name makes it clear this is a forward engineering approach, and whilst it was very strongly business driven, it would be fair to say that the business modeling components were less well worked than the architecture, design and delivery. And as described we have very naturally, as part of the process of supporting large enterprise initiatives, expanded the framework capabilities and embraced an inherently Agile approach.
The principles underlying the framework now include:
– business capability based modularity
– pervasive service architecture
– continuous modernization
– service evolution not (one time) delivery
– model driven architecture and development
– everything is inherently agile – iterative, evolving, and narrowly focused on specific business capability delivery.
So to answer my own question, we clearly need a new framework. But it’s a very different practice framework to the ones that we are are accustomed to.
In our natural evolutionary process we recognized that the original (SAE) framework was merely one component of a much broader body of knowledge and practices. The new framework is goal driven, not discipline driven and incorporates the entire value chain of capabilities. But the capabilities are not standalone, they are effectively services that are executed in a way that supports the overall business goals of the enterprise, domain or program. We refer to this as Service Oriented Application Modernization (SOAM).
Interestingly this is not a monolithic framework. It’s vital to treat the framework as a set of capabilities with defined services and dependencies. Some might say, “eating our own dog food”. In this way we don’t make the same mistake as legacy frameworks such as TOGAF, that are very difficult to keep current.
Finally what happens to the existing discipline based frameworks and standards? Of course they can be used in conjunction with the SOAM framework. But we do need to be careful not to just inherit monolithic capabilities. Increasingly we find it necessary to do this very selectively in order to use capabilities that fit and support the value chain. Perhaps in time the various disciplines will recognize the monolithic nature of their practices, and decompose and modernize into more goal oriented components. Meanwhile, in SOAM we have demonstrated that it is possible to reinvent the wheel.
Some SOAM Components:
Agile Service Factory
Agile Modernization
Conceptual (Agile) Business Modeling
SOA Reference Framework
Ref 1: Beware the New Silos
I note healthy discussion around whether Agile is Dead [ref 1]. And while I may sympathize (sic) with many of the comments, particularly the commercial trivialization of education, the core issue must surely be the difficulty of adopting de facto Agile practices to support real world enterprise programs and projects. My experience is most of the advice and guidance out there is predicated on scaling the de facto Agile development methods. And this isn’t the best place to start.
An exception is Dean Leffingwell’s SAFe, [ref 2] which does introduce the idea of portfolio, program and project perspectives and intentional architecture. I recommend this framework as an intelligent set of practices, but for me it doesn’t go far enough because it is still primarily about development practices. This is the core problem – that Agile is development specific and practices only. In the enterprise, Agile development needs to be an integral part of a bigger ecosystem spanning business design, architecture, requirements, modernization and operational transformation practices plus architecture, delivery and modernization disciplines.
I am indebted to Dave Thomas whose recent blog, [ref 1] includes a worthy successor to the Agile Manifesto, reducing the original, development specific values and principles to a minimalist, more generic set. He says; “here is how to do something in an agile fashion”
– Find out where you are
– Take a small step towards your goal
– Adjust your understanding based on what you learned
– Repeat
And this is more useful in the enterprise context because it is relevant to a broader set of activities than purely software development.
The diagram below is an outline maturity model template for Agile in the enterprise. It suggests there are four key views that need to be part of the transformation. In addition to agile practices we need to be equally focused on what elements of agile architecture are required for an enterprise. What the agile delivery framework is and how the existing application portfolio will be modernized to progressively eliminate the duplication and complexity present in every enterprise on the planet.
People & Process. Much of the dissatisfaction with Agile arises from the limitations of the basic Agile practices, and the need to compromise these in an enterprise context. Both DAD and RUP (yes it’s an iterative method) are examples of extended or hybrid practices that introduce coordination, phasing and other disciplines that are more acceptable in enterprises that require traceability, governance and compliance with pre-existing life cycle practices. Enterprise frameworks such as Leffingwell’s SAFe as discussed and Everware-CBDI’s SOAM are examples of frameworks that adhere more closely to the purity of Agile principles while addressing enterprise specific needs. SAFe provides a framework which is more strongly Lean, coordinating portfolio, program and project activity to meet agile release train demand. SOAM provides a complementary, full life cycle process framework for software service modernization and delivery.
Agile Architecture. There is a requirement to articulate the enterprise requirements for agility as a reference architecture for business agility. In today’s fast moving world core architecture for the business, services, implementations, technology and deployments needs to be:
– under continuous development using Agile principles
– derived from the assessment of business needs for response to change, and constantly updated to reflect competitive and technology opportunities and threats.
– mapped to service architectures, patterns, policies and modernization strategies
– modeled using MDA/MDD to allow delivery as consistent architecture runways for portfolio and demand management, programs and projects.
Agile Delivery Framework. Most enterprises have a well-defined delivery framework of tools, repositories, templates etc that are designed to support well established QA and delivery policies. This is one of the most common inhibitors that Agile projects in the enterprise have to overcome. In an enterprise Agile context that framework must be realigned to provide maximum automation of life cycle management and governance so that key enterprise requirements for integrity can be met without loss of productivity. Similarly the development activity must be structured so that developers can extend the architecture runway with business solution specific rules and behaviors in a managed fashion which preserves the integrity of the architecture. Everware-CBDI has pioneered this runway extension capability implemented as a model driven (MDA/MDD) capability in which the runway code is generated and provided to developers enabling very significant productivity gains in both forward engineering and even more so in iteration.
Agile Modernization. Finally in an enterprise context the elephant in the room is the existing or legacy portfolio. Unless this elephant is addressed, the enterprise will continue to create more and more complexity, increase costs and reduce response times to change. What’s required is a discipline of continuous, Agile modernization. That means, using Dave Thomas’s minimalist manifesto [above] every portfolio item, program and project must include steps to find out the current situation and address minimum goals that reduce complexity and support a progressive modernization strategy. Without that, all enterprise Agile projects will remain narrow focus and simply add technical debt.
I suggest that while Agile is not dead in the enterprise it is certainly struggling to survive. This is because Agile practices alone will be suffocated at birth by enterprise realities of consistency and integrity; or turned into narrow focus, standalone projects; or morphed into BAU. I really don’t want to enter into a debate about nouns or verbs, life is too short. IMO Agile has considerable momentum and it can be morphed into a ground breaking concept that delivers enterprise business agility.
References
1 Agile is Dead (Long Live Agility)
2 Dean Leffingwell SAFe
I note healthy discussion around whether Agile is Dead [ref 1]. And while I may sympathize (sic) with many of the comments, particularly the commercial trivialization of education, the core issue must surely be the difficulty of adopting de facto Agile practices to support real world enterprise programs and projects. My experience is most of the advice and guidance out there is predicated on scaling the de facto Agile development methods. And this isn’t the best place to start.
An exception is Dean Leffingwell’s SAFe, [ref 2] which does introduce the idea of portfolio, program and project perspectives and intentional architecture. I recommend this framework as an intelligent set of practices, but for me it doesn’t go far enough because it is still primarily about development practices. This is the core problem – that Agile is development specific and practices only. In the enterprise, Agile development needs to be an integral part of a bigger ecosystem spanning business design, architecture, requirements, modernization and operational transformation practices plus architecture, delivery and modernization disciplines.
I am indebted to Dave Thomas whose recent blog, [ref 1] includes a worthy successor to the Agile Manifesto, reducing the original, development specific values and principles to a minimalist, more generic set. He says; “here is how to do something in an agile fashion”
– Find out where you are
– Take a small step towards your goal
– Adjust your understanding based on what you learned
– Repeat
And this is more useful in the enterprise context because it is relevant to a broader set of activities than purely software development.
The diagram below is an outline maturity model template for Agile in the enterprise. It suggests there are four key views that need to be part of the transformation. In addition to agile practices we need to be equally focused on what elements of agile architecture are required for an enterprise. What the agile delivery framework is and how the existing application portfolio will be modernized to progressively eliminate the duplication and complexity present in every enterprise on the planet.
People & Process. Much of the dissatisfaction with Agile arises from the limitations of the basic Agile practices, and the need to compromise these in an enterprise context. Both DAD and RUP (yes it’s an iterative method) are examples of extended or hybrid practices that introduce coordination, phasing and other disciplines that are more acceptable in enterprises that require traceability, governance and compliance with pre-existing life cycle practices. Enterprise frameworks such as Leffingwell’s SAFe as discussed and Everware-CBDI’s SOAM are examples of frameworks that adhere more closely to the purity of Agile principles while addressing enterprise specific needs. SAFe provides a framework which is more strongly Lean, coordinating portfolio, program and project activity to meet agile release train demand. SOAM provides a complementary, full life cycle process framework for software service modernization and delivery.
Agile Architecture. There is a requirement to articulate the enterprise requirements for agility as a reference architecture for business agility. In today’s fast moving world core architecture for the business, services, implementations, technology and deployments needs to be:
– under continuous development using Agile principles
– derived from the assessment of business needs for response to change, and constantly updated to reflect competitive and technology opportunities and threats.
– mapped to service architectures, patterns, policies and modernization strategies
– modeled using MDA/MDD to allow delivery as consistent architecture runways for portfolio and demand management, programs and projects.
Agile Delivery Framework. Most enterprises have a well-defined delivery framework of tools, repositories, templates etc that are designed to support well established QA and delivery policies. This is one of the most common inhibitors that Agile projects in the enterprise have to overcome. In an enterprise Agile context that framework must be realigned to provide maximum automation of life cycle management and governance so that key enterprise requirements for integrity can be met without loss of productivity. Similarly the development activity must be structured so that developers can extend the architecture runway with business solution specific rules and behaviors in a managed fashion which preserves the integrity of the architecture. Everware-CBDI has pioneered this runway extension capability implemented as a model driven (MDA/MDD) capability in which the runway code is generated and provided to developers enabling very significant productivity gains in both forward engineering and even more so in iteration.
Agile Modernization. Finally in an enterprise context the elephant in the room is the existing or legacy portfolio. Unless this elephant is addressed, the enterprise will continue to create more and more complexity, increase costs and reduce response times to change. What’s required is a discipline of continuous, Agile modernization. That means, using Dave Thomas’s minimalist manifesto [above] every portfolio item, program and project must include steps to find out the current situation and address minimum goals that reduce complexity and support a progressive modernization strategy. Without that, all enterprise Agile projects will remain narrow focus and simply add technical debt.
I suggest that while Agile is not dead in the enterprise it is certainly struggling to survive. This is because Agile practices alone will be suffocated at birth by enterprise realities of consistency and integrity; or turned into narrow focus, standalone projects; or morphed into BAU. I really don’t want to enter into a debate about nouns or verbs, life is too short. IMO Agile has considerable momentum and it can be morphed into a ground breaking concept that delivers enterprise business agility.
References
1 Agile is Dead (Long Live Agility)
2 Dean Leffingwell SAFe
Subhead: Modernization is NOT Optional!
Once again customers of The Royal Bank of Scotland (RBS) and its subsidiaries Ulster, Natwest etc are in trouble! Unable to use their debit cards, or access their funds! And the new Chief Executive Ross McEwan admitted today that RBS has failed to invest properly in its systems for decades! RBS has given no explanation of the crash, except that it was not related to the high volumes of transactions on Cyber Monday.
Actually the root problem for RBS, and many other banks and other very large organizations, is NOT that they haven’t invested in their systems for decades. The problem is that they have allowed the systems to evolve without good architecture and governance. Banking systems in particular used to be leading edge. But over the years the core systems that actually run the bank, have become badly out of date. Instead of modernizing the core systems, many banks have responded to demand for new products and services by surrounding the old legacy systems with new bolt on systems. As a result the old systems are held together with sticky tape and sealing wax, and modified piecemeal to adjust to new requirements, and the new systems create more and more interfaces and dependencies (including duplicated, hard coded business rules), so that eventually the systems resemble a hair ball. And as you know, a hairball is a tightly packed cylinder of fur, which also includes all manner of foreign particles, which can be quite hazardous in humans and animals. In systems terms we would more usually refer to the resulting mess as a monolithic systems portfolio.
The root problem therefore is twofold:
1. The increasing complexity and risk inherent in making change.
2. Increasing horizon of any change.
And the issues become apparent either when extremely complex operational processes execute incorrectly, or when changes in process, software or environment are made, often unavoidably without full knowledge of all the implications due to lack of experience, documentation or even code!
Although I have no doubt that in all the large banks there are fierce debates about the IT budget, strangely the solution to this problem isn’t about the amount of money that needs to be spent. Rather it’s about how the organization embraces a real modernization program, which is led by good architecture and managed with strong governance.
Sadly in the IT industry the term modernization has come to mean something quite specific – the re-platforming or technology upgrade of some application. Whereas modernization should really encompass a much broader remit including:
1. Establishing a lingua franca between business and IT so that modernization issues and implications are genuinely understood by both business and IT management.
2. Defining a business systems architecture that a) facilitates transformation with low risk and b) progressively develops an inherently agile architecture, that can evolve continuously.
3. A roadmap for progressively rationalizing the business systems portfolio to comply with the architecture.
4. Implementing an integrated business and IT organization that owns the business systems.
5. Implementing knowledge management around business systems that ensures integrity and consistency and ownership of processes, information and rules.
6. Implementing a continuous Agile modernization and ongoing evolution process in both business and IT.
7. Establishing coordination, governance and risk management that ensures architectural integrity at all stages of the life cycle.
It would be easy for business management to think that the sort of problems experienced by RBS and other banks as IT problems. The above list suggests this is nonsense. The only way to properly resolve the issues is for business management to accept co-responsibility for IT. And it’s no accident that the first point in the list above is to ensure that business management understand the issues they are dealing with, and can communicate effectively with the IT organization. This isn’t just attending an education class; it’s embracing a common language that allows critical decisions to be taken on a properly balanced understanding of business and IT assumptions, costs and risks.
Retailers, manufacturers, power companies, logistics companies, government departments etc etc should not be complacent or smug when they see the problems at the banks. In fact almost all very large organizations have these problems. And the message for all enterprises is Modernization is not Optional!
Links:
RBS Crash – Management Prefer Offshoring to Modernization?
CBDI Journal Reports on Modernization
Subhead: Modernization is NOT Optional!
Once again customers of The Royal Bank of Scotland (RBS) and its subsidiaries Ulster, Natwest etc are in trouble! Unable to use their debit cards, or access their funds! And the new Chief Executive Ross McEwan admitted today that RBS has failed to invest properly in its systems for decades! RBS has given no explanation of the crash, except that it was not related to the high volumes of transactions on Cyber Monday.
Actually the root problem for RBS, and many other banks and other very large organizations, is NOT that they haven’t invested in their systems for decades. The problem is that they have allowed the systems to evolve without good architecture and governance. Banking systems in particular used to be leading edge. But over the years the core systems that actually run the bank, have become badly out of date. Instead of modernizing the core systems, many banks have responded to demand for new products and services by surrounding the old legacy systems with new bolt on systems. As a result the old systems are held together with sticky tape and sealing wax, and modified piecemeal to adjust to new requirements, and the new systems create more and more interfaces and dependencies (including duplicated, hard coded business rules), so that eventually the systems resemble a hair ball. And as you know, a hairball is a tightly packed cylinder of fur, which also includes all manner of foreign particles, which can be quite hazardous in humans and animals. In systems terms we would more usually refer to the resulting mess as a monolithic systems portfolio.
The root problem therefore is twofold:
1. The increasing complexity and risk inherent in making change.
2. Increasing horizon of any change.
And the issues become apparent either when extremely complex operational processes execute incorrectly, or when changes in process, software or environment are made, often unavoidably without full knowledge of all the implications due to lack of experience, documentation or even code!
Although I have no doubt that in all the large banks there are fierce debates about the IT budget, strangely the solution to this problem isn’t about the amount of money that needs to be spent. Rather it’s about how the organization embraces a real modernization program, which is led by good architecture and managed with strong governance.
Sadly in the IT industry the term modernization has come to mean something quite specific – the re-platforming or technology upgrade of some application. Whereas modernization should really encompass a much broader remit including:
1. Establishing a lingua franca between business and IT so that modernization issues and implications are genuinely understood by both business and IT management.
2. Defining a business systems architecture that a) facilitates transformation with low risk and b) progressively develops an inherently agile architecture, that can evolve continuously.
3. A roadmap for progressively rationalizing the business systems portfolio to comply with the architecture.
4. Implementing an integrated business and IT organization that owns the business systems.
5. Implementing knowledge management around business systems that ensures integrity and consistency and ownership of processes, information and rules.
6. Implementing a continuous Agile modernization and ongoing evolution process in both business and IT.
7. Establishing coordination, governance and risk management that ensures architectural integrity at all stages of the life cycle.
It would be easy for business management to think that the sort of problems experienced by RBS and other banks as IT problems. The above list suggests this is nonsense. The only way to properly resolve the issues is for business management to accept co-responsibility for IT. And it’s no accident that the first point in the list above is to ensure that business management understand the issues they are dealing with, and can communicate effectively with the IT organization. This isn’t just attending an education class; it’s embracing a common language that allows critical decisions to be taken on a properly balanced understanding of business and IT assumptions, costs and risks.
Retailers, manufacturers, power companies, logistics companies, government departments etc etc should not be complacent or smug when they see the problems at the banks. In fact almost all very large organizations have these problems. And the message for all enterprises is Modernization is not Optional!
Links:
RBS Crash – Management Prefer Offshoring to Modernization?
CBDI Journal Reports on Modernization
I was interested to read the recent McKinsey report on disruptive technologies. McKinsey identifies twelve potentially economically disruptive technologies including the Mobile Internet, Automation of Knowledge work, Internet of Things, Advanced Robotics, Next generation genomics and so on. The report also calls out general purpose technologies as ones that propel steep growth trajectories (think Steam or Internet) – that can be applied across economies and leveraged in many more specific disruptive technologies. Not surprisingly they don’t include software development in either list. The closest they come is with the automation of knowledge work, but this is restricted to artificial intelligence, machine learning and natural interfaces like voice recognition that automate many knowledge worker tasks that have long been regarded as impossible or impracticable for machines to perform.
Is this omission something we should be concerned about I wonder? While software development “practices” have been developing very rapidly with the adoption of Agile methods, it is a reasonable conclusion that software development “technologies ” are not undergoing dramatic changes that might qualify as disruptive. Yes there’s lots going on; in fact there’s a profusion of new languages, frameworks and databases, many open source initiatives, that are progressively specializing development technology. In addition there are significant advances in life cycle management and test technologies. But there isn’t any indication that these new technologies will have high economic impact in terms of dramatic improvement in productivity or quality. Or have a significant impact on the vast economic problem inherent in the worlds legacy systems. Rather there’s a huge proliferation of development diversity and some might say complexity.
Don’t get me wrong, I am not looking for a problem to solve. It’s clear that while smaller Agile projects are fine for tightly targeted problems, most organizations have struggled to scale Agile to larger projects and or enterprise class projects. The increase in dependencies and complexities become overwhelming and the probability of failure increases proportionately.
What’s needed, by larger projects is not process automation, but automation of the deliverable that allows the project to manage the dependencies at the model AND deliverable level. This raises the level of abstraction and can deliver dramatic productivity and quality improvements. As it happens there is a technology that can do this, but strangely it seems to be something that many people have already consigned to the trash heap of “been there done that”. I’m talking about Model Driven Development (MDD). There are many reasons why MDD has not succeeded in gaining widespread acceptance. It is actually extremely complex and requires considerable investment to establish. And in fairness it has been promoted primarily as a deliverable transformation and code generation tool. And many people will say, Oh NO!!! That’s just reinventing Case Tools all over again and we don’t want to go there.
But before we consign this technology to the trashcan of yesterday’s technologies, we need to take a hard look at what you can do if:
A. you have leaf node detail models in the asset repository that are tightly bound to execution deliverables.
B. you use best practice modern architecture with all functionality delivered as service bearing capabilities that minimize dependencies.
C. you can automate to a significant extent the population of the repository with harvested knowledge about legacy applications at that same leaf node level of detail.
D. you can run large scale, full life projects with full iteration of business, architecture, design and development models. (note here this doesn’t mean fully integrated and transformed models, we have gotten a lot cleverer over the years.)
In an Agile context this allows you to iterate functionality at extremely low cost, both in delivery and evolution life cycle stages. In fact experience shows it transforms the development project into an evolutionary approach in which you can really architect and build what you know and evolve to the optimal solution.
Model driven as a concept has been around a long time. Most developers (tell me they) don’t like model driven because it won’t handle complexities; because it diminishes developers’ jobs to be more mundane; that it produces poor code and so on and so forth. But the McKinsey report speaks to the inexorable progress of technology and the inevitability that as technology changes peoples jobs change or disappear. You are either on the train or under it.
Right now Agile MDD is probably only justifiable for the very large, complex projects. But as the case studies start showing higher success rates, with dramatic increases in productivity and quality, and the level of up-front investment is reduced as the capability is productized, we can expect to see the MDD project footprint to expand dramatically. Again the McKinsey report is incredibly bullish on the economic outlook for technology, and information technology in particular as the key general purpose enabling technology, and it’s clear that Agile processes alone are inadequate to support the ever increasing demand.
Being disciplined is for school kids; it’s time we got smart about how we deliver complex services and systems at scale.
McKinsey & Company: Disruptive technologies: Advances that will transform life, business, and the global economy. “Not every emerging technology will alter the business or social landscape—but some truly do have the potential to disrupt the status quo, alter the way people live and work, and rearrange value pools.”
I was interested to read the recent McKinsey report on disruptive technologies. McKinsey identifies twelve potentially economically disruptive technologies including the Mobile Internet, Automation of Knowledge work, Internet of Things, Advanced Robotics, Next generation genomics and so on. The report also calls out general purpose technologies as ones that propel steep growth trajectories (think Steam or Internet) – that can be applied across economies and leveraged in many more specific disruptive technologies. Not surprisingly they don’t include software development in either list. The closest they come is with the automation of knowledge work, but this is restricted to artificial intelligence, machine learning and natural interfaces like voice recognition that automate many knowledge worker tasks that have long been regarded as impossible or impracticable for machines to perform.
Is this omission something we should be concerned about I wonder? While software development “practices” have been developing very rapidly with the adoption of Agile methods, it is a reasonable conclusion that software development “technologies ” are not undergoing dramatic changes that might qualify as disruptive. Yes there’s lots going on; in fact there’s a profusion of new languages, frameworks and databases, many open source initiatives, that are progressively specializing development technology. In addition there are significant advances in life cycle management and test technologies. But there isn’t any indication that these new technologies will have high economic impact in terms of dramatic improvement in productivity or quality. Or have a significant impact on the vast economic problem inherent in the worlds legacy systems. Rather there’s a huge proliferation of development diversity and some might say complexity.
Don’t get me wrong, I am not looking for a problem to solve. It’s clear that while smaller Agile projects are fine for tightly targeted problems, most organizations have struggled to scale Agile to larger projects and or enterprise class projects. The increase in dependencies and complexities become overwhelming and the probability of failure increases proportionately.
What’s needed, by larger projects is not process automation, but automation of the deliverable that allows the project to manage the dependencies at the model AND deliverable level. This raises the level of abstraction and can deliver dramatic productivity and quality improvements. As it happens there is a technology that can do this, but strangely it seems to be something that many people have already consigned to the trash heap of “been there done that”. I’m talking about Model Driven Development (MDD). There are many reasons why MDD has not succeeded in gaining widespread acceptance. It is actually extremely complex and requires considerable investment to establish. And in fairness it has been promoted primarily as a deliverable transformation and code generation tool. And many people will say, Oh NO!!! That’s just reinventing Case Tools all over again and we don’t want to go there.
But before we consign this technology to the trashcan of yesterday’s technologies, we need to take a hard look at what you can do if:
A. you have leaf node detail models in the asset repository that are tightly bound to execution deliverables.
B. you use best practice modern architecture with all functionality delivered as service bearing capabilities that minimize dependencies.
C. you can automate to a significant extent the population of the repository with harvested knowledge about legacy applications at that same leaf node level of detail.
D. you can run large scale, full life projects with full iteration of business, architecture, design and development models. (note here this doesn’t mean fully integrated and transformed models, we have gotten a lot cleverer over the years.)
In an Agile context this allows you to iterate functionality at extremely low cost, both in delivery and evolution life cycle stages. In fact experience shows it transforms the development project into an evolutionary approach in which you can really architect and build what you know and evolve to the optimal solution.
Model driven as a concept has been around a long time. Most developers (tell me they) don’t like model driven because it won’t handle complexities; because it diminishes developers’ jobs to be more mundane; that it produces poor code and so on and so forth. But the McKinsey report speaks to the inexorable progress of technology and the inevitability that as technology changes peoples jobs change or disappear. You are either on the train or under it.
Right now Agile MDD is probably only justifiable for the very large, complex projects. But as the case studies start showing higher success rates, with dramatic increases in productivity and quality, and the level of up-front investment is reduced as the capability is productized, we can expect to see the MDD project footprint to expand dramatically. Again the McKinsey report is incredibly bullish on the economic outlook for technology, and information technology in particular as the key general purpose enabling technology, and it’s clear that Agile processes alone are inadequate to support the ever increasing demand.
Being disciplined is for school kids; it’s time we got smart about how we deliver complex services and systems at scale.
McKinsey & Company: Disruptive technologies: Advances that will transform life, business, and the global economy. “Not every emerging technology will alter the business or social landscape—but some truly do have the potential to disrupt the status quo, alter the way people live and work, and rearrange value pools.”