David Sprott

Systems Failure at RBS – Again

by

Subhead: Modernization is NOT Optional!

Once again customers of The Royal Bank of Scotland (RBS) and its subsidiaries Ulster, Natwest etc are in trouble! Unable to use their debit cards, or access their funds! And the new Chief Executive Ross McEwan admitted today that RBS has failed to invest properly in its systems for decades! RBS has given no explanation of the crash, except that it was not related to the high volumes of transactions on Cyber Monday.

Actually the root problem for RBS, and many other banks and other very large organizations, is NOT that they haven’t invested in their systems for decades. The problem is that they have allowed the systems to evolve without good architecture and governance. Banking systems in particular used to be leading edge. But over the years the core systems that actually run the bank, have become badly out of date. Instead of modernizing the core systems, many banks have responded to demand for new products and services by surrounding the old legacy systems with new bolt on systems. As a result the old systems are held together with sticky tape and sealing wax, and modified piecemeal to adjust to new requirements, and the new systems create more and more interfaces and dependencies (including duplicated, hard coded business rules), so that eventually the systems resemble a hair ball. And as you know, a hairball is a tightly packed cylinder of fur, which also includes all manner of foreign particles, which can be quite hazardous in humans and animals. In systems terms we would more usually refer to the resulting mess as a monolithic systems portfolio.

The root problem therefore is twofold:
1. The increasing complexity and risk inherent in making change.
2. Increasing horizon of any change.

And the issues become apparent either when extremely complex operational processes execute incorrectly, or when changes in process, software or environment  are made, often unavoidably without full knowledge of all the implications due to lack of experience, documentation or even code!

Although I have no doubt that in all the large banks there are fierce debates about the IT budget, strangely the solution to this problem isn’t about the amount of money that needs to be spent. Rather it’s about how the organization embraces a real modernization program, which is led by good architecture and managed with strong governance.

Sadly in the IT industry the term modernization has come to mean something quite specific – the re-platforming or technology upgrade of some application. Whereas modernization should really encompass a much broader remit including:
1. Establishing a lingua franca between business and IT so that modernization issues and implications are genuinely understood by both business and IT management.
2. Defining a business systems architecture that a) facilitates transformation with low risk and b) progressively develops an inherently agile architecture, that can evolve continuously.
3. A roadmap for progressively rationalizing the business systems portfolio to comply with the architecture.
4. Implementing an integrated business and IT organization that owns the business systems.
5. Implementing knowledge management around business systems that ensures integrity and consistency and ownership of processes, information and rules.
6. Implementing a continuous Agile modernization and ongoing evolution process in both business and IT.
7. Establishing coordination, governance and risk management that ensures architectural integrity at all stages of the life cycle.

It would be easy for business management to think that the sort of problems experienced by RBS and other banks as IT problems. The above list suggests this is nonsense. The only way to properly resolve the issues is for business management to accept co-responsibility for IT. And it’s no accident that the first point in the list above is to ensure that business management understand the issues they are dealing with, and can communicate effectively with the IT organization. This isn’t just attending an education class; it’s embracing a common language that allows critical decisions to be taken on a properly balanced understanding of business and IT assumptions, costs and risks.

Retailers, manufacturers, power companies, logistics companies, government departments etc etc should not be complacent or smug when they see the problems at the banks. In fact almost all very large organizations have these problems. And the message for all enterprises is Modernization is not Optional!

Links:
RBS Crash – Management Prefer Offshoring to Modernization?
CBDI Journal Reports on Modernization

Are Service Architects and Designers like Cobbler’s Children?

by

During October I ran a survey asking Architects, Designers and Project Managers about Service Specification practices. I have just completed the analysis and report, see link below. There are some interesting conclusions:
1. DIY is the dominant service specification approach.
Respondents report there’s a wide range of approaches to service specification in use, and the most common approach by far (50%) is “do it yourself”. Sure, DIY will mean that practitioners are like magpies, they pick up ideas from a variety of sources, and create their own capability. But this, by any measure, must represent a failure of industry standards. While consistency of specification meta data may not be a primary goal for designers and developers, architects should consider a range of risks including use of development automation support tools, standardization of technical components of outsourcing contracts, opportunity to find/reuse pre-existing services, and crucially standardization of tooling. Which takes me onto the next point . . .

2. Documents and Spread-sheets still dominate tool usage for service planning, reporting and management. 
In the early stages of service adoption almost everyone simply uses the tools that are available to hand. But as the service portfolio evolves and matures these tools become a strategic liability in areas of planning, reuse management, portfolio and asset management, governance and reporting. I note not one respondent mentioned the EA Repository in context with service support tools. And again this is closely connected with the third interesting conclusion . . .

3.   64% dissatisfied with tool support. 
There was a very high level of dissatisfaction with tools, particularly in the context of communicating service specifications between the different roles and groups across the organization. We all appreciate high quality service specifications are the result of multi-disciplinary efforts, spanning Business Analysis, Architecture, Design, Developers, Product Management, IT Service Management, Outsourcing, Procurement,  Governance etc . So this is much more than reporting, there’s an evident need for modern tools that facilitate the collaboration process.

See Service Specification Practices Survey 2013 – Report

Are Service Architects and Designers like Cobbler’s Children?

by

During October I ran a survey asking Architects, Designers and Project Managers about Service Specification practices. I have just completed the analysis and report, see link below. There are some interesting conclusions:
1. DIY is the dominant service specification approach.
Respondents report there’s a wide range of approaches to service specification in use, and the most common approach by far (50%) is “do it yourself”. Sure, DIY will mean that practitioners are like magpies, they pick up ideas from a variety of sources, and create their own capability. But this, by any measure, must represent a failure of industry standards. While consistency of specification meta data may not be a primary goal for designers and developers, architects should consider a range of risks including use of development automation support tools, standardization of technical components of outsourcing contracts, opportunity to find/reuse pre-existing services, and crucially standardization of tooling. Which takes me onto the next point . . .

2. Documents and Spread-sheets still dominate tool usage for service planning, reporting and management. 
In the early stages of service adoption almost everyone simply uses the tools that are available to hand. But as the service portfolio evolves and matures these tools become a strategic liability in areas of planning, reuse management, portfolio and asset management, governance and reporting. I note not one respondent mentioned the EA Repository in context with service support tools. And again this is closely connected with the third interesting conclusion . . .

3.   64% dissatisfied with tool support. 
There was a very high level of dissatisfaction with tools, particularly in the context of communicating service specifications between the different roles and groups across the organization. We all appreciate high quality service specifications are the result of multi-disciplinary efforts, spanning Business Analysis, Architecture, Design, Developers, Product Management, IT Service Management, Outsourcing, Procurement,  Governance etc . So this is much more than reporting, there’s an evident need for modern tools that facilitate the collaboration process.

See Service Specification Practices Survey 2013 – Report

Service Factory 2.0

by

In July 1989 the Harvard Business Review published a seminal article by Chase and Garvin titled The Service Factory [1]. They argued that “The factory of the future is not a place where computers, robots, and flexible machines do the drudge work. . . the next generation, then, will compete by bundling services with products, anticipating and responding to a truly comprehensive range of customer needs.”

Since the publication some 25 years ago, much has happened to validate Messrs Chase and Garvin’s thesis. The world has manifestly transformed into a “service based world”. By some accounts the service sector grew to over 80% of the US economy by 2000. There are various implementations of the Service Factory concept; we might observe that the ubiquitous Call Centers represent a scalable, repeatable factory model. Similarly in the technology world services, or APIs, are a central part of many IT architectures, and some firms have adopted a service factory model by separating service and solution delivery.

But it must be said that for most enterprises there remains a significant gap between the business and software services. Over 25 years ago Chase and Garvin recognized this as an issue. They use the example of how 200 years ago horse-drawn carriages were mostly made by craftsmen, and the most successful carriage maker was invariably the most accommodating for the customer. But as mass production overtook craftsmanship customers came to value standardized, lower price more than higher price, personalized products. Gradually manufacturing became separated from pre and post-sale customer facing activities. In recent years in some industry sectors, there has been a noticeable reversal of this trend, and mass customization is widely practiced. However in many industries, notwithstanding Agile methods focus on customer involvement, the design of business services remains a discrete activity from the architecture and design of IT services.

This is starting to change. The Mobile revolution and the Internet of Things will inevitably cause a convergence of business and software services. I have described this as “turning the business inside out”. In future it will be a very rare business service that isn’t delivered by a software service, at least as an option or complement. It’s a racing certainty that Call Centers and other conventional service delivery models are going to go the same way as Telephone Exchanges, Typing Pools and Airline Reservations Departments! In this fast emerging world, “everything is a service”. And this will have a profound impact on the shape of the Service Factory of tomorrow.

The Service Factory 2.0 will be a software factory separate from solution delivery projects. As discussed some firms already employ this organizing model in order to architect and design services to standardize core business information and rules for use in many processes.

The software factory concept has been in use for some time, particularly in conjunction with the Product Line concept, where common components are delivered using a framework of tools, repeatable processes and patterns, that are reused in solutions supporting the Product Line. The Service Factory 2.0 is a specialization of the software factory, with a framework that is specific to services. However the most effective service factory will also be a “business service” delivery model in which the life cycle of the business, software and IT service perspectives are integrated.

We can anticipate a number of critical innovations that will enable the Service Factory 2.0:

1. Everything is a Service All business capabilities are delivered as modular software services which are integral to the holistic business service offering.
2. Automated Requirements Capture. The requirements activity is automated in a manner that facilitates the involvement of the real stakeholders in the specification of the business, software and IT service, in a language that is entirely natural to them and to engage meaningfully in distributed Agile processes.
3. Automation of Common Service Patterns. The Service Factory frameworks that bootstrap service implementations will increasingly automate many of the common architectural and business model styles. Yet unlike the de facto application package products, the Service Factory will facilitate the extension and customization of the common pattern to allow competitive differentiation for the customer organization, as well as ongoing flexibility.
4. Service Portfolio Management. The Service Factory must support the management of the “business service” and provide tools that integrate demand management, architecture and governance over a composable and constantly evolving business and software service portfolio.
6. Pattern R&D. An important role of the service factory will be to continuously evolve patterns to enable and support innovation in styles of business service together with increased scope of automation coverage
7. Architecture Managed Iteration. Pattern based development delivers code that is always compliant with the architecture, but also with meta data that allows full automation of configuration management. This facilitates continuous, low effort iteration in the delivery, test, DevOps cycle and evolution in production.

In their paper, Chase and Garvin quote Drucker who noted that the imperatives of information-based competition are breaking down barriers within businesses and making functional divisions obsolete. To my mind, the primary characteristic of Service Factory 2.0 will be the realization of the business, software and IT service as a composite offering, architected, designed and delivered as a business asset.

[1]  The Service Factory, Chase, Garvin, HBR

Link: Everware-CBDI Service Factory as a Servicehttp://agileservicefactory.com

Service Factory 2.0

by

In July 1989 the Harvard Business Review published a seminal article by Chase and Garvin titled The Service Factory [1]. They argued that “The factory of the future is not a place where computers, robots, and flexible machines do the drudge work. . . the next generation, then, will compete by bundling services with products, anticipating and responding to a truly comprehensive range of customer needs.”

Since the publication some 25 years ago, much has happened to validate Messrs Chase and Garvin’s thesis. The world has manifestly transformed into a “service based world”. By some accounts the service sector grew to over 80% of the US economy by 2000. There are various implementations of the Service Factory concept; we might observe that the ubiquitous Call Centers represent a scalable, repeatable factory model. Similarly in the technology world services, or APIs, are a central part of many IT architectures, and some firms have adopted a service factory model by separating service and solution delivery.

But it must be said that for most enterprises there remains a significant gap between the business and software services. Over 25 years ago Chase and Garvin recognized this as an issue. They use the example of how 200 years ago horse-drawn carriages were mostly made by craftsmen, and the most successful carriage maker was invariably the most accommodating for the customer. But as mass production overtook craftsmanship customers came to value standardized, lower price more than higher price, personalized products. Gradually manufacturing became separated from pre and post-sale customer facing activities. In recent years in some industry sectors, there has been a noticeable reversal of this trend, and mass customization is widely practiced. However in many industries, notwithstanding Agile methods focus on customer involvement, the design of business services remains a discrete activity from the architecture and design of IT services.

This is starting to change. The Mobile revolution and the Internet of Things will inevitably cause a convergence of business and software services. I have described this as “turning the business inside out”. In future it will be a very rare business service that isn’t delivered by a software service, at least as an option or complement. It’s a racing certainty that Call Centers and other conventional service delivery models are going to go the same way as Telephone Exchanges, Typing Pools and Airline Reservations Departments! In this fast emerging world, “everything is a service”. And this will have a profound impact on the shape of the Service Factory of tomorrow.

The Service Factory 2.0 will be a software factory separate from solution delivery projects. As discussed some firms already employ this organizing model in order to architect and design services to standardize core business information and rules for use in many processes.

The software factory concept has been in use for some time, particularly in conjunction with the Product Line concept, where common components are delivered using a framework of tools, repeatable processes and patterns, that are reused in solutions supporting the Product Line. The Service Factory 2.0 is a specialization of the software factory, with a framework that is specific to services. However the most effective service factory will also be a “business service” delivery model in which the life cycle of the business, software and IT service perspectives are integrated.

We can anticipate a number of critical innovations that will enable the Service Factory 2.0:

1. Everything is a Service All business capabilities are delivered as modular software services which are integral to the holistic business service offering.
2. Automated Requirements Capture. The requirements activity is automated in a manner that facilitates the involvement of the real stakeholders in the specification of the business, software and IT service, in a language that is entirely natural to them and to engage meaningfully in distributed Agile processes.
3. Automation of Common Service Patterns. The Service Factory frameworks that bootstrap service implementations will increasingly automate many of the common architectural and business model styles. Yet unlike the de facto application package products, the Service Factory will facilitate the extension and customization of the common pattern to allow competitive differentiation for the customer organization, as well as ongoing flexibility.
4. Service Portfolio Management. The Service Factory must support the management of the “business service” and provide tools that integrate demand management, architecture and governance over a composable and constantly evolving business and software service portfolio.
6. Pattern R&D. An important role of the service factory will be to continuously evolve patterns to enable and support innovation in styles of business service together with increased scope of automation coverage
7. Architecture Managed Iteration. Pattern based development delivers code that is always compliant with the architecture, but also with meta data that allows full automation of configuration management. This facilitates continuous, low effort iteration in the delivery, test, DevOps cycle and evolution in production.

In their paper, Chase and Garvin quote Drucker who noted that the imperatives of information-based competition are breaking down barriers within businesses and making functional divisions obsolete. To my mind, the primary characteristic of Service Factory 2.0 will be the realization of the business, software and IT service as a composite offering, architected, designed and delivered as a business asset.

[1]  The Service Factory, Chase, Garvin, HBR

Link: Everware-CBDI Service Factory as a Service

SURVEY: Service Specification Usage

by
Question: What’s the difference between a Web Service, an API and an SOA Software Service?

Answer: The Web Service and the API are technical interfaces, that may or may not be well-formed services that comply with SOA principles of loose coupling, autonomy, encapsulation (of the service), reusability and composability. A well-formed Software Service will usually have some form of Service Specification that defines and governs the compliance with SOA principles.

From observation, the use of Service Specifications by architects and designers is highly variable. In our work at Everware-CBDI we have encouraged more formality of specification over many years in order to increase the quality of delivered services. We have done this by making templates,UML profiles and methodology guidance freely available. Today we are actively engaged in delivering automation methodology and tools at all stages of the delivery life cycle. We plan to deliver some specification capabilities on the same freely available basis. We are therefore interested to learn what others are doing and we are inviting participation in a survey to establish some independent data around how services are specified.
The survey is intended for architects, designers and project managers.
–  It should be very quick to complete; about 10 minutes.
–  Participation is anonymous. Just click here to commence
If you would like to get more involved, please:
     a.      Join the CBDI Forum LinkedIn Group and or
     b.      Register with the Everware-CBDI site
We will keep these groups up to date and of course publish results of the survey in due course.
————————————————————
Everware-CBDI Rich Service Specification:
Download the CBDI-SAE Rich Service Specification Template and view the Service Specification Reference Framework

SURVEY: Service Specification Usage

by
Question: What’s the difference between a Web Service, an API and an SOA Software Service?

Answer: The Web Service and the API are technical interfaces, that may or may not be well-formed services that comply with SOA principles of loose coupling, autonomy, encapsulation (of the service), reusability and composability. A well-formed Software Service will usually have some form of Service Specification that defines and governs the compliance with SOA principles.

From observation, the use of Service Specifications by architects and designers is highly variable. In our work at Everware-CBDI we have encouraged more formality of specification over many years in order to increase the quality of delivered services. We have done this by making templates,UML profiles and methodology guidance freely available. Today we are actively engaged in delivering automation methodology and tools at all stages of the delivery life cycle. We plan to deliver some specification capabilities on the same freely available basis. We are therefore interested to learn what others are doing and we are inviting participation in a survey to establish some independent data around how services are specified.
The survey is intended for architects, designers and project managers.
–  It should be very quick to complete; about 10 minutes.
–  Participation is anonymous. Just click here to commence
If you would like to get more involved, please:
     a.      Join the CBDI Forum LinkedIn Group and or
     b.      Register with the Everware-CBDI site
We will keep these groups up to date and of course publish results of the survey in due course.
————————————————————
Everware-CBDI Rich Service Specification:
Download the CBDI-SAE Rich Service Specification Template and view the Service Specification Reference Framework

Agile SOA in the Digital Economy

by

Are you and your enterprise a prisoner of the past? I don’t mean legacy applications and technologies, I mean today’s business processes and applications. I work with many different enterprises and what’s common to the great majority is the centrality of business processes and applications, and the difficulty in evolving these existing solutions.

Actually I am frequently amazed at the understanding of many business managers. I marvel at how they use the lingua franca of their applications to describe their business. I will readily admit that when I first meet someone like this it’s a bit scary, because their vocabulary is like a foreign language. But frequently I find that it’s also a foreign language to their colleagues and it represents a rather primitive form of power play! Believe me. And that vocabulary of course often pervades the business process also. But the even scarier thing is that these organizations don’t realize they are locked into yesterday; or looking in the rear view mirror is you prefer. And just like Fred Brooks mythical beasts, struggling against the grip of the tar pits [1], they will eventually be overwhelmed by the complexity and their inability to change. Yes they may be delivering Cloud based Web and mobile applications to their customers, but are they just adding to the inherent business complexity?

I observe smart, successful companies making major mistakes as they enter the digital economy. First they set up an eServices project or division. This is treated as an innovation center and separated from the core business, in order to get to market quickly. But of course when they get to market the new products don’t integrate with the core business. Sure there’s application and service integration, anyone can patch old and new together at that level; but what about the way the business works; the business model and the vocabulary used, the opportunities for channel switching, and the development of distinctive sales and customer support systems and internal and external company culture that transcends the technology channels. And the ability to evolve the old and new in a way that they complement each other?

This problem is visible in the decreasing agility of organizations. Many have adopted Agile development but, and I say this as a certified Scrum Master, how many Agile projects think about the vocabulary and integrated business model issue? Yes Agile projects generally deliver faster, better and cheaper, but are they basically adding to the size and eventual grip of the tar pit? Just getting there faster!

In the digital economy enterprises must turn themselves inside-out! Expose their core business capabilities as services through multiple, interconnected channels for internal and external use. Today’s SOA best practice is primarily inwards looking. What’s required is a new form of business model that details the new world from the customers’ perspective. And this needs to be reflected in the way the business operates internally also.

The new business model needs to be a radical departure from de facto practices in business architecture, enterprise architecture or business requirements. And it needs to be developed to govern Agile development projects. Key characteristics include:
– A service oriented business model that transcends business and IT.
– Understandable by all stakeholders
– Owned by the business.
– All enterprise capabilities are (eventually) published as externalized business services and supported by common software services
– Implementation independent models
– Developed as part of an Agile process – initial scoping sprint, followed by drill down modeling sprints by domain and or capability; delivering just sufficient detail to charter Agile delivery projects.

[1] The Mythical Man-Month, Fred Brooks, 1972

Agile SOA in the Digital Economy

by

Are you and your enterprise a prisoner of the past? I don’t mean legacy applications and technologies, I mean today’s business processes and applications. I work with many different enterprises and what’s common to the great majority is the centrality of business processes and applications, and the difficulty in evolving these existing solutions.

Actually I am frequently amazed at the understanding of many business managers. I marvel at how they use the lingua franca of their applications to describe their business. I will readily admit that when I first meet someone like this it’s a bit scary, because their vocabulary is like a foreign language. But frequently I find that it’s also a foreign language to their colleagues and it represents a rather primitive form of power play! Believe me. And that vocabulary of course often pervades the business process also. But the even scarier thing is that these organizations don’t realize they are locked into yesterday; or looking in the rear view mirror is you prefer. And just like Fred Brooks mythical beasts, struggling against the grip of the tar pits [1], they will eventually be overwhelmed by the complexity and their inability to change. Yes they may be delivering Cloud based Web and mobile applications to their customers, but are they just adding to the inherent business complexity?

I observe smart, successful companies making major mistakes as they enter the digital economy. First they set up an eServices project or division. This is treated as an innovation center and separated from the core business, in order to get to market quickly. But of course when they get to market the new products don’t integrate with the core business. Sure there’s application and service integration, anyone can patch old and new together at that level; but what about the way the business works; the business model and the vocabulary used, the opportunities for channel switching, and the development of distinctive sales and customer support systems and internal and external company culture that transcends the technology channels. And the ability to evolve the old and new in a way that they complement each other?

This problem is visible in the decreasing agility of organizations. Many have adopted Agile development but, and I say this as a certified Scrum Master, how many Agile projects think about the vocabulary and integrated business model issue? Yes Agile projects generally deliver faster, better and cheaper, but are they basically adding to the size and eventual grip of the tar pit? Just getting there faster!

In the digital economy enterprises must turn themselves inside-out! Expose their core business capabilities as services through multiple, interconnected channels for internal and external use. Today’s SOA best practice is primarily inwards looking. What’s required is a new form of business model that details the new world from the customers’ perspective. And this needs to be reflected in the way the business operates internally also.

The new business model needs to be a radical departure from de facto practices in business architecture, enterprise architecture or business requirements. And it needs to be developed to govern Agile development projects. Key characteristics include:
– A service oriented business model that transcends business and IT.
– Understandable by all stakeholders
– Owned by the business.
– All enterprise capabilities are (eventually) published as externalized business services and supported by common software services
– Implementation independent models
– Developed as part of an Agile process – initial scoping sprint, followed by drill down modeling sprints by domain and or capability; delivering just sufficient detail to charter Agile delivery projects.

[1] The Mythical Man-Month, Fred Brooks, 1972

Will Microsoft’s Reorganization Work?

by

I note Microsoft are reorganizing, again. From a products divisional organization to functional. Evidently they want to copy firms like Apple that have been demonstrably successful at delivering integrated products. The problem for Microsoft is that do…