4 years, 10 months ago

Scaled Agile Factory – Going Beyond the Usual Suspects

Abstract: If you address the question of how to scale Agile projects by considering what framework to use, you are only looking at one aspect of the problem. Scaling is all about coordination – managing enterprise considerations and cross program dependencies, and the defacto frameworks (SAFe, LeSS and DAD) focus on the people and process dimensions. However, in combination with a factory approach you may be able to automate many of the compliance and dependency management issues.

The question of how to scale Agile development has been around a while. In January 2015 I commented [1] on a clear trend in which my customers were voicing concerns about loss of consistency; inability to govern; lack of coordination and increasing time to market. Since then I have observed many large organizations adopting SAFe (Scaled Agile Framework) perhaps because it appears to be the only game in town, or perhaps there’s good marketing or there’s strength in numbers. But the criticisms of SAFe [2] haven’t gone away. The central and continuing concern is that SAFe compromises core Agile principles of self-organizing, cross functional teams that have full responsibility for the delivery of potentially shippable increments. And that SAFe looks suspiciously like WaterScrumFall because there’s a high overhead of portfolio, value stream and project management and in all probability intentional architecture. Now I have mixed opinions on SAFe; I see positives in value streams and product management which are important. And for organizations that have large programs, highly organized, structured approaches will be seen as lower risk, and indeed something that takes them some way down the Agile path, without straying too far from conventional management comfort zones. But the outcomes are unlikely to be inherently “agile”.

What SAFe does, is provide a rather conventional approach for a complex problem that most enterprises have – that is to deliver high integrity solutions that can work in an enterprise context that demands cross project dependency management, consistent data and reference architecture, collaboration with the existing portfolio complexities, compliance with enterprise standards and governance etc.

There are alternatives. Craig Larman and Bas Vodde in their books [3] and more recently with their LeSS initiative [4] have pursued a very different path that starts with Scrum and scales by understanding the needs for coordination while adhering to core Scrum principles. In their forthcoming book [5] they say, “LeSS is (1) lightweight, (2) simple to understand, and (3) difficult to master—due to essential complexity”. And this allows us to contrast the different approaches; while SAFe clearly works at some level, it has its roots in conventional large scale project management, whereas LeSS is lightweight, but requires much deeper understanding of the systems dynamics because of the inherent complexity of all the coordination requirements.
So the real question underlying Scaling Agile should be, “can we address some of the coordination requirements in a manner that reduces complexity and eliminates some of the need for additional layers of management or events?”

In my post Service Factory 2.0 [6] I describe the conceptual background of the Software Factory, ideas pioneered by my old friend Keith Short and Jack Greenfield while they were at Microsoft. Today these have evolved and become specialized around a framework of tools, repeatable processes and patterns for creation and assembly of services – manifest as first order components with formal interfaces. If you consider that all core business functionality is increasingly composed of services and their operations, this provides us with a reference architecture that by design implements separation of concerns.  Figure 1 below is a conceptual view of the scope of the service factory in terms of managed objects.

Figure 1
Naturally there will be other patterns involved in any solution including UX and workflow; but for most enterprise class solutions, a high proportion of the functionality should comprise services and business rules. This allows a consistent, highly automated approach to architecture, requirements specification, design and test. The factory effectively implements a proven service reference architecture as a platform which can be customized for individual programs, projects and technologies. While the factory platform is similar to an architecture runway in scope and intent, because it is model based the factory framework enables continuous evolution of the platform capabilities that are automatically inherited into work in progress factory code allowing program wide changes to be incorporated, often with minimal or no change to the service specific code. And the factory is a repository of the life cycle artifacts that can be leveraged for various management tasks including governance of traceability, impact analysis, etc. 
Let’s consider the key coordination and management activities typically required in a scaled agile program and how the factory may facilitate that. In Figure 2 below I have used the SAFe layers of portfolio, program and team, albeit without the value stream level. But apart from that, the view is intended to be framework neutral. The diagram illustrates how the Factory provides a full life cycle backplane that establishes the underlying (meta) model that ensures all the participants are in compliance with core concepts from portfolio planning through to delivery and production. The populated model is therefore able to provide essential information particularly about dependencies, but equally complete traceability and governance between business requirements and delivered services and rules.  
Figure 2
In the LeSS framework, Larman and Vodde disregard constructs such as ART (Agile Release Train) and 8 to 12 week PIs (Program Increments) and also the Hardening Sprint, preferring to use the standard Scrum duration of 2 to 4 weeks, with continuous integration.  We might assume these management constructs (ARTs and PIs) were invented because of the difficulty of effective inter team communications, and the overhead that creates. Larman and Vodde have a lot of good ideas about how to manage effective inter Teams communications, but their approach is essentially based on feature or story dependencies. In the factory environment inter-team dependencies can be more accurately based on service operation dependencies. But this is just the tip of the iceberg, in terms of how the factory can be leveraged to manage activity. The service specification becomes a pivotal work product; as it becomes available other teams and team members have access to detailed behaviour information that can inform and facilitate working ahead for developing depending services, test case development including automation assistance in case development, plus devops. 
Another huge issue in Agile projects is managing technical debt. In the factory environment the standard patterns generate all the infrastructure and shared code, and when changes take place, as discussed above, teams can perform an architecture update, inheriting all the changes to bring their project in line with the latest reference architecture. And because the factory is driven by the specification level, independent of technology it is inherently late binding, allowing change if technology with minimum impact.  
By utilizing work products that are part of a defined process based on the full life cycle reference architecture, the factory approach reduces the essential complexity of the scaling task. All moving parts are under management. This doesn’t detract from the inherent purity of the Agile approach. Rather, the defined process provides clarity over the minimum necessary “planning work” (intentional architecture, rules definition and business requirements). And the reference architecture and factory process provide a firm foundation on which emergent architecture can be more effectively carried out by the cross functional delivery team. This is likely to encourage programs to adopt a hybrid approach to their organizing model, embracing elements of LeSS like guidance to optimize multiple concurrent Scrum based sprints with continuous delivery, and to complement them with minimum necessary elements of SAFe, particularly those that are required to communicate to more conventional stakeholders. But, at least for mid-sized projects, maybe up to 10 teams, it seems overkill to adopt the formality of PI Planning and everything that comes with it.
In this post I have focused on the scaled Agile approach for projects, but it’s also useful to understand the factory has other important outcomes. First the highly modular nature of the reference architecture (see Figure 1) leads to “solutions” that are inherently agile. Because solutions are comprised primarily of services and rules with strong formal contracts, implemented using component based implementation patterns, the horizon of change is likely to be very constrained. Further the specification approach in which services, rules, data and processes are defined independent of implementation and technology provides high levels of visibility to the business, and high transparency and traceability of the implemented solution. Finally, we have a way to realize architecture in inherently agile solutions without Big Upfront Design, or indeed Big Upfront Organization, that will of course never become legacy because they can be continuously evolved. 
[2] Reasoned criticisms of SAFe
[3] Scaling Lean & Agile, Craig Larman and Bas Vodde, Addison Wesley, 2009
7 years, 4 months ago

Service Factory 2.0

In July 1989 the Harvard Business Review published a seminal article by Chase and Garvin titled The Service Factory [1]. They argued that “The factory of the future is not a place where computers, robots, and flexible machines do the drudge work. . . the next generation, then, will compete by bundling services with products, anticipating and responding to a truly comprehensive range of customer needs.”

Since the publication some 25 years ago, much has happened to validate Messrs Chase and Garvin’s thesis. The world has manifestly transformed into a “service based world”. By some accounts the service sector grew to over 80% of the US economy by 2000. There are various implementations of the Service Factory concept; we might observe that the ubiquitous Call Centers represent a scalable, repeatable factory model. Similarly in the technology world services, or APIs, are a central part of many IT architectures, and some firms have adopted a service factory model by separating service and solution delivery.

But it must be said that for most enterprises there remains a significant gap between the business and software services. Over 25 years ago Chase and Garvin recognized this as an issue. They use the example of how 200 years ago horse-drawn carriages were mostly made by craftsmen, and the most successful carriage maker was invariably the most accommodating for the customer. But as mass production overtook craftsmanship customers came to value standardized, lower price more than higher price, personalized products. Gradually manufacturing became separated from pre and post-sale customer facing activities. In recent years in some industry sectors, there has been a noticeable reversal of this trend, and mass customization is widely practiced. However in many industries, notwithstanding Agile methods focus on customer involvement, the design of business services remains a discrete activity from the architecture and design of IT services.

This is starting to change. The Mobile revolution and the Internet of Things will inevitably cause a convergence of business and software services. I have described this as “turning the business inside out”. In future it will be a very rare business service that isn’t delivered by a software service, at least as an option or complement. It’s a racing certainty that Call Centers and other conventional service delivery models are going to go the same way as Telephone Exchanges, Typing Pools and Airline Reservations Departments! In this fast emerging world, “everything is a service”. And this will have a profound impact on the shape of the Service Factory of tomorrow.

The Service Factory 2.0 will be a software factory separate from solution delivery projects. As discussed some firms already employ this organizing model in order to architect and design services to standardize core business information and rules for use in many processes.

The software factory concept has been in use for some time, particularly in conjunction with the Product Line concept, where common components are delivered using a framework of tools, repeatable processes and patterns, that are reused in solutions supporting the Product Line. The Service Factory 2.0 is a specialization of the software factory, with a framework that is specific to services. However the most effective service factory will also be a “business service” delivery model in which the life cycle of the business, software and IT service perspectives are integrated.

We can anticipate a number of critical innovations that will enable the Service Factory 2.0:

1. Everything is a Service All business capabilities are delivered as modular software services which are integral to the holistic business service offering.
2. Automated Requirements Capture. The requirements activity is automated in a manner that facilitates the involvement of the real stakeholders in the specification of the business, software and IT service, in a language that is entirely natural to them and to engage meaningfully in distributed Agile processes.
3. Automation of Common Service Patterns. The Service Factory frameworks that bootstrap service implementations will increasingly automate many of the common architectural and business model styles. Yet unlike the de facto application package products, the Service Factory will facilitate the extension and customization of the common pattern to allow competitive differentiation for the customer organization, as well as ongoing flexibility.
4. Service Portfolio Management. The Service Factory must support the management of the “business service” and provide tools that integrate demand management, architecture and governance over a composable and constantly evolving business and software service portfolio.
6. Pattern R&D. An important role of the service factory will be to continuously evolve patterns to enable and support innovation in styles of business service together with increased scope of automation coverage
7. Architecture Managed Iteration. Pattern based development delivers code that is always compliant with the architecture, but also with meta data that allows full automation of configuration management. This facilitates continuous, low effort iteration in the delivery, test, DevOps cycle and evolution in production.

In their paper, Chase and Garvin quote Drucker who noted that the imperatives of information-based competition are breaking down barriers within businesses and making functional divisions obsolete. To my mind, the primary characteristic of Service Factory 2.0 will be the realization of the business, software and IT service as a composite offering, architected, designed and delivered as a business asset.

[1]  The Service Factory, Chase, Garvin, HBR

Link: Everware-CBDI Service Factory as a Servicehttp://agileservicefactory.com

7 years, 4 months ago

Service Factory 2.0

In July 1989 the Harvard Business Review published a seminal article by Chase and Garvin titled The Service Factory [1]. They argued that “The factory of the future is not a place where computers, robots, and flexible machines do the drudge work. . . the next generation, then, will compete by bundling services with products, anticipating and responding to a truly comprehensive range of customer needs.”

Since the publication some 25 years ago, much has happened to validate Messrs Chase and Garvin’s thesis. The world has manifestly transformed into a “service based world”. By some accounts the service sector grew to over 80% of the US economy by 2000. There are various implementations of the Service Factory concept; we might observe that the ubiquitous Call Centers represent a scalable, repeatable factory model. Similarly in the technology world services, or APIs, are a central part of many IT architectures, and some firms have adopted a service factory model by separating service and solution delivery.

But it must be said that for most enterprises there remains a significant gap between the business and software services. Over 25 years ago Chase and Garvin recognized this as an issue. They use the example of how 200 years ago horse-drawn carriages were mostly made by craftsmen, and the most successful carriage maker was invariably the most accommodating for the customer. But as mass production overtook craftsmanship customers came to value standardized, lower price more than higher price, personalized products. Gradually manufacturing became separated from pre and post-sale customer facing activities. In recent years in some industry sectors, there has been a noticeable reversal of this trend, and mass customization is widely practiced. However in many industries, notwithstanding Agile methods focus on customer involvement, the design of business services remains a discrete activity from the architecture and design of IT services.

This is starting to change. The Mobile revolution and the Internet of Things will inevitably cause a convergence of business and software services. I have described this as “turning the business inside out”. In future it will be a very rare business service that isn’t delivered by a software service, at least as an option or complement. It’s a racing certainty that Call Centers and other conventional service delivery models are going to go the same way as Telephone Exchanges, Typing Pools and Airline Reservations Departments! In this fast emerging world, “everything is a service”. And this will have a profound impact on the shape of the Service Factory of tomorrow.

The Service Factory 2.0 will be a software factory separate from solution delivery projects. As discussed some firms already employ this organizing model in order to architect and design services to standardize core business information and rules for use in many processes.

The software factory concept has been in use for some time, particularly in conjunction with the Product Line concept, where common components are delivered using a framework of tools, repeatable processes and patterns, that are reused in solutions supporting the Product Line. The Service Factory 2.0 is a specialization of the software factory, with a framework that is specific to services. However the most effective service factory will also be a “business service” delivery model in which the life cycle of the business, software and IT service perspectives are integrated.

We can anticipate a number of critical innovations that will enable the Service Factory 2.0:

1. Everything is a Service All business capabilities are delivered as modular software services which are integral to the holistic business service offering.
2. Automated Requirements Capture. The requirements activity is automated in a manner that facilitates the involvement of the real stakeholders in the specification of the business, software and IT service, in a language that is entirely natural to them and to engage meaningfully in distributed Agile processes.
3. Automation of Common Service Patterns. The Service Factory frameworks that bootstrap service implementations will increasingly automate many of the common architectural and business model styles. Yet unlike the de facto application package products, the Service Factory will facilitate the extension and customization of the common pattern to allow competitive differentiation for the customer organization, as well as ongoing flexibility.
4. Service Portfolio Management. The Service Factory must support the management of the “business service” and provide tools that integrate demand management, architecture and governance over a composable and constantly evolving business and software service portfolio.
6. Pattern R&D. An important role of the service factory will be to continuously evolve patterns to enable and support innovation in styles of business service together with increased scope of automation coverage
7. Architecture Managed Iteration. Pattern based development delivers code that is always compliant with the architecture, but also with meta data that allows full automation of configuration management. This facilitates continuous, low effort iteration in the delivery, test, DevOps cycle and evolution in production.

In their paper, Chase and Garvin quote Drucker who noted that the imperatives of information-based competition are breaking down barriers within businesses and making functional divisions obsolete. To my mind, the primary characteristic of Service Factory 2.0 will be the realization of the business, software and IT service as a composite offering, architected, designed and delivered as a business asset.

[1]  The Service Factory, Chase, Garvin, HBR

Link: Everware-CBDI Service Factory as a Service