#eavoices

Agile is not Dead, it’s Morphing

by

I note healthy discussion around whether Agile is Dead [ref 1]. And while I may sympathize (sic) with many of the comments, particularly the commercial trivialization of education, the core issue must surely be the difficulty of adopting de facto Agile practices to support real world enterprise programs and projects. My experience is most of the advice and guidance out there is predicated on scaling the de facto Agile development methods. And this isn’t the best place to start.

An exception is Dean Leffingwell’s SAFe, [ref 2] which does introduce the idea of portfolio, program and project perspectives and intentional architecture. I recommend this framework as an intelligent set of practices, but for me it doesn’t go far enough because it is still primarily about development practices. This is the core problem – that Agile is development specific and practices only. In the enterprise, Agile development needs to be an integral part of a bigger ecosystem spanning business design, architecture, requirements, modernization and operational transformation practices plus architecture, delivery and modernization disciplines.

I am indebted to Dave Thomas whose recent blog, [ref 1] includes a worthy successor to the Agile Manifesto, reducing the original, development specific values and principles to a minimalist, more generic set. He says; “here is how to do something in an agile fashion”
– Find out where you are
– Take a small step towards your goal
– Adjust your understanding based on what you learned
– Repeat
And this is more useful in the enterprise context because it is relevant to a broader set of activities than purely software development.

The diagram below is an outline maturity model template for Agile in the enterprise. It suggests there are four key views that need to be part of the transformation.  In addition to agile practices we need to be equally focused on what elements of agile architecture are required for an enterprise. What the agile delivery framework is and how the existing application portfolio will be modernized to progressively eliminate the duplication and complexity present in every enterprise on the planet.

People & Process. Much of the dissatisfaction with Agile arises from the limitations of the basic Agile practices, and the need to compromise these in an enterprise context. Both DAD and RUP (yes it’s an iterative method) are examples of extended or hybrid practices that introduce coordination, phasing and other disciplines that are more acceptable in enterprises that require traceability, governance and compliance with pre-existing life cycle practices. Enterprise frameworks such as Leffingwell’s SAFe as discussed and Everware-CBDI’s SOAM are examples of frameworks that adhere more closely to the purity of Agile principles while addressing enterprise specific needs. SAFe provides a framework which is more strongly Lean, coordinating portfolio, program and project activity to meet agile release train demand. SOAM provides a complementary, full life cycle process framework for software service modernization and delivery.

Agile Architecture. There is a requirement to articulate the enterprise requirements for agility as a reference architecture for business agility. In today’s fast moving world core architecture for the business, services, implementations, technology and deployments needs to be:
– under continuous development using Agile principles
– derived from the assessment of business needs for response to change, and constantly updated to reflect competitive and technology opportunities and threats.
– mapped to service architectures, patterns, policies and modernization strategies
– modeled using MDA/MDD to allow delivery as consistent architecture runways for portfolio and demand management, programs and projects.

Agile Delivery Framework. Most enterprises have a well-defined delivery framework of tools, repositories, templates etc that are designed to support well established QA and delivery policies. This is one of the most common inhibitors that Agile projects in the enterprise have to  overcome. In an enterprise Agile context that framework must be realigned to provide maximum automation of life cycle management and governance so that key enterprise requirements for integrity can be met without loss of productivity. Similarly the development  activity must be structured so that developers can extend the architecture runway with business solution specific rules and behaviors in a managed fashion which preserves the integrity of the architecture. Everware-CBDI has pioneered this runway extension capability implemented as a model driven (MDA/MDD) capability in which the runway code is generated and provided to developers enabling very significant productivity gains in both forward engineering and even more so in iteration.

Agile Modernization.  Finally in an enterprise context the elephant in the room is the existing or legacy portfolio. Unless this elephant is addressed, the enterprise will continue to create more and more complexity, increase costs and reduce response times to change. What’s required is a discipline of continuous, Agile modernization. That means, using Dave Thomas’s minimalist manifesto [above] every portfolio item, program and project must include steps to find out the current situation and address minimum goals that reduce complexity and support a progressive modernization strategy. Without that, all enterprise Agile projects will remain narrow focus and simply add technical debt.

I suggest that while Agile is not dead in the enterprise it is certainly struggling to survive. This is because Agile practices alone will be suffocated at birth by enterprise realities of consistency and integrity; or turned into narrow focus, standalone projects; or morphed into BAU. I really don’t want to enter into a debate about nouns or verbs, life is too short.  IMO Agile has considerable momentum and it can be morphed into a ground breaking concept that delivers enterprise business agility.

References
1 Agile is Dead (Long Live Agility)
2 Dean Leffingwell SAFe

Whole-Brained Business Analysis – New Metaphor Required

by
I’ve been guilty using the much debated ‘Left vs Right brain’ metaphor to explain what I believe is needed. By way of example, Alec Sharp (@alecsharp), Sally Bean  (@Cybersal), Roy Grubb  (@roygrubb) and I have been Tweeting about Concept Modeling vs Concept Mapping. Alec is keen to get Data Modelers to abstract their thinking up from physical Data Models by thinking conceptually and I have been encouraging Business Analysts to think similarly when gathering requirements. This has meant that we both find that we need to introduce a different mindset: one that encourages more creative & inclusive discussion atthe initial   discovery and play-back stage of the Requirements-Solution Design journey. I expect the Agile/XP community will declare this to be their philosophy (and nothing new) and they’re probably right. But rather than get caught-up in ‘IT-centric’ methods, I’d rather think of it as a way to better understand any requirements for change – regardless of the Software Development Life-Cycle. I’d rather see such thinking applied to all aspects of business change – people, process, practice, policy and … technology.


Tried-and-tested analytical techniques should not be abandoned, they just need to be augmented with others that, in my experience, help expand ideas and produce resilient, coherent and business-value-creating solutions.  Both side of the equation are equally important. However, I’m finding (through experiment) that the more creative techniques are more engaging – simply more fun and inclusive – and, this alone, can, in my recent experience, dramatically improve business outcomes. 

In attempts to explain the need for a more ‘whole-brained’ approach, I’ve been following the lead of the ‘Design Thinking’ community in referring to both Theory X and Theory Y from MIT Sloan and the Left-brain Right-brain metaphor. This, however, is fraught with problems due, in large part to the findings of the University of Utah who debunk such binary thinking (as I was reminded by Rob England – @theitskeptic).

So I’m in a quandary: on the one hand I find that an X-Y, Left-Right, metaphor is a simple way to convey the difference between, say, Analysis vs. Synthesis, on the other hand, however, I run the risk of aligning with outdated concepts being fundamental reconsidered by neuroscientists. 

I guess the Complexity Science community might say that I’m talking about the difference between ‘Complex Adaptive’  vs. ‘Complicated’ systems, but, again, academic debate makes coming up with a simple metaphor next to impossible.

Has anyone found an alternative metaphor for a more balanced approach to Business Analysis and Enterprise Architecture?

Importantly, I’m keen to avoid the impression that people are to be seen as fundamentally one way or another. My observation is that it is the practice of Business Analysis/Enterprise Architecture that needs to be more ‘Whole-brained’ – not the individuals per se.

To get the discussion rolling, I’d like to hear views on:
  • A good Business Analyst or Enterprise Architecture must be a balance of Left-X(Reliability – Doing-things-Right) and Right-Y (Validity – Doing-the-right-thing)
  • We’ve spent to much time of methods that attempt to industrialise EA (the TOGAF 9.0 manual runs to around 800 pages in the attempt) and BAs are too often focused on methods focus on an ‘IT solution’ rather that the Whys and Whats of the current or desired business behavior
  • We need to spend more time on developing pattern-based storytelling skills in BAs and EAs to deliver break-through changes and allow for innovation in TO-BE models.
  • Economic churn and environmental challenges warrant more Y-minded thinking (with appropriate X-controls)
  • The world can’t be fully explained or governed algorithmically (thank god!)– not while values and trust dominate the way organisations function.


 

What is a method?

by

(Americans often refer to a method with the term “methodology”, which is not entirely correct semantically, as it would mean “the science of methods”) Examples of methods are ORM, RUP, and one could argue Scrum or agile approaches like DAD. Inspired by the book by Ian Graham et.al. The OPEN Process Specification, I share the […]

What’s Wrong with Universal Credit? On The Rooting of the Particular in the Universal

by

According to Hegel, the administration of a corporation’s affairs by its own supervisors will often be inept, “for although they know and have before them their own distinct interests and affairs, they have a less complete grasp of the connection between these and more remote conditions and universal points of view”. (GWF Hegel, Elements of the Philosophy of Right, 1820)

Therefore, one of the challenges for enterprise architecture is to maintain a pragmatic balance between the particular and the universal, especially when many of the decision-makers (as Hegel observed) tend to see things from a narrow short-term perspective.

So at first sight, we might think that the UK Government initiative known as Universal Credit represents a welcome exception to this narrow short-term thinking. What a brilliant idea to consolidate all benefit systems into a single simpler system!

The devil, of course, is in the detail. Mark Ballard explains:

Universal Credit would not only re-engineer the complex administration of £70bn social security payments to 8m households, merging six benefits systems across two government departments and local authorities throughout the country. It would also rely on councils up and down the country making their own systems and processes compatible. It would depend upon HM Revenue & Customs completing its own income tax system reforms of unprecedented ambition, Real-Time Information. And HMRC would in turn depend on employers, banks and payroll software suppliers reforming their computer systems and processes as well. Universal Credit would have to navigate this exponentially explosive collection of risk factors. This was a spaghetti junction of high-stakes computer gambits from which Universal Credit would have to pull a benefits payment that was reliable and secure. (Mark Ballard, Universal Credit failures put coalition ICT strategy in purdah. Computer Weekly September 2013)

What went wrong? Agile Development? Procurement? Interdepartmental warfare? Bureaucracy?

“There was an extremely strong command and control culture at the DWP, which goes against agile. We were trying to alleviate that – but it wasn’t working,” said a source. “The fundamental problem was procurement,” said an anonymous participant. “Our hands were tied because of procurement. If you don’t set up the contract properly, you are on a hiding to nothing.” (Mark Ballard, Why agile development failed for Universal Credit, Computer Weekly July 2013)

Iain Duncan Smith is stuck between using traditional waterfall and systems integration, advocated by DWP, and “agile hacking”, advocated by GDS.  (Bryan Glick, The question that matters on Universal Credit: Do you believe Iain Duncan Smith? Computer Weekly Feb 2014, comment by John Alexander)

Any system which is designed to operate with a smaller bureaucracy will be opposed tooth and nail by the bureaucrats mandated to implement it. (Benedict Brogan, Whitehall is shuddering over Universal Credit problems Telegraph February 2014, comment by Littlegrayman)

And was it just poor execution, or was the vision faulty? Paul Spicker thinks the whole project was doomed.

It is easy to blame the IT when things go wrong, but when people are asked to do impossible things, it should not be surprising if they do not deliver. … The most basic flaw rests in the idea that we can “personalise” benefits for millions of people. There are just too many moving parts; and in a system with millions of iterations, anything that can go wrong will go wrong. (Paul Spicker, Universal Credit: Don’t blame the IT Computer Weekly Feb 2014)

(I want to make two observations here. Firstly, requirements are not additive. Even if each requirement makes sense on its own, that doesn’t mean that the whole set of requirements makes sense. And secondly, requirements that make sense in some contexts don’t make sense in other contexts. For example, retail may be able to achieve a good level of personalization, because it doesn’t matter much if a customer sometimes gets the “wrong” promotional voucher. But it matters a lot if people get the “wrong” benefits, so the stakes are much higher.)

And even the Spectator (a right-wing weekly magazine) complains about ministerial incompetence and inflexibility.

The Universal Credit fiasco exemplifies Duncan Smith’s narcissistic failure to admit and remedy mistakes. As Computer Weekly — a far better guardian of the taxpayer than the Conservative backbenches or press, incidentally – has said, Duncan Smith proceeded with a vast and complicated IT project without learning the lessons from the IT disasters of the Labour years. (Nick Cohen, The conservative case against Iain Duncan Smith, Spectator June 2014)

John Naughton compares Universal Credit with Obamacare, and detects the same problem with both.

How is it that governments stuffed with able and conscientious civil servants screw up so spectacularly whenever IT is involved? … The strange thing about this is that you wouldn’t need to have been a geek to spot the problem with healthcare.gov. You just had to think architecturally about it. Yet apparently nobody in the administration did. The same applies to the post-9/11 decision to link all the previously separate US security databases into one giant file to which at least 250,000 people had access, one of whom happened to be Bradley (now Chelsea) Manning. (John Naughton, Obamacare, universal credit… why do governments make such a mess of IT? Observer December 2013)

You don’t have to be a geek. You just have to think architecturally. And thinking architecturally means – among other things – anticipating and resolving the kind of problems faced by projects like these.

Standing up for the universal point of view doesn’t justify a Stalinist approach to smoothing out the unavoidable complexities of dealing with real individuals and their messy lives. Is this a failure for IT, or a failure for bureaucracy? And can we tell the difference?

Updated 3 July 2014