(Note: I’ve added an addendum to this post)
It has been many years that we have lived with the concept of technical debt. Martin Fowler did a good job of describing the concept in a 2003 article on his wiki. Basically, the idea is that you …
(Note: I’ve added an addendum to this post)
As I’m about to complete my share of a longer engagement on using Lean principles to improve the processes at an online services firm, it occurred to me that the efforts we undertook to properly embed Architecture practices into their Scrum process were novel. I haven’t seen much written about how to do this in practice, and I imagine others may benefit from understanding the key connection points as well. Hence this post.
First off, let me be clear: Agile software development practices are not at all averse to software architecture. But let’s be clear about what I mean by software architecture. In an agile team, most decisions are left to the team itself. The team has a fairly short period of time to add a very narrow feature (described as a user story) to a working base of code and demonstrate that the story works. The notion of taking a couple of months and detailing out a document full of diagrams that explains the architecture of the system: pretty silly.
The value of software architecture is that key decisions are made about the core infrastructure of the system itself: where will generalizations lie? Will a layered paradigm be used, and if so, what are the responsibilities of each layer? What modules will exist in each layer and why will they be created? How will the responsibilities of the system be divided up among the layers and components? How will the modules be deployed at scale? How will information flow among the modules and between the system and those around it?
The way these questions are answered will indicate what the architecture of the system is. There are many choices here, and the “correctness” of any choice is a balance between competing demands: simplicity, security, cost, flexibility, availability, reliability, usability, correctness, and many more. (These are called “System Quality Attributes”). Balancing between the system quality attributes takes thought and careful planning.
So when does this happen in an agile process?
Let’s consider the architect’s thinking process a little. In fact, let’s break the software architecture process into layers, so that we can divide up the architectural responsibility a little. You have three layers of software architectural accountabilities. (Repeat: I’m talking about Software Architecture, not Enterprise Architecture. Please don’t be confused. Nothing in this post is specific to the practice of Enterprise Architecture). All this is illustrated in the diagram below. (Click on the diagram to get something a little more readable.
At the top, you have the Aligning processes of software architecture. These processes consider the higher levels of enterprise architecture (specifically the business and information architecture) to create To-Be Software Models of the entire (or relevant) software ecosystem. If you’ve ever seen a wall chart illustrating two dozen or more software systems with connectors illustrating things like flow of data or dependencies, you’ve seen a model of the type I’m referring to. Creating and updating these kinds of diagrams is a quarterly or semi-annual process and reflects the gradual changes in the strategy of the enterprise.
In the middle, you have the Balancing processes of software architecture. These processes consider the needs of a single system but only from the level of deciding why the software will be divided up into modules, layers, and components, how that division of responsibility will occur, and what the resulting system will look like when deployed in specific technologies in a specific environment. All of this can be conveyed in a fairly short document that is rich in diagrams with a small amount of text explaining the choices. This occurs once when a system is moving forward, and the software architecture can be developed alongside the first couple of sprints as input to the third and subsequent sprints.
At the bottom, you have the Realization processes of software architecture. This is where the architecture becomes software, and this is where decisions are made about the choice of specific design patterns, the appropriate level of configurability vs. simplicity, and the ability to demonstrate whether the actual intent of the architecture occurs in practice. In agile, this layer occurs within the team itself. The software architect can offer advice about what patterns to use, but it is up to the team to realize that advice and/or decide not to implement it. The team will very likely implement the software architecture as described, but may choose to improve upon it.
What does the process look like
There are many visualizations of scrum running around. Some are described in books, others in papers or blog posts. Most share some common elements. There is a product backlog that, through the magic of sprint planning, the team extracts a subset for the sprint. This becomes the sprint backlog. The illustrations then tend to show the various rhythms of Sprint as cycles (sprint cycles and daily cycles), ending with a demonstration and retrospective.
In order to illustrate and train a team on all elements, including the business analysis elements, we chose to be a bit more explicit about the steps PRIOR to sprint planning, including the processes of creating and improving the stories prior to the start of a sprint. (as above, click on the image to enlarge it).
Astute observers will notice that we added a step that we are calling “pre-sprint story review.” This is a meeting that occurs one week prior to the start of a sprint. It is called by the product owner and he or she invites “senior chickens” (architects, user experience leads, development and test leads, and any other “non-team” members that want a say in the sprint.
In that week prior to sprint planning, those folks, working with the product owner, can improve the stories, add constraints, refine the description and acceptance criteria. And here’s where the architects get to play. Architects fulfilling the role of “Balancing” in the model above will have (or can create) an outline document describing the architecture of the software system, and can “link” that document to the SPECIFIC STORIES that are impacted by that design.
(Note: assuming you are using a tool like Microsoft’s Team Foundation Server, that fully enables Scrum in multiple forms, this is a nearly trivial activity since a document can be easily linked to any story. Enough advertising.)
So is an architect a chicken or a pig? Answer: depends on what “layer” the architecture is at. The top two layers are chickens. The third layer, realization, is done by the team itself. The person on the team may or may not have the title of “designer”. (I’d prefer that they did not, but that’s just because I believe that ALL team members should be trained to fulfill that role. In reality, the skill may not be wide spread among team members). Therefore, the third layer is done by the pigs.
I hope this provides some insight into how a team can embed software architecture into their scrum cycles. As always, I’m interested in any feedback you may wish to share.
I’ve been looking at different ways to implement the ATAM method these past few weeks. Why? Because I’m looking at different ways to evaluate software architecture and I’m a fan of the ATAM method pioneered at the Software Engineering Institute at Carnegie Mellon University. Along the way, I’ve realized that there is a flaw that seems difficult to address.
Different lists of criteria
The ATAM method is not a difficult thing to understand. At it’s core, it is quite simple: create a list of “quality attributes” and sort them into order, highest to lowest, for the priority that the business wants. Get the business stakeholders to sign off. Then evaluate the ability of the architecture to perform according to that priority. An architecture that places a high priority on Throughput and a low priority on Robustness may look quite different from an architecture that places a high priority on Robustness and a low priority on Throughput.
So where do we get these lists of attributes?
A couple of years ago, my colleague Gabriel Morgan posted a good article on his blog called “Implementing System Quality Attributes.” I’ve referred to it from time to time myself, just to get remind myself of a good core set of System Quality Attributes that we could use for evaluating system-level architecture as is required by the ATAM method. Gabriel got his list of attributes from “Software Requirements” by Karl Wiegers.
Of course, there are other possible lists of attributes. The ISO defined a set of system quality attributes in the standard ISO 25010 and ISO 25012. They use different terms. Instead of System Quality Attributes, there are three high level “quality models” each of which present “quality characteristics.” For each quality characteristic, there are different quality metrics.
Both the list of attributes from Wiegers, and the list of “quality characteristics” from the ISO are missing a key point… “Time to release” (or time to market).
The missing criteria
One of the old sayings from the early days of Microsoft is: “Ship date is a feature of the product.” The intent of this statement is fairly simple: you can only fit a certain number of features into a product in a specific period of time. If your time is shorter, the number of features is shorter.
I’d like to suggest that the need to ship your software on a schedule may be more important than some of the quality attributes as well. In other words, “time-to-release” needs to be on the list of system quality attributes, prioritized with the other attributes.
How is that quality?
I kind of expect to get flamed for making the suggestion that “time to release” should be on the list, prioritized with the likes of reliability, reusability, portability, and security. After all, shouldn’t we measure the quality of the product independently of the date on which it ships?
In a perfect world, perhaps. But look at the method that ATAM proposes. The method suggests that we should created a stack-ranked list of quality attributes and get the business to sign off. In other words, the business has to decide whether “Flexibility” is more, or less, important than “Maintainability.” Try explaining the difference to your business customer! I can’t.
However, if we create a list of attributes and put “Time to Release” on the list, we are empowering the development team in a critical way. We are empowering them to MISS their deadlines of there is a quality attribute that is higher on the list that needs attention.
For example: let’s say that your business wants you to implement an eCommerce solution. In eCommerce, security is very important. Not only can the credit card companies shut you down if you don’t meet strict PCI compliance requirements, but your reputation can be torpedoed if a hacker gets access to your customer’s credit card data and uses that information for identity theft. Security matters. In fact, I’d say that security matters more than “going live” does.
So your priority may be, in this example:
- Testability, and
This means that the business is saying something very specific: “if you cannot get security or usability right, we’d rather you delay the release than ship something that is not secure or not usable. On the other hand, if the code is not particularly maintainable, we will ship anyway.”
Now, that’s something I can sink my teeth into. Basically, the “Time to Release” attribute is a dividing line. Everything above the line is critical to quality. Everything below the line is good practice.
As an architect sitting in the “reviewer’s chair,” I cannot imagine a more important dividing line than this one. Not only can I tell if an architecture is any good based on the criteria that rises “above” the line, but I can also argue that the business is taking an unacceptable sacrifice for any attribute that actually falls “below” the line.
So, when you are considering the different ways to stack-rank the quality attributes, consider adding the attribute of “time to release” into the list. It may offer insight into the mind, and expectations, of your customer and improve your odds of success.
There are benefits to taking an idea from one domain and applying it to another. We all know of the famous case of “software patterns” that emerged from the concept of architectural patterns developed by Christopher Alexander for the world of…