The digital enterprise at all costs does not pay
If we look at the digital the way we look at enterprise architecture, we have to project the big picture at the end of the tunnel, the digital enterprise.
Aggregated enterprise architecture wisdom
If we look at the digital the way we look at enterprise architecture, we have to project the big picture at the end of the tunnel, the digital enterprise.
Organizations do not work, in real life, like they work on paper. On paper, there are departments (all shaped like a neat rectangle) and business processes with neat inflows and outflows of responsibility and information. On paper, you improve things by modeling things on paper, and then moving things around, on paper, then teaching people to follow the process that your neat paper diagrams represent.
In real life, there are human beings and the tools that they use. Sometimes the tools move information from one person to another. Sometimes, they just aid in communication. People meet and get to know other people, and they learn to trust some, and distrust others. Some folks have different measures and motivations and just “pass by” one another. Some subset of these people will have shared cultural values and expectations. There may be many cultures in an organization: both because the organization is in multiple places, and because people from multiple places join an organization. Also, “business culture” arises as leaders achieve successes and people learn to use certain “cultural expectations” to get things done efficiently.
Reality is a lot messier than pretty rectangles.
Enterprise Architects apply science and engineering and aesthetics to the challenge of organizational change. We are unique in that most other “change artists” are not focused on engineering and some even ignore science. (see Daniel Pink’s TED Talk on the Surprising Science of Motivation). Few even know how to spell aesthetics. Yet, when you are dealing with systems that contain and include people, you have to use aesthetics, and you are ill prepared for success if you ignore science. Engineering is a mindset as much as a class of methods. It involves applying the things that science has discovered and using that understanding to build great (and sometimes terrible) things. Engineers build on ideas and use them, often experimenting on systems that are too complex and intertwined for “pure science” to get arms around.
As Enterprise Architecture is such a young science, we have relied to heavily on the “boxes and lines” model of enterprises, and not enough on the messy but important sociocultural view of an enterprise. We find it easier to document, and model, and even simulate, processes as though people were interchangeable and their relationships didn’t matter.
That is just lazy.
It is time to get up off our collective butts and start working out ways to understand sociocultural influences, relationships, and architectures. We have to build ways to detect, measure, and consider these structures when we measure capabilities, or improve processes, or suggest automations, or evaluate business models, or any of the two dozen things that EA’s do.
The value of EA often comes to an executive in the form of a reasoned opinion that is based on data that no one else is looking at. Let’s consider the possibility that examining sociocultural influences can provide interesting opinions that an executive will find valuable.
We should consider sociocultural information if:
Think about it. Do you believe that any of those statements are false? I can find ample examples for each one. So if sociocultural interactions matter, why are we not tracking them, learning from them, using them to make decisions?
It’s only hard because we haven’t tried.
(This post inspired by the many similar pleas shared by J.D. Beckingham in social media).
Organizations do not work, in real life, like they work on paper. On paper, there are departments (all shaped like a neat rectangle) and business processes with neat inflows and outflows of responsibility and information. On paper, you improve things by modeling things on paper, and then moving things around, on paper, then teaching people…
As I’m about to complete my share of a longer engagement on using Lean principles to improve the processes at an online services firm, it occurred to me that the efforts we undertook to properly embed Architecture practices into their Scrum process were novel. I haven’t seen much written about how to do this in practice, and I imagine others may benefit from understanding the key connection points as well. Hence this post.
First off, let me be clear: Agile software development practices are not at all averse to software architecture. But let’s be clear about what I mean by software architecture. In an agile team, most decisions are left to the team itself. The team has a fairly short period of time to add a very narrow feature (described as a user story) to a working base of code and demonstrate that the story works. The notion of taking a couple of months and detailing out a document full of diagrams that explains the architecture of the system: pretty silly.
The value of software architecture is that key decisions are made about the core infrastructure of the system itself: where will generalizations lie? Will a layered paradigm be used, and if so, what are the responsibilities of each layer? What modules will exist in each layer and why will they be created? How will the responsibilities of the system be divided up among the layers and components? How will the modules be deployed at scale? How will information flow among the modules and between the system and those around it?
The way these questions are answered will indicate what the architecture of the system is. There are many choices here, and the “correctness” of any choice is a balance between competing demands: simplicity, security, cost, flexibility, availability, reliability, usability, correctness, and many more. (These are called “System Quality Attributes”). Balancing between the system quality attributes takes thought and careful planning.
So when does this happen in an agile process?
Let’s consider the architect’s thinking process a little. In fact, let’s break the software architecture process into layers, so that we can divide up the architectural responsibility a little. You have three layers of software architectural accountabilities. (Repeat: I’m talking about Software Architecture, not Enterprise Architecture. Please don’t be confused. Nothing in this post is specific to the practice of Enterprise Architecture). All this is illustrated in the diagram below. (Click on the diagram to get something a little more readable.
At the top, you have the Aligning processes of software architecture. These processes consider the higher levels of enterprise architecture (specifically the business and information architecture) to create To-Be Software Models of the entire (or relevant) software ecosystem. If you’ve ever seen a wall chart illustrating two dozen or more software systems with connectors illustrating things like flow of data or dependencies, you’ve seen a model of the type I’m referring to. Creating and updating these kinds of diagrams is a quarterly or semi-annual process and reflects the gradual changes in the strategy of the enterprise.
In the middle, you have the Balancing processes of software architecture. These processes consider the needs of a single system but only from the level of deciding why the software will be divided up into modules, layers, and components, how that division of responsibility will occur, and what the resulting system will look like when deployed in specific technologies in a specific environment. All of this can be conveyed in a fairly short document that is rich in diagrams with a small amount of text explaining the choices. This occurs once when a system is moving forward, and the software architecture can be developed alongside the first couple of sprints as input to the third and subsequent sprints.
At the bottom, you have the Realization processes of software architecture. This is where the architecture becomes software, and this is where decisions are made about the choice of specific design patterns, the appropriate level of configurability vs. simplicity, and the ability to demonstrate whether the actual intent of the architecture occurs in practice. In agile, this layer occurs within the team itself. The software architect can offer advice about what patterns to use, but it is up to the team to realize that advice and/or decide not to implement it. The team will very likely implement the software architecture as described, but may choose to improve upon it.
What does the process look like
There are many visualizations of scrum running around. Some are described in books, others in papers or blog posts. Most share some common elements. There is a product backlog that, through the magic of sprint planning, the team extracts a subset for the sprint. This becomes the sprint backlog. The illustrations then tend to show the various rhythms of Sprint as cycles (sprint cycles and daily cycles), ending with a demonstration and retrospective.
In order to illustrate and train a team on all elements, including the business analysis elements, we chose to be a bit more explicit about the steps PRIOR to sprint planning, including the processes of creating and improving the stories prior to the start of a sprint. (as above, click on the image to enlarge it).
Astute observers will notice that we added a step that we are calling “pre-sprint story review.” This is a meeting that occurs one week prior to the start of a sprint. It is called by the product owner and he or she invites “senior chickens” (architects, user experience leads, development and test leads, and any other “non-team” members that want a say in the sprint.
In that week prior to sprint planning, those folks, working with the product owner, can improve the stories, add constraints, refine the description and acceptance criteria. And here’s where the architects get to play. Architects fulfilling the role of “Balancing” in the model above will have (or can create) an outline document describing the architecture of the software system, and can “link” that document to the SPECIFIC STORIES that are impacted by that design.
(Note: assuming you are using a tool like Microsoft’s Team Foundation Server, that fully enables Scrum in multiple forms, this is a nearly trivial activity since a document can be easily linked to any story. Enough advertising.)
So is an architect a chicken or a pig? Answer: depends on what “layer” the architecture is at. The top two layers are chickens. The third layer, realization, is done by the team itself. The person on the team may or may not have the title of “designer”. (I’d prefer that they did not, but that’s just because I believe that ALL team members should be trained to fulfill that role. In reality, the skill may not be wide spread among team members). Therefore, the third layer is done by the pigs.
I hope this provides some insight into how a team can embed software architecture into their scrum cycles. As always, I’m interested in any feedback you may wish to share.
Without the capability of linking application development with the infrastructure such as servers, middleware and network your organization will most likely be in more trouble than the decision-makers think. Information technologies have a profound impact on the organization’s ability to reinvent its business model(s) and the products it produces and since organizations in various industries […]
I did a scan around the web to figure out what many of the leading thinkers were saying about IT project failure and the root causes. Numbers varied between 20% and 80% of projects failing to deliver on their business case. The root cause analysis that follows from these failure numbers spends a lot of time looking at the IT project, but most forget to look at the causes of failure that are outside the project’s control.
In the diagram below, the central blue box represents causes of IT project failure that are inside the control of the IT team. As you can see, there are many more causes of IT project failure that are OUTSIDE that blue box than inside it. Yet, countless articles have been written on the factors INSIDE the box. I think it is time we take a slightly wider lens to the problem!
The factors outside the project are as important, or more important, than the ones inside the box. I have worked on many projects over the years, and if I look back at the ones that ended up cancelled or scrapped, the reasons were not ones in the blue box. They were usually ones from the top box: where the project should not have begun in the way that it did. Let’s look at these six factors:
Each of these conditions has the potential to kill an IT project. I would suggest that MANY IF NOT MOST of the failures of IT projects can be traced to one or more of these conditions, but these conditions rarely get counted in the statistics for “Causes of IT Project Failure.” Why? Because, in most cases, projects that suffer from these conditions are either never funded, or are reworked so that the political problem is simply avoided. The project business case does not reflect the problem, so the criteria for failure (doesn’t meet the business case) is never met. Efforts are made to avoid (but not address) the problem before the business case is written!
This is the world of the Enterprise Architect. These are the kinds of “failure” that fill the eyes and ears of an Enterprise Architect. If an EA focuses on only these six causes, he or she will deliver real, tangible, and unique value to their enterprise without ever overlapping the roles and responsibilities of an IT architect, business analyst, or technologist.
Big data will continue to spread with emerging associative terms like big data expert, big data technologies, etc. Here are E.G. Nadhan’s top 5 reasons why the term big data has stuck, and why it may be appropriate, after all… Continue reading →
A vendor recently asked me a question, and not for the first time, that amounted to “why won’t you buy these delicious software licenses for our amazing identity-management suite, which provides comprehensive solutions to the challenges faced by North American corporations in dealing with compliance requirements and bringing together disconnected versions of identity from across […]
Steady growth of service oriented practices and the continued adoption of cloud computing across enterprises has resulted in the need for integrating out to the cloud. When doing so, we must take a look back in time at the evolution of integration so…
Enterprise Service Bus and Espresso One of the few really-good boutique coffee-roasters and espresso specialists in Auckland labels its beans “esb”, for “ethically-sourced beans”. It’s something of a stretch, but espresso is important and beneficial in fundamental, foundational ways not dissimilar to the importance and benefits an Enterprise Service Bus can bring to integrated organisations. […]
Value management through TOGAF 9.1 and ArchiMate 2.0. Continue reading →