10 months, 15 days ago

The Digital Future: Services Oriented Architecture and Mass Customization, Part 3 B

From Previous PartsPart 1 discussed the four ages of mankind.  The first was the Age of Speech; for the first time humans could “learn by listening” rather than “learn by doing”; that is, data could be accumulated, communicated, and stored by ve…

1 year, 9 months ago

Hurricanes and an Architecture for Process

The idea for this post came from a previous post on this blog regarding enterprise architecture and religious organizations.

Definitions or Definitional Taxonomies

There are three definitions that are needed to understand this post; the definition of Architecture, process using the OODA Loop, and the Hierarchy of Knowledge.  These are somewhat personnel definitions and you may interpolate them into your model universe.

Architecture

Architecture—A functional design of a product, system, or service incorporating all the “must-do” and “must-meet” requirements for that product, system, or service.

OODA Loop

All processes supporting the mission, strategies, or infrastructure of any organization fall into the OODA model of Col. John Boyd.  The OODA, as discussed in several previous posts includes four steps: Observe, Orient, Decide, and Act.  I will discuss this shortly (below) using the example of the prediction of a hurricane.

A Taxonomy for the Hierarchy of Knowledge

I defined the following taxonomy of knowledge based on much reading and some thinking.  I will demonstrate what I mean for each definition by using it in the appropriate place in my example of the hurricane prediction.
Datum—some observation about the Universe at a particular point in four dimensions

Data—a set consistent datum

Information—patterns abstracted from the data.

Knowledge—identified or abstracted patterns in the information.

Wisdom—is the understanding of the consequences of the application of knowledge

Process Architecture and Forecasting Hurricanes

Since the process for predicting or forecasting hurricanes is most likely to be familiar to anyone watching weather on TV, it is the best example I can think of to illustrate how the OODA Loop process architecture works with the taxonomy of knowledge.

Observe

Initially, data is gathered by observing some aspect of the current state of the Universe.  This includes data about the results of their previous actions.  In point of fact,

Datum—some observation about the Universe at a particular point in four dimensions

Data—a set consistent datum

Obviously observations of current temperature, pressure, humidity, and so on, are basic to making weather forecasts, like the prediction of a hurricane.  And each observation requires latitude, longitude, height above sea level, and the exact time to be useful. So one datum, or data point would include, temperature, latitude, longitude, height, and time.

When the temperature is measured at many latitudes and longitudes concurrently, it produces a set of temperature data.  If other weather measurement, like pressure and humidity, are taken at the same locations at the same time, then you have several data sets with which to make a forecast (or a composite weather data set for a particular time).  Because this is a recurrent measurement process, the weather folk continue to build a successively larger composite weather data set.  But large data sets don’t make a forecast of a hurricane.

Orient

The Orient step in the process is inserting the new data into an individual’s model of how the world (Universe) works (descriptive) or should work (prescriptive).  These models are sometimes called paradigms.  Rules from Governance enable and support the Orient step, by structuring the data within the individual’s or organization’s model.  Sometimes these model or paradigms stick around far after they have demonstrated to be defective, wrong, or in conflict with most data and other information.  An example would be the model of the Universe of the Flat Earth society.

Information—patterns abstracted from the data.  This is the start of orienting the observations, the data and information.  Pattern analysis, based on the current model converts data into information is derived from the organization’s model of its environment or Universe.  For hurricane forecasting this would mean looking for centers of low pressure within the data.  It would also include identifying areas with high water temperatures in the temperature data, areas of high humidity, and so on.  This and other abstractions from the data provide the information on which to base a forecast.  But it is still not the prediction of a hurricane.

Knowledge—identified, combined, or abstracted patterns in the information.  Using the same paradigm, environmental model, or model Universe, people analyze and abstract patterns within the information.  This is their knowledge within the paradigm.  In weather forecasting the weather personnel uses the current paradigms (or information structures) to combine and abstract the knowledge that a hurricane is developing.

When they can’t fit information into their model, they often discard as aberrant, an outlier, or as an anomaly.  When enough information doesn’t conveniently fit into their model the adherents have a crisis.  In science, at least, this is the point of a paradigm shift.  And this is what has happened to weather forecasting models over the past one hundred and fifty years.  The result is that the forecasting has gotten much better, though people still complain when a storm moves off its track by fifty miles and causes unforecast wind, rain, and snow events.

Decide

Once the organization or individual has the knowledge, he or she uses input their knowledge within their models of the Universe to make decisions.

Wisdom—is the understanding of the consequences of the application of knowledge. 

This is the hard part of the OODA Loop because it’s difficult to understand both the consequences and the unintended consequences of a decision.  If your paradigm, environment, or Universe model is good, or relatively complete, then you’re more likely to make a good decision.  More frequently than not people make decisions that are “Short term smart and long term dumb.” 

Part of the reason is that they are working with a poor, incomplete or just plain wrong paradigm (view of the world or universe).  This is where the Risk/Reward balance comes in.  When choosing a path forward, what are the risks and rewards with each path?  [Sidebar:  A risk is an unknown and it is wise to understand that “you don’t know what you don’t know”.]  For weather forecasters it’s whether or not to issue a forecast for a hurricane.  To ameliorate the risk that the are wrong, weather forecasters have invented a two part type of forecast, a hurricane watch and a hurricane warning. [Sidebar: It’s split more by giving a tropical storm watch and warning.]

The reason for this warning hierarchy is that the weather forecasters and services are wise enough to know the public’s reaction to warnings that don’t pan out and lack of warnings that do; they understand the risks.  So when they give a hurricane warning, they are fairly sure that the area they have given the warning for will be the area affected by the hurricane.

Act

Once the decision is made people act on those decisions by planning a mission, strategies, and so on within their paradigm.

For governments and utilities in the U.S. this means putting preplanned hurricane preparations into effect.  For people this generally means boarding up the house, stocking up on food, water, and other essentials or packing and leaving.  And for the drunk beach comber this means grabbing the beers out of the frig, and either heading to the hills or back to the beach to watch the show.

Programmed and Unprogrammed Processes

In the 1980s I wrote a number of articles on process types.  After doing much research and much observation, I came to the obvious conclusion that there are two types of processes, Programmed, and those that I will call Unprogrammed and which includes design, discovery, and creative processes. [Sidebar: These three sub-types differ in that design processes start from a set of implicit or explicit customer requirements, discovery processes start from inconsistencies in data or in thinking and modeling the data, and creative really have no clear foundation and tend to be intuitive, chaotic, and apparently random.] 

Programmed Processes

Programmed processes are a set of repeatable activities leading to a nearly certain outcome.  Almost all processes are of this type.  The classic example is automobile manufacturing and assembly.  Since they are repeatable how does the process architecture model apply?
The short answer is to keep them repeatable and to increase their cost efficiency (that is making them less expensive in terms of time and money to create the same outcome).

Keeping Programmed Processes Rolling

To keep your truck or car operating requires maintenance.  How much and when is an OODA Loop process.  For example, you really should check your tires, both for wear and pressure regularly; otherwise the process of driving can become far too exciting.  So you are in the “observe” step.
As the tires wear out or lose pressure too often, the data set becomes information triggering the “orient” step in the process.  At this point a driver starts to gather data on which tires are wearing fastest and where on the tire they are wearing.  This last point may indicate that tire rotation is all that’s needed.
Based on this data our driver models the information (using the computer between his or her ears) to determine how much longer it’s safe to drive on the tires.  Based on results of that model, the driver will “decide” to buy tires, rotate tires, or wait a little longer.  [Sidebar: Actually, many drivers will go through the buy/wait decision based on additional data, the cost of new tires, and their budget.]  The driver will then “act” on the decision, either get new tires or wait (which is really an action).
The point here is that like everything else in our Universe, over time, everything breaks down; so repeatable processes require an OODA Loop process to maintain them.

More Cost Efficient

However, the OODA Loop process architecture elucidated here is important for programmed processes for another reason.  For repeatable processes the key OODA process is always how to make the faster, cheaper, or making a higher quality product at the same or lower price, and producing it in the same or less time.
This is famously what Adam Smith described in his pin example in the first chapter of “The Wealth of Nations”. By creating an assembly line process that he called “the division of labour”, he demonstrated that the same number of workers produce hundreds of more pins.  He went on to describe that each worker might now invent (or innovate, that is, refine or enhance) tools for that worker’s job. [Sidebar: Henry Ford went on to prove this in spades or should I say Model-T’s.]  [Sidebar: To me the worker’s inventing or innovating tools to help them in their job is quite interesting and often missed.  Tools are process or productivity multipliers.  In the military they are called force multipliers; you always want your enemy to bring a knife to your gunfight, because a rifle multiplies your force.  Likewise, the tooling in manufacturing can increase the quality of the product, the uniformity of what is produced, while reducing the cost and time to produce…fairly obvious.]
Both the “division of labour” and the creation of tooling require the use of the Architectural OODA Loop, which means that increasing the cost efficiency of manufacturing uses the OODA Loop with the knowledge taxonomy.

Unprogrammed Process

Unprogrammed processes have stochastic (creative and unplanned activities within them).  There are really three sub-types of unprogrammed processes, design, discovery, and creative. The key differences between the design, discovery, and creative processes are the whether the process has customer requirements driven and what the requirements are.

Design

The design process has customer requirements.  [Sidebar: As I’m using it, the design process also includes custom development, implementation, and manufacturing.]  It uses a semi-planned process, that is, program or project planning creates a plan to meet the objectives, but with latitude for alternative activities because there is significant risk.  The actual design activities within the programmed process are stochastic, from a program management perspective.  That is, the creative element of these activities makes less predictable, and therefore with programmatic risk with respect to cost and schedule.  Therefore, program management must use (some form or) the OODA architecture to manage the program or project.
The stochastic activities are themselves OODA Loop processes.  The designers have to identify (observe) detailed data of the functions they are attempting to create (the “must do” requirements), while working to the specifications (the”must meet” requirements) for the product, system, or service.  The designers then have to “orient” these (creating a functional design or architecture for the function), “decide” which of several alternate proposed designs is “the best”. Finally, they “act” on the decision.

Discovery

The requirements for research, risk reduction, and root cause analysis are generally unclear and may be close to non-existent.  One of my favorite research examples, because it’s well known, and because it so clearly proves the point is the Wright Brothers researching and developing the aircraft.  In 1898 the brothers started their research efforts.  Starting with data and information then publicly available they built a series of kites. With each kite, they collected additional data and new types of data.  They then used this to reorient their thinking and their potential design for the next kite.  They found so many problems with the existing data that they created the wind tunnel to create their own data sets on which to base their next set of designs.  By the end of 1902 they had created a glider capable of controllable manned flight and by 1903 they created the powered glider known as the Wright Flyer.  It took them at least two more cycles of the OODA Loop to develop a commercially useful aircraft.
Risk reduction also uses the OODA Loop.  A risk is an unknown.  It requires some type of research to convert the unknown into a known.  There are four alternative.  First, the project/research team must decide whether or not to simply “accept” the risk. Many times the team orients the observed risk in a risk/reward model and accepts the risk.  However, another to orient is to determine if there is knowledge or knowledgeable people to convert the unknown into a known. So “transferring” the risk is one way to reduce the risk.  A second method is to “avoid” the risk.  This means redesigning the product, system, or service, or changing the method for achieving the goal or target.  The final way is to “mitigate” the risk.  This is nothing short of or more than creating a research project (see above) to convert the unknown into a Known.
Likewise, root cause analysis is a research effort.  However, the target of this analysis is to identify why some function or component is not working the way it is supposed to work.  Again, its observing the problem or issue, that is gathering data, orienting it through modeling the functions based on the data, deciding what the cause is or causes are for the problem and how to “fix” the problem, and then acting on the decision.  Sound a whole lot like the OODA Loop.

Creative

Creative processes like theory building (both through extrapolation and interpolation), and those of the “fine arts” that come from emotions also use the Architecture of Process defined in this post.  For some theories and all fine arts, “the requirements” come from emotion, based on an intuitive belief structure [Sidebar: a religious structure, see my post on architecture of religion].  This intuitive structure provides “the data, information, and knowledge” on which the creativity builds.
However, for scientific theories, at least, they are sometimes based on inconsistencies in the results of the current paradigm.  The theorist attempts to define a new structure that will account for these aberrant data.

Why make a Deal about the Obvious?

To many it might seem that I’m making a big deal about what is obvious.  If it is the case, why hasn’t it been inculcated into enterprise architecture and used in the construction and refinement of processes and tooling to support the charter, mission, objective, strategies and tactics of all organizations?  Using formal architectural models like the architecture for process, presented in this post, will enable the enterprise architects to “orient” their data and information more clearly making Enterprise Architecture that much more valuable.
2 years, 1 month ago

Agility is not Speed

Agility is the ability to change direction quickly.

The paradox of agility is that it is that the slower you are moving the faster you can change direction.

Just going really fast, in the belief that speed is agility, can lead to a nasty, sudden stop.

image

Photo by Dan Masa

https://www.flickr.com/photos/danmasa/27160870210/in/dateposted/

creativecommons.org 2.0

3 years, 27 days ago

Agile Development And Data Management Do Coexist

A frequent question I get from data management and governance teams is how to stay ahead of or on top of the Agile development process that app dev pros swear by. New capabilities are spinning out faster and faster, with little adherence to ensuring compliance with data standards and policies.

Well, if you can’t beat them, join them . . . and that’s what your data management pros are doing, jumping into Agile development for data.

Forrester’s survey of 118 organizations shows that just a little over half of organizations have implemented Agile development in some manner, shape, or form to deliver on data capabilities. While they lag about one to two years behind app dev’s adoption, the results are already beginning to show in terms of getting a better handle on their design and architectural decisions, improved data management collaboration, and better alignment of developer skills to tasks at hand.

But we have a long way to go. The first reason to adopt Agile development is to speed up the release of data capabilities. And the problem is, Agile development is adopted to speed up the release of data capabilities. In the interest of speed, the key value of Agile development is quality. So, while data management is getting it done, they may be sacrificing the value new capabilities are bringing to the business.

Let’s take an example. Where Agile makes sense to start is where teams can quickly spin up data models and integration points in support of analytics. Unfortunately, this capability delivery may be restricted to a small group of analysts that need access to data. Score “1” for moving a request off the list, score “0” for scaling insights widely to where action will be taking quickly.

Read more

3 years, 27 days ago

Agile Development And Data Management Do Coexist

A frequent question I get from data management and governance teams is how to stay ahead of or on top of the Agile development process that app dev pros swear by. New capabilities are spinning out faster and faster, with little adherence to ensuring compliance with data standards and policies.

Well, if you can’t beat them, join them . . . and that’s what your data management pros are doing, jumping into Agile development for data.

Forrester’s survey of 118 organizations shows that just a little over half of organizations have implemented Agile development in some manner, shape, or form to deliver on data capabilities. While they lag about one to two years behind app dev’s adoption, the results are already beginning to show in terms of getting a better handle on their design and architectural decisions, improved data management collaboration, and better alignment of developer skills to tasks at hand.

But we have a long way to go. The first reason to adopt Agile development is to speed up the release of data capabilities. And the problem is, Agile development is adopted to speed up the release of data capabilities. In the interest of speed, the key value of Agile development is quality. So, while data management is getting it done, they may be sacrificing the value new capabilities are bringing to the business.

Let’s take an example. Where Agile makes sense to start is where teams can quickly spin up data models and integration points in support of analytics. Unfortunately, this capability delivery may be restricted to a small group of analysts that need access to data. Score “1” for moving a request off the list, score “0” for scaling insights widely to where action will be taking quickly.

Read more

5 years, 1 month ago

The Next Phase of Agile Development

Guest post by Alan Morrison It’s human nature for people to build sturdy structures to shield themselves from the unpredictability of the elements. But, if you are too sheltered for too long, you weaken your ability to continuously confront change. That’s the dilemma facing IT departments. Change is raining down on them, and they are having trouble continuously adapting. A term is gaining momentum in the IT community to describe an ideal state for IT […]

5 years, 5 months ago

Placing Architecture Properly into Scrum Processes

As I’m about to complete my share of a longer engagement on using Lean principles to improve the processes at an online services firm, it occurred to me that the efforts we undertook to properly embed Architecture practices into their Scrum process were novel.  I haven’t seen much written about how to do this in practice, and I imagine others may benefit from understanding the key connection points as well.  Hence this post.

First off, let me be clear: Agile software development practices are not at all averse to software architecture.  But let’s be clear about what I mean by software architecture.  In an agile team, most decisions are left to the team itself.  The team has a fairly short period of time to add a very narrow feature (described as a user story) to a working base of code and demonstrate that the story works.  The notion of taking a couple of months and detailing out a document full of diagrams that explains the architecture of the system: pretty silly. 

The value of software architecture is that key decisions are made about the core infrastructure of the system itself: where will generalizations lie?  Will a layered paradigm be used, and if so, what are the responsibilities of each layer?  What modules will exist in each layer and why will they be created? How will the responsibilities of the system be divided up among the layers and components?  How will the modules be deployed at scale?  How will information flow among the modules and between the system and those around it?

The way these questions are answered will indicate what the architecture of the system is.  There are many choices here, and the “correctness” of any choice is a balance between competing demands: simplicity, security, cost, flexibility, availability, reliability, usability, correctness, and many more.  (These are called “System Quality Attributes”).  Balancing between the system quality attributes takes thought and careful planning.

So when does this happen in an agile process?

Let’s consider the architect’s thinking process a little.  In fact, let’s break the software architecture process into layers, so that we can divide up the architectural responsibility a little.  You have three layers of software architectural accountabilities.  (Repeat: I’m talking about Software Architecture, not Enterprise Architecture.  Please don’t be confused.  Nothing in this post is specific to the practice of Enterprise Architecture).  All this is illustrated in the diagram below.  (Click on the diagram to get something a little more readable. 

image

At the top, you have the Aligning processes of software architecture.  These processes consider the higher levels of enterprise architecture (specifically the business and information architecture) to create To-Be Software Models of the entire (or relevant) software ecosystem.  If you’ve ever seen a wall chart illustrating two dozen or more software systems with connectors illustrating things like flow of data or dependencies, you’ve seen a model of the type I’m referring to.  Creating and updating these kinds of diagrams is a quarterly or semi-annual process and reflects the gradual changes in the strategy of the enterprise. 

In the middle, you have the Balancing processes of software architecture.  These processes consider the needs of a single system but only from the level of deciding why the software will be divided up into modules, layers, and components, how that division of responsibility will occur, and what the resulting system will look like when deployed in specific technologies in a specific environment.  All of this can be conveyed in a fairly short document that is rich in diagrams with a small amount of text explaining the choices.  This occurs once when a system is moving forward, and the software architecture can be developed alongside the first couple of sprints as input to the third and subsequent sprints.

At the bottom, you have the Realization processes of software architecture.  This is where the architecture becomes software, and this is where decisions are made about the choice of specific design patterns, the appropriate level of configurability vs. simplicity, and the ability to demonstrate whether the actual intent of the architecture occurs in practice.  In agile, this layer occurs within the team itself.  The software architect can offer advice about what patterns to use, but it is up to the team to realize that advice and/or decide not to implement it.  The team will very likely implement the software architecture as described, but may choose to improve upon it.

What does the process look like

There are many visualizations of scrum running around.  Some are described in books, others in papers or blog posts.  Most share some common elements.  There is a product backlog that, through the magic of sprint planning, the team extracts a subset for the sprint.  This becomes the sprint backlog.  The illustrations then tend to show the various rhythms of Sprint as cycles (sprint cycles and daily cycles), ending with a demonstration and retrospective.

In order to illustrate and train a team on all elements, including the business analysis elements, we chose to be a bit more explicit about the steps PRIOR to sprint planning, including the processes of creating and improving the stories prior to the start of a sprint.  (as above, click on the image to enlarge it).

image

Astute observers will notice that we added a step that we are calling “pre-sprint story review.”  This is a meeting that occurs one week prior to the start of a sprint.  It is called by the product owner and he or she invites “senior chickens” (architects, user experience leads, development and test leads, and any other “non-team” members that want a say in the sprint. 

In that week prior to sprint planning, those folks, working with the product owner, can improve the stories, add constraints, refine the description and acceptance criteria.  And here’s where the architects get to play.  Architects fulfilling the role of “Balancing” in the model above will have (or can create) an outline document describing the architecture of the software system, and can “link” that document to the SPECIFIC STORIES that are impacted by that design. 

(Note: assuming you are using a tool like Microsoft’s Team Foundation Server, that fully enables Scrum in multiple forms, this is a nearly trivial activity since a document can be easily linked to any story.  Enough advertising.)

So is an architect a chicken or a pig?  Answer: depends on what “layer” the architecture is at.  The top two layers are chickens.  The third layer, realization, is done by the team itself.  The person on the team may or may not have the title of “designer”.  (I’d prefer that they did not, but that’s just because I believe that ALL team members should be trained to fulfill that role.  In reality, the skill may not be wide spread among team members).  Therefore, the third layer is done by the pigs. 

I hope this provides some insight into how a team can embed software architecture into their scrum cycles.  As always, I’m interested in any feedback you may wish to share.

5 years, 7 months ago

Unraveling the Developer Bias in Agile Development Practices

There is a contradiction that shows up in many books and articles on agile software development.  .  And I’m going to rant a little on one of them: the “developer” bias in Agile software practices.

Before I begin my rant, I’d like to tell you where it comes from.  I am an Enterprise Architect.  I am also an agilest.  I am a certified Scrummaster (for many years) and have been on many agile projects.  I’ve seen success, and I’ve seen failure.  I know that agile is good, but not good enough to overcome people who are determined to undermine change. 

I am currently engaged on a project to help an organization rewire their business processes.  All of their core processes.  Some of them are in software development… and that is where agile comes in, but some are in other areas (sales, account management, customer service, etc).  I see it all, not just the software engineering bits.

To make this work, we created a set of principles that we use to drive the conversation around transforming the company.  We train people on the principles.  We connect change to the principles.  The principles are developed from the agile mindset but are not the same as the ones traditionally tied to the agile manifesto. 

One is focused heavily on the notion of an empowered team.  As the trainer, I’ve embedded these principles into my head.  Kinda hard not to.  So when I’m reading a book on agile practices, and the author says something that violates one of these principles, it stands out.  Big time.

And hence the rant.  I was reading Mike Cohn’s book on agile stories and I came across a passage that reflects a kind of mindset that does NOT build empowered teams.  And I went off.  The rest of this post is my rant.

On one hand, there is a basic expectation that we should empower the team to solve their problems using their skills and training.  On the other hand, there is a bias towards having developers involved in every part of a business process, from sales through requirements to project management and out to delivery and customer service.  The net result: we are empowering developers to perform every role.  No one else is empowered.  They are, in fact, not trusted at all.

At some level, agile practices like XP and Scrum are written, and promoted, by software developers, so it is understandable that these authors believe that software developers are a necessary part of every process.

But they are not. 

So what’s the problem

Organizations intentionally create specific roles and groups of people.  Those roles are defined to take the most advantage of the intrinsic motivations of people when performing their duties.  The things that motivate a person to become a project manager, for example, are rarely the same things that motivate a person to become a software engineer.  Motivation matters. If you are not motivated to excellence, you will not perform with excellence.

While it is fine for an individual person to be able to perform well in a couple of roles, it is absurd to build a system or business process that expects all people in one role to be able to perform another role well.  In specific, it is absurd to build processes where a technologist is required to develop requirements, or manage a project, or plan the delivery of systems.  Those processes can be performed just fine with motivated people who are trained and experienced in those activities.  However, to empower a team, people in these roles must be respected as well. 

Agile software, that so values people over processes and tools, demonstrates an utter lack of respect for the individuals who choose to excel in non-technical roles by asserting the need for developers to be involved in every step of their process.

For example, in XP (and sometimes in Scrum), user stories are described as being developed by a “customer team” that includes developers.  In the book “User Stories Applied,” Mike Cohn describes a process where a series of user roles are developed.  From those roles, the stories will be drawn.  The following is a quote from the book:

To identify user roles, the customer and as many of the developers as possible meet in a room with either a large table or a wall to which they can tape or pin cards.  It’s always ideal to include the whole team for the user role modeling that initiates a project, but not necessary.  As long as a reasonable representation of the developers is present along with the customer, you can have a successful session. — “User Stories Applied” by Mike Cohn

I’m an agilest, and this statement takes my breath away.  Why?  Because the SKILL of developing user roles is a good skill to have, but not a necessary skill for all developers to have.  Yet the author (whom I respect) assumes that the activity cannot be performed successfully without developers in the room!  At the same time, the text assumes that the task can be performed by developers exclusively (no one else is necessary at all).

The time that we spend training developers to be good at creating user stories is time that we are NOT spending training them on concurrent processing, idempotent messaging, and query optimization.  We don’t train business analysts or engagement managers on C# design patterns, so why should we train developers on user role modeling and requirements analysis?  It’s a total lack of understanding of the value of roles, and it is a bias that works against success.

What’s the solution: respect

 

Roles are not boundaries.  In any team, there are roles.  More importantly, as a member of a team, I have to trust my team members to fulfill their roles well.  If they need help, I TRUST THEM TO ASK FOR HELP.  When they do, I am happy to “cross roles” and provide help.  When I provide help, I am doing so from the perspective of a less-skilled resource, and under the supervision of the accountable role.  (e.g. if I “help” a tester, but don’t document a defect according to team standards, how much “help” am I actually providing?)

Roles are empowering.  If my team member does not ask for help, I trust each one to perform his or her role with excellence.  If I don’t trust them, how can they develop the excellence that the company expects?  How can they be recognized and rewarded for excellence if our processes and methods show a lack of trust in their roles to perform well without a developer present? 

Roles demonstrate respect.  As an agilest, I value people over processes.  That means I value the contribution of individual experts on my team to do their work with excellence, and I will show them respect for the work that they do, and the unique value that they contribute.  I will do so by allowing them to do their work without the “supervision and assistance” of a developer.

If they need me, they will ask for me.  Until then, I will get to doing what I do best.  I have a job to do, after all… and it is mine to do well.

Why this matters

Agile software development solves a problem.  The problem is that we are including waste in our processes, and that waste is causing software to be delivered late or with poor quality. 

If we optimize our processes, we can reduce waste. If we include developers in every process, we are not optimizing our processes.  We are, in fact, replacing one kind of waste with another. 

5 years, 7 months ago

AGILE architecture vs. agile ARCHITECTURE

As an architect involved in an agile implementation (my current gig), you can imagine how interested I was to see that there’s a new book on Agile Architecture, and perhaps how disappointed I was to see that it focused on SOA and Cloud.  That’s not to put down SOA or the cloud.  I’m a huge fan of both.  But it wasn’t the area of agility that I was hoping that a book, with that title, would address.  The misunderstanding was mine, not the authors.  I haven’t read the book yet, but I’m sure I will.

That moment of misunderstanding crystallized a thought: how even a two word phrase like “agile architecture” had two completely different meanings.  The opening scene of the movie “The Hobbit, An Unexpected Journey,” puts a rather humorous twist on this idea, when Gandalf introduces himself to Bilbo Baggins (who has apparently forgotten having met him as a boy).

Bilbo: Good Morning

Gandalf: What do you mean? Do you wish me a good morning, or mean that it is a good morning whether I want it or not;

Bilbo: <stunned silence>

Gandalf: Or that you feel good this morning; or that it is a morning to be good on?

Bilbo: All of them at once, I suppose.

Of course, in Enterprise Architecture, we have the same problem.  Does Enterprise Architecture mean “the practice of using technology architecture at an enterprise-wide scale,” or does it mean “the practice of using architectural ideas to shape the enterprise itself?” 

And after a bit of stunned silence, perhaps it means

“Creating an architecture to describe the externalities of an enterprise to set its context and improve the relationships it has with customers, partners, and suppliers?” 

All of them at once, I suppose.

Having just re-watched the Hobbit movie on my morning flight, these bits connected up in my head.

I’m proud to be both an architect of agility (applying the principles of agility to the processes of a business so that the business achieves the ability to change its own processes in response to agile demands), as well as a person who can craft technology architecture that reflects the notion of agility itself (technology that can be set up to change rapidly in response to business events).

All of them at once, I suppose.

5 years, 10 months ago

All Effective Enterprise Architects Are Agile

I explained to one of my clients recently that there is a perception of animosity between the Enterprise Architecture community and the Agile community.  Both sides make assumptions about the other, often assumptions that are simply unfair.  For example, many in the EA community think of “agile practices” as an opportunity to develop software without any architecture at all, while many in the agile software development community think of architecture as one of the “big design up front” guys who oppose their principles and practices.  Of course, it is not difficult to find people who fit those unfair descriptions, but I’d like to point out how these two viewpoints are similar.

I believe that effective Enterprise Architecture must be approached from an agile standpoint. 

First off, what does it mean to be agile?  We can always look to the agile manifesto for some guidance, but more recent publications do a good job of filling in some of the details as well.  I include a number of things in the notion of “being agile.”  These are not just from the agile manifesto, but also Kanban and the Theory of Constraints, Systems Thinking, Six Sigma, Scrum, eXtreme Programming, and value stream analysis.

  • A focus on performing high-value activities, and removing low-value activities.
  • A focus on empowering the people who make things to make decisions about how those things should be made.
  • A focus on developing small increments of actual value on a frequent basis and getting direct feedback on them.
  • A focus on making sure that one thing is “done” before moving to the next, so that we reduce “debt” as we go.
  • A focus on modern practices that remove ‘deployment impedance’ like test driven development and continuous integration.

I follow the terminology of Sam Guckenheimer in calling this the “Agile Consensus.” 

We have to recognize that the “agile consensus” is an approach, not a methodology.  It is a way of thinking about dealing with problems.  More importantly, it is a way for dealing with complex problems.  The diagram below comes from Ken Schwaber (inventor of Scrum) who adapted it from Strategic Management and Organisational Dynamics, by Ralph D. Stacey.

image

When we look at the problems that software is being used to address, and if we look at the process of writing software itself, we have to recognize that both areas of problems typically fall into the category of “complex.”  Not always.  Some software is simply configured and configured simply.  Some problems that software addresses are simple problems.  However, most of the discussions around software development are fueled by people addressing complex problems because that is where most of the software development community works.  It is the bread and butter of software development: solving complex problems in a complex way.

Enterprise architecture also deals with complex problems.  EA models and information are also complex to build and manage.  In this way, EA is very similar to software development.  EA solves complex problems in a complex way.

In order for EA to be effective, it has to use the same mentality as agile software development.

When I speak with enterprise architects who are actually doing the job of EA, big themes quickly arise:

  • do the work that is highly valuable and don’t do the work that the business doesn’t value
  • build consensus where it actually lives, not where the “people from above” believe that it does
  • deliver value quickly and in increments that your stakeholders understand
  • leave your repository in a good state of completeness with all the data that others need to use it
  • build in the deliverable artifacts into the business processes that need it, so that you get immediate feedback

If this looks like the list above, that is intentional.  I am trying to point out that Enterprise Architects use agile ideas, even if they often don’t use the term “agile” to convey the message. 

Why use these techniques?  They work for complex problems. 

Can someone think of a more complex problem than helping to move an organization towards their goals? 

EA addressed complex problems.  Agile thinking helps them to do it.

6 years, 6 months ago

On the road to a Business Architecture Manifesto

One very powerful metaphor that has reverberated throughout the technical community, in the past few years, was the Agile Manifesto.  Created by a group of folks who wanted to communicate the principles that drove their thinking, the Agile Manifesto has been a very useful tool for deciding if a particular practice is being done well.  I think it may be time to build one for the Business Architecture space.

That said, I am by myself, sitting in my living room.  I am in no position to speak for the community of business architects.  So, this submission is a suggestion for content that could be useful when the conversation begins.  It is my personal opinion about the principles of business architecture.  I would hope to bring this material to a group of other BA practitioners, as my contribution, to develop a full consensus on business architecture manifesto.  

First off, in order to develop principles for business architecture, we need to describe the problem that we are trying to solve. 

The problem that business architecture solves

Business architecture is a relatively new field that addresses an old problem.  Most business people recognize the underlying truth: the structure and practices of your organization directly impacts your ability to deliver the intended value.  Whether we are talking about a military service, a civilian government agency, a non-profit organization, or a for-profit business, the structures and processes that a leader chooses to employ will impact the results that the organization will produce.  That includes both intended and unintended results.  So the basic problem is this: how do we deliver on our mission while maintaining our values?

Business architecture gets to deal with a slice of that problem.  As people, we need to organize around a common shared mission.  We need to know what we want, and we need to go get it.  Humans can be pretty haphazard.  Business architecture does not address every issue.  Business architecture attempts to answer this question: what is the optimal way to organize?  Business architecture typically does NOT answer questions around the integration of corporate controls, or supporting activities like how to find staff to fill new roles.  Business architecture is focused on the narrow slice of “how to organize.” 

So why do we need business architecture to solve this problem?  There are literally hundreds of good, well researched, books that offer useful insight for solving this problem.  Why use a business architecture approach?  Because BA brings a novel approach, one based on the rigorous application of conceptual traceability, process improvement, information science, and mathematics.  While most of the business analysis methods prior to business architecture were founded, fundamentally, in social science, mechanical engineering, and even education, business architecture focuses on the newer sciences that have emerged in the computerized age. 

How does business architecture solve the problem

Business architecture’s unique value proposition is to focus on answering the questions of business structural and organizational effectiveness in a way that is rigorous, quick, clear, consumable, and value-focused. 

We are uncovering better ways of developing business insight by doing it and helping others do it.  Through this work, we have come to value:

Consistently reusable methods over Piecemeal assortment of best practices

Rapid insight over Deeply accurate models

Clear choices over Nuanced decision trees

Consumable deliverables over Consistency with external frameworks

Value-driven prioritization over Justification of the status quo

 

That is, while there is value in the items on the right, we value the items on the left more.

 

To break that down:

  • Repeatability, Reuse, and Rigor.  There are many ways to understand a business.  Business architects will expect you to pick one of those ways (one conceptual model) and then stick to it.  The rigor comes from sticking to the model.  If your enterprise is focused on creating a smooth customer experience, then the business architect will leverage the customer experience work done elsewhere, and will drive a business stakeholder to follow along rather than make something up.  While products must be creatively and freely developed, the organization itself must be architected and engineered.  Rigor matters.
     
  • Rapid Insight. There are many ways to analyze a business.  Business architects will work to reduce the overhead of their analysis methods so that they can produce valuable answers in a very timely manner.  Business people are not rewarded for taking a long time to do an excellent job.  Most will be better rewarded if they do a reasonably good job in a shorter timeframe.  While accuracy is great, the value of information is inversely proportional to the time needed to produce it.  Speed matters.
     
  • Clear Choices. If a business person cannot tell what the recommendation is, they won’t follow it.  If the business architect cannot produce insight that is clear for the business stakeholder, the architect will not effect change.  It is not good enough for a business architect to be quick and correct… they must also be clear. 
    The amount of information, and the coarseness of the decisions, depends on the level of the stakeholder.  At any level, a decision maker should be provided a short list of options (often 2 or 3) where the distinctions between them are clear.  This rule applies at all levels of the organization.  One strategy from a senior manager may require a choice among three different tactics for a department head to choose from.  No one person needs to be concerned with the entire decision tree, except perhaps the business architect himself.   The ability to make decisions is proportional to the clarity of the choices.  Business architecture favors clarity over nuance.
     
  • Consumable Deliverables.  In order for business architects to be successful, they must deliver a plan for the execution of business strategy.  That plan has to be something that the impacted stakeholders can understand and use.  In other words, the output of business architecture must be consumable.  Reams of technical detail are rarely useful.  At the other end of the spectrum, vague goals and promises of value may be just as inappropriate.  Recommendations must be provided using words and metaphors that the actual impacted business stakeholders understand.  They must be provided using forms and templates that the existing organization will recognize and can quickly use.  While consistency with frameworks and practices are important, business architects value consumability more.
     
  • Priority based on Business Value.  Business architects can spend their time on many tasks.  In addition, they can recommend that the organization spend time on many tasks.  Sometimes, even an efficient use of business architecture would be a waste of time if the resulting advice is unlikely to deliver strategic insight.  The selection of tasks, which to do now and which to do later, is of critical importance to a business architect.  While all supporting tasks can be justified, business architects will give priority to tasks that directly lead to actionable, consumable, value-driven business advice.

 

I’m always looking for insight and feedback from the community, so please feel free to add your comments. 

Please note: if your comment is long, the software will sometimes have trouble.  Write it in notepad or Word first, and then cut and paste into the comment edit window.  Don’t be afraid to send it more than once.  I will delete duplicates.  If all else fails, e-mail your comment to me and I’ll put it in.