One week ago

The Digital Future: Services Oriented Architecture and Mass Customization, Part 4

From Previous PartsPart 1 discussed the four ages of mankind.  The first was the Age of Speech; for the first time humans could “learn by listening” rather than “learn by doing”; that is, data could be accumulated, communicated, and stored by verb…

17 days ago

The Digital Future: Services Oriented Architecture and Mass Customization, Part 3 B

From Previous PartsPart 1 discussed the four ages of mankind.  The first was the Age of Speech; for the first time humans could “learn by listening” rather than “learn by doing”; that is, data could be accumulated, communicated, and stored by ve…

1 month, 11 days ago

The Digital Future: Services Oriented Architecture and Mass Customization, Part 2

From Part 1There have been four ages of mankind.  The first was the Age of Speech; for the first time humans could “learn by listening” rather than “learn by doing”; that is, data could be accumulated, communicated, and stored by verbal communicat…

11 months, 21 days ago

Hurricanes and an Architecture for Process

The idea for this post came from a previous post on this blog regarding enterprise architecture and religious organizations.

Definitions or Definitional Taxonomies

There are three definitions that are needed to understand this post; the definition of Architecture, process using the OODA Loop, and the Hierarchy of Knowledge.  These are somewhat personnel definitions and you may interpolate them into your model universe.

Architecture

Architecture—A functional design of a product, system, or service incorporating all the “must-do” and “must-meet” requirements for that product, system, or service.

OODA Loop

All processes supporting the mission, strategies, or infrastructure of any organization fall into the OODA model of Col. John Boyd.  The OODA, as discussed in several previous posts includes four steps: Observe, Orient, Decide, and Act.  I will discuss this shortly (below) using the example of the prediction of a hurricane.

A Taxonomy for the Hierarchy of Knowledge

I defined the following taxonomy of knowledge based on much reading and some thinking.  I will demonstrate what I mean for each definition by using it in the appropriate place in my example of the hurricane prediction.
Datum—some observation about the Universe at a particular point in four dimensions

Data—a set consistent datum

Information—patterns abstracted from the data.

Knowledge—identified or abstracted patterns in the information.

Wisdom—is the understanding of the consequences of the application of knowledge

Process Architecture and Forecasting Hurricanes

Since the process for predicting or forecasting hurricanes is most likely to be familiar to anyone watching weather on TV, it is the best example I can think of to illustrate how the OODA Loop process architecture works with the taxonomy of knowledge.

Observe

Initially, data is gathered by observing some aspect of the current state of the Universe.  This includes data about the results of their previous actions.  In point of fact,

Datum—some observation about the Universe at a particular point in four dimensions

Data—a set consistent datum

Obviously observations of current temperature, pressure, humidity, and so on, are basic to making weather forecasts, like the prediction of a hurricane.  And each observation requires latitude, longitude, height above sea level, and the exact time to be useful. So one datum, or data point would include, temperature, latitude, longitude, height, and time.

When the temperature is measured at many latitudes and longitudes concurrently, it produces a set of temperature data.  If other weather measurement, like pressure and humidity, are taken at the same locations at the same time, then you have several data sets with which to make a forecast (or a composite weather data set for a particular time).  Because this is a recurrent measurement process, the weather folk continue to build a successively larger composite weather data set.  But large data sets don’t make a forecast of a hurricane.

Orient

The Orient step in the process is inserting the new data into an individual’s model of how the world (Universe) works (descriptive) or should work (prescriptive).  These models are sometimes called paradigms.  Rules from Governance enable and support the Orient step, by structuring the data within the individual’s or organization’s model.  Sometimes these model or paradigms stick around far after they have demonstrated to be defective, wrong, or in conflict with most data and other information.  An example would be the model of the Universe of the Flat Earth society.

Information—patterns abstracted from the data.  This is the start of orienting the observations, the data and information.  Pattern analysis, based on the current model converts data into information is derived from the organization’s model of its environment or Universe.  For hurricane forecasting this would mean looking for centers of low pressure within the data.  It would also include identifying areas with high water temperatures in the temperature data, areas of high humidity, and so on.  This and other abstractions from the data provide the information on which to base a forecast.  But it is still not the prediction of a hurricane.

Knowledge—identified, combined, or abstracted patterns in the information.  Using the same paradigm, environmental model, or model Universe, people analyze and abstract patterns within the information.  This is their knowledge within the paradigm.  In weather forecasting the weather personnel uses the current paradigms (or information structures) to combine and abstract the knowledge that a hurricane is developing.

When they can’t fit information into their model, they often discard as aberrant, an outlier, or as an anomaly.  When enough information doesn’t conveniently fit into their model the adherents have a crisis.  In science, at least, this is the point of a paradigm shift.  And this is what has happened to weather forecasting models over the past one hundred and fifty years.  The result is that the forecasting has gotten much better, though people still complain when a storm moves off its track by fifty miles and causes unforecast wind, rain, and snow events.

Decide

Once the organization or individual has the knowledge, he or she uses input their knowledge within their models of the Universe to make decisions.

Wisdom—is the understanding of the consequences of the application of knowledge. 

This is the hard part of the OODA Loop because it’s difficult to understand both the consequences and the unintended consequences of a decision.  If your paradigm, environment, or Universe model is good, or relatively complete, then you’re more likely to make a good decision.  More frequently than not people make decisions that are “Short term smart and long term dumb.” 

Part of the reason is that they are working with a poor, incomplete or just plain wrong paradigm (view of the world or universe).  This is where the Risk/Reward balance comes in.  When choosing a path forward, what are the risks and rewards with each path?  [Sidebar:  A risk is an unknown and it is wise to understand that “you don’t know what you don’t know”.]  For weather forecasters it’s whether or not to issue a forecast for a hurricane.  To ameliorate the risk that the are wrong, weather forecasters have invented a two part type of forecast, a hurricane watch and a hurricane warning. [Sidebar: It’s split more by giving a tropical storm watch and warning.]

The reason for this warning hierarchy is that the weather forecasters and services are wise enough to know the public’s reaction to warnings that don’t pan out and lack of warnings that do; they understand the risks.  So when they give a hurricane warning, they are fairly sure that the area they have given the warning for will be the area affected by the hurricane.

Act

Once the decision is made people act on those decisions by planning a mission, strategies, and so on within their paradigm.

For governments and utilities in the U.S. this means putting preplanned hurricane preparations into effect.  For people this generally means boarding up the house, stocking up on food, water, and other essentials or packing and leaving.  And for the drunk beach comber this means grabbing the beers out of the frig, and either heading to the hills or back to the beach to watch the show.

Programmed and Unprogrammed Processes

In the 1980s I wrote a number of articles on process types.  After doing much research and much observation, I came to the obvious conclusion that there are two types of processes, Programmed, and those that I will call Unprogrammed and which includes design, discovery, and creative processes. [Sidebar: These three sub-types differ in that design processes start from a set of implicit or explicit customer requirements, discovery processes start from inconsistencies in data or in thinking and modeling the data, and creative really have no clear foundation and tend to be intuitive, chaotic, and apparently random.] 

Programmed Processes

Programmed processes are a set of repeatable activities leading to a nearly certain outcome.  Almost all processes are of this type.  The classic example is automobile manufacturing and assembly.  Since they are repeatable how does the process architecture model apply?
The short answer is to keep them repeatable and to increase their cost efficiency (that is making them less expensive in terms of time and money to create the same outcome).

Keeping Programmed Processes Rolling

To keep your truck or car operating requires maintenance.  How much and when is an OODA Loop process.  For example, you really should check your tires, both for wear and pressure regularly; otherwise the process of driving can become far too exciting.  So you are in the “observe” step.
As the tires wear out or lose pressure too often, the data set becomes information triggering the “orient” step in the process.  At this point a driver starts to gather data on which tires are wearing fastest and where on the tire they are wearing.  This last point may indicate that tire rotation is all that’s needed.
Based on this data our driver models the information (using the computer between his or her ears) to determine how much longer it’s safe to drive on the tires.  Based on results of that model, the driver will “decide” to buy tires, rotate tires, or wait a little longer.  [Sidebar: Actually, many drivers will go through the buy/wait decision based on additional data, the cost of new tires, and their budget.]  The driver will then “act” on the decision, either get new tires or wait (which is really an action).
The point here is that like everything else in our Universe, over time, everything breaks down; so repeatable processes require an OODA Loop process to maintain them.

More Cost Efficient

However, the OODA Loop process architecture elucidated here is important for programmed processes for another reason.  For repeatable processes the key OODA process is always how to make the faster, cheaper, or making a higher quality product at the same or lower price, and producing it in the same or less time.
This is famously what Adam Smith described in his pin example in the first chapter of “The Wealth of Nations”. By creating an assembly line process that he called “the division of labour”, he demonstrated that the same number of workers produce hundreds of more pins.  He went on to describe that each worker might now invent (or innovate, that is, refine or enhance) tools for that worker’s job. [Sidebar: Henry Ford went on to prove this in spades or should I say Model-T’s.]  [Sidebar: To me the worker’s inventing or innovating tools to help them in their job is quite interesting and often missed.  Tools are process or productivity multipliers.  In the military they are called force multipliers; you always want your enemy to bring a knife to your gunfight, because a rifle multiplies your force.  Likewise, the tooling in manufacturing can increase the quality of the product, the uniformity of what is produced, while reducing the cost and time to produce…fairly obvious.]
Both the “division of labour” and the creation of tooling require the use of the Architectural OODA Loop, which means that increasing the cost efficiency of manufacturing uses the OODA Loop with the knowledge taxonomy.

Unprogrammed Process

Unprogrammed processes have stochastic (creative and unplanned activities within them).  There are really three sub-types of unprogrammed processes, design, discovery, and creative. The key differences between the design, discovery, and creative processes are the whether the process has customer requirements driven and what the requirements are.

Design

The design process has customer requirements.  [Sidebar: As I’m using it, the design process also includes custom development, implementation, and manufacturing.]  It uses a semi-planned process, that is, program or project planning creates a plan to meet the objectives, but with latitude for alternative activities because there is significant risk.  The actual design activities within the programmed process are stochastic, from a program management perspective.  That is, the creative element of these activities makes less predictable, and therefore with programmatic risk with respect to cost and schedule.  Therefore, program management must use (some form or) the OODA architecture to manage the program or project.
The stochastic activities are themselves OODA Loop processes.  The designers have to identify (observe) detailed data of the functions they are attempting to create (the “must do” requirements), while working to the specifications (the”must meet” requirements) for the product, system, or service.  The designers then have to “orient” these (creating a functional design or architecture for the function), “decide” which of several alternate proposed designs is “the best”. Finally, they “act” on the decision.

Discovery

The requirements for research, risk reduction, and root cause analysis are generally unclear and may be close to non-existent.  One of my favorite research examples, because it’s well known, and because it so clearly proves the point is the Wright Brothers researching and developing the aircraft.  In 1898 the brothers started their research efforts.  Starting with data and information then publicly available they built a series of kites. With each kite, they collected additional data and new types of data.  They then used this to reorient their thinking and their potential design for the next kite.  They found so many problems with the existing data that they created the wind tunnel to create their own data sets on which to base their next set of designs.  By the end of 1902 they had created a glider capable of controllable manned flight and by 1903 they created the powered glider known as the Wright Flyer.  It took them at least two more cycles of the OODA Loop to develop a commercially useful aircraft.
Risk reduction also uses the OODA Loop.  A risk is an unknown.  It requires some type of research to convert the unknown into a known.  There are four alternative.  First, the project/research team must decide whether or not to simply “accept” the risk. Many times the team orients the observed risk in a risk/reward model and accepts the risk.  However, another to orient is to determine if there is knowledge or knowledgeable people to convert the unknown into a known. So “transferring” the risk is one way to reduce the risk.  A second method is to “avoid” the risk.  This means redesigning the product, system, or service, or changing the method for achieving the goal or target.  The final way is to “mitigate” the risk.  This is nothing short of or more than creating a research project (see above) to convert the unknown into a Known.
Likewise, root cause analysis is a research effort.  However, the target of this analysis is to identify why some function or component is not working the way it is supposed to work.  Again, its observing the problem or issue, that is gathering data, orienting it through modeling the functions based on the data, deciding what the cause is or causes are for the problem and how to “fix” the problem, and then acting on the decision.  Sound a whole lot like the OODA Loop.

Creative

Creative processes like theory building (both through extrapolation and interpolation), and those of the “fine arts” that come from emotions also use the Architecture of Process defined in this post.  For some theories and all fine arts, “the requirements” come from emotion, based on an intuitive belief structure [Sidebar: a religious structure, see my post on architecture of religion].  This intuitive structure provides “the data, information, and knowledge” on which the creativity builds.
However, for scientific theories, at least, they are sometimes based on inconsistencies in the results of the current paradigm.  The theorist attempts to define a new structure that will account for these aberrant data.

Why make a Deal about the Obvious?

To many it might seem that I’m making a big deal about what is obvious.  If it is the case, why hasn’t it been inculcated into enterprise architecture and used in the construction and refinement of processes and tooling to support the charter, mission, objective, strategies and tactics of all organizations?  Using formal architectural models like the architecture for process, presented in this post, will enable the enterprise architects to “orient” their data and information more clearly making Enterprise Architecture that much more valuable.
1 year, 8 days ago

Enterprise Architecture, Systems Engineering, and Regulations: Process Rudders or Process Brakes

Regulations: The “Must Meet” Requirements

Regulations directly affect all organizations, products, systems, and services. Further, they can be a rudder guiding the organization or a brake causing the organization to stop making any progress in meeting its charter, goal, or in completing its mission.  This post discusses laws, rules, specifications, standard, regulations (or simply regulations) as a part of any enterprise architecture or systems engineering effort.

The Customer Requirements Identification and Management tool that I’ve developed, CARRMA®, uses the concept of Must Meet requirements to store both the regulations and the metrics for meeting the regulation.  If more than one project has a particular regulation imposed on it, the CARRMA®’s data store will allow for reuse.

Regulations the rudders of an organization

Any commandment, law, rule, specification, standard, or regulation creates process friction, by its very nature.  It inhibits what can be done or defines what must be done. For example, saying “Thou Shall not Kill” means that it’s not nice to end a vehemently intense discussion by bouncing your opponent “six feet under”, though that might be very satisfying at the moment.  That is, killing is not a good solution to an intra-group disagreement since it doesn’t promote understanding, knowledge and therefore value growth of the group and doesn’t instill trust with other groups.

Hoping that I’ve made the point that regulations curb action or ensure action, by an over-the-top example, a regulation acts like a rudder, making it difficult for a process and thus a strategy to go one way, thereby making it easier to go another.  This, in turn, may directly affect the organization’s charter, mission, or goal.  In any rational system it should enable both the charter/mission/goal and the strategies and processes for achieving these.
Fortunately or more importantly, unfortunately systems and organizations built by humans are not entirely rational and sometimes not rational at all.  If the economy is the engine that powers the ship-of-state in an organization, then each commandment, law, rule, specification, standard, or regulation enacted is a rudder with its own wheel guided by part of the crew steering the organization toward their own Avalon.

Arrow’s Paradox and Catch 22s

At some point the number of rudders pointing at all points of the compass are such that the organization be it private or a ship-of-state either comes to a complete halt or turns in tight circles.  The rudders have formed a damn that effectively stops all forward progress of the organization, which no amount of churning by the organization’s economic engine can overcome.  Some of these regulation rudders are small and some are very large.  Twist the large ones hard and the ship brakes to a crawl and it may not turn at all. 

Arrow’s Paradox

A bigger problem is that too many regulation rudders will simply cause the organizational ship to stop and not make any of the ports.  Dr. Kenneth Arrow’s is known for “Arrow’s Impossibility Theorem” [Sidebar: For which he won the Nobel Prize], which is also known as Arrow’s Paradox.  He demonstrated mathematically that if an organization has three or more goals that they want to optimize, they will be able to only optimize on two (or I suspect they can sub-optimize on all three).

The point here isn’t that organizations can’t have more than one goal or objective; it is that if the organization attempts to define more than two (or maybe three) objectives using the “policy/regulation” strategy, the organization will slow down, wobble around, and achieve none of the objectives.

If you add in that in a democracy the goals and objectives change as controlling constituencies change, normally you end up with more and more laws, regulations, rules, and standards each attempting to use its regulatory rudder to change the course of the ship-of-state to reach the objective for which it was enacted.  The net result is that either the ship-of-state will end up on the rocks or going in circles.

From personal experience with federal contracts and from a recent DoD report, I can illustrate the problem.  One goal, which probably should be the only goal of DoD contracting, is to provide the most effective weapons in the world for the US Military.  That is, the weapons should be the greatest force multiplier.

However, the contracting office is faced with a second objective (a second rudder) of acquiring these weapons cost efficiently.  There are two metrics for cost efficiency, the initial cost (including research, development, and construction), and life-cycle costs (maintenance, upgrades, and disposal).  The question for the contracting officer is, “Do you contract for the cheapest initial cost product or for the one that will cost the least over the product’s projected lifespan?”

Then there is a third rudder in the form of the dependability of the product, system, or service. “Dependability encompasses all of the “illities”, including reliability, maintainability, serviceability, and so on.  Each of these has metrics and standards that must be met.  In some cases the metrics for the standard or policy is either untestable in a timely manner or in a few infeasible or impossible to meet.  All of these rudders will force addition time and expense into the effort.

A fourth rudder is reconfigurability and upgradeability of the product, system, or service.  Before the US Civil War, the rate of change of weapons and support systems was such that weapons and weapons systems had no need for this “Must Meet” requirement/policy.  The weapon would be worn out long be the technology changed.  However, since then it’s obvious that technology has and is continuing to accelerate (to the point that for many systems, they must be upgraded before their development and implementation is completed).  These continue to increase the initial design costs. [Sidebar: Services Oriented Architecture can reduce these costs greatly for IT systems, and modular systems/product can do the same for hardware of all types.]

A fifth rudder, and first one that is politically/social motivated as well as costly is the implied policy that all congressional districts and states should have jobs related to weapons and intelligence system development, especially where the senator or representative sits on committees dealing with budgets and military programs.  A recent DoD study has shown that this can add 20 to 25 percent to the cost of a product, system, or service for the DoD.
Then comes the politically motivated socially liberal welfare policy rudders (those intended to regulate social welfare and social change).  For weapons and intelligence tools, these require that a certain percent of the work on the product be from female owned companies and another percentage be from “minority” owned businesses.  [Sidebar:  The social actives’ idea was that the only way these groups could break into DoD contracting was through regulation.  I think they were correct because most of the work they did that I observed over the 25 years I was associate with government contracted engineering demonstrated beyond a doubt that they were incompetent to compete with a level playing field.  Many times the prime contractor had to supply the engineering capability to complete the job over schedule and way over cost.]  While it isn’t Politically Correct to attempt to define how much money was spent on this contracting welfare, from personal experience I expect that it is very significant.

The point of this section is not to discuss the problems with the regulations and informal policies of DoD contracting, rather it’s to demonstrate that as more laws, policies, regulations, business rules, standards, and so on (the “Must Meet” requirements) are imposed on a program, especially, extraneous ones, that both the effectiveness of product and the cost efficiency of the project or program are reduced.  And at some point there are so many “Must Meet” requirements that the effort, even at the enterprise architectural level will fail.

Catch-22

The extreme case of Arrow’s Paradox is the famous Catch-22 where two regulations are diametrically opposed leaving whatever effort, project, program, organization, or enterprise going in circles and making no progress in any direction. Even with a ship-of-state the size of the United States (which is a supertanker sized economy), given enough Catch-22s and nothing will get done; too many steersmen, too many rudders, and too many goals (targets, harbors, or whatever).

A good current and everyday example of regulatory Catch-22 is deicing roads.  Having icy sidewalks can be very exciting and occasionally lethal—so it really is not a good thing.  So deicing the sidewalks is mandatory.  Deicing calls for the use chemicals like sodium chloride (salt).

The problem is that there are regulations (must meet) requirements for the use and storage of “salt” because it “pollutes the environment” (and it does, you should see my grass near the sidewalk and road).  So you must use chemicals to save lives but you must not to save the environment.  Two regulatory “must meet” requirements (rudders) are in opposition, one to save lives and one to save the environment.  This is a small example of a big problem that can and will bring the economic ship-of-state to a dead stop.

Reducing the Number of Regulatory Brakes but Keeping the Rudders

To get any organization to at least head to a goal it should be clear that removing internal policies that interfere with the attainment of the goal is necessary.  For large organizations with many sub-organizations, the issue becomes one of identifying which regulations guide the organization in the direction its charter, goal, or mission state and which are braking it to a stop.  For many organizations, but especially democratic style governments, there will always conflicting goals and missions and therefore conflicting regulations.  So how should a large organization or government determine which are rudders and which are brakes?
To my mind, this is a good place to apply Enterprise Architecture and the architectural model that I set out both in my book and in this blog.  The nice thing about that architectural model is that it can start as a static model that can be used to identify customer requirements and end up as a dynamic model of the enterprise (even the Ship-of-State).  As such, it can identify policies that are braking or causing bottlenecks in the processes enabling the strategies for attaining the goal or mission.  Until you can dynamically model the enterprise, you will never really be able to identify the unintended consequences and negative externalities of any policy, standard, or regulation.  Nor, as the goal or mission changes can you identify those policies, standards, or regulations that truly impede progress in the changed direction (though many politicians in the organization will be able to tell you, or so they believe).

For those policies and standards internal to the organization, the leadership should be able to understand which regulations support the organization’s strategies and process and which don’t.  Additionally, the leadership can propose changes, deletions, and new regulations, which the enterprise architect can then model to determine the likely consequences, both intended and unintended.  Once the enterprise architects have oriented the changes the leadership proposes [Sidebar: See the OODA loop] the leadership can then choose what internal policies, standards, and regulations to change and which changes to implement with a much lower risk while seeing the their organization move more quickly toward achieving its goal.  [Sidebar: The modeling will also show where leaders and managers of sub-units are working on their own agenda which might or might not be steering toward the overall goal.  Remember the Systems Engineering axiom, “Optimizing the sub-systems sub-optimizes the system.”]

For governments, especially for the legislative branch, architectural modeling is particularly important both to determine conflicting laws, regulations, rules, standards and codes.  If, as the architectural models mature and their predictions help to make better decisions, there may even be fewer vehemently intense discussions about which laws, regulations, rules, standards, and codes to enact and which to remove or rewrite…Interesting.
1 year, 19 days ago

If You Want to Create an Enterprise Architecture; Don’t!

One of the last presentations I made as an Enterprise Architect for a major DoD contractor was to the Chief Architect of the US Veterans Administration.  I walked in with a fully prepared presentation that was to take about 10 minutes of the time allotted to our team only to find the Chief Architect cutting the presentation off with a question, “How do we go about creating an IT architecture for the VA?”  Even though I had a very good answer and had applied it on a couple of occasion, my mind blanked.  I want to share with you his problem and the answer I should have given.


The Problem

The problem that the Chief Architect of the VA has is the same problem that plagues CA’s of all large organization and most of medium and smaller organizations.  That question is base on the very logical idea very much the analog of the idea that before you start changing the plumbing, you should know design of the current plumbing; that is, before you can create a “to be” or “next step” architecture you need to have a “current architecture”.  Obviously, if you don’t know which pipes connect where and start making changes to the plumbing you could end up with some very interesting and exciting results for which you may need to call your insurance company.  Likewise, if you want to improve the effectiveness and/or the cost efficiency of the organizational processes and information systems, most Enterprise Architects assume they must first define and delimit the “as-is” processes and information systems for the organization.


The conundrum is that, in today’s technological environment, by the time an IT architecture team has mapped out (structured and ordered) an “as is” architecture, some, most, or all of the elements and data of the architecture will be obsolete and out of date.  For something as large as a major corporation, a department within a state or the federal government, the cost and effort involved would require a tour de force on a very large perhaps unprecedented scale.  This cost and level of effort would be such that the senior management would cut funding to the effort as a waste of time and money, since having an “as-is” architecture by itself produces little in value to the organization.


As can be found in the literature, there are many ways to “solve” or at least ameliorate the problem of creating an “as-is” architecture.  For example, one of the best, that almost works, is to chop the organization into its components and create an “as-is” architecture for each component separately.  Then try to stitch the architectures together.  I’ve tried this and it works up to a point.


There is a truism in Systems Engineering, Systems Architecture, and Enterprise Architecture, “Optimizing the sub-systems will sub-optimize the system”.  I have demonstrated this to many people many times and experienced it several times.  But this is crux of the problem for those that try to create an Enterprise Architecture for a large organization.


The Solution

The simple answer is “Don’t”.  That is, “Don’t attempt to create an “as is” architecture for an organization, especially a large organization, because it will create itself with the proper procedures in place.  So how would I do it?


1.
 Define, delimited, and structured an initial set of classes and attributes for the Organization’s Enterprise Architecture.  These should include:

  • Its Charter, Mission, Goal

  • Its Strategies for achieving its charter, mission, or goal

  • Its Processes supporting its strategies

  • Its Tooling and infrastructure

  • Its Governance that affects any of the above, including:

  • Internal Policies and Standards

  • External Regulations and Standards

I worked with one Enterprise Architecture database that had over fifty classes, each class with ten or more attributes.  This was a fairly mature architecture.  My recommendation, don’t try to think of all the classes you may need or all of the attributes for each class; that’s way over thinking.  Instead, start simple and add through the cycles

2. Once you have designed and structured the initial set of classed and attributes, create a data base structured according to the design.


3.
 Here is the key to creating an “As-Is” Architecture by not creating it…Huh?  Design and implement processes to capture the current state of strategies, processes, and tooling/infrastructure as part of review of funding for revision and upgrades to the current systems and processes.  


4.
 When personnel in the organization propose a project insist that these personnel demonstrate the value of the process or procedure that they intend to update or upgrade. The “value” would include demonstrating which of and how the current product, system, or service enables the processes, strategies, and charter, mission, or goal of the organization. My experience has been that the initial attempts will be fuzzy and incomplete, but that as the number of proposed projects in the database (which is generally called the Asset Management System and on which the “as-is” architecture is built) increases both the completeness and clarity of the current enterprise architecture will increase.


The reason I say “Don’t” try to create an “as-is” architecture is that
 every 3 to 7 years every component of the organization’s information system will need replacement.  This means that within 3 to 5 years simply by documenting and structuring the inputs from all of the efforts the organization’s “as-is” architecture will be synergistically created (and at minimal cost) [Sidebar: There will be some cost because the project proposers will need to think through how their current charter, mission, or goal and the strategies they support links to and supports the overall charter, mission, or goal of the organization.  This is not necessarily a bad thing.]  For large organizations, no matter how much time or effort is put into attempting to create an “As-Is” Enterprise Architecture, it will take a minimum of a year and a great deal of funding; so it simply makes no sense.


As this Enterprise Architecture evolves you will begin to see a number of things that managers want to obfuscate or hide completely.  For example, a number of processes and component or sub-organizations will be demonstrated to be obsolete.  In this case obsolete means that the process or component organization no longer supports any of the organization’s strategies or its goal.  Since managers want to build or at least keep their fiefdoms they will not appreciate this much.  Additionally, it will demonstrate which internal policies, regulations, and standards help the organization and which hurt it in meeting its goal.  Again, the gatekeepers of these policies, regulations, and standards will object–strenuously.  


But there are two more insidious problems that a good “As-Is” Enterprise Architecture will reveal, nepotism and the famous “Catch-22s”.  


Nepotism

Nepotism in this case is more broadly defined than what most people think of as “nepotism”.  In the sense I mean, nepotism can include creating a non-level economic playing field. In all large organizations, but especially in the U.S. Federal government (probably in all governments) the type of nepotism I’m identifying is rampant.  In fact a December 2016 report from the Department of Defense highlights what most federal employees and DoD contractors have known for years, because representatives and senators will only vote for a large program if their district or state gets a part of it, the DoD estimates that the cost of the program increases approximately 20 percent.  This is “jobs welfare” on a massive scale.  Some major defense contractors have plants in every state for just this reason, not because it make any sense from a cost efficiency perspective.  Further, Congress had passed laws to ensure that minority and female owned businesses.  The reason is that minorities and women scream that the good old boy network doesn’t allow them to compete for sub-contracts [Sidebar: Actually the reason for the “good ole boy” network is that the prime contractors have sub-contractors that actually know what their doing.  In my experience, many times primes will “encourage”–read subsidize–inexperienced and frequently incompetent minority and female owned businesses in order to meet these regulations imposed on their proposals.]  Again, this is a form of social welfare to ensure all political constituents that scream loudly are appeased.  This adds up to the DoD being one of the larger governmental welfare organizations. [Sidebar: While, seemingly, I’ve picked on government organizations, especially the U.S. DoD, and while I have found that all governmental organizations in a democracy will have this type of nepotism.  This is what lobbing is all about.  Only when it goes so far that it’s plain to all and when it’s not openly enacted into law that we call it graft and corruption.]  And it’s not only governments that suffer from this type of nepotism, all large organizations have the same problems, though generally on a smaller scale.  For example, sometimes the nepotism is written into union contracts.  Along with finance engineering, the auto industry in Detroit suffered a near collapse due to contractual nepotism.

This presents a problem for any Enterprise Architect.  The as-is architecture will highlight the nepotism of this type more clearly than any report.  The Enterprise Architect won’t need to report it to the management, it will be self-evident.  I’ve experienced a situation, as I suspect many of you have, where the management kills the messenger in order to not address the problem.  In my case, three times I’ve been chased off programs when I reported that the effort was subsidizing silliness.


Catch-22

The second significant problem that policies, regulations, and standards become contradictory to each other or in combination make it impossible for the organization to achieve its goal.  Again, a good enterprise architecture will highlight these, though frequently, when management from one generation of technology with its set of policies and standards, finds the next upon them, they will refuse to resend or modify the existing regulations, preferring instead, again, to kill the messenger.  So like Systems Engineers, I’ve found that enterprise architects are only respected by other enterprise architects.


5.
 When the development and implementation team completes a project, and once it goes into operation, then as a final step in their effort, they should review the data they gave to the enterprise architect, revising the data to accurately reflect the “as-built” instead of the “as-proposed”.  The as-built documentation must include all component, assembly or functional, and customer acceptance testing, and all post production required changes.  This documentation will inevitably lead to additional class attributes of the Asset Management System and structure in the enterprise architecture.


6.
 As the Asset Management System and the Enterprise Architecture matures, management should prepare for a paradigm shift in the way projects and other efforts are proposed.  This is where Enterprise Architecture really demonstrates how it can make the organization both more effective and cost efficient.


A mature enterprise architecture can serve as the basis for a dynamic business or organizational process model for the organization.  Management can use this model to identify obsolete processes, (and as discussed) policies, regulations, and standards; ones that the organization should eliminate.  Additionally, with the help of the Enterprise Architect, management can identify missing or inhibiting processes and tools, and identify bottlenecks and dams in process flows.


Further, they can model what happens when the missing and inhibiting processes and tools are added or when the bottlenecks are eliminated or reduced.  This modeling will then indicate where there is a need for new efforts and to some degree the effectiveness and cost efficiency of such efforts.  It’s a paradigm shift in that no longer to component or sub-units of the organization propose changes.  Instead, senior management working with the Enterprise Architect and the component or sub-units will identify and fund efforts.  They now have a way to measure the potential of the change in meeting the organizational goal, which means senior management has a better way of managing organizational change.


Finally, once management has identified targets for change or upgrade, the enterprise architect together with a system architect can define alternatives to meet the effort’s requirements.  They can model alternative process and tooling changes to forecast which has the lowest potential risk, the highest potential return, the least disruption of current activities, lowest initial cost, lowest lifecycle cost, the most adaptable or agile, or any number of other targets defined by the senior management.  This will make the organization much more cost efficient, and perhaps more cost effective; and this is the purpose of Enterprise Architecture,


To sum up, using this six step, high-level process is an effective way to create both an Asset Management System (an “As-Is” Architecture) and an effective Enterprise Architecture process; perhaps the only way.  

2 years, 5 months ago

Enterprise Modernization – The Next Big Thing!

“We tend to think the strong will survive, but a virus is a very small thing that kills big things.” Horace Dediu, Clayton Christensen Institute, speaking about the fall of Nokia.

All enterprises, be they large or small, national or multinational, commercial or government agency, American or Chinese, Japanese or European, are carrying the dead weight of their history and almost certainly continuing to add unnecessary complexity and excessive cost that will progressively reduce effectiveness, with the potential to trigger existential crises. Newer enterprises including Internet platform and Cloud based companies are not immune from this effect. As Horace Dediu has commented, Nokia is a classic case of a large enterprise brought from market leadership to irrelevance and zero value in an extremely short space of time.

In the past, modernization has become synonymous with technically related “application modernization” The term “modernization” is commonly used to refer to information systems, applications and technologies. And it is very common for systems or applications to be “modernized” because a technology has come to the end of its life, or an application is so antiquated that it cannot support newer business process patterns such as multi-channel and mobile. More generally organizations clearly implement modernizing programs without necessarily using the modernizing name. For example transformation programs or projects will often involve considerable elements of modernization. Similarly adoption of Agile methods may be considered a form of process modernization. Or the introduction of DevOps practices, enterprise architecture, master data management, life cycle management etc. But it is notable that each of these forms of modernization are discipline centric. It is very rare for an enterprise to undertake concerted effort to understand what modernization really means for a specific enterprise and to plan and execute modernization that delivers inter-discipline collaboration in support of specific modernization objectives and goals.

Each enterprise has unique modernization needs. A bank may see a primary modernization goal to establish core banking systems that reduce the risk of negative customer impact, such as delays or errors in transaction processing. Also to be able to introduce price and product change in days or weeks. A healthcare company may similarly see the requirement to change prices rapidly but equally the need to rapidly implement new legislation. A retail enterprise may see the ability to interact with customers using processes that span multiple technology, shopping and delivery channels, and to be able to influence the customer decision making process to achieve maximum customer satisfaction.  A key element apparent in all these modernizing goals is that modernization is not about achieving a new plateau of capability and functionality. Rather it is about enabling continuous, short cycle time response to change, targeted at the very specific areas that are mission critical for the individual enterprise.

The problem with modernization is that it is widely perceived as slow, very expensive and high risk because the core business legacy systems are hugely complex as a result of decades of tactical change projects that inevitably compromise any original architecture. But modernization activity must not be limited to the old, core systems; I observe all enterprises old and new, traditional and internet based delivering what I call “instant legacy” [Note 1] generally as outcomes of Agile projects that prioritize speed of delivery over compliance with a well-defined reference architecture that enables ongoing agility and continuous modernization.

What’s required is a modernization approach and strategy that progressively breaks out business and technology components that establish highly independent units of business capability that comply with a reference model and architecture that ensures architecture agility and implements clear fire breaks between the components.

But as discussed, enterprises are extremely reluctant to undertake modernization as they see it as all cost and risk. Transformation projects are widely viewed in the same way. And anyway immediate business priorities always trump housekeeping!

There are various aspects to a successful modernization approach. But the most important are:
1. Define Agility Vectors. Identify the top priorities for business agility and integrate relevant actions into all business projects. Here are some examples:
a. Radical improvement (time and cost) in response to legislative change
b. Generalization of existing capabilities to support new products and services
c. Dramatic reduction in new product time to market
d. Integration of existing capabilities to leverage disparate channels
e. Separation of common and customer/channel specific capabilities
I refer to these as “vectors”. Mission critical goals that will require actions and change right across the organization. Single projects are generally not going to cut it. Therefore the vectors provide a mechanism to exert influence (demand shaping) over the incoming application demand management pipeline and to select and coordinate multiple project actions.
2. Mandate Reference Architecture for Agility. Establish a reference architecture for business agility that defines the minimum necessary architecture compliance to ensure continuous business evolution. Mandate that all core business capabilities are implemented as software services and components.
3. Integrate Agile Architecture AND methods. Implement Agile practices that give equal weight to agile architecture and agile project delivery. Demonstrate small, incremental delivery steps, business capability by capability, service by service.
4. Govern business agility. Automate governance by establishing model driven architecture and development tooling that ensures compliance with the reference architecture.
5. Integrate Business and IT. Practice conceptual business modelling to establish common business and IT vocabulary independent of existing applications, align business and software services and reengineer the organization around business capabilities and services.
6. Get top level sponsorship for the Enterprise Modernization Roadmap. Recognize enterprise modernization as a major business priority and elevate to the highest levels of the enterprise to make it happen.

The enormous, disruptive creativity of Silicon Valley is transforming how companies make decisions, store data, reach potential customers, outsource activities, processes and capabilities, and how people communicate, make friends, protest and shop. No enterprise is immune from this effect.

Accordingly, modernization today means reinventing the way enterprises work, the business model and systems, and being genuinely agile. You only do this by ripping up today’s world and turning it into a genuinely flexible structure of independent components that can flex and morph continuously.

Application Modernization should be long dead and buried. Enterprise Modernization is an existential priority for all enterprises, not just those with mainframes and COBOL!

Note 1: Understanding Agile Adoption Failure

2 years, 5 months ago

Enterprise Modernization – The Next Big Thing!

“We tend to think the strong will survive, but a virus is a very small thing that kills big things.” Horace Dediu, Clayton Christensen Institute, speaking about the fall of Nokia.

All enterprises, be they large or small, national or multinational, commercial or government agency, American or Chinese, Japanese or European, are carrying the dead weight of their history and almost certainly continuing to add unnecessary complexity and excessive cost that will progressively reduce effectiveness, with the potential to trigger existential crises. Newer enterprises including Internet platform and Cloud based companies are not immune from this effect. As Horace Dediu has commented, Nokia is a classic case of a large enterprise brought from market leadership to irrelevance and zero value in an extremely short space of time.

In the past, modernization has become synonymous with technically related “application modernization” The term “modernization” is commonly used to refer to information systems, applications and technologies. And it is very common for systems or applications to be “modernized” because a technology has come to the end of its life, or an application is so antiquated that it cannot support newer business process patterns such as multi-channel and mobile. More generally organizations clearly implement modernizing programs without necessarily using the modernizing name. For example transformation programs or projects will often involve considerable elements of modernization. Similarly adoption of Agile methods may be considered a form of process modernization. Or the introduction of DevOps practices, enterprise architecture, master data management, life cycle management etc. But it is notable that each of these forms of modernization are discipline centric. It is very rare for an enterprise to undertake concerted effort to understand what modernization really means for a specific enterprise and to plan and execute modernization that delivers inter-discipline collaboration in support of specific modernization objectives and goals.

Each enterprise has unique modernization needs. A bank may see a primary modernization goal to establish core banking systems that reduce the risk of negative customer impact, such as delays or errors in transaction processing. Also to be able to introduce price and product change in days or weeks. A healthcare company may similarly see the requirement to change prices rapidly but equally the need to rapidly implement new legislation. A retail enterprise may see the ability to interact with customers using processes that span multiple technology, shopping and delivery channels, and to be able to influence the customer decision making process to achieve maximum customer satisfaction.  A key element apparent in all these modernizing goals is that modernization is not about achieving a new plateau of capability and functionality. Rather it is about enabling continuous, short cycle time response to change, targeted at the very specific areas that are mission critical for the individual enterprise.

The problem with modernization is that it is widely perceived as slow, very expensive and high risk because the core business legacy systems are hugely complex as a result of decades of tactical change projects that inevitably compromise any original architecture. But modernization activity must not be limited to the old, core systems; I observe all enterprises old and new, traditional and internet based delivering what I call “instant legacy” [Note 1] generally as outcomes of Agile projects that prioritize speed of delivery over compliance with a well-defined reference architecture that enables ongoing agility and continuous modernization.

What’s required is a modernization approach and strategy that progressively breaks out business and technology components that establish highly independent units of business capability that comply with a reference model and architecture that ensures architecture agility and implements clear fire breaks between the components.

But as discussed, enterprises are extremely reluctant to undertake modernization as they see it as all cost and risk. Transformation projects are widely viewed in the same way. And anyway immediate business priorities always trump housekeeping!

There are various aspects to a successful modernization approach. But the most important are:
1. Define Agility Vectors. Identify the top priorities for business agility and integrate relevant actions into all business projects. Here are some examples:
a. Radical improvement (time and cost) in response to legislative change
b. Generalization of existing capabilities to support new products and services
c. Dramatic reduction in new product time to market
d. Integration of existing capabilities to leverage disparate channels
e. Separation of common and customer/channel specific capabilities
I refer to these as “vectors”. Mission critical goals that will require actions and change right across the organization. Single projects are generally not going to cut it. Therefore the vectors provide a mechanism to exert influence (demand shaping) over the incoming application demand management pipeline and to select and coordinate multiple project actions.
2. Mandate Reference Architecture for Agility. Establish a reference architecture for business agility that defines the minimum necessary architecture compliance to ensure continuous business evolution. Mandate that all core business capabilities are implemented as software services and components.
3. Integrate Agile Architecture AND methods. Implement Agile practices that give equal weight to agile architecture and agile project delivery. Demonstrate small, incremental delivery steps, business capability by capability, service by service.
4. Govern business agility. Automate governance by establishing model driven architecture and development tooling that ensures compliance with the reference architecture.
5. Integrate Business and IT. Practice conceptual business modelling to establish common business and IT vocabulary independent of existing applications, align business and software services and reengineer the organization around business capabilities and services.
6. Get top level sponsorship for the Enterprise Modernization Roadmap. Recognize enterprise modernization as a major business priority and elevate to the highest levels of the enterprise to make it happen.

The enormous, disruptive creativity of Silicon Valley is transforming how companies make decisions, store data, reach potential customers, outsource activities, processes and capabilities, and how people communicate, make friends, protest and shop. No enterprise is immune from this effect.

Accordingly, modernization today means reinventing the way enterprises work, the business model and systems, and being genuinely agile. You only do this by ripping up today’s world and turning it into a genuinely flexible structure of independent components that can flex and morph continuously.

Application Modernization should be long dead and buried. Enterprise Modernization is an existential priority for all enterprises, not just those with mainframes and COBOL!

Note 1: Understanding Agile Adoption Failure

3 years, 13 days ago

Understanding Agile Adoption Failure

The most common concern our customers voiced in 2014 was the unexpected outcomes of Agile projects. They don’t talk about failure as such. But they do talk about loss of consistency; inability to govern; lack of coordination AND THE INCREASING TIME TO MARKET caused by these precise issues.

I was struck by the results of the Agile Adoption Experiences Survey 2014 published by Scott Ambler. The really significant result to me is that 40% of respondents rates their organizations adoption of Agile as neither a success or a failure. Add to this the categories of Failure, Great Failure and Too early to tell and you have 58% that are not successful! This synchs with my customer feedback referred to above.

The advice my colleagues and I give when customers approach us looking for answers to these questions, is to look at how architecture is integrated into Agile projects. And there are some key areas that we look for in our assessments:
1. Is there a good reference architecture and associated contextual patterns?
2. Are there clear policies attached to work products together with the rationale?
3. Are developers and architects working as a community of interest to evolve the reference architecture, patterns and policies?
4. Are the reference architecture, patterns and policies integrated into the tooling and the architecture runway?
5. Is the architecture runway model based – allowing it to provide a reusable design time platform to be evolved by projects?

Agile projects can be successful in an enterprise situation. But architecture and governance need to be coordinated for consistency and mechanisms (automation) enforced to ensure consistency.

I wonder why the Agile Adoption survey didn’t ask any questions along these lines?

3 years, 13 days ago

Understanding Agile Adoption Failure

The most common concern our customers voiced in 2014 was the unexpected outcomes of Agile projects. They don’t talk about failure as such. But they do talk about loss of consistency; inability to govern; lack of coordination AND THE INCREASING TIME TO MARKET caused by these precise issues.

I was struck by the results of the Agile Adoption Experiences Survey 2014 published by Scott Ambler. The really significant result to me is that 40% of respondents rates their organizations adoption of Agile as neither a success or a failure. Add to this the categories of Failure, Great Failure and Too early to tell and you have 58% that are not successful! This synchs with my customer feedback referred to above.

The advice my colleagues and I give when customers approach us looking for answers to these questions, is to look at how architecture is integrated into Agile projects. And there are some key areas that we look for in our assessments:
1. Is there a good reference architecture and associated contextual patterns?
2. Are there clear policies attached to work products together with the rationale?
3. Are developers and architects working as a community of interest to evolve the reference architecture, patterns and policies?
4. Are the reference architecture, patterns and policies integrated into the tooling and the architecture runway?
5. Is the architecture runway model based – allowing it to provide a reusable design time platform to be evolved by projects?

Agile projects can be successful in an enterprise situation. But architecture and governance need to be coordinated for consistency and mechanisms (automation) enforced to ensure consistency.

I wonder why the Agile Adoption survey didn’t ask any questions along these lines?

3 years, 7 months ago

Mapping Agile Architecture

Jason Bloomberg recently published a mind map for Agile Architecture. It’s a nice map that sketches top level thinking and I welcome that. It prompted me to do a drill down.

Mind maps are useful in that they are, by definition free form and intended to support brain storming. The downside is obvious – they are generally inconsistent and cause modelers’ intense frustration! Caveat emptor over, I fully agree with Jason that we need a dual interpretation of Agile  – that is Agile practices and Agile Architecture, and I have written about this on many occasions. Also that the entire motivation is about business agility. On this last point my mind map is clearly a little more technical than Jason’s, and on reflection I think that is because it’s essential to converge the business and technology concerns.

For example, the map suggests a strong capability centric approach to interpret the business morphology. However this is insufficient; the technology must also establish appropriate levels of implementation independence that will facilitate the pluggability of business capabilities. Similarly you might think that considerations regarding the platform and delivery technology (such as MDA/MDD) are irrelevant to business concerns. However the platform and platform delivery technology are potentially massive drivers of rapid iteration and ongoing change, because they encapsulate common application level infrastructure and common services, so understanding the “business” standardization and localization model is crucial to delivering agility through this structure.

References:
Related posts: 
3 years, 7 months ago

Mapping Agile Architecture

Jason Bloomberg recently published a mind map for Agile Architecture. It’s a nice map that sketches top level thinking and I welcome that. It prompted me to do a drill down.

Mind maps are useful in that they are, by definition free form and intended to support brain storming. The downside is obvious – they are generally inconsistent and cause modelers’ intense frustration! Caveat emptor over, I fully agree with Jason that we need a dual interpretation of Agile  – that is Agile practices and Agile Architecture, and I have written about this on many occasions. Also that the entire motivation is about business agility. On this last point my mind map is clearly a little more technical than Jason’s, and on reflection I think that is because it’s essential to converge the business and technology concerns.

For example, the map suggests a strong capability centric approach to interpret the business morphology. However this is insufficient; the technology must also establish appropriate levels of implementation independence that will facilitate the pluggability of business capabilities. Similarly you might think that considerations regarding the platform and delivery technology (such as MDA/MDD) are irrelevant to business concerns. However the platform and platform delivery technology are potentially massive drivers of rapid iteration and ongoing change, because they encapsulate common application level infrastructure and common services, so understanding the “business” standardization and localization model is crucial to delivering agility through this structure.

References:
Related posts: