8 years, 24 days ago

Changing the Congressional Budget Office to the Congressional Enterprise Architecture Office

The Current Mission of the CBO
Currently, the mission of the Congressional Budget Office (CBO):

is to provide Congress with objective, timely, nonpartisan analyses needed for economic and budget decisions and the information and estimates required for the Congressional budget process” [from CBO TESTIMONY Statement of Robert D. Reischauer Director Congressional Budget Office before the Joint Committee on the Organization of Congress,[the] Congress of the United States]
The director broke that into three operating strategies:
1. Helping the Congress formulate a budget plan;
2. Helping the Congress stay within that plan; and,
3. Helping the Congress consider policy issues related to the budget and the economy.
The Problem with the Current Mission
This Mission and Strategies is part of the economic, political, and social problems currently facing the United States, and, potentially, a source for the solution of those problems. The reason that the CBO is part of the problem is that finance engineering has influenced Congress to emphasize the “financial” part of an overall Enterprise Architecture. That is, Congress proposes functional and component changes to the US Federal Government and the CBO responds with an analysis, which Congress can choose to spin-doctor to its political purposes. Consequently, Congress can choose to support any industry; examples include agriculture (subsidies) and housing (mortgage deductions, etc.), gambling (gambling deductions), and so on.
A Solution the Congressional Enterprise Architecture Office
As I demonstrate in my book, Organizational Economics: The Formation of Wealth, the body performing the controlling (see IDEF0 post) and governing functions of any organization has three Missions, Security, Standards, and Infrastructure.  This is particularly true of any organization that has a spatial domain.  These missions appear in the Preamble of the US Constitution and throughout that document.
Given these three high-level missions, and my discussion of the role and responsibilities of the Enterprise Architect (as a sub-discipline of Systems Engineering), what the US Federal Government needs is a real implementation of the FEA Framework and a formal Enterprise Architecture process.  This process aligns the departments’, agencies’, and other organizations’ of the Federal Government with the three high-level missions of government. 

Additionally, Enterprise Architecture proposes where develop, transform, reform, end or otherwise change the organizations’ missions, strategies, processes, and tooling.  For the US Federal Government, (or any other organization of this scope and size), the EA process must be recursive, but traceable and integratable.  The CBO is in the position with some of the responsibilities for doing this. 

Why not have Congress empower them as the Congressional Enterprise Architecture Offiice?

8 years, 27 days ago

SOA, Cloud Computing, and Event and Model Driven Architecture

SOA and Cloud Computing, the Predicate to Model and Event Driven ArchitectureIn a recent post (see Functions Required in the Cloud PaaS Layer to Support SOA), I discussed two SOA patterns and two Cloud Computing patterns and showed how they are, in fac…

8 years, 27 days ago

SOA, Cloud Computing, and Event and Model Driven Architecture

SOA and Cloud Computing, the Predicate to Model and Event Driven ArchitectureIn a recent post (see Functions Required in the Cloud PaaS Layer to Support SOA), I discussed two SOA patterns and two Cloud Computing patterns and showed how they are, in fac…

8 years, 1 month ago

Housing, Finance, and Government: Three "industries" that produce Minimal Value

The thesis of this post is that it is pretty silly to base an economy, like that of the United States, on housing, finance, and government, which is what Wall St. and Pennsylvania Ave. seem to want to do.Types of IndustriesAll organizations are constru…

8 years, 1 month ago

Housing, Finance, and Government: Three "industries" that produce Minimal Value

The thesis of this post is that it is pretty silly to base an economy, like that of the United States, on housing, finance, and government, which is what Wall St. and Pennsylvania Ave. seem to want to do.

Types of Industries

All organizations are constructed from three types of sub-organizations, which are within their domain.  The Domains would normally be considered as political unit as per example, a city, county, state, or country.  However, even in private organizations, these types of organizations exist, within the organization’s functions and departments.  These organizational categories[i] are:
·         Primary Industry – Organizations that are in an industry that creates a product or service that is exported beyond the boundaries of the domain within which it is produced. 
·         Secondary Industry – Organizations that are in an industry that enables and supports one or more of the processes of the primary industry within the domain it operates.
·         Tertiary Industry – Organizations that in an industry that enable and supports both the primary and secondary industries by providing services that support the environment in the domain within which the primary and secondary industries operate.
As I demonstrate in my book, Organizational Economics: The Formation of Wealth, the primary industry (or industries) is the economic engine that forms the value of the organization for other organizations. Hamel and Prahalad called the turbine of this engine, the organization’s core competence.[ii] It produces the value for the organization.  All other “industries” enable and support this engine.  For example, the economic engine and primary industry for Detroit Michigan, has been and continues to be the automotive industry; in “silicon valley” it’s information technology, the State of Iowa is agriculture, and so on.
Secondary industries are sub-contractors and suppliers of hardware, software, and services to the primary industries.  These industries would include auto parts suppliers, tool manufacturers, transportation within the organizational domain, and other organizations directly supporting the primary industry or industries.
Tertiary industries are organizations that enable and support the personnel, or the domain’s infrastructure.  Schools, colleges, and universities, banks and other financial services, municipal services (e.g., electric, communications, roads and bridges, sewer, water, and so on), food stores, and other stores, hospitals and other medical services, restaurants, fast food outlets, and so on.  In other words, the majority of economic activities within an organizational domain.  Additionally, tertiary industries includes all types of construction.  It also includes the defense (see   Security a Mission of Government).  These industries are where most of the economic activity of an organization occurs.
Some organizational theoreticians include quaternary industries as a category.  These activities include standards and policies (see Standards a Mission of Government) and infrastructure (see Infrastructure a Mission of Government and Organizational Control).

Types of Value

In the first chapter of Organizational Economics, I describe three types values, knowledge value, capacity value, and political value. 
Knowledge value (see Knowledge Value) is value created by an increasing knowledge-base and includes research and development (invention and innovation), and knowledge transfer (education). Products based on new scientific discoveries and transferred into production are the most high valued.  Unique user interface designs like the iPhone or innovative medicines are examples of knowledge value. 
Capacity value (see Capacity Value) is “more of the same” value.  Once a product has been perfected and competitors have brought out versions, then what Adam Smith called “the invisible hand” starts to force reduction in cost of the product.  Many economists refer to the as commoditization of a product, but its value is in capacity production—which produces capacity value.
Political value (see Political Value) is of two types, mediating and exploitive.
·         Mediating (or mediated) political value is created by reducing the organization’s internal process friction.  Examples of mediating political value include contracts, laws, customs, codes, standards, policies, and so on.  In the military, mediating political value (reduction in process friction) comes from “the rules of engagement” (e.g., don’t shoot your fellow military).  The reduction in process friction is very often the difference between a process adding value and a process absorbing value.  The regulation of markets (and the processes of markets, themselves) is such an example.
·         Exploitive political value is indirect or “siphoned” value.  It is caused by someone in the position of responsibility or authority using the position for the reaping of value to their own benefit; “The Lord of the Manor” is the archetypal example, those these include dictators, lobbyists, bankers, day traders, and many judges and legislators.  Further, as I describe in my book, in many cases it includes various religious authorities.

Housing, Finance, and Government as Value creators

My thesis is  that housing, finance, and government either do not create value or very little value.  I base this on the understanding on how these fit within the dimensions described in the previous sections.

Housing

A house is worth a house.  While that seems to be a tautology (and it is), too many people forgot that during “the housing bubble”.  What that saying means is that the value of the house is only what value it imparts to the consumer of the house’s value.  The house is never worth more than when it was built, unless it is maintained and upgraded.  And even when it is upgraded the value of house begins to decrease as it is used (what’s being used, at the most abstract is its value).  The problem, recently, has been that governments tend to inflate their money supply—money being a reserve of value.  With the inflation of money (that is, the decrease in the value of money) the price of a house to increases—though its value remains the same; it’s worth one house.  Likewise, when the housing market “goes down”, the price of the house goes down, but the value remains the same; one house.
House construction and remodeling is a tertiary economic activity.  It produces some capacity value (more of the same value) for the builder and construction workers, but once completed and purchased, it starts loosing value.  In giving the people of the organization a place to live, a house supports the secondary and primary industries of the organization.
Obviously, this is not an activity that enables and supports the formation of wealth for an organization.  Consequently, basing an economy on housing, or at least a significant portion of an economy is foolish and silly.  Yet, in the period from 1995 to 2007, that is what many Americans built the perceived wealth on, and what the United States did.

Finance

Finance includes two subtypes; banks and markets.  The Wall Streeters, (e.g., bankers, hedge fund managers, stockbrokers, pension fund managers and so on) have forgotten that a bank is a value battery and “a market” is the transfer point for the value.
Banks dilute stored value of money through investments that increases risk and potentially increases the amount value through the implementation of discoveries and inventions as new products, systems, or services.  In and of itself, investing cannot increase the amount of value only reduces it.  Only when the money is invested in innovative ideas or the production capability (seeROI Vs VOI) does the value increase, so that, for example, loaning money for a house does not increase the value of the house or create value of any sort.   However, if a bank loans money to a farmer to buy seed or farming implements, the bank has made an investment that does create capacity value—food.  Consequently, banks are tertiary activities that do not produce an increase value, but they loan their repository of potential value (Money) to primary and secondary activities that do.
In the process of each transaction, the bankers siphon off some of the value as a “transaction” fee.  This siphoning is converting potential value into exploitive political value; and exploitive political value is value that is quickly destroyed.
Markets have two missions.  The first is to measure the value of a material, product, or organization. The second is to transform value from real to potential and back; that is trade materials or stocks for money (potential value) or money for materials and stocks.  “Making a market” does both of these; and in this Internet age, anyone can do this.  That is, the person can buy commodities, hold them, and sell them.  In the process, the price of the commodity (be it materials, products, or stocks) converges on a price.
Again, market are tertiary activities that can convert knowledge and capacity value into potential value and the reverse.  And, again, the “market makers” and “stock brokers” that siphon a percentage off, because they are “providing a service” (which to some degree they are), are converting some of the value and potential value into exploitive political value.  Unfortunately, a good many Wall Streeters have turned the markets into legal mega-slot machines, gaming them through “day trading” and even “micro-second trading” to siphon off a much value as possible as quickly as possible, converting it into exploitive political value.

Government

According to my Book, Organizational Economics: The Formation of Wealth, and as note above in this post, a government has three  missions—security, standards, and infrastructure (see. Internal and External security, standards, and infrastructure are mediating political value and all three are tertiary activities, that is, necessary but not sufficient conditions for the growth of value within the domain of the organization.  Further, the second and third activity can be Quaternary.  That is activities, like the enactment of laws and determination of regulations, policies, and standards that enable the standards and infrastructure activities.  These activities are very susceptible to manipulation for personal gain.  The personnel that enact or fund the activities can enjoy an extreme amount of exploitive political value, as I describe in my book.  In the past, it has been the lord of the manor, dictator, duke, king, emir,  priest, shaman, rabbi, Imam, or other religious leader.  Today, lobbyists must be included as they encourage the lawmakers to create uneven economic playing fields that favor one activity or one industry over another; this includes unions and other “not for profit” organizations as well as economic organizations. Consequently, mediated political value is at best much more easily converted into exploitive than either knowledge or capacity value, and is the catalyst for the conversion of these.
In this age, “Entitlements” are the single biggest place that creates exploitive political value.  These safety nets drain value from the infrastructure portion of government.  They are popular because the exploitive value goes into the pockets of the many rather than the few and popular with politicians because Entitlements buy votes.  But, entitlements are unsustainable for any organization as Greece and Italy have proven, and like the United States is likely to prove, now that the population is addicted to Entitlements.  For example, the occupy Wall St. movement feels that all college graduates are “entitled” to jobs (so what value is art history or black studies to an economic organization?).

The Net Result

Too much “unearned income” in too few wallets; too much “Entitlement income” in too many wallets.  I think what I’ve shown is that having an economy based on housing, finance, and government, like that toward which the United States is heading, is a sure recipe for going out of business.
We still have time, but do we have the leadership?

[i]These categories of industries were generally accepted in the 1920s onward, as primary: mining, and agriculture, secondary, manufacturing, and tertiary, services—these definitions are outdated and don’t get at the underlying concepts.  Therefore, I’ve redefined them for a more general meaning of the concepts.
[ii]G. Hamel and C. Prahalad, Competing for the Future: Breakthrough Strategies for Seizing Control of Your Industry and Creating the Markets of Tomorrow, (Boston: Harvard Business School Press, 1994).

8 years, 1 month ago

Systems Engineering and System Architecture in an Agile and Short-cycle Transformation Process (Second Try)

I haven’t updated the blog in a while; but I’ve been working. Here is a copy of a paper that I will present at an INCOSE Meeting in Detroit early in November.  I tried posting a link to a PDF version of the paper and it doesn’t seem to work, so here is a copy.

Bob

Abstract

For a Systems Engineering and Architecture process to tame uncertainty in a continuous change organizational and technology environment requires that the transformation process be agile.  Agility is “the ability to successfully respond to unexpected challenges and opportunities.”  Fundamental to agile systems and engineering and architecture is that assumption that “not all requirements are known upfront”.  This means that the process must allow the customer to add requirements throughout the transformation process.
In this paper, first, I will present a short-cycle CMMI Level 3 software development process, based on Extreme Programming, and anecdotal results from over 100 efforts using the process.  Second, I will generalize this process to include systems engineering and system architecture procedures and functions many types of agile and short-cycle development and transformation efforts.  Third, I will discuss the implications of this process on the roles of the program manager, systems engineer, and system architect.

What and Why Short Agile/Short-cycle Development?

This paper will describe, somewhat anecdotally and somewhat analytically, my experience with agile and short-cycle development and transformational processes. Then it will discuss what I’ve found relative to the change in emphasis in the roles and responsibilities of a project’s team members.
Since the terms “agile” and “short-cycle” are among the many hyped and over used terms, first, let me define what I mean by the terms. 
·         Short-cycle is “One implementation cycle Every 1 to 3 months” depending on the type of development or transformation, as I will describe later.
·         Agility is “The ability to successfully respond to unexpected challenges and opportunities”.[1]
Additionally, the reason I include both “development” and “transformation” is that they are somewhat different from a process perspective.[2]  In development, the team is creating an entirely new product, while in transformation; the team starts with a current product and revises or transforms it.
So why is it important that projects migrate to agile/short-cycle development and transformation processes?  To meet the challenge of “better”, “faster”, ”cheaper” that all customers want.[3]
·         Better” means producing a higher quality product, whether the “product” is a product, service, or process.  Quality is defined as “Conformance to [customer] requirements” by Phil Crosby, in the seminal work on the topic, entitled, Quality is Free.[4]  Any project that focuses on requirements will produce a highly effective, therefore, high quality product.  In any project, “Better” is the responsibility of the systems engineer and the system architect for that project.
·         “Faster” means creating or implementing a useable product in the short period.  Most customers prefer a usable part of a complex system in operation early, than the complete system and wait.  For example, Henry Ford did not wait until electric start was invented to start manufacturing the Model T; yet he sold a fair number of them.  Though his company and others manufactured a number of models before the Model T, many automotive historians would argue that the Model T Ford was the Initial Operating Capability (IOC) for the automotive industry.[5]
·         Cheaper” means more effective and more cost efficient, not the lowest cost.  There is a big difference that “finance engineering” tends to forget.  A stone from the backyard, a hammer, and a nail gun can all drive a nail, so why do carpenters building a house choose a nail gun, by far, the most expensive option?  The reason is that a nail gun can drive nails much more uniformly (i.e., more effectively) and that one carpenter can drive more nails much faster than using a hammer (i.e., more cost effective).

An Example: The Web Application Development Methodology (WADM)[6]

In late 1999 and early 2000, after the group supporting the organization I worked for announced a software CMM Level 3 process, it almost had a rebellion on its hands among the software developers.  The reason was that it could take longer to do the “necessary” paperwork than it took to do the job for small to medium projects (40 to 700 hours).  In the case of the minor project it nearly always took longer.
True of all large organizations, the internal information systems organization I worked for formed a team to “reduce” the SW CMM intermediate artifacts (the paperwork that is used only during the process, but has no value  beyond the end of the project—things like status reports).  Looking at the requirements (in the following section), I realized that a radical rethinking was required to meet them.  I had been researching the Software Engineering Institute’s “evolutionary spiral process” and other Rapid Application Development (RAD) processes and concluded that the simplest approach would be to add functions and documentation to the simplest short-cycle process of them all (eXtreme Programming—XP).  XP started from the premise that developers should develop and not write documentation, which apparently made it the opposite of SW CMM applied to a waterfall process.  In using this approach, but simplifying the process documentation of XP—which for me proved almost indecipherable—and the adding functions and documentation as needed to meet, the SW CMM, and later CMMI level 3 created a highly usable processes and one that was low cost with high customer satisfaction, as I will show.

The requirements

The requirements (or capabilities) for this new process and system had four high-level requirements; they should customer focused, produce a high-quality product, in a short time span, at low cost.

Customer Focus

What the team meant by the requirement for customer focus was:
·         Customer has ownership of, and a high degree of visibility into, the project.  In order for a customer to feel ownership of the project and its deliverable, the product, the customer needs insight into what is going on in the project and a feeling that the product is being influenced by the customer’s inputs (in the form of requirements).  This requires a:
·         High level of customer commitment and involvement in the development team.  The customer must feel like a team member and be treated as such by the rest of the team.  The customer’s role on the team is as the source of the requirements.  This means that the:
·         Customer can add, delete, modify and/or reprioritize functions as the application is being developed. Giving the customer the ability to add, delete, and modify or reprioritize the requirements during the process enables the customer to identify “new” real requirements that were forgotten in the original RFP or contract, but are of higher utility to the customer than the ones in the original source.  Consequently, this:
·         Meets customer’s real demand due to the customer’s degree of influence and control over the development effort.

Produces High Quality Product

What the team meant by the requirement for producing a high quality product is based on Phil Crosby’s definition of quality cited earlier,
·         Product has minimum number of defects.  There are two types of defects.  First, the product does not meet the customer’s requirements (fails validation (i.e., Customer acceptance tests) or second, there is a basic flaw in a function or component of the product.  These defects are minimized by clear identification of requirements—as required above—and clear and consistent high quality verification and validation procedures.  This means that:
·         Testing and defect prevention are integral to the process.  In fact, it means that regression testing is to used, that is, retesting for each build.

Short time Span

What the team meant by short time span is that:
·         The development cycle is divided into successive, short-duration builds.  A short-duration build is one that produced a build every month to six weeks.  Further,
·         The first build and each successive build are usable.  We did not consider it a build if it was not useable.

Low Cost

·         No unnecessary functions are allowed.  Functions that are not required are not developed.  Many times, especially for software, the designers, developers, and implementers add functions to the product that customer’s requirements do not call for.[7] Most of the time, creating functions that are not called for in the requirements means additional time removing them from the product and retesting.  Another requirement for low cost is to ensure that:
·         Project documentation and reviews are minimal.  This is key to a more effective process.  I did this by eliminating all intermediate artifacts, like status reports, and reviews like all of the XDRs (PDR, CDR, and so on) and all PMRs.  Finally, the last requirement of the team for low cost was that the process should:
·         Work with virtual teams.  This was important in the organization I worked for at the time because it was nation-wide.  It had well trained, but idle team members in West Texas, while at the same time, it had a work overload in Baltimore.  Both idle team members and overloaded team members, increases the cost to the organization.  Enabling the team members to work virtually, greatly reduces both costs.

WADM Structure

Figure 1 shows the structure of the WADM process.  It consists of an initialization phase and three cycles; Development, Design, and Construction.
Figure 1 – The WADM Process

Development Cycle

The objective of the Development Cycle is to ensure that the customer’s requirements for development (in priority order by the customer) are understood and met (that is identify customer requirements and validate that the requirements are met). Its duration is approximately one month to six weeks.  It is made up of:
·         A Planning Game[8] where the customer decides on the priority of all unfulfilled requirements and the team, including the customer, decides on which of the highest priority requirements will be implemented during the next cycle.  In many ways the Planning Game replaces the Program Management Review (PMR), except the Planning Game’s emphasis is on meeting the customer’s system requirements, and not its programmatic requirements of cost and schedule.
·          Three to four Design Cycles

Design Cycle

The objective of the Design Cycle is to design an application to meet a prioritized subset of the customer requirements.  Its duration is approximate one week.  It is made up of:
·         A Design Cycle Planning Meeting where the customer reviews the progress of the design in meeting the current requirements and recommends changes.  Sometimes, the developers can incorporate the changing during the next series of construction cycles and sometimes these are really new requirements.
·         Generally five Construction Cycles

Construction Cycle

The objective of the Construction Cycle is to code and unit test modules of the application to meet the current requirements.  Its duration is daily.  It is made up of:
·         A Standup Meeting that is supposed to be 15 minutes long, but that sometimes runs to a half hour.
·         Construction Activities where the developers actually code.
The WADM process meets all of the requirements for an Agile/Short-cycle development or transformation process.  First, it is a Short-Cycle process because a new usable version of the software product every 4 to 6 weeks.  Second it is Agile because it is based on the principle that “Not all the requirements are known upfront”; therefore: it allows the customer to identify new requirements anytime in the process and allows the customer to reprioritize the requirements every 4 to 6 weeks.

Systems Engineering Functions in the WADM

When compared with his or her role and responsibility in a waterfall-base development process the importance of Systems Engineer is much greater in the WADM.  The emphasis in the WADM is on the customers system requirements.  In 2000, I realized there were two types of requirements, the “must perform” and the “must meet”.[9]  Consequently the WADM requirements management procedures use a combination of: Use Cases[10] “One way one type of actor will use the product”, which are the “must perform” requirements, and Design Constraints, which are the “Must Meet” requirements (e.g., standards, regulations, codes, policies, etc.).
Additionally, the Systems Engineer must perform Risk Management procedures.  As defined in the WADM process this is management of the “unknowns in the design”.  These risks frequently arise at the daily standup meetings and can, in most cases, be disposed of at the same meeting or during the next construction cycle.
Finally, the systems engineer must perform the Verification and Validation procedures.  The procedures, themselves must be base on scenarios of the use cases (a scenario is a data instantiated version of the use case, or the verification procedures documented in the design constraints.  Of particular importance for defect minimization is a regression verification and validation procedure, that is, going through all of the verification and validation procedures from the previous development cycles.

Experience with WADM

My experience with the WADM process falls into three part, the pilots, what happened in production, and why were there failures.

WADM Pilots

Starting in 2001, before the process was rolled into production, we had two pilots.  The first we called the Health Visibility Management System (HVMS).  It is a management dashboard for programs and projects graded against program metrics.  Several large programs had their own management dashboard systems.  This made it cumbersome for executive management.  Further, the metric categories were not in same, in the same order, and were not based on uniform data or reporting methods.  Consequently, our customer representative, a former COO of the corporation led a customer team to resolve these issues.  Initially, this project was to provide a framework within which the executive management could gain access to a more uniform set of data from the other HVMS systems.  According to my best estimates of the initial set of requirements, this should have taken about 3 cycles (3 months).  However, as we developed the first three releases, it became obvious to the customer that a) the system was usable, and b) they had many more requirements.  The net result was the initial 3 month project continued at least 7 years, the last time I checked.  Further, our customer representative stated, “This is the only way I’ll ever build software again.”
A second project was the development of Financial and Asset Management System for Data Networks.  At the time, the corporation had merged with nine major organizations and close to 50 small organizations over a 10 year time frame.  At the same time, the data network grew to 8 times its initial size, yet the number of personnel managing “the phone bills” for the corporation had stayed the same.  The result was they were working 80 hours or more, they were getting further behind, and they were catching fewer billing errors.
After I gathered their first set of use cases (their “must perform” requirements), I estimated that it would take at least 6 development cycles (with the personnel available) to meet them.  Unfortunately, the networks department had only enough money for 3 cycles.  So we pared back the requirements to their highest priority set that would make a useable product.  We developed the initial datastore, security, and inputs displays during the first cycle.  While, we were developing some initial reports, in the second cycle, they were inserting data into the datastore.  In this insertion process, the network personnel found enough errors to more than pay for the additional 3 cycles.  By the end of the first six months, also the end of the financial year, the personnel were back to working 40 hours a week, they were catching most of the charging errors, and they could provide senior and executive management timely and correct reports on the data network costs.  Consequently, the management had more requirements.  The last I checked, the project had continued 5 years.  The customer said “This is the first time I got what I wanted without multiple defects or changes and added cost.”

Production (2003 to 2005)

The WADM process was roll into production in late 2002.  By 2005, well over 500 small and medium sized efforts used the process successfully.  The reason it was so successful in its adoption throughout the internal organization was that it met its requirements, better, faster, cheaper.
However, I was called in on two projects that used it that didn’t meet those requirements.  In both cases, the reason is “they didn’t follow the process”.  Instead the program managers attempted to follow the waterfall process and identify all of the requirements upfront and set up a project schedule based on those requirements and stick to the schedule.  The result was they each cost more than four times as much as estimated and did not roll out even the initial iteration of the product.

Shortcomings

The WADM process has some major shortcomings.  Chief among these is that it is designed and implemented only for software development efforts.  It is not adaptable for hardware development or for hardware/software integration efforts.
Further, it cannot handle the complexity of large systems because it lacks systems architecture procedures (functional design processes), component requirement allocation procedures and tradeoff study procedures.  However, these two major limitations can be addressed.

The Generalize Agile Development and Transformation Process

The Generalize Agile Development and Transformation process is envisioned to eliminate the shortcomings of the WADM process.  It requirements includes the WADM Process Requirements of being customer focused, producing a high quality product, in a short time span, and at low cost; all of which were discussed in the previous section.
Additionally, it must support the development or transformation of both hardware and software products.  This would include:
·         System Architecture procedures to develop functional requirements (or specification) and the ordering of these to create a system architecture (or functional design) and the allocation of the functions to components.
·         A more integrated risk management functions for traceability
·         And a Tradeoff study procedure that is traceable to the requirements.
Figure 2 shows the process.
Figure 2 – Generalized Agile Development and Transformation

The Process

This process is made up of the three cycles of the WADM process, plus a fourth System implementation Cycle.  As shown in Figure 2, this process has a Construction Cycle, a Design Cycle, and a Development Cycle or all of its software.  However, additionally it has the System Implementation Cycle.  This cycle has many functions familiar to systems engineers.
The objective of the system implementation cycle is to ensure that the customer’s requirements for implementation of the system are understood and met (that is identify customer requirements and validate that the requirements are met for hardware and the integration of hardware and software).  The duration of this cycle is 3 to 6 months depending on the feasibility of useable hardware and its integration with software and other hardware.  It is made up of Systems Engineering, Detailed Design, Implementation and Procurement Activities.
As was noted in an earlier section, hardware of all types has frequently been developed using this type of short-cycle process.  The first model might only be a prototype or an X-model to try out a new concept.  That is followed by a number of test articles culminating in a fully functional produce.  Even then the process continues with block point upgrades and so on.  Sometimes the Systems Implementation Cycles are treated by program management as a single cycle.  But a close look at the process frequently reveals a hidden short-cycle process.[11] In taking advantage of this “short-cycle attribute of a large process, the development or transformation process can be much more agile.

Changing Roles and Responsibilities in Short-cycle Processes

To take advantage of this attribute within all programs, the team must revise their thinking about the various roles and responsibilities of the team members.  As mentioned earlier the key change in turning a waterfall-based process into an agile/short-cycle process is to its assumption that, “all requirements are known upfront”.  In and of itself this is a heroic assumption that, for all but the simplest effort, is false.  Further, it minimizes the agility of development or transformation effort, by adding all of the intermediate artifacts and reviews required for a formal waterfall process.
However, a truly agile process that takes advantage of short-cycles assumes that “not all requirements are known upfront”.  Any short-cycle process that does not make this assumption will have greatly increased management cost and schedule problems inherent in the waterfall process.

Program Management

With the assumption that “not all requirements are known upfront” the role and responsibilities of the program manager are radically altered.  For example:
·         Project Planning is minimized.  How can a project be fully scheduled and resources identified for unknown requirements?  Instead, the project plan should:
o   Emphasize Project Goals, Missions, Strategies, etc.
o   Emphasize the Systems Engineering Management Plan (SEMP)
·         Minimizes all intermediate artifacts and reviews.
o   There should be no need for status reports or Program Management Reviews (PMRs) since the customer is involved on the effort on at least a weekly basis.
o   There should be no need for Engineering Change Orders (ECOs), since requirements change is built into the process and procedures.
·         There is no need for scheduling, in the traditional waterfall sense.
o   Only high-level schedule possible over the System Implementation Cycle
o   Detailed scheduling is irrelevant (and a waste of resources) because schedules can change on a weekly or monthly bases.
·         Earned Value Analysis is irrelevant in its present form.  Traditional EVA will not work since there is no way of knowing what percentage of the work is complete when the requirements are continuously changing.  Instead, an alternative to EVA will need to be requirements completion based.
·         Agile/Short-cycle Contract Management cannot use a contract based on meeting the initial set of contracted requirement.  Agile/Short-cycle development and transformation processes require “Level of Effort”(LOE) Contracts, since there is no way to guarantee that all of the contracted requirements will be met—more importantly additional  requirements may be found in the process and some initial requirements may not be met during the contract.  Still, the customer will be more satisfied if the product the customer pays for, actually meets his or her highest priority needs.
·         Procurement is very important in an agile/short cycle process because the timing of the procurement of hardware and software is very important.  If the procurement is made too early the equipment or software will set around for awhile, at least, and as the requirements change the team may find that the equipment is not needed or a different version is needed.  On the other hand, if the equipment is procured too late, the project may come to a halt due to needing that hardware or software (though this is less true of short-cycle efforts and waterfall-based efforts because the functions needing the equipment can be reprioritized and be implemented later in the effort).

Systems Engineering

 With the assumption that “not all requirements are known upfront” the role and responsibilities of the systems engineer changes as well.
·         The Requirements Management procedures become seminal to successful agile/short-cycle development and transformation process.  The Requirements Management procedures must enable the customer to add requirements at any time in any cycle of the process.  It must also enable traceability from the product, through the design to the customer’s system requirements.
·         Risk Management must manage risk; risks are unknowns in the design and are very important to identify early.  With agile/short-cycle development and transformation processes this is particularly true because of the need to estimate the cost of each requirement (in the case of WADM, and the Generalized version in this paper, the use cases and design constraints)
·         Configuration Management is a must since a new version will be rolled out every one to three months.  Since the project team wants to insure that the version and components rolled out are the version and components verified and validated, disciplined configuration management in an imperative.  Configuration Management must include managing the Verification and Validation information and documentation to ensure proper regression testing.
·         Verification and Validation procedures must trace directly to the requirements at all levels, customer (A Spec), functional (B Spec) and Component (C Spec).  And both Verification and Validation procedures must be regressive, that is, they must include all prior verification or validation procedures.
From this you can get the idea that for an agile/short-cycle development and transformation effort there is much less emphasis on Program Management procedures and much more on Systems Engineering.

Summary

Agile/Short-cycle development and transformation can increase quality, decrease cost, and reduce the schedule concurrently, but only if the procedures, functions, roles, and responsibilities are changed and executed in a highly disciplined manner. To quote from a 25+ year veteran software developer who had worked 5 years in a SW CMM Level 3 environment: “In the first iteration of this process (the WADM), I thought that documenting all that requirements cr_p was more busy work.  I was wrong, this documentation is really helpful”.


[1] This definition comes from Agility Forum’s Virtual Extended Enterprise committee. The Agility Forum is based at Lehigh University.
[2] Unfortunately, I will have to leave the description of the differences for another presentation.
[3] An aircraft engineer had a saying posted on the wall of his office, “Cost, quality, schedule; choose two”, all customer prefer all three.
[4] Phil Crosby, Quality is Free.  Penguin Group, New York, 1980.
[5] For a discussion of the growth of the automotive industry, see my paper, Industrial Location Behavior and Spatial Evolution, Journal of Industrial Economics, Vol. 5 (1977) pp. 295-312
[6] In 2006 The Northrop Grumman Corporation applied for a patent on this process.  The process has been presented at several open forums since.  It is a CMMI level 3 conformant process
[7] It does also occur in hardware.  One famous example was the WWII M-3 tank where it showed up with a police siren.
[8] For more detail of the Planning Game, looks for papers on XP.
[9] While I will not claim to be the only one that came up with this concept for requirements, I did come up with it independently of others over a period of some 25 years of tortured and frustrating attempts to find a good method to identify requirements.  I had tried use cases without design constraints and found the shortcoming in that and thus backed into the concept of design constraints.
[10] This came directly from XP.
[11] The aircraft Kelly Johnson took advantage of the inherent short-cycle process within large hardware development efforts when he designed the U-2 and the SR-71.
8 years, 1 month ago

What does developing an IT Strategy mean?


I have observed many situations where a c-level person was supposed to document an IT Strategy in a short period of time in order to prepare the following year’s annual budget. Very often lacking much supporting documented business information the result is a weak strategy, sometimes ignored by the user’s community, the key stakeholders.

A weak IT strategy can be costly and wasteful, especially for resource-constrained organizations that operate with minimal budget, tools, knowledge and people.  It also implies that organizations cannot respond to changing business requirements rapidly enough. The absence of strategic anticipation causes organizations to be inefficiently reactive, forcing them to work in a constant state of catch-up.

An IT Strategy should answer the following questions:

  • Are we doing the right things with technology to address the organization’s most important business priorities and continuously deliver value to the clients?
  • Are we making the right technology investments?
  • Do we measure what is the real value to the organization derived from that technology?
  • Is our current Information Technology agile enough; flexible to continuously support a successful organization?
  • Is our Information Technology environment properly managed, maintained, secured, able to support the clients, and is it cost effective?
  • Can our strategy support current and future business needs?

Quite often the first thing we should consider when writing such a document is the targeted audience and its content. Different people with varying roles and responsibilities may read an IT Strategy within a company, so the document may need to serve several different purposes. It is not easy to pitch a strategy to different levels in the hierarchy within an organization, and at the appropriate level of detail. Sometimes it is too detailed and does not always match the stakeholder’s needs.

An IT Strategy is an iterative process to align IT capabilities with the business strategy and requirements:

  • It is a documented and approved process (part of the organization’s governance framework)
  • It is iterative (it needs to be frequently be revisited). Traditionally, IT strategies are updated and communicated on an annual basis, usually to meet budget cycles. This should be considered the minimum review period. If an emerging technology surfaces that has the potential to enhance the business, it should be investigated and communicated to the business as soon as possible. A one-year cycle may be too late.
  • It is a strong alignment of business and IT capabilities rather than designing IT solutions to support business requirements
    • Assuming that both business and IT capabilities drive each other
    • Assuming that business drives IT and not the other way around
  • The IT Strategy sets future direction for IT function in the organization
    • Ensuring that the IT budget is spent on value creation activities for the business
    • Creating shareholder value
    • Helping to maximize the return on IT investments
  • The IT Strategy may include sub-elements such as:
    • Infrastructure strategy
    • Application strategy
    • Integration strategy
    • Service strategy
    • Sourcing strategy
    • Innovation strategy

This pyramid diagram can be used to illustrate the IT strategy and vision, and how the technology and business strategies are totally aligned. At the top of the pyramid is the enterprise overarching vision. Aligned below that is how IT supports the vision by becoming a premier IT organization in creating competitive advantage for the clients. The IT vision is in turn supported by three pillars: integration, improvement, and innovation.

To deliver this , the business strategy should clearly be articulated and documented taking into account some IT aspects. There are different ways of gathering these business inputs.
The first approach is a very classical one where you develop a questionnaire in business terms which asks each business unit to identify their future requirements for infrastructure growth, taking into account capacity and availability requirements. This extracts the data you need for business driven strategy.

This questionnaire may include some of the following examples of questions:

1. What are your top 5 business “pain” points? These are things that you wish you had a solution for
2. What are your top 5 business objectives? These can be short term or long term, can be driven by revenue, cost, time, time to market, competitive advantage, risk or various other reasons
3. How do you plan to achieve these objectives?
4. What will we gain by leveraging IT Capabilities across the business?
5. What is in the way of achieving your business imperatives?
6. Can IT help achieve your business imperatives?
7. How much do you spend on IT capabilities?
8. What is your technology ROI?
9. Does your company have a plan for technology?
10. Does your business plan include a technology plan?
11. Where is IT being used across your business unit?

The second approach would be the use of Enterprise Architecture that I will explain later on.

With this input you may now start to consider the structure of your document. It may look similar to this example below: An executive summary

  • An introduction
    • The purpose
    • The background
    • The Business drivers
    • The Organizational drivers
    • The IT drivers
  • The Business and IT aspects
    • The Business Goals and Objectives
    • The IT approaches and principles
  • The IT components
    • Business application systems
    • IT infrastructure
    • Security and IT Service continuity
  • Structure, organization and management
    • IT Governance
    • Skills, knowledge and education
    • IT Financial management
    • KPIS, measurement and metrics, balance scorecards
  • Technologies opportunities
  • Key issues

And this is where Enterprise Architecture may have to play an important and even crucial role. Some companies I have encountered have an Enterprise Architecture team, and in parallel, somebody called an IT Strategist. Frequently the connection is non-existing or quite weak. Other organizations may also have a Strategic Planning unit, again without any connection with the Enterprise Architecture team.

An Enterprise Architecture must play the important role of assessing; existing IT assets, architecture standards and the desired business strategy to create a unified enterprise-wide environment – where the core hardware and software systems are standardised and integrated across the entire organisation’s business processes, to greatly enhance competitive advantage and innovation. The IT Strategy details the technologies, application and the data foundation needed to deliver the goals of the corporate strategy, while Enterprise Architecture is the bridge between strategy and execution; providing the organising logic to ensure the integration and standardisation of key processes that drive greater agility, higher profitability, faster time to market, lower IT costs, improved access to shared customer data and lower risk of mission-critical systems failures.

As a real example, TOGAF 9 is perfect way to produce that IT Strategy document during the Phase F: Migration Planning.

Below you will find the relationship between some phases of the ADM and the structure of the above document. It needs to be said that we should probably use a Strategic architecture level to deliver a first version of the document, which then could be reviewed with Segment or Capability architectures.

ContentExamplesEnterprise Architecture and TOGAF
An executive summary
An introduction (This document must be business oriented)
ContentExamplesEnterprise Architecture and TOGAF
The purposeKey elements of the scope, audience, time horizonThe Preliminary phase is about defining ‘‘where, what, why, who, and how” Enterprise Architecture will be done and will provide all information. It also creates the conditions and context for an Architecture Capability
The backgroundBusiness problems, constraints (financial, resources, IT, legal, etc.)This is covered by the Phase A: Architecture Vision. An Architecture Vision sets stage for each iteration of ADM cycle.

-Provides high-level, aspirational view of target the sponsor uses to describe how business goals are met and stakeholder concerns are addressed
-Provides an executive summary version of full Architecture
-Drives consensus on desired outcome

The Business Scenarios is used to discover and document business requirements, identify constraints, etc.

The Business driversMarket conditions, competition, consumer trends, new customers, products sales, costs savings, incremental services revenues, drivers related to the IT functionIn the Phase A: Architecture Vision, we:

Identify business goals and strategic drivers
-Ensure that descriptions used are current
-Clarify any areas of ambiguity
Define constraints
-Enterprise-wide constraints
-Architecture project-specific constraints

The Organizational driversProfitability, financial performance, change in strategic objectives, end of the product development life cycle, mergers and acquisitions, staffs, risks
The IT driversNew or obsolete technologies, updatesConsidering that IT is part of the Business, these drivers should also be considered in that phase
The Business and IT aspects
The Business Goals and ObjectivesMarket growth, entering new markets, addressing manufacturing capacitiesIn the Phase A: Architecture Vision, we:

Identify business goals and strategic drivers
-Ensure that descriptions used are current
-Clarify any areas of ambiguity
-Define constraints
-Enterprise-wide constraints
-Architecture project-specific constraints

The IT approaches and principlesIT standards, development, implementation, delivery, testing, consolidation, maturity, best practicesStandards should be documented in the SIB (Standard Information Base)

When we define the Architecture Governance Framework during the Preliminary Phase, we identy the various touch points with existing other frameworks in the organization
IT principles should have already have been defined by the IT department

The IT components
Business application systemsBaseline (main applications: ERP, CRM, customers facing systems). Future plans, concerns, time period, priorities)This will be addressed by Phase C: Information Systems based on the Statement of Architecture Work, output from the Phase A
IT infrastructureBaseline (servers, network , middleware, technical services)This will be addressed by Phase D: Technology Architecture based on the Statement of Architecture Work, output from the Phase A
Security and IT Service continuityIssues, challenges, opportunities related to security, security principles, controlsSecurity concerns are addressed during all phases of the ADM
Structure, organization and management
IT GovernanceBest practices, frameworks, management and monitoring, resource management, portfolio management, vendors management, IT service management, project management, etc.IT Governance will be considered when the Architecture Governance Framework is defined. There will obviously be touch points between the ADM and some other best practices used by the organization. IT Governance is defined outside of the Enterprise Architecture programme
Skills, knowledge and educationSkills, knowledge and educationEnterprise Architecture skills will have to be addressed by the Architecture Capability Framework. Other skills may also be identified independently of the Enterprise Architecture programme
IT Financial managementIT budget, costs structures, measurement and metrics, targets, areas needing investments, etc.This is addressed is outside of the Enterprise Architecture programme
KPIS, measurement and metrics, balance scorecardsIT performance measurements on SMART objectives ((Specific, Measurable, Achievable, Realistic, & Time bound)Every governance frameworks may have its own KPIs. Enterprise Architecture KPIs may be added to that list.
Technologies opportunitiesEmerging technologies, business related benefitsThis can be done in parallel of the Enterprise Architecture programme
Key issues and initiativesSummary or link to the IT Project portfolioThis can be done in parallel of the Enterprise Architecture programme

Color legend
Direct relationship with Enterprise Architecture
Indirect relationship with Enterprise Architecture
Produced somewhere else

The next step would be the review of the IT Strategy document by the main stakeholders who would accept or reject technology opportunities. This could also be used as an important source of information for the Strategic Planning exercise (please refer to another post for additional information: “How Strategic Planning relates to Enterprise Architecture? “).

Once the IT Strategy has been reviewed, amended and authorised (which should in reality already be approved, as it is the result of various ADM cycles and the output of Phase F: Migration planning), the multi-disciplinary programme team for the implementation during Phase G: Implementation Governance, will deliver the solutions to the business.

As already mentioned previously, the outline strategies will be refined and expanded with a low level of detail when addressing Segment and Capability architectures. This is the part that meets the first challenge described above, where we need different levels of detail for different stakeholders. The documents should be hierarchical, with the ability to drill down to lower levels as more detail is required.

One of the main reasons for developing an Enterprise Architecture with TOGAF 9 is to support the business by providing the fundamental technology and process structure for an IT Strategy. Enterprise Architecture is the superset that represents both Business and IT Strategy; this is reflected in Enterprise Architecture’s basic structure of strategy, business architecture and technology/information architecture. One can certainly do an IT Strategy without Enterprise Architecture, but Enterprise Architecture cannot be done without an IT Strategy; the same would apply to business strategy/business architecture.

8 years, 1 month ago

The Chief Process Office of an Agile Organization

The IDEF0 Pattern and the Organization’s Value EngineIn my book, Organizational Economics: the Formation of Wealth, I use the IDEF0 pattern to model the organization.  This pattern has three internal components, Control, Process, and Mechanisms (t…