8 years, 8 months ago

Systems Engineering and System Architecture in an Agile and Short-cycle Transformation Process (Second Try)

I haven’t updated the blog in a while; but I’ve been working. Here is a copy of a paper that I will present at an INCOSE Meeting in Detroit early in November.  I tried posting a link to a PDF version of the paper and it doesn’t seem to work, so here is a copy.



For a Systems Engineering and Architecture process to tame uncertainty in a continuous change organizational and technology environment requires that the transformation process be agile.  Agility is “the ability to successfully respond to unexpected challenges and opportunities.”  Fundamental to agile systems and engineering and architecture is that assumption that “not all requirements are known upfront”.  This means that the process must allow the customer to add requirements throughout the transformation process.
In this paper, first, I will present a short-cycle CMMI Level 3 software development process, based on Extreme Programming, and anecdotal results from over 100 efforts using the process.  Second, I will generalize this process to include systems engineering and system architecture procedures and functions many types of agile and short-cycle development and transformation efforts.  Third, I will discuss the implications of this process on the roles of the program manager, systems engineer, and system architect.

What and Why Short Agile/Short-cycle Development?

This paper will describe, somewhat anecdotally and somewhat analytically, my experience with agile and short-cycle development and transformational processes. Then it will discuss what I’ve found relative to the change in emphasis in the roles and responsibilities of a project’s team members.
Since the terms “agile” and “short-cycle” are among the many hyped and over used terms, first, let me define what I mean by the terms. 
·         Short-cycle is “One implementation cycle Every 1 to 3 months” depending on the type of development or transformation, as I will describe later.
·         Agility is “The ability to successfully respond to unexpected challenges and opportunities”.[1]
Additionally, the reason I include both “development” and “transformation” is that they are somewhat different from a process perspective.[2]  In development, the team is creating an entirely new product, while in transformation; the team starts with a current product and revises or transforms it.
So why is it important that projects migrate to agile/short-cycle development and transformation processes?  To meet the challenge of “better”, “faster”, ”cheaper” that all customers want.[3]
·         Better” means producing a higher quality product, whether the “product” is a product, service, or process.  Quality is defined as “Conformance to [customer] requirements” by Phil Crosby, in the seminal work on the topic, entitled, Quality is Free.[4]  Any project that focuses on requirements will produce a highly effective, therefore, high quality product.  In any project, “Better” is the responsibility of the systems engineer and the system architect for that project.
·         “Faster” means creating or implementing a useable product in the short period.  Most customers prefer a usable part of a complex system in operation early, than the complete system and wait.  For example, Henry Ford did not wait until electric start was invented to start manufacturing the Model T; yet he sold a fair number of them.  Though his company and others manufactured a number of models before the Model T, many automotive historians would argue that the Model T Ford was the Initial Operating Capability (IOC) for the automotive industry.[5]
·         Cheaper” means more effective and more cost efficient, not the lowest cost.  There is a big difference that “finance engineering” tends to forget.  A stone from the backyard, a hammer, and a nail gun can all drive a nail, so why do carpenters building a house choose a nail gun, by far, the most expensive option?  The reason is that a nail gun can drive nails much more uniformly (i.e., more effectively) and that one carpenter can drive more nails much faster than using a hammer (i.e., more cost effective).

An Example: The Web Application Development Methodology (WADM)[6]

In late 1999 and early 2000, after the group supporting the organization I worked for announced a software CMM Level 3 process, it almost had a rebellion on its hands among the software developers.  The reason was that it could take longer to do the “necessary” paperwork than it took to do the job for small to medium projects (40 to 700 hours).  In the case of the minor project it nearly always took longer.
True of all large organizations, the internal information systems organization I worked for formed a team to “reduce” the SW CMM intermediate artifacts (the paperwork that is used only during the process, but has no value  beyond the end of the project—things like status reports).  Looking at the requirements (in the following section), I realized that a radical rethinking was required to meet them.  I had been researching the Software Engineering Institute’s “evolutionary spiral process” and other Rapid Application Development (RAD) processes and concluded that the simplest approach would be to add functions and documentation to the simplest short-cycle process of them all (eXtreme Programming—XP).  XP started from the premise that developers should develop and not write documentation, which apparently made it the opposite of SW CMM applied to a waterfall process.  In using this approach, but simplifying the process documentation of XP—which for me proved almost indecipherable—and the adding functions and documentation as needed to meet, the SW CMM, and later CMMI level 3 created a highly usable processes and one that was low cost with high customer satisfaction, as I will show.

The requirements

The requirements (or capabilities) for this new process and system had four high-level requirements; they should customer focused, produce a high-quality product, in a short time span, at low cost.

Customer Focus

What the team meant by the requirement for customer focus was:
·         Customer has ownership of, and a high degree of visibility into, the project.  In order for a customer to feel ownership of the project and its deliverable, the product, the customer needs insight into what is going on in the project and a feeling that the product is being influenced by the customer’s inputs (in the form of requirements).  This requires a:
·         High level of customer commitment and involvement in the development team.  The customer must feel like a team member and be treated as such by the rest of the team.  The customer’s role on the team is as the source of the requirements.  This means that the:
·         Customer can add, delete, modify and/or reprioritize functions as the application is being developed. Giving the customer the ability to add, delete, and modify or reprioritize the requirements during the process enables the customer to identify “new” real requirements that were forgotten in the original RFP or contract, but are of higher utility to the customer than the ones in the original source.  Consequently, this:
·         Meets customer’s real demand due to the customer’s degree of influence and control over the development effort.

Produces High Quality Product

What the team meant by the requirement for producing a high quality product is based on Phil Crosby’s definition of quality cited earlier,
·         Product has minimum number of defects.  There are two types of defects.  First, the product does not meet the customer’s requirements (fails validation (i.e., Customer acceptance tests) or second, there is a basic flaw in a function or component of the product.  These defects are minimized by clear identification of requirements—as required above—and clear and consistent high quality verification and validation procedures.  This means that:
·         Testing and defect prevention are integral to the process.  In fact, it means that regression testing is to used, that is, retesting for each build.

Short time Span

What the team meant by short time span is that:
·         The development cycle is divided into successive, short-duration builds.  A short-duration build is one that produced a build every month to six weeks.  Further,
·         The first build and each successive build are usable.  We did not consider it a build if it was not useable.

Low Cost

·         No unnecessary functions are allowed.  Functions that are not required are not developed.  Many times, especially for software, the designers, developers, and implementers add functions to the product that customer’s requirements do not call for.[7] Most of the time, creating functions that are not called for in the requirements means additional time removing them from the product and retesting.  Another requirement for low cost is to ensure that:
·         Project documentation and reviews are minimal.  This is key to a more effective process.  I did this by eliminating all intermediate artifacts, like status reports, and reviews like all of the XDRs (PDR, CDR, and so on) and all PMRs.  Finally, the last requirement of the team for low cost was that the process should:
·         Work with virtual teams.  This was important in the organization I worked for at the time because it was nation-wide.  It had well trained, but idle team members in West Texas, while at the same time, it had a work overload in Baltimore.  Both idle team members and overloaded team members, increases the cost to the organization.  Enabling the team members to work virtually, greatly reduces both costs.

WADM Structure

Figure 1 shows the structure of the WADM process.  It consists of an initialization phase and three cycles; Development, Design, and Construction.
Figure 1 – The WADM Process

Development Cycle

The objective of the Development Cycle is to ensure that the customer’s requirements for development (in priority order by the customer) are understood and met (that is identify customer requirements and validate that the requirements are met). Its duration is approximately one month to six weeks.  It is made up of:
·         A Planning Game[8] where the customer decides on the priority of all unfulfilled requirements and the team, including the customer, decides on which of the highest priority requirements will be implemented during the next cycle.  In many ways the Planning Game replaces the Program Management Review (PMR), except the Planning Game’s emphasis is on meeting the customer’s system requirements, and not its programmatic requirements of cost and schedule.
·          Three to four Design Cycles

Design Cycle

The objective of the Design Cycle is to design an application to meet a prioritized subset of the customer requirements.  Its duration is approximate one week.  It is made up of:
·         A Design Cycle Planning Meeting where the customer reviews the progress of the design in meeting the current requirements and recommends changes.  Sometimes, the developers can incorporate the changing during the next series of construction cycles and sometimes these are really new requirements.
·         Generally five Construction Cycles

Construction Cycle

The objective of the Construction Cycle is to code and unit test modules of the application to meet the current requirements.  Its duration is daily.  It is made up of:
·         A Standup Meeting that is supposed to be 15 minutes long, but that sometimes runs to a half hour.
·         Construction Activities where the developers actually code.
The WADM process meets all of the requirements for an Agile/Short-cycle development or transformation process.  First, it is a Short-Cycle process because a new usable version of the software product every 4 to 6 weeks.  Second it is Agile because it is based on the principle that “Not all the requirements are known upfront”; therefore: it allows the customer to identify new requirements anytime in the process and allows the customer to reprioritize the requirements every 4 to 6 weeks.

Systems Engineering Functions in the WADM

When compared with his or her role and responsibility in a waterfall-base development process the importance of Systems Engineer is much greater in the WADM.  The emphasis in the WADM is on the customers system requirements.  In 2000, I realized there were two types of requirements, the “must perform” and the “must meet”.[9]  Consequently the WADM requirements management procedures use a combination of: Use Cases[10] “One way one type of actor will use the product”, which are the “must perform” requirements, and Design Constraints, which are the “Must Meet” requirements (e.g., standards, regulations, codes, policies, etc.).
Additionally, the Systems Engineer must perform Risk Management procedures.  As defined in the WADM process this is management of the “unknowns in the design”.  These risks frequently arise at the daily standup meetings and can, in most cases, be disposed of at the same meeting or during the next construction cycle.
Finally, the systems engineer must perform the Verification and Validation procedures.  The procedures, themselves must be base on scenarios of the use cases (a scenario is a data instantiated version of the use case, or the verification procedures documented in the design constraints.  Of particular importance for defect minimization is a regression verification and validation procedure, that is, going through all of the verification and validation procedures from the previous development cycles.

Experience with WADM

My experience with the WADM process falls into three part, the pilots, what happened in production, and why were there failures.

WADM Pilots

Starting in 2001, before the process was rolled into production, we had two pilots.  The first we called the Health Visibility Management System (HVMS).  It is a management dashboard for programs and projects graded against program metrics.  Several large programs had their own management dashboard systems.  This made it cumbersome for executive management.  Further, the metric categories were not in same, in the same order, and were not based on uniform data or reporting methods.  Consequently, our customer representative, a former COO of the corporation led a customer team to resolve these issues.  Initially, this project was to provide a framework within which the executive management could gain access to a more uniform set of data from the other HVMS systems.  According to my best estimates of the initial set of requirements, this should have taken about 3 cycles (3 months).  However, as we developed the first three releases, it became obvious to the customer that a) the system was usable, and b) they had many more requirements.  The net result was the initial 3 month project continued at least 7 years, the last time I checked.  Further, our customer representative stated, “This is the only way I’ll ever build software again.”
A second project was the development of Financial and Asset Management System for Data Networks.  At the time, the corporation had merged with nine major organizations and close to 50 small organizations over a 10 year time frame.  At the same time, the data network grew to 8 times its initial size, yet the number of personnel managing “the phone bills” for the corporation had stayed the same.  The result was they were working 80 hours or more, they were getting further behind, and they were catching fewer billing errors.
After I gathered their first set of use cases (their “must perform” requirements), I estimated that it would take at least 6 development cycles (with the personnel available) to meet them.  Unfortunately, the networks department had only enough money for 3 cycles.  So we pared back the requirements to their highest priority set that would make a useable product.  We developed the initial datastore, security, and inputs displays during the first cycle.  While, we were developing some initial reports, in the second cycle, they were inserting data into the datastore.  In this insertion process, the network personnel found enough errors to more than pay for the additional 3 cycles.  By the end of the first six months, also the end of the financial year, the personnel were back to working 40 hours a week, they were catching most of the charging errors, and they could provide senior and executive management timely and correct reports on the data network costs.  Consequently, the management had more requirements.  The last I checked, the project had continued 5 years.  The customer said “This is the first time I got what I wanted without multiple defects or changes and added cost.”

Production (2003 to 2005)

The WADM process was roll into production in late 2002.  By 2005, well over 500 small and medium sized efforts used the process successfully.  The reason it was so successful in its adoption throughout the internal organization was that it met its requirements, better, faster, cheaper.
However, I was called in on two projects that used it that didn’t meet those requirements.  In both cases, the reason is “they didn’t follow the process”.  Instead the program managers attempted to follow the waterfall process and identify all of the requirements upfront and set up a project schedule based on those requirements and stick to the schedule.  The result was they each cost more than four times as much as estimated and did not roll out even the initial iteration of the product.


The WADM process has some major shortcomings.  Chief among these is that it is designed and implemented only for software development efforts.  It is not adaptable for hardware development or for hardware/software integration efforts.
Further, it cannot handle the complexity of large systems because it lacks systems architecture procedures (functional design processes), component requirement allocation procedures and tradeoff study procedures.  However, these two major limitations can be addressed.

The Generalize Agile Development and Transformation Process

The Generalize Agile Development and Transformation process is envisioned to eliminate the shortcomings of the WADM process.  It requirements includes the WADM Process Requirements of being customer focused, producing a high quality product, in a short time span, and at low cost; all of which were discussed in the previous section.
Additionally, it must support the development or transformation of both hardware and software products.  This would include:
·         System Architecture procedures to develop functional requirements (or specification) and the ordering of these to create a system architecture (or functional design) and the allocation of the functions to components.
·         A more integrated risk management functions for traceability
·         And a Tradeoff study procedure that is traceable to the requirements.
Figure 2 shows the process.
Figure 2 – Generalized Agile Development and Transformation

The Process

This process is made up of the three cycles of the WADM process, plus a fourth System implementation Cycle.  As shown in Figure 2, this process has a Construction Cycle, a Design Cycle, and a Development Cycle or all of its software.  However, additionally it has the System Implementation Cycle.  This cycle has many functions familiar to systems engineers.
The objective of the system implementation cycle is to ensure that the customer’s requirements for implementation of the system are understood and met (that is identify customer requirements and validate that the requirements are met for hardware and the integration of hardware and software).  The duration of this cycle is 3 to 6 months depending on the feasibility of useable hardware and its integration with software and other hardware.  It is made up of Systems Engineering, Detailed Design, Implementation and Procurement Activities.
As was noted in an earlier section, hardware of all types has frequently been developed using this type of short-cycle process.  The first model might only be a prototype or an X-model to try out a new concept.  That is followed by a number of test articles culminating in a fully functional produce.  Even then the process continues with block point upgrades and so on.  Sometimes the Systems Implementation Cycles are treated by program management as a single cycle.  But a close look at the process frequently reveals a hidden short-cycle process.[11] In taking advantage of this “short-cycle attribute of a large process, the development or transformation process can be much more agile.

Changing Roles and Responsibilities in Short-cycle Processes

To take advantage of this attribute within all programs, the team must revise their thinking about the various roles and responsibilities of the team members.  As mentioned earlier the key change in turning a waterfall-based process into an agile/short-cycle process is to its assumption that, “all requirements are known upfront”.  In and of itself this is a heroic assumption that, for all but the simplest effort, is false.  Further, it minimizes the agility of development or transformation effort, by adding all of the intermediate artifacts and reviews required for a formal waterfall process.
However, a truly agile process that takes advantage of short-cycles assumes that “not all requirements are known upfront”.  Any short-cycle process that does not make this assumption will have greatly increased management cost and schedule problems inherent in the waterfall process.

Program Management

With the assumption that “not all requirements are known upfront” the role and responsibilities of the program manager are radically altered.  For example:
·         Project Planning is minimized.  How can a project be fully scheduled and resources identified for unknown requirements?  Instead, the project plan should:
o   Emphasize Project Goals, Missions, Strategies, etc.
o   Emphasize the Systems Engineering Management Plan (SEMP)
·         Minimizes all intermediate artifacts and reviews.
o   There should be no need for status reports or Program Management Reviews (PMRs) since the customer is involved on the effort on at least a weekly basis.
o   There should be no need for Engineering Change Orders (ECOs), since requirements change is built into the process and procedures.
·         There is no need for scheduling, in the traditional waterfall sense.
o   Only high-level schedule possible over the System Implementation Cycle
o   Detailed scheduling is irrelevant (and a waste of resources) because schedules can change on a weekly or monthly bases.
·         Earned Value Analysis is irrelevant in its present form.  Traditional EVA will not work since there is no way of knowing what percentage of the work is complete when the requirements are continuously changing.  Instead, an alternative to EVA will need to be requirements completion based.
·         Agile/Short-cycle Contract Management cannot use a contract based on meeting the initial set of contracted requirement.  Agile/Short-cycle development and transformation processes require “Level of Effort”(LOE) Contracts, since there is no way to guarantee that all of the contracted requirements will be met—more importantly additional  requirements may be found in the process and some initial requirements may not be met during the contract.  Still, the customer will be more satisfied if the product the customer pays for, actually meets his or her highest priority needs.
·         Procurement is very important in an agile/short cycle process because the timing of the procurement of hardware and software is very important.  If the procurement is made too early the equipment or software will set around for awhile, at least, and as the requirements change the team may find that the equipment is not needed or a different version is needed.  On the other hand, if the equipment is procured too late, the project may come to a halt due to needing that hardware or software (though this is less true of short-cycle efforts and waterfall-based efforts because the functions needing the equipment can be reprioritized and be implemented later in the effort).

Systems Engineering

 With the assumption that “not all requirements are known upfront” the role and responsibilities of the systems engineer changes as well.
·         The Requirements Management procedures become seminal to successful agile/short-cycle development and transformation process.  The Requirements Management procedures must enable the customer to add requirements at any time in any cycle of the process.  It must also enable traceability from the product, through the design to the customer’s system requirements.
·         Risk Management must manage risk; risks are unknowns in the design and are very important to identify early.  With agile/short-cycle development and transformation processes this is particularly true because of the need to estimate the cost of each requirement (in the case of WADM, and the Generalized version in this paper, the use cases and design constraints)
·         Configuration Management is a must since a new version will be rolled out every one to three months.  Since the project team wants to insure that the version and components rolled out are the version and components verified and validated, disciplined configuration management in an imperative.  Configuration Management must include managing the Verification and Validation information and documentation to ensure proper regression testing.
·         Verification and Validation procedures must trace directly to the requirements at all levels, customer (A Spec), functional (B Spec) and Component (C Spec).  And both Verification and Validation procedures must be regressive, that is, they must include all prior verification or validation procedures.
From this you can get the idea that for an agile/short-cycle development and transformation effort there is much less emphasis on Program Management procedures and much more on Systems Engineering.


Agile/Short-cycle development and transformation can increase quality, decrease cost, and reduce the schedule concurrently, but only if the procedures, functions, roles, and responsibilities are changed and executed in a highly disciplined manner. To quote from a 25+ year veteran software developer who had worked 5 years in a SW CMM Level 3 environment: “In the first iteration of this process (the WADM), I thought that documenting all that requirements cr_p was more busy work.  I was wrong, this documentation is really helpful”.

[1] This definition comes from Agility Forum’s Virtual Extended Enterprise committee. The Agility Forum is based at Lehigh University.
[2] Unfortunately, I will have to leave the description of the differences for another presentation.
[3] An aircraft engineer had a saying posted on the wall of his office, “Cost, quality, schedule; choose two”, all customer prefer all three.
[4] Phil Crosby, Quality is Free.  Penguin Group, New York, 1980.
[5] For a discussion of the growth of the automotive industry, see my paper, Industrial Location Behavior and Spatial Evolution, Journal of Industrial Economics, Vol. 5 (1977) pp. 295-312
[6] In 2006 The Northrop Grumman Corporation applied for a patent on this process.  The process has been presented at several open forums since.  It is a CMMI level 3 conformant process
[7] It does also occur in hardware.  One famous example was the WWII M-3 tank where it showed up with a police siren.
[8] For more detail of the Planning Game, looks for papers on XP.
[9] While I will not claim to be the only one that came up with this concept for requirements, I did come up with it independently of others over a period of some 25 years of tortured and frustrating attempts to find a good method to identify requirements.  I had tried use cases without design constraints and found the shortcoming in that and thus backed into the concept of design constraints.
[10] This came directly from XP.
[11] The aircraft Kelly Johnson took advantage of the inherent short-cycle process within large hardware development efforts when he designed the U-2 and the SR-71.
8 years, 8 months ago

What does developing an IT Strategy mean?

I have observed many situations where a c-level person was supposed to document an IT Strategy in a short period of time in order to prepare the following year’s annual budget. Very often lacking much supporting documented business information the result is a weak strategy, sometimes ignored by the user’s community, the key stakeholders.

A weak IT strategy can be costly and wasteful, especially for resource-constrained organizations that operate with minimal budget, tools, knowledge and people.  It also implies that organizations cannot respond to changing business requirements rapidly enough. The absence of strategic anticipation causes organizations to be inefficiently reactive, forcing them to work in a constant state of catch-up.

An IT Strategy should answer the following questions:

  • Are we doing the right things with technology to address the organization’s most important business priorities and continuously deliver value to the clients?
  • Are we making the right technology investments?
  • Do we measure what is the real value to the organization derived from that technology?
  • Is our current Information Technology agile enough; flexible to continuously support a successful organization?
  • Is our Information Technology environment properly managed, maintained, secured, able to support the clients, and is it cost effective?
  • Can our strategy support current and future business needs?

Quite often the first thing we should consider when writing such a document is the targeted audience and its content. Different people with varying roles and responsibilities may read an IT Strategy within a company, so the document may need to serve several different purposes. It is not easy to pitch a strategy to different levels in the hierarchy within an organization, and at the appropriate level of detail. Sometimes it is too detailed and does not always match the stakeholder’s needs.

An IT Strategy is an iterative process to align IT capabilities with the business strategy and requirements:

  • It is a documented and approved process (part of the organization’s governance framework)
  • It is iterative (it needs to be frequently be revisited). Traditionally, IT strategies are updated and communicated on an annual basis, usually to meet budget cycles. This should be considered the minimum review period. If an emerging technology surfaces that has the potential to enhance the business, it should be investigated and communicated to the business as soon as possible. A one-year cycle may be too late.
  • It is a strong alignment of business and IT capabilities rather than designing IT solutions to support business requirements
    • Assuming that both business and IT capabilities drive each other
    • Assuming that business drives IT and not the other way around
  • The IT Strategy sets future direction for IT function in the organization
    • Ensuring that the IT budget is spent on value creation activities for the business
    • Creating shareholder value
    • Helping to maximize the return on IT investments
  • The IT Strategy may include sub-elements such as:
    • Infrastructure strategy
    • Application strategy
    • Integration strategy
    • Service strategy
    • Sourcing strategy
    • Innovation strategy

This pyramid diagram can be used to illustrate the IT strategy and vision, and how the technology and business strategies are totally aligned. At the top of the pyramid is the enterprise overarching vision. Aligned below that is how IT supports the vision by becoming a premier IT organization in creating competitive advantage for the clients. The IT vision is in turn supported by three pillars: integration, improvement, and innovation.

To deliver this , the business strategy should clearly be articulated and documented taking into account some IT aspects. There are different ways of gathering these business inputs.
The first approach is a very classical one where you develop a questionnaire in business terms which asks each business unit to identify their future requirements for infrastructure growth, taking into account capacity and availability requirements. This extracts the data you need for business driven strategy.

This questionnaire may include some of the following examples of questions:

1. What are your top 5 business “pain” points? These are things that you wish you had a solution for
2. What are your top 5 business objectives? These can be short term or long term, can be driven by revenue, cost, time, time to market, competitive advantage, risk or various other reasons
3. How do you plan to achieve these objectives?
4. What will we gain by leveraging IT Capabilities across the business?
5. What is in the way of achieving your business imperatives?
6. Can IT help achieve your business imperatives?
7. How much do you spend on IT capabilities?
8. What is your technology ROI?
9. Does your company have a plan for technology?
10. Does your business plan include a technology plan?
11. Where is IT being used across your business unit?

The second approach would be the use of Enterprise Architecture that I will explain later on.

With this input you may now start to consider the structure of your document. It may look similar to this example below: An executive summary

  • An introduction
    • The purpose
    • The background
    • The Business drivers
    • The Organizational drivers
    • The IT drivers
  • The Business and IT aspects
    • The Business Goals and Objectives
    • The IT approaches and principles
  • The IT components
    • Business application systems
    • IT infrastructure
    • Security and IT Service continuity
  • Structure, organization and management
    • IT Governance
    • Skills, knowledge and education
    • IT Financial management
    • KPIS, measurement and metrics, balance scorecards
  • Technologies opportunities
  • Key issues

And this is where Enterprise Architecture may have to play an important and even crucial role. Some companies I have encountered have an Enterprise Architecture team, and in parallel, somebody called an IT Strategist. Frequently the connection is non-existing or quite weak. Other organizations may also have a Strategic Planning unit, again without any connection with the Enterprise Architecture team.

An Enterprise Architecture must play the important role of assessing; existing IT assets, architecture standards and the desired business strategy to create a unified enterprise-wide environment – where the core hardware and software systems are standardised and integrated across the entire organisation’s business processes, to greatly enhance competitive advantage and innovation. The IT Strategy details the technologies, application and the data foundation needed to deliver the goals of the corporate strategy, while Enterprise Architecture is the bridge between strategy and execution; providing the organising logic to ensure the integration and standardisation of key processes that drive greater agility, higher profitability, faster time to market, lower IT costs, improved access to shared customer data and lower risk of mission-critical systems failures.

As a real example, TOGAF 9 is perfect way to produce that IT Strategy document during the Phase F: Migration Planning.

Below you will find the relationship between some phases of the ADM and the structure of the above document. It needs to be said that we should probably use a Strategic architecture level to deliver a first version of the document, which then could be reviewed with Segment or Capability architectures.

ContentExamplesEnterprise Architecture and TOGAF
An executive summary
An introduction (This document must be business oriented)
ContentExamplesEnterprise Architecture and TOGAF
The purposeKey elements of the scope, audience, time horizonThe Preliminary phase is about defining ‘‘where, what, why, who, and how” Enterprise Architecture will be done and will provide all information. It also creates the conditions and context for an Architecture Capability
The backgroundBusiness problems, constraints (financial, resources, IT, legal, etc.)This is covered by the Phase A: Architecture Vision. An Architecture Vision sets stage for each iteration of ADM cycle.

-Provides high-level, aspirational view of target the sponsor uses to describe how business goals are met and stakeholder concerns are addressed
-Provides an executive summary version of full Architecture
-Drives consensus on desired outcome

The Business Scenarios is used to discover and document business requirements, identify constraints, etc.

The Business driversMarket conditions, competition, consumer trends, new customers, products sales, costs savings, incremental services revenues, drivers related to the IT functionIn the Phase A: Architecture Vision, we:

Identify business goals and strategic drivers
-Ensure that descriptions used are current
-Clarify any areas of ambiguity
Define constraints
-Enterprise-wide constraints
-Architecture project-specific constraints

The Organizational driversProfitability, financial performance, change in strategic objectives, end of the product development life cycle, mergers and acquisitions, staffs, risks
The IT driversNew or obsolete technologies, updatesConsidering that IT is part of the Business, these drivers should also be considered in that phase
The Business and IT aspects
The Business Goals and ObjectivesMarket growth, entering new markets, addressing manufacturing capacitiesIn the Phase A: Architecture Vision, we:

Identify business goals and strategic drivers
-Ensure that descriptions used are current
-Clarify any areas of ambiguity
-Define constraints
-Enterprise-wide constraints
-Architecture project-specific constraints

The IT approaches and principlesIT standards, development, implementation, delivery, testing, consolidation, maturity, best practicesStandards should be documented in the SIB (Standard Information Base)

When we define the Architecture Governance Framework during the Preliminary Phase, we identy the various touch points with existing other frameworks in the organization
IT principles should have already have been defined by the IT department

The IT components
Business application systemsBaseline (main applications: ERP, CRM, customers facing systems). Future plans, concerns, time period, priorities)This will be addressed by Phase C: Information Systems based on the Statement of Architecture Work, output from the Phase A
IT infrastructureBaseline (servers, network , middleware, technical services)This will be addressed by Phase D: Technology Architecture based on the Statement of Architecture Work, output from the Phase A
Security and IT Service continuityIssues, challenges, opportunities related to security, security principles, controlsSecurity concerns are addressed during all phases of the ADM
Structure, organization and management
IT GovernanceBest practices, frameworks, management and monitoring, resource management, portfolio management, vendors management, IT service management, project management, etc.IT Governance will be considered when the Architecture Governance Framework is defined. There will obviously be touch points between the ADM and some other best practices used by the organization. IT Governance is defined outside of the Enterprise Architecture programme
Skills, knowledge and educationSkills, knowledge and educationEnterprise Architecture skills will have to be addressed by the Architecture Capability Framework. Other skills may also be identified independently of the Enterprise Architecture programme
IT Financial managementIT budget, costs structures, measurement and metrics, targets, areas needing investments, etc.This is addressed is outside of the Enterprise Architecture programme
KPIS, measurement and metrics, balance scorecardsIT performance measurements on SMART objectives ((Specific, Measurable, Achievable, Realistic, & Time bound)Every governance frameworks may have its own KPIs. Enterprise Architecture KPIs may be added to that list.
Technologies opportunitiesEmerging technologies, business related benefitsThis can be done in parallel of the Enterprise Architecture programme
Key issues and initiativesSummary or link to the IT Project portfolioThis can be done in parallel of the Enterprise Architecture programme

Color legend
Direct relationship with Enterprise Architecture
Indirect relationship with Enterprise Architecture
Produced somewhere else

The next step would be the review of the IT Strategy document by the main stakeholders who would accept or reject technology opportunities. This could also be used as an important source of information for the Strategic Planning exercise (please refer to another post for additional information: “How Strategic Planning relates to Enterprise Architecture? “).

Once the IT Strategy has been reviewed, amended and authorised (which should in reality already be approved, as it is the result of various ADM cycles and the output of Phase F: Migration planning), the multi-disciplinary programme team for the implementation during Phase G: Implementation Governance, will deliver the solutions to the business.

As already mentioned previously, the outline strategies will be refined and expanded with a low level of detail when addressing Segment and Capability architectures. This is the part that meets the first challenge described above, where we need different levels of detail for different stakeholders. The documents should be hierarchical, with the ability to drill down to lower levels as more detail is required.

One of the main reasons for developing an Enterprise Architecture with TOGAF 9 is to support the business by providing the fundamental technology and process structure for an IT Strategy. Enterprise Architecture is the superset that represents both Business and IT Strategy; this is reflected in Enterprise Architecture’s basic structure of strategy, business architecture and technology/information architecture. One can certainly do an IT Strategy without Enterprise Architecture, but Enterprise Architecture cannot be done without an IT Strategy; the same would apply to business strategy/business architecture.

8 years, 8 months ago

The Chief Process Office of an Agile Organization

The IDEF0 Pattern and the Organization’s Value EngineIn my book, Organizational Economics: the Formation of Wealth, I use the IDEF0 pattern to model the organization.  This pattern has three internal components, Control, Process, and Mechanisms (t…

8 years, 8 months ago

Satisficing Behavior

In economic theory, the is an assumption that customers and consumers decide on products that satisfy their demand.  This has simplified customer behavior sufficiently for microeconomics theory to create mathematical models.  For th…

8 years, 8 months ago

Gartner: Top 10 Strategic Technologies for 2012

Gartner recently published the top 10 technologies and trends that will be strategic for most organizations in 2012. Gartner defines a strategic technology as one with the potential for significant impact on the enterprise in the next three years. It may be an existing technology that has matured and/or become suitable for a wider range of uses. It may also be an emerging technology that offers an opportunity for strategic business advantage for early adopters or with potential for significant market disruption in the next five years. These technologies impact the organization’s long-term plans, programs and initiatives. They are as following;

Media Tablets and Beyond. Users can choose between various form factors when it comes to mobile computing. No single platform, form factor or technology will dominate and companies should expect to manage a diverse environment with two to four intelligent clients through 2015.

Mobile-Centric Applications and Interfaces. The user interface (IU) paradigm in place for more than 20 years is changing. UIs with windows, icons, menus, and pointers will be replaced by mobile-centric interfaces emphasizing touch, gesture, search, voice and video.

Contextual and Social User Experience. Context-aware computing uses information about an end-user or objects environment, activities, connections and preferences to improve the quality of interaction with that end-user or object.

Internet of Things. The Internet of Things (IoT) is a concept that describes how the Internet will expand as sensors and intelligence are added to physical items such as consumer devices or physical assets and these objects are connected to the Internet.

App Stores and Marketplaces. Application stores by Apple and Android provide marketplaces where hundreds of thousands of applications are available to mobile users. Gartner forecasts that by 2014, there will be more than 70 billion mobile application downloads from app stores every year.

Next-Generation Analytics. Analytics is growing along three key dimensions:

  1. From traditional offline analytics to in-line embedded analytics.
  2. From analyzing historical data to explain what happened to analyzing historical and real-time data from multiple systems to simulate and predict the future.
  3. from structured and simple data analyzed by individuals to analysis of complex information of many types (text, video, etc…) from many systems

Big Data. The size, complexity of formats and speed of delivery exceeds the capabilities of traditional data management technologies; it requires the use of new or exotic technologies simply to manage the volume alone.

In-Memory Computing. Gartner sees huge use of flash memory in consumer devices, entertainment equipment and other embedded IT systems. In addition, it offers a new layer of the memory hierarchy in servers that has key advantages — space, heat, performance and ruggedness among them.

Extreme Low-Energy Servers. The adoption of low-energy servers potentially delivering 30 times or more processors in a particular server unit with lower power consumption vs. current server approaches.

Cloud Computing. Cloud is a disruptive force and has the potential for broad long-term impact in most industries. While the market remains in its early stages in 2011 and 2012, it will see the full range of large enterprise providers fully engaged in delivering a range of offerings to build cloud environments and deliver cloud services.

“These top 10 technologies will be strategic for most organizations, and IT leaders should use this list in their strategic planning process by reviewing the technologies and how they fit into their expected needs,” said David Cearley, vice president and Gartner fellow.

The complete related Gartner press release can be accessed here.

8 years, 9 months ago

What is Business Technology Management and how does it relate to Enterprise Architecture?

Business Technology Management (BTM) is not a methodology but I would say a concept, or eventually the aggregation of several guidelines and techniques. It is also described as a management science which aims to unify business and technology business strategies with the aim of extracting the full potential value of business technology solutions. In a nutshell, it allows you to unify business and technology decision making. Sounds familiar?
Pragmatically it corresponds to a group of various services intended to help businesses communities. BTM can include different methods such as IT planning, Project and Portfolio management (e.g. PMI, Prince 2), Balance Scorecards, Business support, Database services, disaster recovery, network management, security, document service, and frameworks. BTM delivers a set of guiding principles known as capabilities and defines the expected characteristics of an organization according to five levels of a maturity mode like CMMi. While these methods/methodologies have recognized strengths, they represent a piecemeal approach. There is a need to integrate these capabilities to achieve that strategic business technology alignment because most of these methods do not really focus on the goals and objectives of an enterprise. Balance Scorecard is a performance measurement methodology, Six Sigma or Lean are quality improvement methodologies mostly used in manufacturing, and so on…
BTM may sound like an evolved IT governance concept, where business and IT are in tune in an effort to support and realise the enterprise strategy. But does it really differ from an Enterprise Architecture which sometimes may also be considered as being the glue between various methods/methodologies?
Some questions may quickly arise…Is BTM just “better IT Governance” or simply a different way of naming an Enterprise Architecture? And does TOGAF® support BTM? BTM like Enterprise Architecture aligns activities which remain pure business and some pure technology, but most activities intertwine business and technology such that they become indistinguishable. It also guides and supports enterprises to these various states.
The precepts of Business Technology Management have been developed and refined by BTM experts working with such think tanks as the BTM Institute and the International Institute of Business Technologies (IIBT).

Business Technology Management addresses four critical dimensions of enterprise-wide strategy
1. Process
This first dimension refers to the institution of a set of robust, flexible and repeatable processes, broadly defined as:
General quality of Business Practice: Doing the right things
Efficiency: Doing things efficiently, quickly with little redundancy
Effectiveness: Doing things well
The TOGAF® 9 ADM is an example of such processes with its associated governance framework.
2. Organisation
Management processes are more likely to succeed when it refers to the establishment of appropriate organisational structures, establishing a structure in which every member understands the scope and responsibilities of his or her role, and decision rights. Something perfectly addressed during the Phases Preliminary and A of TOGAF®.
Organizational structures may include
· Participative bodies involving senior level business and technology participants on a part-time but routine basis (e.g. Business Technology Investment Board). TOGAF® suggests the creation of an architecture board who participate with the key business stakeholders.
· Centralized bodies requiring specialized dedicated technology staff (e.g. PMO).
· Need-based bodies involving rotational assignments dealing with particular efforts (e.g. PMO, Project Management teams).
Both last bodies would be identified during Phase F: Migration Planning and Phase G: Implementation Governance
3. Information
Valid, effective, timely provision of information is a prerequisite in effective decision making. Information must be delivered in a way that is comprehensible to non specialists as well. Data and metrics must be available. This would be addressed by the Communication Plan defined in the TOGAF® Architecture Vision’s phase taking into account the stakeholders needs, the communication mechanisms and timetable. Measurements and metrics may be included for strategic and operational objectives.
4. Technology
Effective technology can help connect the other three dimensions. The idea is that technology plays a vital role in all processes and can enable timely information sharing, improve co-ordination between members of an organisation and makes processes easier to execute. This covers automation of tasks, reporting, analytics and integration between management systems. In Enterprise Architecture, this would be covered by the interoperability requirements identified by the business and the identification of appropriate solutions in the TOGAF® Phases E and F, such as BPM suites and BI products.

Business Technology Management (BTM) Capabilities
A BTM capability is defined as a competency achieved by combining each of the above dimensions and creating repeatable management processes that are executed with the appropriate organizational structures, using an effective information architecture.
Business Technology Management defines 17 of these specific capabilities, each grouped into one of four functional areas.
Governance and Organisation:  These capabilities are focused on the enterprise’s CIO and business executives concerned with enterprise wide governance of business technology. It ensures that business technology decisions are effectively identified and executed, meet the needs of the business, manages the risk and give proper consideration to regulatory, legal and industry requirements. TOGAF® addresses all of this in the Preliminary Phase and the Architecture Vision, where an enterprise architecture governance framework is created.
Managing Technology Investments : This sits with PMO and business executives who are concerned with the selection and execution of the right business technologies initiatives and fulfil their objectives. The enterprise understands its current IT capabilities, what is currently available and what it is working on for the future. This is equally addressed during the Phase F: Migration Planning.
Strategy & Planning: These capabilities ensure that the CIO and business executives make the most appropriate moves to synchronise technology and business, both reducing complexity and planning for future developments. Enterprise Architecture and TOGAF 9 undoubtedly support these capabilities; you may refer to the previous article “How Strategic Planning relates to Enterprise Architecture”.
Strategic Enterprise Architecture: This capability must be developed to support this functional area, ensure that appropriate information and documentation exists that can describe current and future business technology environment within the enterprise. As we observed, TOGAF® as an Enterprise Architecture framework includes most of the capabilities mentioned above!
The BTM Maturity Model
A maturity model describes how well an enterprise performs a particular set of activities. These capabilities are useless without a method by which to measure their effectiveness. The BTM Maturity model is aligned with the de-facto standard from CMMi and use the five levels of maturity of all the four dimensions. Here again the Architecture Capability Maturity Model from TOGAF® 9 may be used to evaluate these capacities. We would identify the area most in need for improvement.
Level 1: enterprises execute some strategic business technology management processes in ad-hoc way. These enterprises typically manage processes in a simple task-based manner.
Level 2: enterprises attempt to assemble information for major decisions, and refer to IT on decisions for technology implications.

Level 3:
enterprises are ‘functional’ in BTM.

Level 4:
enterprises have achieved full BTM implementation. Their capabilities ensure that there is strong alignment between business and technology decision making.

Level 5:
enterprises have achieved the ‘Holy Grail’ of BTM. They are good enough to know when to change the rules to maintain strategic advantages over competitors.
To implement its business strategy, the enterprise requires particular operational capabilities as described above and clearly it appears that Business Technology Management can be supported by Enterprise Architecture. TOGAF® 9 is in reality addressing all of these 17 BTM capabilities grouped in functional areas, identified by the four dimensions and work as a management framework to clarify required enterprise business needs. Companies having implemented BTM should consider using TOGAF® 9 as the way of rightfully pursuing alignment of technology with the business and support a Business-Agile enterprise.