April 24, 2002
I'm sitting here in an airplane high above the as I write this, and I'm thankful that the business owners, marketers, sales people, architects, designers and developers of this aircraft understand the meaning of Business System Performance and the criticality of predictable deployability of capability.
To an aircraft company, business system performance and "predictable deployability" are not a surprise or an afterthought; it's the most important aspect of their job, and meeting the goals of performance reliably and verifiably is key to success. For them, performance isn't just an IT problem, it's something the entire organization focuses on 24/7, designed in from the start. Aircraft manufacturers understand not only the technical aspects of performance, but also the business perspective. They understand that for the business system (in this case an airplane) to be successful, it must perform a complex set of services reliably, repeatedly and satisfy the business performance requirements as well as the technical ones. In short, they recognize the systemic importance of performance at every phase, from product vision through production.
No surprises, please.
The goal of most IT managers is avoiding surprises when deploying new systems, releases to existing systems or scaling up capabilities. Surprises on production systems can create CLMs (Career Limiting Moments), and therefore should be avoided at all costs. That is why most good IT shops invest in quality assurance, testing, release planning, production staging and configuration management facilities. Why is it then that when most systems are released to production, they don't behave as expected? Often it's because requirements for business system performance were not adequately understood by the business owners or the IT developers, or both.
Actually, production surprises should come as no surprise if performance and deployability factors are not considered as a systemic quality of the entire organization. The truth is that although avoiding production surprises is critical to success, few organizations actually have processes in place--or procedures to avoid them.
Fire, Ready, Aim...
So how come in our industry the record of developing and providing business systems that perform is so poor? One of the reasons may be that when many companies think about system performance, it's often looked at as an IT problem to be dealt with at the network or infrastructure layer, late in the process--after business requirements, design and development are done. However, when trying to put a finger on what performance actually means or how much performance is necessary, answers are often difficult to come by. The truth is that if the entire organization is not focused on performance, achieving it is difficult if not impossible.
Without clear performance targets, it's not surprising that systems often don't perform to expectations. To deal with business system performance properly, determine the proper requirements, measure performance levels and predict performance requirements for the future, it must be looked at and assessed from the business perspective. Otherwise, the results will be off the mark either operationally, financially (cost) or both. The point here is that business system performance requirements and metrics need to be defined with respect to the business context where it is used. Which brings us to the point of this article: using the business view to understand performance from the business perspective early in the process, and utilizing it as a reference point through all phases of deployment.
Okay, so what is Business System Performance?
Business system performance can be defined as how well the system meets the needs of the business. Business needs can functional, performance, scalability, accessibility, etc. Many of these requirements are driven by revenue or cost reduction targets often only identified in the business plan. Understanding the relationship of business system performance factors to achieving the goals in the business plan is critical to business system performance requirements. However, for this brief presentation, let's focus business system performance requirements and metrics in two categories:
Service Level Performance Factors
Functional Performance Factors
All these factors affect Business system performance differently. Generally, service level performance issues show up as Service and Response time requirements. Functional performance issues show up as usability issues and workflow problems. Both show up as user complaints, low customer acceptance and performance issues when the business system is not meeting expectations.
Assessing Business System Performance
To properly assess business system performance, it is important to have a basic understanding of the business requirements for the system and how it fits into the enterprise architecture (both organizationally, and from the process perspective). The assessment must include components to assess the effectiveness of the system within the business, application, data/information and IT architectures to be sure to capture all the performance criteria and constraints that need to be considered in developing an overall assessment of the system performance.
The business architecture understanding helps to give context to how the business system in question supports the business organization, and the up-stream and down-stream business processes. The application architecture understanding helps to clarify the logical, physical and deployment constraints to performance. The data/information architecture provides an understanding of what information is available, where it is accessed from and how the application modifies and extends the information model. Last (but not least), knowledge of the IT architecture is required to understand how the physical system is deployed and accessed from inside and outside the enterprise.
Business System Performance Program Assessment
An overall performance program needs to address serviced level and functional performance logically to insure that they are addressed properly and in a complimentary fashion.
Classically, IT support organizations are driven in kind of a knee-jerk mode to address performance issues, and usually focus on
Step 1: Define the meaning of System Performance
In order to accurately assess business system performance and begin to determine optimal system configuration requirements, it is important to document and agree on what the desired state of the system is, and also to have a clear documented understanding on the "current state" of the system. Additionally, it is fundamentally important to understand what "performance" is.
Step 2: Understand the High Level Business Requirements
An understanding of the
Step 3: Understand Perceived Business System Deficiencies
In order to "quantify" the requirements for the desired state, an understanding of the system deficiencies must be derived from the business user and IT support communities. This understanding helps to complete the picture of the "desired state" of the system. Often, this step is limited to focusing on
Step 4: Understand the Current State
Now with the context of system performance understood as a point of reference, the current state of performance can be defined. This is to define, document and agree on the current "AS-IS" state of the system from both a business and IT perspective. The scope of the current state definition should include both functional performance and
Step 5: Classification of Issues
At this point in the process, issues can begin to be classified into logical groups. This is a very important step. It is here where the scope of the performance program and the requirements for the test environment can begin to be identified. Performance issues are separated into functional and
Step 6: Understand the Desired State
For an overall performance program to be successful, it is also fundamental to have a firm understanding of the desired state of the system. This understanding must include near term (within the current technology and architecture) and long term (with new and/or updated technology and architecture) states of the system. This understanding helps to understand the scope of the performance program as well as the metrics under which the program will be measured. With the understanding of both the near term and long term desired states, issues can be further classified as those that can be handled within the current design, technology and architecture, and those that can only be handled by updating any or all three. Additionally, by documenting and agreeing on the desired state, realistic expectations can be set by both the business user and IT community.
Step 7: Understand the Criteria for Performance Measurement
Once there is a definition of system performance requirements, user perceptions and current and desired system states--and the issues are documented and classified--the performance program assessment effort focus needs to shift to the identification of the identifiable and visible components that contribute to business system performance (system constraints, interfaces and stress points). This understanding is required to determine the scope and nature of the measurement criteria and instrumentation that will be required to determine the causes of the stress, and the metrics on which they will be measured. This understanding helps to determine how and where the system will be measured, and the types of instrumentation that will be required to monitor the program and measure its effectiveness
Step 8: Understand Current Approach to Performance
It is also important to have a clear understanding of the current approach to performance tuning. This includes the logical process of identifying and addressing performance issues, but also the configuration of the test environment, the tools currently available and how they are used (how applications are instrumented).
Step 9: Review Performance Measures Taken to Date
No assessment could be complete without a thorough review of the performance measures taken to date and their effectiveness. This is to insure that any next-step planning considers only measures that have been effective, and only consider new measures that have not been taken before, and that may present additional opportunities to improve overall system performance.
It is required to have a thorough review of the performance measures taken to date and their effectiveness. It is important that the information reviewed is validated as fact. While often with the best of intensions, anecdotal performance information is derived and used as a basis for moving forward. This is frequently dismissed as '" KNOW where the problem is," and no point in going over the old ground. It is important to have a consistent factual basis for making changes. This is to insure that any next-step planning considers only measures that have been effective, and only consider new measures that have not been taken before and that may present additional opportunities to improve overall system performance.
Step 10: Brainstorming
Now the fun begins. Once the program has reached this point, it is important to gather the team together to discuss the system performance from all perspectives (business, application, information, IT) and facilitate a brainstorming session to identify the opportunities for performance improvement and to classify them into near-term, mid-term and long-term programs. During these sessions, using a collaborative approach, a coordinated picture of the overall performance program opportunities can be developed. This "opportunity matrix" can then be used by program managers (both business and IT) to scope the performance program, understand the timeframes for implementation and also gain a high level of understanding on what choices for investment are possible.
Step 11: Action Plan Development
The last step in the Business System Performance Program Assessment process is the development of the action plan and definition of next steps. The action plan uses the opportunity matrix developed during the brainstorming sessions as a basis. The action plan is designed to take full advantage of the assessment and identify three tracks for further action:
Track 1: Immediate Action Plan
This plan is to address any near-term, high-priority or critical issues. This usually includes the identification of a performance SWAT team to analyze, design, implement, test and deploy production fixes.
Track 2: Long Term Action Plan
This plan is designed to address longer term performance enhancements. These are generally broken down into two classes:
Track 3: Ongoing Performance Maintenance
This is the plan for continuous improvement of the performance program and system performance enhancement activities. Instantiating a program of continuous improvement helps to develop a coordinated approach to business system performance, and makes performance part of the overall process of business system development.
Put them all together and what have you got?
The business system assessment process described here provides a basic framework of how to establish the QoS requirements for performance as a systemic quality from both the business and IT perspectives. By assessing business system performance requirements early in the lifecycle (for both new and evolving systems), system performance criteria and expectations can be defined and set early enough so that architectural, technical, design, testing and deployment planning can consider performance needs holistically. This greatly improves the quality of the process and allows time for good decisions to be made in time to plan for their implementation. When this process is followed and added to the overall business system development methodology, the result is effective systems and smooth deployments.
Jeff Schwartz is Technical Director for Tanning Technology and is responsible for the application of