Measurement Bias Analysis

Bias can be defined as the difference between the average result of taking measurements many times and the true value of what is being measured. Measurement error and/or bias must be minimized so as not to adversely influence the interpretation of measurement results. The goals, recommended by the business reengineering team, must be measured regularly and evaluated to ensure the enterprise is on track to achieve reengineering or redesign goals and/or to monitor the effectiveness of all activities. (See Quality Measurement.) It takes a concerted effort to introduce a measurement program to sustain the enterprise as it moves forward. All measurement programs need to measure the effectiveness of the measures being used.

The performance measures used to evaluate effectiveness must not be biased or mislead the enterprise in their measure of customer satisfaction. It is imperative to review the basis for and the derivation of each performance measurement used on a periodic basis. Measurements should be changed if such changes will provide unbiased results.

Applications

• To identify the bias and inaccuracies of chosen measurements.
• To identify the precision of criteria.
• To provide confidence in the measurements being used.
• To normalize measurements to new standards.

1. Select an appropriate sampling interval.
2. Take sample and collect data.
3. Analyze, using statistical models.
4. Determine degree of bias.
5. Revise measures as appropriate.

Depending on the criteria being measured, select an appropriate sampling interval to test for bias. (Various statistical sampling techniques can be used to sample the measurement detail.) Collect the data and analyze the results. Determine the degree of bias and revise measures as required.

One method for removing bias is to perform a statistical analysis that correlates the customer's real experience with the same customer's reaction. For example, question the customer immediately after he has used a product or service. Ask him to comment on all of the customer needs that were identified at the beginning of the program and to relate his feelings about the experience. The analysis of this data can tell the levels of performance on each characteristic that correlates with high satisfaction and the relative strength of correlations for each characteristic.

Review the customer needs and the measurement of the enterprise's effort to meet those needs at least once a year. Revise if necessary.

References

1. Kazuo Ozeki & Tetsuichi Asaka. Handbook of Quality Tools, The Japanese Approach. Productivity Press, 1990.
2. Richard C. Whiteley. The Customer Driven Company, Moving from Talk to Action. Addison-Wesley.