Metrics Management in Automation Projects

 

Introduction

Projects have been executed for decades for many reasons including customer requirements, technological advancements and compliance requirements etc. The success of the projects has been driven by mostly by conformance to plan for plan driven projects and the value of delivery for agile projects. There are many factors that attribute to the success of the projects and had helped project managers steer projects in right directions, take corrective and preventive actions. Metrics is one of the most important aspects project management which can assess if your existing project or program is doing enough to justify your existence. A metric, by definition, is any type of measurement used to measure some quantifiable component of performance. A metric can be collected through observation, such as delay in days, or number of defects; or the metric can be derived from directly observable quantities, such as defects per “x” lines of code, a cost performance index (CPI), or a schedule performance index (SPI) Metric is also called as an indicator, or a key performance indicator (KPI). We will see how metrics can aid automation projects realize its goals and objectives.

Why Automation?

We’ve witnessed so far application or product being built to solve a customers’ problem. Most of the applications including e-commerce are now being based on human centered design or user centric. Today’s systems are simply becoming too big and complex to run completely manually, and working without automation is largely unsustainable for many enterprises across all industries. Moreover, automation enables the ability to adapt and improve the service experience without manual intervention. Engineers and high-skill IT personnel will be tasked with identifying automation targets and variables in each workflow, gathering inputs and outputs, as well as simplifying and troubleshooting automated workflows.

What to Automate?

This is one of the most common inquiries around automation, as well as its primary barrier to implementation. While the answer will be business-specific, a logical starting point for many will be repeatable and mundane processes. Identify tasks within each department—be it marketing, finance, sales, or even legal—that people manually perform on a daily basis, and ask if they could be automated. Frequently, these repetitive tasks consume a considerable amount of time from junior team members; organizations with an accurate and granular look at their service delivery, combined with solid bench marking metrics, will be able to hone in on opportunities for automation.

Managing Automation Projects

Every organization has its own approach for managing automation projects and have come up with assessments and frameworks to deliver benefits to their clients on their automation journey. There are multiple phases associated with the assessment framework and the below framework can help any automation projects reach its goal and measure the maturity level.

The phases are,

• Kick-off

This is where the what part of the automation use cases are defined. Risk are identified along with mitigation plans.

• Ideation

The use cases for automation are brainstormed for BOTs development.

• Assessment

Use cases are finalized based on different techniques.

• Configuration

Architecture is finalized along with dependencies with related components.

• Release & Transition

BOTs are certified, deployed and marked for transition to operations team.

• Monitor & Support

Monitor logs, failures, exceptions and get customer feedback on implementation

• Benefits Realization

ROI, Productivity improvement, FTE reductions details are gathered.

How Metrics can help Automation projects realize its goals?

Measurement is key to successful project management. As the old adage says, “You can't manage what you don't measure.” Collecting and measuring data is at the heart of any worthwhile endeavor. With a New Year’s resolution to lose weight, you may find yourself counting calories or—go you! —watching the pounds drop on the scale. Race car drivers track their miles to the second.

There are generally 6 factors that managers generally measure to create metrics that determine project success:

1.  Benefits resulting from the capability delivered by a project
2. Time/Schedule to deliver project output
3. Cost to deliver project output    
4. Scope of work to deliver project output
5. Quality of deliverables and quality of process (customer satisfaction)
6. Risks including uncertainty or threats to project success

Automation metrics are no different than traditional project metrics but have few variations on what metrics needs to be tracked to effectively manage the projects and deliver the desired outcomes.

Automation metrics can be chosen based on the type of the projects being executed as listed above and area applicable for RPA/IPA/ITPA projects as well.

• Understand the purpose or goal of the project or work.
• Determine what critical success factors need to be fulfilled in order for you to succeed and achieve the goal.
• Take each critical success factor for the project or program and identify how you will measure its fulfillment.

Conclusion

With successful automation workflows, additional customers and services can be covered by the existing team, pushing margins higher while improving the consistency and speed of the service. Ultimately, people can do more with less. The metrics management approach and defined set of metrics may be suitable for a particular automation team and not suitable for another team depending on the nature of the automation, but it has worked out pretty good for me so far.