How would you measure an increase 'Customer Satisfaction' (for a new application replacing a manual process) when no 'current state' measure has been captured?
Scott RugglesIT Project Manager| Maricopa County Department of TransportationPhoenix, Az, United States
When delivering application, we often measure 'time saved' or '$ saved' but we also are delivering applications used by the 'customers of our customers'. Since the process has been manual (paper forms) there was no 'measure of satisfaction' to capture, though after the launch of a new application we want to SHOW the increase in customer satisfaction for making the process more efficient and easier to use.
What advice would you suggest to capture that? Would we need to survey users BEFORE starting the project to establish a baseline? how large of a sample would be necessary, and how long should we let that go on?
Project Manager| AWR Development (BD) Ltd. Cox's Bazer , Bangladesh
Great question, Scott. When no baseline exists, I’ve found it useful to reconstruct a proxy baseline before implementation—through quick user surveys, interviews, or even measuring pain points like processing time, errors, and user effort in the current manual process.
After launch, you can then compare using simple satisfaction metrics (CSAT, NPS, ease-of-use scores) along with behavioral indicators like adoption rate and reduction in complaints.
Even a short pre-launch pulse survey can provide enough reference to demonstrate improvement later. Saving Changes...
Titan Bagus BramantyoInformation Technology Project Manager @Bukit VistaSleman, Yogyakarta, Indonesia
Interesting question, Scott.
I think we should measure both how users feel and how the process performs. The first can be captured through satisfaction questions, and the second through operational results like speed, errors, and completion volume.
We can use a short baseline survey for about 2 weeks before launch and the same again after launch, with a target depends on how many users of the app. We can set a general sample target, but we should also ensure it is representative by user type, so each major user group is included and the result is not biased toward one segment.
But, if one user type is small but important, consider to be intentionally oversampled, then interpreted separately, so it is not hidden inside the overall average. Saving Changes...
This is a situation where I think AI would be an excellent resource. I fed in a some inputs and got back both good inputs on writing customer satisfaction surveys, and information on sample sizes, better than I could try and describe. There is software like SurveyMonkey available that is specifically for this purpose, but not necessary if you're not polling on a regular basis.
My statistics is passable but not expert level, however having been through major digital transformation programs, I would advise caution interpreting the data. The results will likely have some heavy biases due to the psychology of how people deal with change. Newer employees who hadn't mastered the old ways or are more comfortable with digital tools will like it more. People who were the previous masters in the legacy paper process often feel that it devalues their skill set. It typically takes about 2 years for people to feel fully comfortable with some of those process changes but some as little as 3-4 months.
That raises and important question about what your survey results may be telling you. Is it measuring satisfaction with the new software, or how different personality types adapt to change? Saving Changes...
Program Manager| HARPER SRLSanto Domingo / Distrito Nacional, Dominican Republic
If there’s no baseline, I create a quick proxy before launch with a short survey or user feedback. Then after launch, I compare simple metrics like satisfaction, ease of use, and adoption to show improvement. Saving Changes...
Community Champion
