Assessing Explainability in Vendor AI Solutions

Artificial Intelligence

Anu Venkatachalam Infrastructure Project Manager| RMIT University Melbourne, Victoria, Australia

Hi Team,

Hope you are doing well.

Keen to get the PM community's thoughts on something that is coming up more often in vendor evaluations.

Many vendor solutions now highlight “AI-driven insights based on operational data feeds” as a key feature. From a delivery and governance perspective, how do you go about assessing the explainability and transparency of these insights during vendor evaluation?

For those who have dealt with similar scenarios:

What questions or artefacts do you typically ask vendors to provide?
Do you rely on proof-of-concepts, model documentation, or other validation methods?
Are there particular governance or risk considerations you ensure are addressed before adoption?

Keen to hear how others in the PM community approach this.

Would also be interested in hearing about any lessons learned or red flags encountered.

Appreciate your inputs.

Thanks

Posted: Mar 14, 2026 3:18 AM

Sort By:

Gwenola Michaud

Community Champion

Project Manager & Advisor| Geosciences & Monitoring Consulting Milano, Italy

Keen also to hear lessons learnt on this front from others as transparency of model and validation through proof of value are key.

Posted: Mar 14, 2026 10:38 AM

Sergio Luis Conte Helping to create solutions for everyone| Worldwide based Organizations Buenos Aires, Argentina

thanks for posting this. Unfortunately talk about AI without understand AI then they jeopardizes the work of lot of people that are working without trying to sell all smoke and mirrors.
So, first thing: AI is based into two key, seminal, basement concepts: human in the loop and data (AI will help to transform data into information. Totally different things). Second, AI is a wide term we are using from more than 50 years ago but always is based on date. Your data and data taking from other sources. Third, depending on your requirements, you will search what type of AI is included into the solution and then all the impacts. For example, if you decided to use generative AI then Responsible AI is a must.

Posted: Mar 14, 2026 10:54 AM

Syed Ashir Riaz

Community Champion

AI-Powered Social Media Strategist

During vendor evaluations, I ask for model documentation, explainability samples, and insist on a POC with real project data; demos never tell the full story. Key governance checks for me are clear accountability, human override capability, and bias monitoring before any adoption sign-off.

Posted: Mar 14, 2026 4:03 PM

Faisal Ahmed Rony Founder & Chief Editor| Total InfoHub Dhaka, Bangladesh

This is a highly relevant discussion, Anu. As we navigate the increasing reliance on AI-driven insights—especially within digital strategy and project execution at Total InfoHub—the explainability of vendor AI solutions becomes paramount. It's not just about compliance; it's about building trust and ensuring that decisions are auditable.

My key concern often revolves around data lineage and understanding the models' core assumptions. Before adopting any AI solution, asking for clear documentation on data sources, validation methods, and potential biases is non-negotiable. Thank you for raising these critical points for the PM community.

Posted: Mar 16, 2026 3:44 AM

Lissette Indhira Pimentel Sosa

Community Champion

Program Manager| HARPER SRL Santo Domingo / Distrito Nacional, Dominican Republic

In vendor evaluations, I usually focus on three things: model transparency, decision traceability, and governance controls. You could ask vendors to explain how the model reaches its outputs, what data sources it relies on, and whether human override is possible.

A proof-of-concept with real operational scenarios is also important. Demos can look impressive, but explainability becomes clearer when the model is tested against real workflows, assumptions, and edge cases.

Posted: Mar 16, 2026 11:37 AM

Please login or join to reply

Assessing Explainability in Vendor AI Solutions

Sponsors

Vendor Events

Guessing is not a strategy: How to build decision velocity with AI and real-time data

Newsletters