Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
As senior PMs, start by scaffolding every AI interaction with RTF and CREATE. Explicitly set the Role (e.g., risk analyst), bound the Task (in/out of scope), and demand a precise Format (sections, tables, JSON). Provide Context: objectives, constraints, stakeholders, timelines, and Definition of Done aligned to your original goal. Make a clear Request and specify Tone (exec brief vs deep dive) and Evaluation criteria: acceptance tests, required sources, and risk thresholds. Require the model to surface assumptions, data gaps, confidence levels, and to ask clarifying questions before proceeding.
To ensure accuracy and relevance, ground the AI on authoritative references: policies, contracts, specs, metrics—prefer retrieval over open‑web. Instruct it to produce options with pros/cons, impacts, reversibility, and monitoring signals, not just a single answer. Demand source‑backed claims with quotations and links, plus “last updated” dates and regional applicability notes. Run a second‑pass verification: self‑check, cross‑model comparison, or targeted spot‑checks of calculations and logic. Add a pre‑mortem: what could be wrong, what evidence would change the recommendation, and how to test fast.
Wrap this in governance and iteration. Protect data (no sensitive info; approved tools only), version your prompts, and keep decision records and audit trails. Establish a review cadence with a scoring rubric (accuracy, relevance, completeness, compliance) and iterate until thresholds are met; automate commodity outputs to free time for stakeholder alignment and risk burn‑down. Monitor model and source drift, retrain prompts as contexts evolve, and log deviations with corrective actions. Close the loop by mapping outputs to owners, dates, and measurable outcomes so the AI remains aligned to the original goals.
I usually try to have a structured and detailed first prompt, but I may also include that GenAI may ask me questions to gather more information that I may have failed to include in the original prompt.
ArunKumar MuniyappaPM Consultant| IBM India Private LimitedSaltlakecity, Ut, United States
Here are some best Practice • Clearly define your objective Be specific about what you want. A well‑framed prompt leads to more accurate and focused output. • Provide context AI performs better when you give background, constraints, examples with your goals. • Break complex tasks into smaller parts Divide big questions or tasks into smaller prompts so the AI can handle each one precisely. • Ask for verification or reasoning Request the AI to explain how it arrived at the answer. This reduces errors and helps you validate the result. • Iteratively refine the prompts If the response isn't accurate, tweak your instructions. Iteration helps align results with your goals. • Provide corrective feedback If the output is wrong or off‑topic, correct it. AI will adjust based on the new input. • Cross‑check important information For critical tasks, verify results using trusted sources or tools. AI can occasionally produce mistakes. • Use constraints and boundaries Specify formats, tone, length, or exclusions to avoid irrelevant or unwanted content. • Avoid ambiguity Clear, direct prompts reduce misinterpretations and improve accuracy. • Ask the AI to state uncertainties You can ask the AI to mention if it’s unsure or lacks sufficient information, helping you assess reliability. • Review results with human judgment AI is a support tool; your own validation ensures final accuracy and alignment with goals.
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
Very Good Question and equivalent responses makes this thread rich with content for AI users
AI is a broader term. Generative AI is just an ancient model but everything "explode" when Google published the new architecture called transformer in 2017. So, with that said, take into account that generative AI is just "predictive test with steroids" just simplifying the model. With that said, two key points has to be taking into account when somebody works with AI: 1-human in the loop. 2-AI without Data (today called data science discipline or big data or whatever) is the same thing that live without oxygen. Talking about generative AI all related to technology has almost not impact with relation to all related to non-technological roles and activities. What you stated about accuracy and things like that are easy to implement because there are a lot inside disciplines like statistics. Most of them to make things "a priori" to prevent instead of cure. Few organizations taking into account that when generative AI environments are put in place almost a new business unit has to be created where roles like lawyers, linguistic, diversity and inclusion specialist must be hire to help on put it in place.
When using AI it is more about giving effective prompts and checking whether the responses are inline to the given prompt or not. If its not relevant we need to make use of Patterns that match the prompt to provide us the most effective way of dealing with responses.
To make sure the result is linked to original request, I will use some techniches such as RTF or CREATE. For somes that requires facts, information research, I often ask AI to cite resource. Saving Changes...
"It is an important and popular fact that things are not always what they seem. For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much -- the wheel, New York, wars and so on -- whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man -- for precisely the same reasons."
PMI Team Member
