Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
1. Define Clear Objectives & Success Metrics
2. Prioritize High-Quality, Representative Data
3. Validate Models Rigorously
4. Implement Human-in-the-Loop (HITL) Oversight
5. Monitor for Drift & Update Continuously
6. Ensure Transparency & Accountability
7. Align with Industry Standards Saving Changes...
Anonymous
AI should augment experts, not replace them—especially where lives, environments, or liabilities are at stake Saving Changes...
Anonymous
I am very new to using AI but early on I started asking AI to provide the source of the data in its response and then I check it out to ensure its validity. It has been a time saver but there is a lot of information out there, so validation is key. Saving Changes...
Be clear on what we need from the AI system and our expectations. Accordingly, frame the prompts. Elaborate as much as possible. Sometimes, iterating and confirming the results multiple times in a slightly different way can be helpful. Ethical considerations and citation reference checks are a must for important papers or research work. Saving Changes...
Vasco MarquesIT Manager| lcs valanticLisboa, Portugal
I tend to look at prompting in the same way that you code software, the same guidelines and strategies that are used to build software can and shoul be used for prompts and respective responses, such as: structure in input and output (expected format), clear and precise goals, decomposition, i.e. breaking down complexity into more simple and manageable pieces, refactoring/ refinement, acceptance testing, i.e. prompt output review by a Subject Matter Expert (SME) are key for an accurate, reliable and usable prompt response as in a working application that meets the users needs Saving Changes...
Start by clearly defining the goals and objectives, integrate new data, provide iteratively more detailed instructions; avoid referencing irrelevant data; also adapt your input in response to the AI output. Saving Changes...
Firstly you need to be precise in what you are expecting as the outcome, i.e. Be clear in defining what you want with the appropriate specificity, information that needs to be considered in aligning with the result you are seeking. Guide the output with relevant data and avoid ambiguity. With the use of the correct tools, you are able to refine and generate the useful outcome you desire. Saving Changes...
I am not an expert in AI. Actually, I am taking my certificate in Prompt Engineering for Project Managers, and according to my PMI course, when you are using an AI system, it is good to use reliability checks, structure checks, know your audience, and provide a strategy formula such as SMART, RTF, PEAR, STAR, and CREATE. This strategy will help to adjust and refine prompts based on Large Language Models (LLM) responses, make them more actionable outputs, and generally make the response dynamic interaction with AI. Saving Changes...
When using AI systems is very hard to set the precision or accuracy of the responses. I love bringing in the Agile mindset here pretty much imagine if you are mentoring someone you do a Q&A and based on the reponses of your Mentee you give the feedback so that Mentee can align his/her thoughts in the direction that we hint similarly review the AI responses and using our rationale judgement
1- Give Feedback to the AI system
2- Rework on your promp and be specific on what is expected
3- Keep it short and conscise, guage the responses and slowly we can tune the AI system in a way to get the best output
4- Now the Tech. Solution that comes in for accuracy is havig specific set of APIs that talk to real and accurate data sources or use 2-3 outputs of LLMs and then analyze and bring the best in output.
Jabi, that was an excellent discussion. I agree with your point of view when you mention the Tech. The solution that comes in for accuracy is having a specific set of APIs that talk to real and accurate data sources or use 2-3 outputs of LLMs and then analyze and bring the best in output.
Definitely, using an LLM analyst provides a better response when integrating two or more outputs and combining them to create accurate, relevant, and aligned outputs with your original goals and desired outputs. Excellent discussion!
PMI Team Member
