Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Evaluating AI literally requires methods and techniques that an average user of these tools cannot perform. While assessing the AI agent in use is essential, it is equally crucial to set clear goals, iteratively refine prompts, question assumptions, and regularly evaluate outputs. So, from the large set of practices, I would say conducting regularly the following:
1. A/B Testing: Compare different AI models or versions in a live environment to see which performs better.
2. Bias and Fairness Testing: Regularly test the AI for biases and ensure it treats all demographic groups fairly.
3. Qualitative Evaluation: Have a team to interact with the AI and provide feedback on its performance. Saving Changes...
Kalu KaluSr. Consultant| North Highland, LondonLondon, United Kingdom
Never assume that AI is perfect, be in the middle to cross check and validate the information provided by AI. Saving Changes...
Garland MobleyChief Systems Engineer| Government AgencyEl Segundo, Ca, United States
I recommend utilizing the CREATE method for AI prompt engineering featuring requested sourced references and provide non sensitive documents, supporting information relative to your request and review it for accuracy and clarity. Saving Changes...
The most important to set up metrics of criterions to be able to evaluate the output of a request precisely. Since various requests has different outputs, it is essential to use suitable prompting patterns as well. Saving Changes...
Yasin Ali ShahPMP®, PMI-RMP® Certified Project Manager| SEPCO Electric Power Construction CorporationRas al khair, Eastern, Saudi Arabia
To ensure AI results are accurate and relevant, clearly define the goals and objectives upfront. Use high-quality, well-labeled data to train the model, ensuring it aligns with the task at hand. Continuously validate the output with real-world data and involve domain experts for review. Regularly fine-tune the model based on feedback and evolving requirements. Be transparent about the limitations of the AI system and cross-check results with human input where necessary. Testing in different scenarios helps identify biases and ensures robustness. Saving Changes...
To ensure the results you receive from AI systems are accurate, relevant, and aligned with your goals; start by defining clear objectives and crafting specific, concise inputs. Vague or broad queries can lead to irrelevant outputs, so refining your prompts is crucial. Remember that AI systems have limitations; always cross-check critical information with reliable sources to verify accuracy. Saving Changes...
Mahmood AbdulrahmanFO and Customer Support Manager| Sevennet Telecom CompanySulymanya -Iraq, Sulymanya, Iraq
There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. Saving Changes...
The preciseness of the result was checked by me by using another analytical tool. In some not so complex cases, it is possible to check by another way of solution. But this is not applicable in complex tasks Saving Changes...
Having a goal/or intent when using AI systems so that one has an idea of what done is and use defintion of done ( original goals) to evalute results. Examine the prompt and results. If the results aren't quite right examine the prompt for clues as to needs to be modified. Look at what worked, what needs to be changed and what could be improved to determine next steps: Prompt refinement, a new prompt,Zoom In or Zoom Out, etc. do the the goals need to change or need more/fewer details. Saving Changes...
Having a goal/or intent when using AI systems so that one has an idea of what done is and use defintion of done ( original goals) to evalute results. Examine the prompt and results. If the results aren't quite right examine the prompt for clues as to needs to be modified. Look at what worked, what needs to be changed and what could be improved to determine next steps: Prompt refinement, a new prompt,Zoom In or Zoom Out, etc. do the the goals need to change or need more/fewer details. Saving Changes...
PMI Team Member
