Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Use the formulas and advice that the PMI Prompt Engineering class offer. In a gist, ensure that you are asking the right question and give the LLM enough background to iterate, learn and make changes. Use the CREATE formula, monitor the answers and continue giving feedback and informtion to the LLM to refine the information you get back.
I believe is a great tool for PMs as it will make the progress reporting and charts that we need to distribute easier to produce.
Have a structured prompt using formulas, ask questions, and iterate until you get the most appropriate response. Saving Changes...
roberta favaroApprentice Project Manager| RBKCLondon, United Kingdom
I am glad I have come across this course as I will need to get better at refining my prompts using the Create formula. I have never used examples or asked AI to reference its sources. That said, I usually ask Copilot or Chat Gpt "Are you sure this answer is correct?" whenever I am not convinced about the outcome. You will be surprised how many times Ai will correct the response. Saving Changes...
George NgugiMarketing Coordinator| VIA Global HealthNairobi, Kenya
It all starts with the basics - spcific and clear instructions with real augmented data to ensure that AI hallucination does not occur and the output is aligned with the objectives of the project team. Saving Changes...
Anonymous
Having useful example or sample data. Although not always easy, a clear ask or asks for what the expected output is required, maybe even down to formatting. Saving Changes...
To ensure the results you receive are accurate, relevant, and aligned with your original goals use, prompts with CREATE format . Use best suitable pattern like Iterative or ReaAct or Flipped or the best in context to your project or Requirements. Validate response and make it clear by making it specific to your goal. Saving Changes...
Emad BadrProject Management| Hassan Allam HoldingDammam, 4, Saudi Arabia
Like anything else, this is done by making a short list of the basic requirements that must be covered. Saving Changes...
Here are some practically effective practices to assure the results we receive from AI systems are accurate, relevant, and aligned with our organizational objectives:
1. Define Clear Objectives: Start by clearly defining the goals you want to achieve with the AI system. Having a well-defined objective will guide the AI in producing results that are aligned with your needs.
2. Use High-Quality Data: Ensure that the data you feed into the AI system is accurate, up-to-date, and relevant. High-quality data is crucial for generating reliable results.
3. Regularly Update Data: Keep your data sets regularly updated to reflect any changes. Stale or outdated data can lead to inaccurate results.
4. Choose the Right Model: Select an AI model that is well-suited to your specific task. Different AI models have different strengths, so choosing the right one is important for achieving your goals.
5. Validate Results: Regularly validate the results produced by the AI against known benchmarks or ground truth. This helps ensure that the AI is performing as expected.
6. Monitor Performance: Continuously monitor the performance of the AI system. Set up metrics and key performance indicators (KPIs) to track its accuracy and relevance.
7. Provide Feedback: Provide feedback to the AI system based on the results it generates. This can help fine-tune the system and improve its performance over time.
8. Understand Limitations: Be aware of the limitations of the AI system. Knowing what the AI can and cannot do will help you manage expectations and use the system more effectively.
9. Involve Human Oversight: Combine AI-generated insights with human expertise. Human oversight can help interpret results, identify errors, and make final decisions.
10. Stay Updated: Keep yourself informed about the latest advancements in AI technology. This will help you leverage new features and improvements that can enhance the performance of your AI system.
By following these best practices, we can maximize the effectiveness of AI systems and ensure that the results we get are accurate, relevant, and aligned with our business objectives. This is just like Deming’s Quality Improvement Life Cycle – Plan-Do-Check-Act (PDCA) Saving Changes...
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
I agree. Using AI in a nonfamiliar area requires proper insight to navigate informational or technical mishaps which it may generate. Saving Changes...
PMI Team Member
