Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Consultant| Canarys Automation LtdBangalore, Karnataka, India
In my experience with AI systems like Generative AI, ensuring the accuracy and relevance of results involves a combination of clear planning and iterative refinement. Here are some best practices I follow:
Define Clear Objectives: Establish specific criteria for what constitutes a successful output. This helps set the foundation for validating results against project goals.
Iterative Testing: Use iterative refinement to tweak prompts or inputs, continuously improving the alignment of AI-generated outputs with expectations.
Human Oversight: Always incorporate human review to validate AI outputs, especially for critical applications. This helps catch subtle inaccuracies or biases.
Diverse Testing Scenarios: Test outputs across a variety of scenarios to identify edge cases or areas where the AI might falter.
Feedback Loops: Regularly gather feedback from end users or stakeholders to refine outputs and ensure the system evolves to meet real-world needs.
By combining these practices, it’s possible to achieve more reliable and goal-aligned results from AI systems.
My experience with AI is very interesting as it continues to learn and evolve. I have found that using very detailed prompts give the best results. For instance, framing the question placing the AI in the specific role e.g Project Manager, Financial Analyst or Farmer etc and specifying the expected parameters desired in the output helps tailor responses while eliminating unnecessary refinements. Sometimes you might need to input a little background information such as numbers, demographics or other relevant information for the AI to draw from to provide a more accurate response.
Additionally, giving feedback (thumbs up and thumbs down) helps the AI ‘learn’ as it understands favored response patterns.
AI is best used as a guide and not to be relied on completely. It is imperative to have a background knowledge of the subject matter to be able to vet the response, otherwise involve subject matter experts. A project manager should be able to tell if an AI-generated RISK ANALYSIS is accurate/detailed or not and be able to make necessary inputs/adjustments if need be. Likewise with other professionals or simply involve an expert to confirm the response.
Rest assured the LLM bot will eventually learn your style. I use ChatGPT and Gemini and often compare responses from both and I found that because I have used ChatGPT for longer, it provides more detailed and tailored responses to PM related prompts than Gemini. However, Gemini is ‘learning’ just as quickly. Saving Changes...
Validating AI responses starts with the design of the prompts. Prompts should be written clearly with the relevant persona, task and contextual information, examples and additional constraints included. Where assumptions are being made, ask AI to state those clearly and provide it with templates or guidelines for the desired output. It is important to ensure that sufficient information is given and the prompt is refined as much as possible, including feedback to previous outputs, to produce the most accurate and satisfactory answer,
AI needs a complete context to answer the query. It’s better to refine output till we get intended result in required format. Saving Changes...
Donna OttDirector of PM/PMO| ETSMaple Shade, Nj, United States
Evaluate and adjust. Two key tasks for a PM to do while working with AI prompts Saving Changes...
Helder ValleProject Manager| OiRio De Janeiro, Rj, Brazil
Having a clear vision of what you need facilitates writing the prompt and making adjustments when necessary. Interacting based on the responses obtained helps achieve better results. Finally, documenting the interactions is a good practice, noting the prompts that best address your questions. Saving Changes...
Be iterative in your queries and compare AI outputs with that of SMEs as well as requests AI to cite its sources of information. Saving Changes...
Anonymous
Provide a role, ask detailed questions, provide examples, and be concise Saving Changes...
Sindhu KCIO| Genomatics Pvt. Ltd.Coimbatore, Tamilnadu, India
I have used ChatGPT 4o many a times! For eg., I have used it to create project presentations or for writing business proposals, getting a comparison table of products which I am buying etc... I have learnt by the many activities to refine, iterate, and get the best output while validating every output I consider final for my work. It has hallucinated sometimes, and with feedback mechanism, it appears to have learnt by my prompts and inputs. Saving Changes...
PMI Team Member
Community Champion
