Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Taking an iterative approach to validating and refining throughout. Perhaps use some of the project documents such as the Project Charter, Scope Definition, and Requirements Traceability Matrix as sanity checks when applying the second 'E' in the CREATE formula to challenge the outputs. Where things don't align, there might be a need to correct the AI with additional information. Or, critical evaluation of such misalignments may be a signal to consider adjustments or outright changes to the project (in which case you'd discuss with stakeholders and follow the project's change control). Saving Changes...
I have had the best outcomes when talking to the AI as if I were training a new hire. This enables the AI to ask the kinds of questions that will often get me thinking about the way I should be doing a given task in view of the objectives of a project or the particular hurdles the team is facing. Then I switch roles and behave like the new hire and let the AI give me feedback. Saving Changes...
Its depends on your purpose of usage. If its personal use, you can verify; if its professional use, verify from industry standards; and if its for specific industry purspose, verify with test cases to pass it. Saving Changes...
We should ensure the prompting technique doesn't stay in a CREATE disguised as RTF. Meaning we provide for example no Examples, or we simply put inconsistent or missaligned data in the Request. Also we should use chain prompting techniques to help GenAI improve from its initial response to something we want (human in the loop). Saving Changes...
Teiichiro InouePM I| Mitsubishi Electric CorporationDuesseldorf, Germany
Jun 11, 2024 11:51 AM
Replying to Mashhood Ahmed
...
have a well structured prompt, understand Project injection, drifting, leaking and AI Hallucination. Here are some common elements of well structure prompt.
●Instruction - a specific task or instruction you want the model to perform
●Context - external information, Persona or additional context that can steer the model to better responses
●Input Data - the input or question that we are interested to find a response for
●Output Indicator - the type or format of the output
●Response Tone – Tone of the response
Thank you for putting this together. I'll make use of it when I need to use it. Saving Changes...
Valerie Williams-SanchezPrincipal Consultant| Valorena Online, L.L.C.Palisades, Ny, United States
Some best practices I have found for ensuring the results I receive are accurate, relevant, and aligned with my original goals is to ask for citations or references used to form the response. With those citations, I then often go back and check the source to ensure the contexts (of the my question and the AI generated response) align, that the resource used is credible and applicable. In taking this approach, I often learn of additonal information that leads to an even better, richer and more aligned bit of information or insight for my purpose. Saving Changes...
Valerie Williams-SanchezPrincipal Consultant| Valorena Online, L.L.C.Palisades, Ny, United States
Some best practices I have found for ensuring the results I receive are accurate, relevant, and aligned with my original goals is to ask for citations or references used to form the response. With those citations, I then often go back and check the source to ensure the contexts (of the my question and the AI generated response) align, that the resource used is credible and applicable. In taking this approach, I often learn of additonal information that leads to an even better, richer and more aligned bit of information or insight for my purpose. Saving Changes...
Following the CREATE Prompt model will help increase accuracy and relevancy of the LLM's response. You can also incorporate prompting patterns to engage w/ the LLM's responses and get them closer and closer to your intended goal. Saving Changes...
Tera MontgomeryProgram Manager| QuarterhillAustin, United States
To ensure AI systems deliver accurate, relevant, and goal-aligned results, project managers should start by crafting clear, precise prompts that reflect the project's objectives and constraints. Regularly validating AI outputs against known benchmarks or expert opinions ensures reliability and minimizes errors. Iterative refinement of inputs and outputs is key to optimizing relevance and alignment. Additionally, combining AI insights with human judgment prevents over-reliance on automation and mitigates risks of bias or inaccuracies. By treating AI as a powerful support tool rather than a sole decision-maker, project managers can harness its value while maintaining control over critical outcomes. Saving Changes...
To ensure that AI systems deliver accurate, relevant, and goal-aligned results, it's essential to start with clear objectives and well-defined parameters for the AI model. Regularly validate and audit the data fed into the system to ensure its quality, consistency, and relevance to the project. It's crucial to monitor the AI’s performance continuously, using metrics and feedback loops to assess its outputs and make adjustments as needed. In addition, leveraging *domain expertise* to interpret the AI’s findings and ensure they align with real-world goals and requirements is key. Finally, iterating and refining the model through ongoing testing and training helps to enhance its accuracy and relevance over time. Saving Changes...
PMI Team Member
