Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Like with any new tool, you need to test the results before you scale up.
Think about if you were to manually model a very complex problem in a spreadsheet. You don't build all the links and formulas first and then evaluate your final output. You build and test sections of the bigger solution first and then add on layers once you have validated the functionality.
This resonates, especially when seeing it as a new model that has many glitches and still needs to learn. This perspective also adjusts expectations on accuracy and time required to actually mine useful responses from the llm. Saving Changes...
HAMEED ALYAJORIAudit and Risk General Manager , Project Manager| Yemen Customs AuthoritySn, Yemen
Improving AI responses requires clear, structured, and context-rich prompts. Using iterative steps and avoiding conflicting requests ensures more accurate results. Adding a refusal breaker pattern helps handle unexpected responses. The more precise the prompt, the better the output. Master the art of prompting to get the most out of AI Saving Changes...
It is important to note that using LLM is an art form and thus important to ensure the following amongst others.
Be specific with the prompts
Provide context for the situation
Provide examples that the LLM can use to generate results.
It is important to experiment, test and refine responses.
Knowing the audience of the response will enable you have clear prompts Saving Changes...
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
BLEU, ROUGE, and METEOR are all automatic evaluation metrics used to assess the quality of machine-generated text, particularly in tasks like machine translation and text summarization. Saving Changes...
Break down the ask to list out how one's brain would compute the problem. Essentially, if I am asking AI to do the job on my behalf, the approach I follow is to break down the ask to steps I would take to solve the problem. I have experienced this as a strategy that gives me the most success when I use AI for my work. Saving Changes...
Brandyn HollowayProject Manager| Amazon.comGlendale, AZ, United States
Building a secure and scrubbed environment that utilizes non-confidential to supply the LLM with context is the most important part of the process. The viability and quality of the outputs you receive are strictly based on what inputs you supply. Good Input = Good Output. Saving Changes...
Dhanwada RaoEnterprise Data Program Manager| ComericaRochester, Mi, United States
Adding a below sentence/phrase to every prompt consistently delivers the desired results. Give it try:
"Ask questions to clarify until you are 95% certain before composing the answer"
Adding this above sentence generate more thought provoking questions and answers. Saving Changes...
PMI Team Member
