Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
Great question. I also find it a challenge to verify outputs that I am not conversant with. I have in the past requested the LLM to provide sources/references and also compared the outcome across various LLMs. Saving Changes...
When using AI systems is very hard to set the precision or accuracy of the responses. I love bringing in the Agile mindset here pretty much imagine if you are mentoring someone you do a Q&A and based on the reponses of your Mentee you give the feedback so that Mentee can align his/her thoughts in the direction that we hint similarly review the AI responses and using our rationale judgement
1- Give Feedback to the AI system
2- Rework on your promp and be specific on what is expected
3- Keep it short and conscise, guage the responses and slowly we can tune the AI system in a way to get the best output
4- Now the Tech. Solution that comes in for accuracy is havig specific set of APIs that talk to real and accurate data sources or use 2-3 outputs of LLMs and then analyze and bring the best in output.
Yes, very true. Thanks for those insights Saving Changes...
LLMs are indexed to be flattery machines - not critical - so to avoid this in my work, I've uploaded this article to my favorite LLMs and asked that they ditch the flattery. (No, I'm not using AI for therapy ;)
Validation is critical when working with AI systems. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement. Saving Changes...
Ahmed MadkourBusiness Analyst III| Eastern Province MunicipalityDammam, 04, Saudi Arabia
Hello Sarah & All,
That's good question, and in order to answer it let's start with the basic concept "Garbage in, garbage out" (GIGO) which mean if I provide unstructured context and not solid, with a genric desired outcome, the AI will provide me the same as well, that's why I usually provide a very spcific scenario with well structred context along with clear required outcome to get what I'm looking for. Saving Changes...
Asha NaikProject Manager| JanisonSydney, Australia
Create your login with a LLM tool of your choice and the train the model with data related to your projects and always be polite, it gives you better answers :) Saving Changes...
Theresa MoutonExecutive Director| Bester Distribution (Pty) LtdAdelaide, Australia
Prompt engineering is a bit like giving GPS directions - the clearer your route, the better your destination. To get accurate and relevant results from AI, be specific about what you want, give context, and set boundaries (like format, tone, or timeframes). Break complex tasks into smaller steps, refine your prompts if needed, and always check the output against your goals - think of it as adjusting your route when your GPS takes a wrong turn. In short, clear inputs = smart outputs. Saving Changes...
Adrian MarableFounder & CEO| Groopwork, LLCConcord, Nc, United States
Chat GPT is doing better with this by actually adding citations inside of the responses, but you can ask AI what is real and what is no within after seeing the response. I would still find other avenues to prove information that may be critical to a project. It will still take less time than trying to develop something yourself.
Before prompt emgineering, I usually spend more time sending requests to the AI and Keep refining it for me to reach my desireable output. but now with the CREATE formula I get almost all of what I need with just one click. The time I spent sending in requests has now been reduced to half, where I will use that half period to build my prompts.
However, What I am most excited about is not just the exposure to the capabilities of the AI models, but the "HUMAN IN THE LOOP" concept, which seems very lacking in most places where AI tools are being utilized. Sometimes when I receive correspondences from other institutions, without even running it through any detection platform, I can tell that it was AI generated, given the generity of it nature. Therefore it's a concept that is very important to follow.
That being said, I will proudly say with my little knowledge of prompts and its engineering through this course, i will champion a small training for my colleagues at work, so that they can also utilize AI tools, which will massively boost our project team efficiency.
PMI Team Member
