Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Human in the loop, the one who is expert in his domain, is of utmost important, similarly have a LLm evaluator that constantly check for gorundedness and accuracy are also very important. Saving Changes...
I always practice the adage, "Specificity is the key to effective communication." I believe this will enable me to apply these new skills to ensure my results are accurate, relevant and aligned with my project goals. Saving Changes...
As a program manager, ensure AI results are accurate and aligned by setting clear objectives, crafting precise prompts, and validating outputs with human judgment. Always cross-check facts, refine iteratively, and avoid inputting sensitive data. Document AI usage and ensure it supports broader program goals, not replaces critical thinking or accountability. Saving Changes...
That's a very good question. In my response, I am assuming that the question refers to an LLM-based chatbot.
From my experience, the best results are achieved the more context I provide to the LLM. This means providing as much information as possible that describes both the project itself and the project context.
A second very important step is the quality of the request, also known as the prompt for the LLM. This is similar to human communication, where the quality of the question determines the quality of the answer. Therefore, a good prompt strategy is required, for example:
1. Data and context about the project
2. The goal of the request
3. The task that the LLM should fulfill
4. The format in which the output should be delivered.
In subsequent requests, it is possible to build on the context and results of the previous request. It is important that this process takes place within a chat, as otherwise the context is lost.
Hi, great answer! I agree with you — interacting with AI is very similar to communicating with people. If you want a specific answer, you need to be clear in both the information you provide in the prompt and in your request.
That's a very good question. In my response, I am assuming that the question refers to an LLM-based chatbot.
From my experience, the best results are achieved the more context I provide to the LLM. This means providing as much information as possible that describes both the project itself and the project context.
A second very important step is the quality of the request, also known as the prompt for the LLM. This is similar to human communication, where the quality of the question determines the quality of the answer. Therefore, a good prompt strategy is required, for example:
1. Data and context about the project
2. The goal of the request
3. The task that the LLM should fulfill
4. The format in which the output should be delivered.
In subsequent requests, it is possible to build on the context and results of the previous request. It is important that this process takes place within a chat, as otherwise the context is lost.
Hi, great answer! I agree with you — interacting with AI is very similar to communicating with people. If you want a specific answer, you need to be clear in both the information you provide in the prompt and in your request.
When using AI systems, it's important to follow a few best practices to ensure the results are accurate, relevant, and aligned with the original goals. Provide clear and specific instructions or questions, vague prompts often lead to vague answers. Always review the AI’s responses critically and cross-check key information with trusted sources when accuracy is essential. Saving Changes...
When using AI systems, it's important to follow a few best practices to ensure the results are accurate, relevant, and aligned with the original goals. Provide clear and specific instructions or questions, vague prompts often lead to vague answers. Always review the AI’s responses critically and cross-check key information with trusted sources when accuracy is essential. Saving Changes...
Some of my items may be redundant but the most important things in my experience so far is:
Be precise and clear.
Be sure you explain jargon or specialized terminology
Provide the context for all of your requests
Be sure you provide the outcomes you are expecting
Experiment and refine as you go
I've found breaking down big problems can be better refined by chunking the whole into natural sections and working to refine each section and then working to put them back together.
I like that you mentioned jargons, explicitly breaking down the request and clearly specifying the company specific jargons do help in getting the desired output. As some of these jargons are very specific to the team and organizations. Saving Changes...
This could be done by fine-tuning the chat context to fine-tuning the model using several strategies, such as Examples or few or many shots.
I'm currently working on a project. In my system Instruction [which Could be the scoping prompt if you are not accessing the API], I have the request and the verification method and criteria, so at the end of each output, I receive the confidence level achieved by AI.
With some training, I developed it further to output only results with an 85% confidence level or else provide an explanation or ask for clarification. This, btw, surprisingly jammed all the previous hallucinations.
That’s a really smart approach. Adding a confidence threshold along with a verification step not only improves output quality but also adds a layer of reliability to the system. I’m definitely taking notes from this thanks for sharing! Saving Changes...
Anonymous
create and iterate it till we find the correct solution Saving Changes...
PMI Team Member
