Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
Subash BalabhadranProject Manager| Kyndryl IndiaBangalore, Karnataka, India
Start with clearly defined Goals. Opt the right model and keep the data quality high. Create very precise context oriented prompts. Validate the iterate the data. Maintain ethical and bias awareness. Saving Changes...
Subash BalabhadranProject Manager| Kyndryl IndiaBangalore, Karnataka, India
Start with clearly defined Goals. Opt the right model and keep the data quality high. Create very precise context oriented prompts. Validate the iterate the data. Maintain ethical and bias awareness. Saving Changes...
Marci HooverSenior Project Manager| Lockton CompaniesCentennial, CO, United States
Be precise in your language
Avoid using acronyms, abbreviations and company jargon
If a complex issue, break it into subtasks
Provide clear examples
Be clear on expected output/outcomes
Do not assume one entry will work; refine as you go
Be willing to try various prompt patterns Saving Changes...
Silvia CastroCommercial Manager| GENPROSão Paulo, Sp, Brazil
Jun 08, 2024 6:40 AM
Replying to Oliver Chitsamatanga
...
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
Good point! Saving Changes...
Silvia CastroCommercial Manager| GENPROSão Paulo, Sp, Brazil
Jun 08, 2024 11:44 AM
Replying to Giorgos Sioutzos
...
Providing the specific context in clear and consise way is essential.
Totally agree Saving Changes...
Silvia CastroCommercial Manager| GENPROSão Paulo, Sp, Brazil
Jun 08, 2024 1:37 PM
Replying to Keith Novak
...
Like with any new tool, you need to test the results before you scale up.
Think about if you were to manually model a very complex problem in a spreadsheet. You don't build all the links and formulas first and then evaluate your final output. You build and test sections of the bigger solution first and then add on layers once you have validated the functionality.
Good point! We have to keep in mind that AI must have data and must be trained, so it has to be verified. Saving Changes...
Silvia CastroCommercial Manager| GENPROSão Paulo, Sp, Brazil
Jun 10, 2024 5:03 PM
Replying to Elmar Saenger
...
That's a very good question. In my response, I am assuming that the question refers to an LLM-based chatbot.
From my experience, the best results are achieved the more context I provide to the LLM. This means providing as much information as possible that describes both the project itself and the project context.
A second very important step is the quality of the request, also known as the prompt for the LLM. This is similar to human communication, where the quality of the question determines the quality of the answer. Therefore, a good prompt strategy is required, for example:
1. Data and context about the project
2. The goal of the request
3. The task that the LLM should fulfill
4. The format in which the output should be delivered.
In subsequent requests, it is possible to build on the context and results of the previous request. It is important that this process takes place within a chat, as otherwise the context is lost.
I´ll keep that answer in my files Saving Changes...
HAO JIE HUStaff Applications Engineer| CyberOptics Private LimitedSingapore, -, Singapore
1. Start with a clear goal.
Be specific with what you want. Vague input = vague output.
2. Always fact-check.
AI isn’t perfect. Cross-check important details before using them, especially in reports or proposals.
3. Refine as you go.
Didn’t get the answer you wanted? Rephrase, narrow down, or give more context. Small tweaks = big improvements.
4. Trust your judgment.
AI is smart, but you know your context better. Treat it like an assistant, not the boss.
5. Beware of bias & made-up info.
AI can sound confident even when it’s wrong. Be extra cautious with sensitive or public content.
Some best practices for ensuring the results you receive from your LLM chatbot are accurate, relevant, and aligned with your original goals:
- use fresh data that runs through an automated system that verifies the data
- employ a well crafted prompt using a formula appropriate (i.e. CREATE) for the desired output, perhaps including Chain of Thought and Chain of Feedback patterns
- review the output for hallucinations
- refine prompt if necessary - use a different prompt or pattern (questions refinement: ask it to rewrite the question for you, provide examples- documents, images, etc.)
- socialize the output for accuracy, use a ReACT prompt if necessary Saving Changes...
Paula BurchAppeal Territory Coordinator| Archdiocese of San AntonioSan Antonio, Tx, United States
Use CREATE and be very specific. Iterate to refine. Saving Changes...
"We cling to our own point of view, as though everything depended on it. Yet our opinions have no permanence; like autumn and winter, they gradually pass away."