Project Management

Please login or join to subscribe to this thread

When using AI systems, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?

linkedin twitter facebook   Artificial Intelligence  
avatar
Sarah Philbrick
PMI Team Member
Director, Learning Design & Development| PMI Asheville, NC, United States

Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.

In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?

Sort By:
< 1 ... 96 97 98 99 100 101 102 103 104 105 106 ... 191 >
Definitely, the RTF formula!
In my opinion is we should CREATE formulars to give instructions to the AI and use strategies to get the most relevant outputs from it as we required . Using a pattern such as React as well we can continuously interact with the AI to get filtered results more accurate relevant and aligned with our goals .
avatar
Anonymous
Providing content in your prompt that is detailed and clear, making a clear ask, and vetting the response from AI are essential. Then iterating on the response from AI is a critical step to refine the response until you have something sufficiently precise and accurate.
avatar
Anonymous
Providing content in your prompt that is detailed and clear, making a clear ask, and vetting the response from AI are essential. Then iterating on the response from AI is a critical step to refine the response until you have something sufficiently precise and accurate.
avatar
Carol Robinson New Britain, CT, United States
To ensure that the results you receive from AI systems are accurate, relevant, and aligned with your original goals, its important to use prompt formulas, develop strategies to validate AI responses, and continuously refine your approach. And of course, practice is key - so keep practicing, practicing, and practicing!
avatar
Babita Ramlal Project Manager| GovTechON Scarborough, Ontario, Canada
Jun 07, 2024 9:24 AM
Replying to Sergio Luis Conte
...
AI is a broader term. Generative AI is just an ancient model but everything "explode" when Google published the new architecture called transformer in 2017. So, with that said, take into account that generative AI is just "predictive test with steroids" just simplifying the model. With that said, two key points has to be taking into account when somebody works with AI: 1-human in the loop. 2-AI without Data (today called data science discipline or big data or whatever) is the same thing that live without oxygen. Talking about generative AI all related to technology has almost not impact with relation to all related to non-technological roles and activities. What you stated about accuracy and things like that are easy to implement because there are a lot inside disciplines like statistics. Most of them to make things "a priori" to prevent instead of cure. Few organizations taking into account that when generative AI environments are put in place almost a new business unit has to be created where roles like lawyers, linguistic, diversity and inclusion specialist must be hire to help on put it in place.
Hi
You will need to do an algorithmic impact assessment before implementation and then evaluate your entire AI system from end to end to ensure there is no drift, bias or other ethical dilemmas (like MechaHitler Grok). some good guidelines can be found in the IEEE AI series of standards.
avatar
Babita Ramlal Project Manager| GovTechON Scarborough, Ontario, Canada
Jun 07, 2024 9:24 AM
Replying to Sergio Luis Conte
...
AI is a broader term. Generative AI is just an ancient model but everything "explode" when Google published the new architecture called transformer in 2017. So, with that said, take into account that generative AI is just "predictive test with steroids" just simplifying the model. With that said, two key points has to be taking into account when somebody works with AI: 1-human in the loop. 2-AI without Data (today called data science discipline or big data or whatever) is the same thing that live without oxygen. Talking about generative AI all related to technology has almost not impact with relation to all related to non-technological roles and activities. What you stated about accuracy and things like that are easy to implement because there are a lot inside disciplines like statistics. Most of them to make things "a priori" to prevent instead of cure. Few organizations taking into account that when generative AI environments are put in place almost a new business unit has to be created where roles like lawyers, linguistic, diversity and inclusion specialist must be hire to help on put it in place.
Hi
Evaluating the performance of commercial AI models is more difficult because those are black box. Third party AI Audits may be useful.
avatar
Anonymous
Bring context and data to the LLM and share an example of what is needed
avatar
Anonymous
Bring context and data to the LLM and share an example of what is needed
avatar
PEGGY HASKINS Upper Marlboro, Md, United States
Jun 08, 2024 6:40 AM
Replying to Oliver Chitsamatanga
...
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well.
The best practice  and protocol to follow  would be to consult subject matter expects  to validate the AI generated response before making critical decisions based on it to avoid any  inherent associated risks which you might be not aware of.
I like the point about having a subject matter expert review GenAI output that I'm not familiar with. GenAI is just like us humans. Sometimes it understands and answers as we need. Sometimes, we may have to walk away, rethink, and rewrite the prompt to ensure the results you receive are accurate, relevant, and aligned with your original goals. We might need to verify results with legal statutes, codes, and procedures. Perhaps we might run the same prompt through several LLMs to see if all return the same results. We might explain what we want to have a peer review of our prompt and rewrite it accordingly to see if we get a better result.
< 1 ... 96 97 98 99 100 101 102 103 104 105 106 ... 191 >

Please login or join to reply

Content ID:
ADVERTISEMENTS

Eighty percent of success is showing up.

- Woody Allen

ADVERTISEMENT

Sponsors