Director, Learning Design & Development| PMIAsheville, NC, United States
Validating and checking outputs is critical when working with AI systems like Generative AI. Such validation approaches may include establishing clear criteria, implementing strong testing protocols, and continuous refinement.
In your experience with AI, what are some best practices for ensuring the results you receive are accurate, relevant, and aligned with your original goals?
When using AI systems, it’s important to follow best practices to ensure the results are accurate, relevant, and aligned with your goals. Start by clearly defining your objectives and crafting specific, well-structured prompts to guide the AI effectively. Always validate the output by cross-checking with reliable sources or applying your own expertise to identify inaccuracies or biases. Provide feedback to the system if possible, refining prompts or parameters to improve results. Additionally, be mindful of the AI’s limitations and consider the context in which it’s being used, ensuring its outputs align with your broader objectives and ethical considerations. Saving Changes...
Akinwale AkinolaHead, Project Management| JNC International LtdSurulere, Lagos, Nigeria
The Knowledge Skills and Abilities of the PM are called into action when interacting with an AI system. AI is a system on steroids as described by Sergio Luis Conte but faster does not mean accurate, thus outputs must be validated, and refined iteratively until a good enough output is achieved. Saving Changes...
Vladimir QuinteroProfessor| Simon Bolivar UniversityBarranquilla, Colombia
First, keep in mind that this is, and will always be, an incremental, unfinished process. Build on previous successes, documenting, reinforcing and improving best practices.
The best way to overcome old data, due to the last training date of the LLM, is to include local data from your project, organization and context.
To improve LLM´s interpretation ability, don´t forget to use, and experiment with, roles, using very specific information/data on the project´s content.
In summary, you need to treat it as a continuous refinement of a simulation. Saving Changes...
From my experience and a prompt engineering course I attended, I have learned that providing context is critical. Clearly explaining the role the AI needs to assume from a "persona" perspective, as well as the situation at hand, helps avoid misunderstandings or incorrect assumptions.
Another important tip is to provide examples for the AI, such as documents it can use as evidence or the specific format required for its response. This approach establishes a clearer pattern, ensuring the AI's output aligns with your expectations. Saving Changes...
In my experience as an R&D leader, ensuring AI-generated results are accurate and aligned with goals involves clearly defining the prompt, continuously validating it, and iteratively refining it. For instance, when using AI to draft research proposals, I refine the prompt to include specific project requirements, ensuring the output is relevant. I also validate the AI’s results by cross-checking them against known benchmarks and expert input, especially when dealing with technical data. Lastly, I continuously refine the AI’s responses based on feedback to better match project needs. Saving Changes...
Terry RitchieDirector, Strategic Program Office| Aptima Inc.Oviedo, Fl, United States
Jun 08, 2024 6:40 AM
Replying to Oliver Chitsamatanga
...
A very good question and also difficult to answer as well. However you have to go to the basics and say as far as you are concerned, how well are you versed with the subject at hand ?. There are facts which the AI will generate and if you can verify these facts the more reliable the generated response will be. The fewer the facts then it means that the Generative AI response is far from meeting your original goals. Then it becomes very critical that you review the accuracy , relevancy and the alignment of the response to your original need. Unfortunately there are no clearly defined metrics that one can use a model to evaluate an AI generated response. So from my personal experience I basically restrict AI to an area where i have sound knowledge of , else it becomes almost impossible to verify details generated by an AI if you venture into unchartered territory. However with long usage and exposure your confidence also tend to increase as well. The best practice and protocol to follow would be to consult subject matter expects to validate the AI generated response before making critical decisions based on it to avoid any inherent associated risks which you might be not aware of.
Agree with the OP. We conduct a multi-faceted verification of outcome accuracy by comparing outcome with known trustworthy sources and subject matter experts as well as comparing outcome with authoritative databases. In addition, we conduct regular team introspections to examine and act on feedback. Saving Changes...
To ensure AI systems provide accurate, relevant, and goal-aligned results in construction project management:
1. Define clear objectives for AI use
2. Ensure data quality and security
3. Select tools that integrate with existing systems
4. Regularly review and update AI models
5. Combine AI insights with human expertise
6. Implement robust data validation processes
7. Continuously monitor AI outputs against project goals Saving Changes...
When using the AI prompt, it is a good practice to be specific and clear about the command. Although this is the first step, it leads to a particular response. The project manager must state the relevant context to understand the task. Next, the tone must be selected, whether professional or casual. Finally, reliability must be checked by asking for references to ensure the answer is credible and reliable. In short, it is better to incorporate the CREATE model for medium-to-advanced project operations. Saving Changes...
PMI Team Member
