Project Management

Please login or join to subscribe to this thread

The Power and Limits of AI: Reliability for organizations and project managers?

linkedin twitter facebook   Artificial Intelligence  
avatar
Tamara Campbell Consultant Seattle, United States
Recently, I experienced the power and limits of AI while working on an e-book project about investment models for major infrastructure. Despite well-constructed queries, the accuracy, framing and quality of the output was incomplete at best. Images and exhibits requirements for the books also demanded a separate AI process, or traditional content creation and editorial to meet the expectations of the project. After reviewing the texts, several more attempts with the same or similar queries were made, each with different output before the content was adequate for synthesis and revision.

How can we be certain of the power of LLMs to produce the intelligence we need to proceed in our workdays? Statistically speaking, the reliability of qualitative data can only be tested with far more queries than I was willing to input during the book project. Organizations seeking business intelligence reliability from qualitative big data compiled over time, will want AI powers that reduce the hassle of 100+ queries to ensure the reliability of the information they are using. AI is an iterative process in the workplace as well, I would suggest. The integration of its full potential for projects and organizational processes can only be achieved over time. Until then, I think it is safe to say, we will still be on the job as the machine learns.  

Are there more power and limits stories about AI changes in the organization or project management strategy?
 
Sort By:
avatar
Kiron Bondale Retired | Mentor| Retired Welland, Ontario, Canada
Tamara -

In general, publicly available LLMs are unlikely to be usable in a mostly hands-off manner given the questionable training data and wide range of variance in the training feedback provided by end users.

Private LLMs on the other hand can be quite accurate if their scope is focused, their training data is clean and users are diligent about providing feedback on the quality of the outputs.

Kiron
avatar
Eric Simms Senior Program Manager Baltimore, Maryland, United States

The problem is the source, not AI. No query, however well-written, can guarantee accurate data from a public source that’s likely replete with misinformation. Also, the subjective nature of qualitative data means results might be technically correct but completely unsuitable for the desired purpose.

avatar
Tamara Campbell Consultant Seattle, United States
Thank you for the conversation!
avatar
Sergio Luis Conte Helping to create solutions for everyone| Worldwide based Organizations Buenos Aires, Argentina
What you state is not about AI. It is just a type of AI: generative AI. No matter that please remember that AI in general, and genAI in particular, will work if and only if a key success factor is taking into account: human in the loop. "Well constructed queries" has sense if and only if you use prompt engineering frameworks. But after that you have to do things like accuracy and trustworthy testing using the same prompt engineering frameworks. By the way LLM is not the same than genAI. At the end, genAi is not more than predictive text with steroids.
avatar
Tamara Campbell Consultant Seattle, United States
Well, I guess one can effectively argue that LLMs are the query and the source of GenAI. The accuracy of machine learning models has to do with replicability (or generalizability) at some level, which is also statistical concept for the measurement of reliability.

An AI model trainer, I know the specificity of artificial intelligence is also reliant on the selective reasoning of humans still at some level, as we "experts" hired to solve theorems with the proper solutions, are the wizard behind the machine. Mathematical formulae already subject to model training, enable the machine to accurately perform the calculus of queried equations; including those involving scenario descriptions (i.e., "Calculate the cost of capital (Ks) equation(s) for an M&A transaction"). are part of the question.

Hence, the magic of LLM. Once an AI is launched for global project manager use, new queries add to the knowledge base and so forth. The efficacy of the project management field will inevitably be a source of key data and LLM proficiency in this regard. Who are the AI experts now?

Please login or join to reply

Content ID:
ADVERTISEMENTS

"What really excites me in a project is when it goes in a way you haven't been before"

- Idris Elba

ADVERTISEMENT

Sponsors







Vendor Events

See all Vendor Events

Newsletters

Jun 3: The Career Problem Facing New PMs
May 20: Your PM Career as a Project Life Cycle
May 13: The Big Lie After Layoffs

See all newsletters