AI IQ

Project managers must embrace new technology, especially when it can improve project performance. Large language models (LLMs) such as ChatGPT, Bard, and llama can be excellent tools if used properly. As with most technology, we must learn how to effectively interact with this software application to optimize the results. Humans are inconsistent when asking questions.

“What is the weather today?”

“What is the temperature?”

“What is it like outside?”

“Is it going to rain or be sunny today?”

The words, phrases, and sentences people say are known as utterances. The LLM evaluates them to determine an intent. For the questions above, the intent is to get a report on the weather. How we ask questions determines the results obtained, and there are tips for using this process effectively. A new area of knowledge is being developed called prompt engineering, which is the ability to make a request to an LLM and obtain the best response.

Prompt Engineering Techniques

1. Chain prompting. This technique is based on LLMs being a conversational technology that remembers previous questions and answers. Based on this characteristic, after you ask a question and receive a response, you can modify your next question. Using this method, you can seek a change to the first response or use feedback to develop a better series of questions.

Project Scenario

Q1. What is the greatest risk to my project?                   

LLM answer 1. The project schedule.

Q2. Why is this such a big risk?                                     

LLM answer 2. There are resource issues where allocated resources have insufficient

experience to complete the tasks on time.

Q3. What is the best way to mitigate this risk?                

LLM answer 3. Perform an assessment for critical path tasks comparing task complexity to

resource capability.

Q4. Will there be residual issues if this risk occurs?        

LLM answer 4. If your schedule is late, there is the potential for additional risks that affect

product quality.

2. Persona replication. This has the potential to be the most exciting and the most dangerous feature of LLMs. By loading content from a specific individual, the LLM can assume the characteristics of the person and respond in that persona. For example, once you load a series of texts by a famous scientist, you ask the LLM to respond based on the manner and knowledge of that person.

Project Scenario

In an agile project, customer feedback is an important factor for iterations. The project manager can load information about the customer (with their permission) to acquire feedback when the customer is unavailable. The process is to load personal background information, experience, organizational responsibilities, emails, messages, and previous decisions. When a sprint is completed, the project manager asks the LLM to respond in the voice of the customer (VoC).

3. Chunking. Sometimes, you need a long response, and the best approach is to break it into smaller segments. For example, you want the LLM to write a movie screenplay. Rather than providing the basic plot and characters and then letting it create an entire movie script, it makes more sense to ask for the first few scenes. Based on the initial response, you can modify the parameters before you ask for the next series of scenes. Chunking is the process of accomplishing your request using a step-by-step approach to provide a better result.

Project Scenario

Instead of asking for an entire project plan, the project manager provides the project type and objective then requests a plan for the first stage, such as design. After reviewing the results, the following request is for details on the implementation stage. Similarly, rather than asking for an entire project management plan, the project manager asks for a sequence of components such as a risk plan, resource plan, and communication plan.

4. Response customization. Additional features known as temperature and frequency penalties allow you to alter the randomness of a response and the number of repetitive words or phrases.

Conclusion

LLMs offer a myriad of capability that has not yet been fully exploited. For example, a project manager can create a status report or an important message and ask the LLM to modify it to eliminate bias or improve clarity. Learning how to collaborate with this technology using prompt engineering techniques improves the project results and the performance of project managers.

Unsupervised learning is a type of AI-based algorithm that relies on characteristics instead of labeled datasets that are used in supervised learning. A typical application is the ability to classify or cluster datasets based on their characteristics. For example, an unsupervised learning algorithm can easily classify fruit based on color, size, and shape. The algorithm does not know what a banana is, but it will create a common group for anything resembling a banana. In projects, three uses of unsupervised learning are for risks, task complexity, and change requests.

1) Risks. Using unsupervised learning to cluster risks might result in finding a common cause for a group of risks or developing a shared mitigation strategy. Clustering risks from several projects can also result in finding a risk on your project that was overlooked.

2) Task Complexity. For this application, tasks are grouped by complexity based on the task definitions. If there is an unusually high number of complex tasks, the project manager needs to evaluate the ability of assigned resources to complete them. Additional training or mentoring from an expert may be required. A review of the resource allocation plan may alleviate any concerns. A high level of complex tasks can also provide an incentive to validate the risk management plan.

3) Change requests. Grouping change requests from previous similar projects can result in being able to forecast expected changes on your project. This proactive approach allows more accurate estimates for budget and schedule. If all changes in your project require additional funding and a shift in the end date, that is good. However, the sheer number of changes or unexpected changes may still result in a deterioration of project performance.

In project management, unsupervised learning finds patterns that a project manager cannot detect. Finding these patterns allows proactive actions to be taken that keep the project on a trajectory for success.

Supervised learning is a commonly used technique in AI-based machine learning. This type of algorithm has two main capabilities: prediction and classification. Prediction occurs when the algorithm receives labeled datasets to predict an outcome. For example, a large number of images labeled as either a cat or a dog is provided as input to an algorithm, and the algorithm builds a model of what a cat is supposed to look like and what a dog is supposed to look like. Now the algorithm can accurately classify a new unlabeled image as a cat or dog. The capability is beyond simply matching, as the software learns all variations to arrive at the correct result.

Think of a project as an image. All the characteristics of a completed project are input to an algorithm, and the project is labeled either a success or failure. The algorithm builds an image of what a successful project looks like. A new project can then be evaluated to verify how close it is to the image of a successful project. Typically, the output is a probability. If the probability is above 90 percent, the project is set up for success. If the probability is lower, several actions can be taken. An organization may want to add the probability of success as another project selection criterion. The starting image of the project does not guarantee success since projects can encounter variances very quickly. Supervised learning can also be used to proactively minimize variances or determine how to resolve them.

There are many situations when an accurate prediction is useful. A machine learning algorithm can predict whether a risk will occur on a project or, even more importantly, whether the risk will cause project failure. An algorithm can predict communication issues, stakeholder management problems, and numerous other potentially failure-inducing situations. Supervised learning allows proactive measures to be taken to keep the project on a path to success.

As evident from the descriptions above, historical data is crucial for an algorithm to create a model. Supervised learning normally requires a significant amount of data to perform accurately. In the field of medicine, vast amounts of datasets are collected. In the world of project management, it is yet to be determined what amount of data is sufficient. Some academic research suggests that as few as 50 datasets are acceptable. However, the number of characteristics captured in each dataset is also essential. The theory that AI requires large amounts of data is based on supervised learning, yet it is not necessarily true for other AI capabilities.

Computers have surpassed humans in the ability to store and process data. Supervised learning is a clever concept that can be effectively deployed to improve project performance.

An explanation of the basic functionality is required to understand how AI can be used in project management. While there is no clear definition of AI with boundaries and limits, describing what AI looks like and how it can be used is helpful. AI for project management has two main components: machine learning (ML) and natural language programming (NLP).

Machine Learning

ML is software that uses datasets to create a model, and based on that model, the algorithm produces a prediction or performs classification. Prediction is the probability that the answer or decision is correct. The results are valid for a single point in time but need to be updated with new data as the project progresses or the data changes. What can you predict? Anything where you have data such as overall project success, quality results, or the probability of achieving the budget.

For ML, the most common learning models are supervised learning, unsupervised learning, and reinforcement learning. These algorithms are incorporated into the software used by organizations to improve project performance.

Natural Language Processing

NLP converts words to data and is extremely useful for document analysis, sentiment analysis, translation, and using a virtual assistant. NLP reviews documents for accuracy and consistency. Sentiment analysis checks verbal or text input and identifies positive or negative feelings in the project team. There may be ethical issues if this is not implemented and used correctly. Virtual assistants determine the intent of a request and deliver a response. Commonly used virtual assistants are Siri, Alexa, or Google Assistant. NLP is also at the forefront of recent developments with large language models (LLMS) such as ChatGPT, Llama, and Bard. However, large language models use NLP and ML algorithms to achieve their amazing capability. The value of NLP is the ability to extract insights from large amounts of data, such as a database of historical projects.

Summary

ML and NLP are valuable AI components that can be combined when using LLMs. Further blogs will review how these algorithms work and how the project methodology is changing to take advantage of this opportunity.

The scope statement defines the product, service, and results to be delivered by a project. The scope statement outlines the boundaries for the project work and provides the input to create the budget and schedule. AI is applied to scope for two primary purposes: to help create the scope statement and to review the content for consistency, checking for errors and omissions.

As Professor Bent Flyvbjerg points out in his book How Big Things Get Done, in an analysis of 258 projects with a budget over $1 billion, only 8.5 percent met the budget and schedule, and a minuscule 0.5 percent were delivered on time, within the budget and achieved the expected benefit. These results suggest that the project methodology needs to be reconsidered. Fortunately, new technology, specifically AI, offers hope for improving project results, and it starts with the scope statement. Both agile and waterfall projects can benefit.

Agile

For agile projects, a study by Accenture found that the root cause of 35 percent of defects in production was due to errors in the requirements document. Based on the wording of user stories, natural language processing (NLP) tools find errors and omissions. For example, the requirements might define the details of a feature, but that feature is never tested. Alternatively, there may be testing for a feature that is only vaguely defined. AI-based scope review algorithms find these inconsistencies up to 20 times faster than a human and are especially useful for requirements that are hundreds of pages long. This is not a distant reality or exaggeration. A government department recently created a software consolidation project, moving the functionality from a variety of different applications into a single software solution. A previous similar project was significantly over budget and schedule. However, the main issue was that it did not perform as expected, creating negative public criticism. For the current software consolidation project, the government acquired an AI-based NLP tool to thoroughly review and correct the user stories before starting the project.

Waterfall

Construction projects have two areas where AI can help provide a more accurate scope document. First, a draft scope document can be automatically created using a large language model (LLM) or a database of previous documents. This saves time and may include items overlooked when the scope is created manually. The second area is like the agile concept. Once a scope statement is documented, an AI-based algorithm using NLP reviews the document to look for errors, omissions, and gaps.

One of the discoveries from the work of Professor Flyvbjerg is that projects that start well are less likely to become project disasters. An accurate scope statement supports that finding. Applying AI to ensure an accurate and comprehensive scope statement significantly improves the project methodology.