AI IQ

Project managers have a significant opportunity to increase the value they bring to an organization by embracing AI. This is especially relevant to data management. Making a machine learning algorithm effective requires determining the best set of project features to use as input. Data scientists are now a scarce resource. Yet, it is doubtful that a data scientist is prepared to understand the nuances of project data.

Here are four practical steps for successful data management when using AI. These steps are designed to be straightforward and manageable, empowering project managers to take control of their data.

1. Determine what project data is available, how it can be accessed, and the format for the files being accessed.
2. Understand how to perform data wrangling. Ensure there is structured project data. This means consistency in the format, and resolving any anomalies, such as blank data fields, multiple entries for a single field, and inconsistent meaning for entries within a field.
3. Become familiar with feature engineering. Determine how to manage data fields that have the same meaning, add data fields that are missing, and improve the details of data that is captured and used by AI-based software
4. Improve your knowledge of basic statistics. Be able to analyze outliers and decide on appropriate action. Understand probability, sample distributions, and causal correlation. You don’t need to be an expert, but basic knowledge is essential.

With increased skills in basic data management, the project manager becomes an integral part of the organization's successful adoption of AI-based tools. Their role becomes significantly more valuable, and their contributions will be recognized as they help navigate the complexities of AI.

Here is a comparison of large language model (LLM) responses to the same prompt. The scenario is a project manager asking for suggestions to recover a project schedule from being late. The input is simple as if the project manager had no time to perform extensive prompt engineering or provide specific current and historical project data. The comparison was made on the same day (June 20) for all LLMs.

The first row is the LLM used. The second row indicates the number of steps in the suggested process, although some had sub-steps or additional suggestions. The recommendations are listed with similar content in the same row. Although the responses are summarized, the wording is intended to reflect the responses as accurately as possible.   

INPUT: “My project completion date is late to schedule. How can I recover the time?”

The table is a summary of each set of responses.

Observations

According to PMBOK 7th ed., the two main schedule compression methods are crashing and fast tracking. Crashing is a specific technique that determines the most effective schedule reduction at the least cost for tasks on the critical path. Fast tracking evaluates if tasks normally performed in sequence, a finish-to-start relationship, can have some or all of a task done in parallel with another task on the critical path. None of the LLMs suggested both of these. CoPilot was the only LLM that used the term “crashing” the schedule. It was disappointing that PMI Infinity did not include this. PMI Infinity and ChatGPT included risk management. ChatGPT and Gemini were the only LLMs that responded with a social concern for the well-being of the project team.

It is important to remember that LLM responses, like those of other AI-based solutions, are determined by the data accessed. From my perspective, ChatGPT offered the most comprehensive recommendations, and CoPilot had the weakest response.

What observations do you think are relevant in this comparison?

In a previous blog titled, An Empirical Study Comparing ChatGPT to Project Managers, I described a scope document study comparing the results of ChatGPT to the ability of project management students. In my research, ChatGPT was faster at identifying errors and more accurate, but it was only about 80 percent accurate. The study also revealed that some errors in the document were not listed by the large language model (LLM).

One of my student groups at the business college in France performed a variation on my work and included it in their major assignment. They compared three LLMs by requesting them to identify errors in a scope document.

Observations

The first observation is that LLMs perform at different speeds, although all are very fast compared to humans. The next observation is that each LLM had a different result. The explanation for different results should be obvious. Since the input document was identical for each LLM, the different results are based on what data each LLM could access. The quality and quantity of data used to train machine learning models are significant factors in producing accurate and reliable results. Acquiring high-quality data is becoming a priority for organizations that train machine learning models.     

I want to thank the students for allowing me to use their assignment content for my blog: Dalton Bent, Carlos Carlson, Allen Jomy, and Vishal Venkata Penjarla.

Their study is one more example of the significance of data for AI technology to be successful.

In May, I delivered a Masterclass at the annual Symposium for the Athens, Greece, PMI chapter. This was an extremely well-organized event, and it was exciting to present live in front of 200 people. There was an impressive lineup of speakers, all discussing the value of AI in project management. All speakers clearly engaged the attendees.

My message to participants was about the knowledge we need as project practitioners to understand and implement AI solutions that improve project performance. Based on my conversations with participants, the overwhelming sentiment was the desire to find out how to take advantage of AI. AI technology does not offer a simple or straightforward solution. It is flexible around solving problems and increasing productivity.

One observation was how having more project experience may not help with AI. While on stage, I polled people, asking them to raise their hands if they had ever used ChatGPT. About one-third indicated they had. Two months earlier, I asked a similar question to over 60 project management students in my classes at a business college in France. I asked how many had used ChatGPT on a regular basis. Every single student raised their hand. Is this evidence of a generational gap that will eventually have an impact on project managers? 

I heard from attendees at the symposium who were frustrated by unclear information about AI technology. How much data is required? What can be done about data security and privacy issues? How can ethical concerns be managed? Most importantly, participants wanted to know how to use AI successfully. In addition to answering their questions, I provided a step-by-step approach that starts with assessing their project data and then thinking about the project issue(s) they are trying to solve.

AI technology has many branches and can solve various problems using different methods. Training and education are the most significant contributions to achieving effective AI solutions in project management.   

Once a project begins, keeping it on budget and on schedule is the biggest challenge for project managers. Facing a variety of colorful status dashboards and a myriad of project metrics, what we really need is an indicator of potential future issues. Machine learning algorithms provide early detection of deteriorating budget and schedule performance, and those metrics must be included or prioritized. However, another metric may be of even greater importance and it is based on genetic algorithms, another form of machine learning. The new factor is called stickiness. Although this is a new concept for managing projects, entrepreneurs and new start-up companies often use it to retain customers. In marketing, stickiness refers to the likelihood that your customer will stay with your brand, make repeat purchases, and upgrade to a newer version of the same product.

From a project perspective, a stickiness factor adds resiliency to maintaining project performance. There are two steps in this process. First, the machine learning algorithm identifies the most critical metrics to prevent the project from deteriorating. Once the stickiness metrics are identified, the project manager determines what actions can be taken to increase the probability that those metrics will remain positive. Sticky factors consistently provide benefits to project progress. This might include a project complexity metric or a stakeholder volatility metric. The value is that the algorithm can identify the right metrics for each project rather than relying on a standard set of common metrics.

This might sound like key performance indicators (KPIs), but KPIs tend to be quantifiable metrics for each critical project area, such as budget, schedule, quality, or risk. KPIs are still important, but stickiness is what AI uses to keep the KPIs on track. Project management needs to be open to a new way of thinking. A machine learning algorithm finds the factors that result in a high level of stickiness, and a genetic algorithm determines the best actions to maintain the metrics.