How do you navigate unexpected data challenges in your projects?

Thank you Nikita Jha for sharing this illustrative experience with data. Since we are encountering "new" problems we have to be creative in the solutions and yours seems to be a great option. I guess that finding the right resources for that fast-tracking approach was a challenge. Did you include specific data-related profiles? Have you detected any new roles needed in the context of project + data? thank you

Hi Nikita, thank you for you contribution. I came across a similar situation for an Oil & gas company. In the case of legacy systems where data reside in data warehouses or data centers, this will increase complexity. Now with solutions from vendors like AWS, Azure or GCP, we can leverage them and minimize the risk

Data quality issues are quite common in complex projects like M&A systems migrations. For example, during a merger I managed during the COVID-19 pandemic, we ran into a big challenge with misaligned data systems between the merging entities.
To tackle this, we first thoroughly analyzed the critical impacts and the potential fallout. Then, with the collaborative spirit of our team, including the principal architect and system owners, we brainstormed and implemented both temporary and permanent solutions. This process not only solved our immediate issues but also strengthened our teamwork and adaptability, underscoring their importance in maintaining data integrity throughout the project.

As an old adage goes ‘garbage in garbage out’, so what you put in determines what comes out’. I did encounter few incidences where Geotech reports were not spot on. One instance the depth of roots was quite deep and extensive. And still in another instance, the first 300m of an access road on a landfill site was as per Geotech the rest we discovered an average of 900mm layer plastics. In both instances time and money were lost on the project in redesign, and a substantial amount of additional earthworks resulted. It was suspected, the existing data was used, or even extrapolated without actual site investigation. Many things could have happened in between when data was first recorded to a time later when the data was used. Indeed, recent updated quality and quantity data can leverage AL usage also on construction projects, but there will always be aspects that requires physical human intervention to verify or observe salient features.

Wow! It sounds like one of my last challenges, even though within a smaller project (all the teams were in the same timezone).

I think you took the best approach, you nailed it.

Thank you very much for your comment.

In my project manager roles, I gather detailed requirements and do my pre-project research to understand data requirements and work with Data SMEs seamlessly. I assigned responsible owners to different / individual data elements but due to un-foreseen circumstances or other project demands, they do not deliver at due dates. When there are late updates or no updates to utilised data, I'm able to pick up the piece of work based on my knowledge and set timelines for data quality for efficient usage. I also assigned deputy owners who deal with uncertainties and pitfalls. These process help mitigate the issue.

Of course, I find that despite basing the project’s design assumptions on trends, the reality of the project can often be affected by any changes in aspects that seemingly have no relation to the project. In those cases, the project planning is reconsidered, as long as it still aims for the same development objectives.

Keeping it very simple initially: Assure that the data resources are accurate- companies can vastly vary on how their data is pulled and distributed. Review lessons learned from prior projects, strong proactive risk analysis, and contingency plans.

I have also encountered unexpected challenges while using data in some projects.
One challenge was related to the data availability, and it affected also the accuracy of the performance analyze provided to customer.
Collaborating with the teams, who were responsible for data availability, we provided improvement plans, looking to mitigate this risk, by creating additional scripts to monitor data availability and correcting in time the performance analyzes delivered to customer.

Si si he encontrado dificultades, principalmente cuando he querido integrar el project profesional, con teams y con power bi.

Thanks for your contribution to the group.

Data fields must be able to coexist. Field names should be consistent and translate the same information. The slight difference in in naming conventions, can and will prolong the task.

I haven't encountered challenges myself due to not yet using GenAI in my projects, so appreciate getting to ready the insightful responses and learn from my peers.

Have you ever encountered unexpected challenges or pitfalls while using data in your projects? How did you navigate the situation and find a resolution?

Have developed many solutions for regulated industries whereby our offshore teams couldn't see the data without masking, scrambling or at all. Had to be quite creative in the use of different environments, demonstrations and development techniques.

Amazing work!

Really interesting insights Sergio Luis Conte Thank you for sharing them. I agree when you say that everything around data rests on other disciplines rather than on project management, but I also think that being data so decisive in projects, and more to be in the future, project managers are already shifting towards an understanding of how data behaves and affects their projects. It is as if until now, data-related issues were just another input coming from specific departments but now, project managers are trying to understand and influence those inputs since they affect their projects. What do you think?

AI integration is still new to us.

however Sergio even that many tools can be applied specialli to AI data, video manipulation, fake news can give you strange allucinations so that human control at the end is the only way that AI unexpected outcomes can be monitered, detected and corrected ....

Very rightly said Luis, Data quality and quantity are critical considerations when leveraging AI in projects because they directly impact the accuracy, reliability, and overall success of AI models.
High-quality data ensures that the inputs to AI systems are clean, consistent, and free from errors or biases, which is essential for producing meaningful and unbiased outcomes.
Adequate quantity of data, on the other hand, provides the AI with enough examples to learn patterns, make predictions, and generalize effectively across various scenarios. Without sufficient high-quality data, AI models can underperform, leading to incorrect results, reduced efficiency, or failed implementations.
Thus, managing and optimizing data quality and quantity is foundational for maximizing the potential of AI in any project.

Claudia,
I believe I believe that 'unexpected data' suggest that there is a break of pattern in data algorithm or progression. This can suggest that the new data encountered are outliers which upon further investigation reveal the reason behind the outlier. It may be worth your effort to find that reason in a means of understanding that data in particular. Additionally, in project management, risk, both foreseen and unforeseen, will give measures to take when unexpected data occurs. Root cause analysis may then increase the scope of previously determined risk. I find Ishikawa diagramming of 8P to a comprehensive enough to establish a good scope of risk. Lastly, I believe that the quality of the data is also established by the validation of the accuracy of the data tested. Depending on the type of data used there are several statistical test to do on both data instruments and data. Some of which are: Gage R&R, One way Anova, Standard Deviation Tests, Kruskal Wallis tests etc. It all depends on your data type and what exactly you want to determine. :

I would say that ensuring applicability of the data is the greatest pitfall. Meaning finding data that is most appropriate for the task at hand.

If I understood well your point I should say that there is no change about what we are doing today related to work with project and program related data.

So, what is the data for? and how the data is coming today to the decision-makers. I'll say a better source is better data, nothing new, right but the source is still changing, so enterprise data awareness and alignment could support the way of handling data into information nowadays.

Thanks for sharing, very insightful...

While I agree that it is up to data scientists to manage AI, they don’t know the ends and outs of the work of project management. I think of it as data scientists organizing the data to be most useful in the models. But the project manager must work in a close partnership to provide the data. It would have to be a team effort to be truely successful. Interested in others thoughts.

In accuracy and duplicated data has always been a problem. We have had client requests to retain personal data for 50 years and with that long a period owner of the data don't remember data on fil3e, This may be a dumb question, but when AI is pulling data from various sources how do you make know that proper security and authorization to use data is in place?

Its always a struggle in large organizations to identify the true owner of data, and prevent replication / creation of outdated data. There are multiple vendors offering data management software - has anyone had any experience using or implementing Illumio ?

Our jobs would be a lot easier if this is all that was required. There are still data fiefdoms out there whose management teams that have to be educated on scrubbing/normalizing data (old terms; same problem).

well articulated.

Quality data without a doubt help solid foundation. It shows the difference between reaching the people and reaching the right people.

Gaurav, you make a great point that prioritization is key. Data informs decision-making and the quantity of data allows companies to make informed decisions, however, poor quality data can lead to flawed analysis and bad conclusions. Clear priorities help make collecting, aggregating, and effectively using data mor efficient. Data security is also extremely important as it ensures that sensitive information is protected from nefarious use and is compliant with any necessary regulatory body. While AI may bring about new challenges, the data challenge will remain the same.

I agree. Historically, while the value of data may not have been as apparent as it is today with the emergence of new technologies, issues related to data security—such as unauthorized access—have always been significant. However, these concerns have become even more critical now due to the immense value placed on data and the substantial consequences of a security breach.

Thank you Visukumar, I completely agree with you. Project managers have the responsibility to make data relevant in their projects. We have the knowledge to do it, let’s understand how data flows in our companies and develop the communication skills to break silos with IT, data scientists…

I would also add the need for continuous engagement of all relevant stakeholders.

I agree Biwi, its essential to have a very good communication between the teams' members and the stakeholders, by leading the project manager, several times the stakeholders (managers, chiefs, suppliers, analyst, etc.), provide poor or incorrect data, in this case is responsibility of PM to implement a set of procedure to correct and validate the data, along the whole project.

Mustafa addressed this issues really well.

Working with data everyday, it is important to ask the data analysts or business intelligence where and how they acquired the data. Ask them if they understand the data based on workflows and operational definitions in different operation areas. Data Quality often becomes compromised when the data pulled from different resources are inadequate when the agent did not understand the workflows, leading to pull the wrong data. Alot of old databases do not have the operational definition for each field, causing poor data quality..This is very important when working with new data analysis or business intelligence.