Masha NehmeIT Program Manager| Central HealthAustin, Tx, United States
Can you share best practices and lessons learned around evaluating and implementing AI tools for your organization? How is/was this different from evaluating or implementing just any other technology vendor? Saving Changes...
Program Manager, PPM&PMO Specialist.| Coppel, Mexico.Culiacán, Sinaloa, Mexico
Hello Masha, in the case of my company, AI tools are being tested to review and correct code for vulnerabilities. They recently showed the results and showed a considerable double-digit increase in productivity of the teams dedicated to this work.
To support this work, suppliers are hired, but with this test of the AI tool the savings are significant and the expertise remains at home. The conclusion was to continue testing now at a more scalable level.
As best practices and lessons learned would be that before acquiring a tool, first make a use case with one of them, and analyze the results, and correct or escalate, as the case may be.
Best Regards!
...
1 reply by Masha Nehme
Apr 04, 2024 5:12 PM
Masha Nehme
...
Thank you, Francisco! Sounds like you ran a pilot first. How did you narrow the scope for the first round of testing? Also, how long did it take for you to see the results?
Saving Changes...
Omar JabbarProject Management and Digital Transformation Consultant| OGreen IT Service Inc.Ontario, Canada
Hi Masha,
In High-Level: This is On Demand Public Transport System
We recently evaluated an AI-enabled integrated voice recognition cloud solution. This might give you an idea to help you choose the right system or tool.
We started with:
● Defining clear objectives and desired outcomes. At a high level (what we have and what we need).
What we have:
1. Working process.
2. Clean data (Historical and Real-time.
3. Governance and methodology.
4. Buy-in from the management for improvement.
5. Resources.
6. Budget.
7. Requirements and user stories.
8. Initial timelines and high-level plan.
What we need:
1. Automate and streamline most of the current processes to:
a. Reduce trip booking wait times.
b. Reduce the load on the contact center staff.
2. Reduce capital infrastructure cost.
3. Scalability and integration with existing systems.
We conducted a thorough lean analysis to evaluate four different solutions and their alignment with the department's vision and mission.
To ensure comprehensive coverage of this exercise, we established a process to identify and evaluate stakeholders and invited representatives from all departments (Legal, Access Control, Privacy, Diversity, Cyber Security, Union, etc.).
As part of this initiative, we collaborated with the procurement department to issue a request for information (RFI) to explore the market, redefine requirements, and update user stories.
This broad overview of only one system replacement incorporating artificial intelligence.
...
1 reply by Masha Nehme
Apr 04, 2024 5:15 PM
Masha Nehme
...
Thank you, Omar. Was anything in the evaluation process unique to Generative AI, or would you say you followed more or less the same process as with other tech tools?
Head of Cloud Software & Services| Ericsson EMEAVictoria Island, Lagos, Nigeria
AI is used in my organization for report generation and also for booking Purchase Order in our system.
It is used for project tracking and performance measurement. However, it has led to a reduction of some human resources which justifies their expectation that AI will affect jobs.
...
1 reply by Masha Nehme
Apr 04, 2024 5:36 PM
Masha Nehme
...
Thanks, Taiwo. Wow, so there already was a workforce reduction because of an AI tool or tools being used. What is your take on the work your organization's Gen AI assistants are doing to generate reports or book POs? (quality, efficiency, etc.)
When evaluating an AI Tool, you should take into account mainly:
* What is the objective when using the AI Tool
*How reliable is the information or product produced by the AI Technology
*Owner of the AI Technology and license requirements
*Cost considerations, according to the budget available
*Adequate interface and adaptation for utilization by the personnel who will use it
A difference between any other technology vendor could be the necessity of evaluating the presence of ethical issues associated with the use of the AI Tool. Saving Changes...
Masha, As highlighted by our colleagues it depends on the objective. A few questions that you might want to ask:
1. Would like to a GPT kind of solution or a mere AI based evaluation that generates dashboards and reports?
2. Who are the target population and what are their pain points? How AI based tool can help them in achieving these goals?
3. Maturity of existing tools in the market
4. Would you be able to train it on properity data and internal methods/frameworks? and do we have enough data for this?
5. Would you like to run pilot first or go with big bang deploment?
6. Most important: acceptance/resistance by stakeholders, some people would be afraid that AI would take over their jobs (and it might happen in a few cases). You might need to provide them with psychological safety about it.
7. And the classic criteria like budget, timeline and ease of deployment, objective and goals for the change, scalability, local data and privacy laws etc Saving Changes...
Candice ShubbieConsultant| PROJECT40 ConsultingOntario, Ca, United States
From my vantage point of view as a consultant working with different companies, I find that organizations who work with more antiquated systems are struggling to update their legacy systems and procedures with basic technology, while more advanced companies are hyper focused on AI at the moment. As a PM, I see the evaluation, implementation, and management of these projects as one in the same.
...
1 reply by Masha Nehme
Apr 04, 2024 5:47 PM
Masha Nehme
...
Thank you, Candice! I have a similar observation having worked with both large companies and small-to-medium size ones - the process for evaluating a new tech tool is the same. The smaller, tech-savvy companies are very focused on incorporating GenAI tools and are more risk-tolerant than the large corporations, but everyone seems to be struggling with focusing on a specific approach long enough so that they can see some results. So many new shiny things to chase!
Saving Changes...
Sergio Luis ConteHelping to create solutions for everyone| Worldwide based OrganizationsBuenos Aires, Argentina
After working on this from more than 30 years I will say: there is no difference Saving Changes...
Masha NehmeIT Program Manager| Central HealthAustin, Tx, United States
Apr 03, 2024 11:38 AM
Replying to Francisco Herrera
...
Hello Masha, in the case of my company, AI tools are being tested to review and correct code for vulnerabilities. They recently showed the results and showed a considerable double-digit increase in productivity of the teams dedicated to this work.
To support this work, suppliers are hired, but with this test of the AI tool the savings are significant and the expertise remains at home. The conclusion was to continue testing now at a more scalable level.
As best practices and lessons learned would be that before acquiring a tool, first make a use case with one of them, and analyze the results, and correct or escalate, as the case may be.
Best Regards!
Thank you, Francisco! Sounds like you ran a pilot first. How did you narrow the scope for the first round of testing? Also, how long did it take for you to see the results? Saving Changes...
Masha NehmeIT Program Manager| Central HealthAustin, Tx, United States
Apr 03, 2024 2:38 PM
Replying to Omar Jabbar
...
Hi Masha,
In High-Level: This is On Demand Public Transport System
We recently evaluated an AI-enabled integrated voice recognition cloud solution. This might give you an idea to help you choose the right system or tool.
We started with:
● Defining clear objectives and desired outcomes. At a high level (what we have and what we need).
What we have:
1. Working process.
2. Clean data (Historical and Real-time.
3. Governance and methodology.
4. Buy-in from the management for improvement.
5. Resources.
6. Budget.
7. Requirements and user stories.
8. Initial timelines and high-level plan.
What we need:
1. Automate and streamline most of the current processes to:
a. Reduce trip booking wait times.
b. Reduce the load on the contact center staff.
2. Reduce capital infrastructure cost.
3. Scalability and integration with existing systems.
We conducted a thorough lean analysis to evaluate four different solutions and their alignment with the department's vision and mission.
To ensure comprehensive coverage of this exercise, we established a process to identify and evaluate stakeholders and invited representatives from all departments (Legal, Access Control, Privacy, Diversity, Cyber Security, Union, etc.).
As part of this initiative, we collaborated with the procurement department to issue a request for information (RFI) to explore the market, redefine requirements, and update user stories.
This broad overview of only one system replacement incorporating artificial intelligence.
Thank you, Omar. Was anything in the evaluation process unique to Generative AI, or would you say you followed more or less the same process as with other tech tools? Saving Changes...
Masha NehmeIT Program Manager| Central HealthAustin, Tx, United States
Apr 03, 2024 3:56 PM
Replying to TAIWO POPOOLA
...
AI is used in my organization for report generation and also for booking Purchase Order in our system.
It is used for project tracking and performance measurement. However, it has led to a reduction of some human resources which justifies their expectation that AI will affect jobs.
Thanks, Taiwo. Wow, so there already was a workforce reduction because of an AI tool or tools being used. What is your take on the work your organization's Gen AI assistants are doing to generate reports or book POs? (quality, efficiency, etc.) Saving Changes...
Community Champion
