Unique Challenges With AI/ML Projects

Tod Bookless works in software development and has run projects for Apple, Amazon, Disney, Microsoft, Starbucks, HP, Intel, Cisco and others.

  Artificial Intelligence  

Traditional software development projects have a well understood workflow. To grossly oversimplify the typical software development lifecycle (SDLC), and ignoring resourcing for the moment, we know that software development projects generally fall into these broad areas:

• Develop requirements and determine which ones are required for launch
• Break down the work into tasks
• Develop the code
• Test the code to see if we met the requirements and fix any launch blocking bugs
• Launch

We can use waterfall or agile processes, we can start testing much earlier, and we can add more milestones, artifacts and reviews for stakeholders, performance, legal compliance, privacy, etc. But at a high level, those are the major tasks that we all need to complete to launch nearly any software project. Artificial intelligence and machine learning (AI/ML) projects, however, tend to break this pattern.

Let’s explore a more specific example. Suppose your company takes in a huge amount of files, or metadata, for some reason. They could be financial documents, legal documents, product descriptions, or any other kind of files. The data might be in XML format, JSON, or it might be in flat files. It doesn’t matter. For this example, let’s assume that the amount of data is so large that it is not really practical for a human, or a team of humans, to review all of it. I have …

Please log in or sign up below to read the rest of the article.