Artificial Intelligence (AI) will continue to evolve and revolutionize industries-that’s not going away. However, implementing AI features or launching projects require thorough planning and safeguards in order to avoid costly, embarrassing, and even dangerous mistakes. Lucky for us, a surprising number of big companies have done the legwork of exploring missteps in AI launches and we can learn from their mistakes, grow, and ensure the success of future AI initiatives.

Let’s start with something relatively harmless- just a little embarrassing -with Instacart’s AI Recipes.

In 2024, Instacart- a popular grocery (and now restaurant) delivery platform- likely wanted to increase their recipe content and chose AI to be the tool to do it. They were able to scale up content, but overlooked the importance of human oversight to ensure suggested recipes not only made sense, but the accompanying AI-generated photos were on point as well. The result: some unpalatable concoctions and borderline disturbing images (e.g. anatomically incorrect depictions of chicken, hot dog slices that looked like tomatoes, etc). AI-image generation has advanced quite a bit, but if you’re familiar with some of the early, off-putting examples,  you can imagine how these would have likely fallen short in wetting someone’s appetite to buy ingredients. This serves as a good reminder that businesses need to ensure AI improves the product or experience – otherwise, they may save some pennies on content creators, but lose in the grand scheme with their customers.

Lesson(s) learned: Implementing AI solutions require human quality control. When the impact causes a negative experience, even when it’s comical, customers may lose trust in the brand and that can be costly to bounce back from.

Instacart wasn’t the only ‘Grocer’ to attempt this feature, but the next example was actually dangerous: Pak ‘n Save’s Meal Planner

A beloved supermarket, Pak ’n Save, wanted to deploy an AI-powered meal planner to assist their customers in getting creative with the ingredients they had. The result: harmful recipes, including one that would produce chlorine gas, with household products. Other infamous “recipes” that were shared with customers include poison bread sandwich and mosquito-repellent roast potatoes. In Pak ‘n Save’s defense, these examples likely, if not all, arose from users getting creative and being funny while testing the meal planner with unlikely ingredients. Unfortunately for Pak ‘n Save, this became a global news story, but it’s not difficult to see how this could have been avoided. Today, their meal planner still has the warning, “Savey Meal-bot uses a generative artificial intelligence to create recipes, which are not reviewed by a human being.”

Lesson(s) learned: Testing. Testing. Testing-AI has shortcomings and even when you think it’s at the highest level, it needs to be explicitly programmed with safety constraints. And then, testing! In general, but especially anything that could remotely affect health and safety, businesses need to be thorough with their risk assessments and human oversight.

‍

Training on Faulty Data with Amazon’s Biased Hiring Tool

Amazon got innovative with their hiring needs and created a resume screening tool using AI. This algorithm required past hiring data to learn from and it’s important to mention that Amazon’s workforce has been predominantly male. Resumes that included “women’s” to describe any of the groups, clubs, or hobbies (e.g. “women’s chess club”) the prospective employee was involved in, the algorithm would downgrade and ultimately score female candidates lower. Luckily for Amazon, they noticed this bias and when they realized they could not adjust the algorithm, they scrapped the tool back in 2017 before it was deployed.

Lesson(s) learned: AI is only as good as the data it’s trained on. If data is flawed, biased, or incomplete, AI will repeat the errors and often even amplify the problem. Regularly audit training data to avoid legal risks and ensure the results are desirable before deployment.

‍

An Expensive Example with Zillow’s House-Flipping Algorithm

The real estate platform that helps buyers and sellers, as well as renters, found a creative way to expand their services with Zillow Offers: a side project of the company that would use AI to find the optimal time to buy houses and flip them for a profit. Three years into the project, Zillow lost over $800 million and had to lay off ~25% of its employees. How? The algorithm regularly overestimated home values but the company still purchased thousands of properties at costly prices. 

Lesson(s) learned: Any dynamic market with complex factors poses a real challenge for AI models. Controlled testing can be the right first step, but it doesn’t ensure your project will work in the real-world with unknown conditions. 

‍

Slow is smooth, smooth is fast. 

The possibilities are endless when it comes to AI and it’s always getting better, but in the meantime, industries must be smart when approaching these initiatives. Clear objectives, rigorous testing, and human oversight are not friendly suggestions - they’re part of the best practice guidelines for successful deployments. For aviation—where safety and reliability are crucial—be cautious, be thoughtful, and be structured with any AI implementation. These are just a couple examples of AI failures (and we’ll undoubtedly get more overtime), to learn from and avoid similarly embarrassing, costly mistakes.

‍