Don't build an AI project without reading this.

Are you excited about ChatGPT's capabilities, or have you just finished your ML course and are ready to use these technologies to build a cool feature?

In both cases, you must know what challenges await you when building your first AI-powered feature. I wish I had known all of this a year ago.

First things first, are you sure you need AI?

Google's rules-of-ml suggest starting with a simple algorithm based on heuristics that will do the job and move to a machine-learning solution only when the heuristics become complex to maintain.

Adding AI to solve a problem brings additional challenges you have to deal with, such as:

• Explainability - why does the solution work this way for a customer
• Data Privacy - are there any 3rd party solutions that process customer's sensitive data
• Maintenance/implementation cost - the thing you build won't likely have the best target performance from day 1, and you'll have to spend more time improving the model

No planning = no success

What is the project about? What customer problem will it solve? What metrics do we expect to improve? What are known system limitations?

Before implementing, it is essential to have answers to all of these questions. The more potential pitfalls you identify during the planning, the better, as the cost of change is relatively low at this stage.

Where are the notes?

You should document everything—problem statement, metrics, desired outcomes, test cases, research log, design document, milestones.

Writing documents allows you to keep fewer things in mind. Other people can join your project quickly or use the results of your work in different activities.

Start small and simple

By keeping the first version of the feature simple, you can build it quickly, measure the impact, learn the insights, and continue iterating. Building a small thing also allows you to establish a baseline performance that you will improve in further iterations.

Creating a system that does many things and handles all possible use cases is an exciting challenge. However, making a skateboard is always good before building a car.

Have you done your research?

If the problem is new to the field and no one has solved it yet, consider researching it. You want to test whether solving the problem is feasible.

The research result is a minimal working prototype showing that an algorithm can solve the problem. Research can also show that there is no feasible solution, which is excellent learning—that's why you do this step at the beginning of the project.

Do not overestimate!

Your proof-of-concept may look fantastic and create hype, but there is a lot of work between the prototype and the production-grade solution that works for all users.

When working with new technology, the best thing you can do is avoid promising to get things done quickly. Take your time, get familiar with the technology and the problem space, split the work into multiple small milestones and estimate them separately.

When you notice that something doesn't go according to plan, communicate with the team and make sure everyone understands that the timeline/project has changed.

Write tests

You must apply the same software engineering principles when developing with AI. Your solution will evolve, and you must ensure it works as expected. Automated tests reduce the time spent manually testing things and give you more time to focus on the problem and solution.

Time for sad news...

Even after months of hard work, your model may perform poorly. This can be frustrating, but it's a part of the ML development process. You must accept that failure is possible and prepare to pivot your approach if necessary.

The important thing to remember is that every failure is an opportunity to learn and improve for the future.

Conclusion

Building an AI-powered feature requires careful planning, research, and implementation. It is crucial to start small, document everything, and communicate regularly with the team. And always remember to assess whether AI is necessary before implementing anything.

Remember that failure is possible, but it's an opportunity to learn and improve. The only people who never fail are those who never try.

Useful resources:

• Why you MUST write automated tests
• Rules of ML by Martin Zinkevich, Google
• How Cost of Development Changes over Time
• Making sense of MVP by Henrik Kniberg