When trying to run and manage an AI project, there are many things to consider when it comes to determining the project’s budget and costs. Every technology project has three main cost aspects: software, hardware, and services. But when it comes to AI projects, you need to consider an additional aspect: data.
Determining the actual cost of your AI project
If you are trying to increase your return on investment, you need to know what the investment part is. You also need to know what the return is. However, while you are guaranteed to spend at least as much as your investment portion of the two, there is no guarantee of return. People underestimate the complexity and cost of working with data. Therefore, it is important to consider the scale of your AI project. Because scale matters a lot when it comes to AI projects.
What is the real cost of AI?
A recent AI Today podcast delving into the cost of an AI project covered the topic of how various aspects of people, process, and technology contribute to the cost of an AI project. Data is at the heart of AI, so the challenge is to go beyond the cost of software, hardware, and services. Specifically, the costs of data collection, preparation, and cleaning can be significant. Data continues to grow, but unfortunately not all data is ready to use in a clean, pristine format. Compounding these data quality and quantity issues is the challenge of accessing the data you need.
AI project managers often underestimate the total cost of an AI system, which can have a significant impact on the project. There are many things to consider when thinking about the total cost of an AI project. One involves building and purchasing AI models. Are you using someone else’s model? Are you going to focus on rapid engineering or fine-tuning? Are you building a Search Augmentation Generation (RAG) solution or do you plan to build your own model?
You also need to consider where your model will be tested and used in the real world. Are you planning to use it in the cloud? Locally? How are you training your model? And of course, you need to consider all aspects of data engineering.
How to reduce the cost of your AI projects
Think big, start small, repeat often is the real mantra for successful AI projects. If you’re starting small, how small is important if you want to control the cost of your project. How small you want to go will vary for each project and organization. Therefore, it is important to understand the scope of the project.
The iteration scope of an AI project has a direct impact on cost. Project iterations should be short. Each iteration takes about two weeks instead of months. This means controlling the scope of your project so you can do short iterations.
One way to control scope is to build using a model that someone else has already built. It has the lowest cost and shortest iteration time. If it’s already available, just use it. This is one of the most effective ways you can start small.
That’s why LLM and foundational models are getting so much attention right now. Costs are low, iteration times are short, and potential return times are very short. Therefore, if you are cost-conscious and your budget is not very large, using someone else’s model is a very good option.
Keep AI costs low
There are some easy and low-cost ways to use someone else’s model. If an API exists, use it. If you have a chat interface, use it. Some are available for free, while others are available for a small subscription fee. You can also build incremental solutions on top of the API that don’t require training new models, such as acquisition augmentation generation (RAG) solutions. If you cannot use the model directly, you can also fine-tune the model or extend it with additional sources.
It’s important to understand that all of these options aren’t free, but they can help keep prices down. Some AI systems may have monthly subscription fees or API costs for using someone else’s models. Additionally, keep in mind that time may be required by analysts, developers, users, and/or citizen developers, and people’s time is not free. There is an associated time and cost associated with generating prompts or inputs for the model. Building RAG or other solutions on top of it requires additional development time.
In addition, you may need to prepare data to input into someone else’s model or to obtain results from that model. If you need to make small adjustments or adjust the model, there will be some cost.
Overall, using someone else’s model can significantly reduce the cost and time it takes to deploy an AI project. However, as with any AI application, you should always check the results. This is especially true for LLMs, as they can have pretty bad outcomes. Sanity checks are very important.
Do you need to spend the time and effort to build your own AI model?
There may be situations where you cannot simply use or sit on someone else’s model. In this case, building your own model may make the most sense. There are many reasons, use cases, and justifications for taking this approach.
Considering the cost of the project, building your own model is usually not the cheapest solution. So if you adopt this “think big, start small, iterate often” mindset and approach to building your models, you can actually save a lot of money by starting small because you can keep the scope down.
When trying to understand and determine the cost of an AI project, you must also consider the cost of the AI ​​service. What types of services will this iteration of an AI project require? Who will be on the current and future AI team for this project? This will depend on the labor costs that need to be paid and the scope of the project. It is important when determining the cost of an AI project because it can also affect the cost of an AI project.
Especially when you think about AI teams, the composition is different when using someone else’s model versus building your own model. If you’re using someone else’s model, you’re likely looking at a “citizen developer”, which is easier to find and more affordable. Recently, efforts in generative AI have been moving rapidly because they can produce results very quickly, even if it is not difficult to find well-trained data scientists and data engineers on the team. Because people realized that it was possible.
However, even if you don’t build your own models, you need to identify who on your team will manage the data and monitor data quality. Garbage is garbage, so we need high-quality data coming in. You also need to identify who will monitor model usage and input/output quality. When it comes to AI, you can’t just set it and forget it. This also includes generative AI. This all impacts the overall cost of your AI project.
On the other hand, if you’re building your own model, you’ll need a data engineering team and a preparation team. When you operationalize a model, you need the right team to build the model and constantly monitor it for data drift and model performance drift. These costs are likely to be fixed costs, so you know exactly how much they will cost you. Therefore, even if the demand for the model actually increases, the cost will not change.
Consider all the factors outlined above when determining the actual cost of your AI project. Understand your team. Understand your budget. For a successful AI strategy, understand your scope and follow best practice methodologies such as CPMAI. (Disclosure: I am the managing partner and co-host of the AI ​​Today podcast)
