Do’s and Don’ts of GenAI Spend

Clayton Christensen wrote in a very provocative article entitled The New Church of Finance, “Empowering innovations transform something that is complicated and expensive into something that is so much more simple and affordable that a much larger population can enjoy it.” Clayton was bemoaning that the generally accepted principles of finance led to underinvestment in empowering innovations. Generative AI (GenAI), such as ChatGPT, is certainly an empowering innovation.

So, what principles should CFOs adhere to as they evaluate generative AI projects?

Don’t focus on short-term ROI but demand short-term success: GenAI is not yet as advanced as AI focused on diagnosis and predictions. With those technologies, we can and should expect ROI in a short period of time. But with generative AI, the focus should be to get the technology in the hands of business users as quickly as possible so that they can try out different use cases. When the internet or PC first started, we could have never predicted the breakthrough use cases. With GenAI, the key is getting end users comfortable with the technology so that they can figure out the best ways to use it to advance their business.

Don’t spend money on inflexible solutions: Many companies are spending money on consulting projects to operationalize a specific use case using a specific technology. The underlying technology in GenAI is evolving rapidly. For example, many projects delivered outstanding results with the initial release of GPT-4. But, as more users started using the system and the response time became too long, OpenAI seemed to adjust the algorithms such that they run faster (and at lower cost to OpenAI) but deliver much worse results. As a result, pilot projects that had been successful started running into trouble and companies had to redo the work on the projects with other models.

The best technology for this space will change 15 times in the next three months. If you haven’t read the leaked Google memo entitled “We Have No Moat, And Neither Does OpenAI,” it is well worth the read. If you are spending money on work that is specific to a model, you are wasting your money. Ask how much of the work will be reusable if a different GenAI technology becomes dominant. Value that flexible part of the project only -- because the GenAI technology-specific part will need to be thrown away soon.

Keep your eye out for areas where AI can drastically improve cost structures: AI will not just bring incremental benefits. In a world designed around human-first processes, AI-first introduces some absurd opportunities. For example, some of the major data warehouse services have usage-based pricing. Essentially, you are being charged for the length of time the data warehouse is focused on answering your questions. But an AI can ask a million questions in the time a human can ask a handful of questions. Thus, in an AI-first process, a million questions cost as much as what you are paying today to ask a handful of questions. Of course, at some point, usage-based pricing from warehouses will change to address AI-first usage patterns. But in the meantime, if you are an early adopter of AI-first processes, you can reduce your effective data warehouse costs by using AI to efficiently analyze data in bursts on serverless architectures and then answering user questions real-time using generative AI. Look for drastic savings like these that can pay for your entire GenAI project in a matter of months.

Maximize organizational learning: As discussed above, we don’t yet know the right use cases and usage patterns for GenAI. So, it is important to set up the GenAI projects to maximize learning. But whose learning? Data scientists and engineers will always be eager to learn new technology and that is a good thing. But GenAI is by definition general purpose and the people best able to come up with useful use cases and evaluate the usefulness of the GenAI output are the business users. So, make sure the tech gets in their hands as quickly as possible. Make it easy for them to experiment with different use cases. Then make it easier for successful use cases to be adopted by other business users as quickly and widely as possible.

But how is that the role of finance leaders? Well, finance leaders are business users, too. Insist on getting your hands dirty with GenAI projects when people ask for budgets for them. If they say the project is too complicated for you to understand, it is probably a science project for data scientists as opposed to something business users will quickly benefit from. Essentially vet the budget request by volunteering to be a guinea pig.