2025 is the year of ‘show me the money’ for GenAI, Snowflake strategist

Organisations have spent the last 18 months experimenting with generative AI. This year will see a whittling down.

2025 is going to be GenAI’s the year of “show me the money,” says Jennifer Belissent, principal data strategist at Snowflake.

Large organisations have spent the last 18 months experimenting with generative AI. This year will see a whittling down, a prioritisation based on what has worked.

Businesses will be focusing on “what can be developed and potentially reused across the organisation,” in order to create greater efficiencies, and also on “being able to measure the business value”.

“I believe that 2025 is going to be about show me the money,” Belissent told Computing. “All right, well, we've put these a few things into production … so show me the money is it generating. Is it what we expected?”

Most projects fail

According to researchers at RAND, more than 80% of AI experiments fail, a rate twice that for projects that do not involve AI. While this is understandable given the novelty of the field, it is unlikely that this situation will be tolerated for much longer; there will be pressure on AI evangelists and IT teams to show up with some positive results. Projects will need to demonstrate they how they align with the broader business strategy, and how and when they will provide a return on investment.

Currently, most AI projects are instigated, managed and paid for by individual lines of business. Within those departments they are cost centres. This year departments will need to justify their investment in new datacentre infrastructure, software and skills. As projects scale up these costs can rise exponentially, increasing the pressure to demonstrate a return. If they cannot, they risks joining the 80% in the fail bin.

Other factors that can stand in the way of successful operationalising of promising experiments include a lack of skills, inflated expectations, regulatory requirements and a dearth of good-quality data with which to fine-tune AI models. There’s a lot that can go wrong.

On the plus side, though, successful projects are leading to an increasing number of lessons learned. Best practices are emerging which are either filtering into in the public domain or being propagated by vendors and consultants to their other customers. These include how to roll out AI, how to set expectations, where to prioritise, and how to benchmark results and measure success.

Data diversity

As models move from experiment to production so the need for trustworthiness increases. They must be trained and fine-tuned with representative data, not just material from sources that are easiest to access. There must be guardrails and safety measures, and to minimise hallucinations, biassed output and dead-end reasoning, LLMs need to understand the context as fully as possible.

“I spend a lot of time talking about data diversity,” said Belissent.

“People are still wringing their hands about the risks of hallucination and bias. How do we mitigate those risks? Well, just as we do as humans. You inform yourself, you ask other people's opinions. You expand your research to make sure that you've got the full context.”

For organisations, “it's about making sure you're using all of your own internal data.”

Practically speaking, providing diverse training data means, breaking silos, flattening walled gardens, ingesting unstructured data, forging data sharing agreements with partner organisations, and possibly creating synthetic data to fill in gaps with the goal of making sure the data is representative.

LLM vendors may provide assurances as to the data used to train their models. At Snowflake, data diversity and accessibility are addressed by supporting open source and interoperable protocols and formats, notable Apache Iceberg, a table format for large analytic datasets that it supports through Snowflake Open Catalog, providing access to Iceberg tables via a variety of data engines. The intention is to avoid lock-in when fine-tuning and inferencing models, and to remove the bottleneck of having to copy data from one place to another.

“So that means that your data is not moving. It's not like you copy it and send it to somebody else,” Belissent explained. “It stays very secure, and it stays governed by the rules that you've set up for it.”

The company’s Arctic LLMs are open source and licensed under Apache2.0 in line with the focus on interoperability.

Adjusting expectations

On skills, organisations can learn a lot from their experiments, not least how to navigate a world characterised by questionable claims. Belissent references an airline which started building its own models but is now looking to take something off the shelf.

“This exercise of actually having your teams build models is a great way for upskilling, so that they know what to evaluate when we start down the route of buying. I thought that was really mature.”

Finally, said Belissent, it’s important to set realistic expectations about what AI can do and where it should be considered. It is wasteful and possibly dangerous to use if for a task where existing systems are working just fine. This means picking the right sized model for the job and not using AI just because you can. “You need to know what are you doing and why. Do you need GenAI for it? Just because we've got hammers doesn't mean that every project is now a nail.”

And a big part of this is education. Just as techies in the past needed to be made aware of the costs of leaving a bunch of cloud instances running over the weekend, so people need to understand the financial and environmental costs of using AI. “It’s really educating people across the organisation and making sure they understand that.”