Is big AI losing its edge? - Ctrl Alt Lead podcast

Is the era of cloud-only AI over?

AI is transforming edge computing with smaller, smarter models and real-world applications.

Artificial intelligence has made great strides in recent years, and its applications are becoming more specialised and focused.

In the latest episode of Ctrl Alt Lead, Paul Williamson, SVP and general manager of the IoT line of business at chipmaker Arm, joins the conversation to explore how AI is transforming edge computing - and vice versa.

Many people are familiar with large language models (LLMs), but Paul discusses the growing role of small language models (SLMs) designed for specific tasks. These "compressed" AI systems are changing the landscape of edge computing, enabling more efficient and lower-latency solutions.

In the episode, Paul explains how AI is reshaping traditional computing applications. For instance, microphones, which once relied on signal processing, now use AI for improved functionality. Meanwhile, edge computing is driving a shift away from massive general-purpose models to smaller, more constrained ones that require less computing power.

Paul also highlights real-world examples where AI is making an impact at the edge. In agriculture, compact AI models are being used for video and image analysis, reducing the need for costly, resource-heavy compute systems. And in Eastern Europe, the synthesis of AI and IoT is protecting both humans and the local wildlife from each other.

This is possible because of new developments in both areas, including AI SLMs and a new approach to the IoT where devices only connect to the cloud when necessary.

The rise of AI at the edge opens up opportunities previously out of reach. However, it also brings challenges, particularly in balancing performance and power efficiency. Paul shares insights into how organisations are addressing these issues, often by turning to external expertise.

Looking to the future, Paul discusses significant changes ongoing in software development now, which he says is already shaping the hardware that powers these innovations.

To access the full podcast watch the video, use the web player, or find Ctrl Alt Lead now on Spotify and Apple Podcasts.