For several years, artificial intelligence (AI) applications, in one form or another, have driven Google search results, tailored our news feeds, run our customer service bots, driven us crazy with predictive text, supported pharmaceutical research, sent us Facebook friend recommendations, and suggested the advertisements that litter our screens. As a technology, AI lived almost exclusively in the software developer’s toolkit, and was mostly unbeknownst to the public.

AI’s relative obscurity in the eyes of the public was not deliberate, however. Consumers are not typically the first audience when it comes to new technologies. Consider that in the post-war tech boom, the mainframe was the reserve of large enterprises such as banks, airlines, governments, and insurers who could afford their price tags that often ran into the millions of dollars. Towering mainframes were the tour de force of the age, and the concept of retail demand for computing was laughable. That was until Steve Wozniak and Steve Jobs made personal computers (PCs) the must-have accessory in the early 80s.

At the time, the cost of this technology was prohibitive, and the processing power was extremely limited. However, as technology improved exponentially, prices fell equally quickly. In 1970, the $4.6 million IBM Mainframe wore the crown of ‘fastest business computer’ with 12.5 MHz of speed on offer. Eleven years later IBM launched its first PC which was powered by Intel’s groundbreaking 8088 microprocessor offering up to 8 MHz of processing speed for $1,565 per PC. By 1997, Apple’s Macintosh Performa gave you a whopping 100 MHz for $1,900, and today my sturdy Dell work PC gives me 1 600 MHz of performance, for a fraction of the cost.

COST OF AI IS PROHIBITIVE, BUT FALLING

Much like mainframes, AI does not come cheap. The training and fine-tuning of large language models (LLMs) such as OpenAi’s GPT were achieved using cutting-edge computer chips which were originally built for computer games. Supplying this computing power is exceptionally lucrative and chip manufacturer, Nvidia, is the only at-scale supplier in town. Nvidia’s $10,000 A100 graphics processing unit (GPU) is credited with the training of OpenAi’s various generative pre-training transformers (GPTs) including the latest GPT-4 which is rumoured to have required around 25,000 A100s – that’s a whopping $250 million capital expenditure. Nvidia’s state-of-the-art processing chip is the $40,000 H100 which OpenAI is currently using to train the, yet to be released, GPT-5.

Unsurprisingly, other chip makers are vying for market share, including AMD which recently announced its latest M1300x as a somewhat better alternative to Nvidia’s H100, while Intel’s Gaudi2 is a valiant attempt for a company playing catch-up. In both cases, AMD and Intel offer a compelling value proposition. However, Nvidia has a competitive advantage. Large existing customers (known as hyperscalers) cannot use Nvidia’s proprietary software package, CUDA, if they use other manufacturers’ chips. If hyperscalers do want to use another supplier, the software development costs of switching from Nvidia to AMD or Intel chips would be prohibitive. However, for smaller and medium-sized companies looking to enter the model-building space AMD and Intel’s opensource software approach is compelling.

The other good news is that deflation in AI LLMs is similar to the deflation experienced in processing power that occurred in the late 20^th century, as highlighted above. As an example, OpenAI charges $1.00 per million input tokens and $2.00 per million output tokens. Mistral AI, a competitor offering similar language models charges $0.65/$1.96 per million input/output tokens, respectively. Further down the pricing ladder Together AI costs $0.60 per million output tokens with no cost per input token. Tokens are the finest granularity in natural language processing by LLMs. This means that for professional programmers, universities or hobbyists, the cost of research and development in AI has deflated significantly in the space of a few months.

AI SOON TO BE UBIQUITOUS

Nothing I’ve said thus far addresses the broad commercial potential that artificial intelligence presents, or rather, why the race to the bottom in LLM pricing should matter to the general population. In fact, we are still in the dawn of the technology’s broad and direct adoption. To date, outside of ChatGPT, and its imitators, few AI-first software products exist for the average consumer yet. This is all about to change.

Microsoft 365 Copilot is arguably the most hotly anticipated AI/GPT software release in 2024. Copilot will be embedded in Microsoft’s suite of applications like Word, Excel and PowerPoint, and incorporate LLMs alongside user’s data to reshape the productivity experience for millions of existing Microsoft users. Furthermore, the incremental subscription costs of Copilot will provide a new revenue stream for Microsoft.

Intel’s Developer Day in September 2023 offered a glimpse into other little-known applications. Scala Biodesign developed an AI-powered computational platform which enables efficient protein design for pharmaceutical, chemical, and industrial applications. Deep Render developed an AI-based data compression technology that claims to be 80 years ahead of traditional compression, and effectively improves internet bandwidth by five times. Rewind.ai is an AI assistant that records PC activity (anything seen, said, or heard) to give users near-perfect recall of their workday for summarisation, searchability, and backup to name a few benefits.

Our anticipation of AI’s impact is even broader, however. The breakthroughs in machine learning and AI workloads will eventually make most existing PC hardware somewhat redundant given the trend towards inference which involves running LLMs natively on PCs as opposed to the cloud. This suggests that realising significant future productivity gains from AI software could usher in a multi-year PC replacement cycle to equip workers with capable machines, which is good news for PC hardware manufactures, but bad news for corporate IT budgets. Crucially, in December 2023, Intel launched its new Intel Core Ultra processors with an integrated neural processing unit (NPU) which addresses the technical requirements to run AI workloads on-device and offline. Acer’s Swift Go 14 laptop is the first PC powered by Intel’s latest high-end chips and some reviews have compared it favourably to the MacBook Air 13 M2, but at a greatly reduced price. Let that sink in.

Further, it appears a wave of boutique AI consumer devices is also in the works following the launch of Humane’s Ai Pin, a screenless device that relies on hand gestures and voice control. The more recent Rabbit R1, is a handheld gadget powered by a ‘Large Action Model’ that can effectively navigate your apps such as Spotify and Uber based on voice commands. Both devices rely heavily on AI, though some believe they are “just an app in physical form”. Critique notwithstanding, the first generation of new devices generally always come across as gimmicky when they are first launched.

Like the incipient PC evolution towards inference, Snapdragon (the dominant mobile chip maker) recently released its 8^th Gen 3 flagship which comes with a credible NPU, not to be outdone by Apple’s A17 iPhone chip with similar hardware. Building smartphones and gadgets such as the Ai Pin with these latest chipsets will gradually make them bona fide AI devices as opposed to just GPT ‘wrappers’.

A NEW BREED OF PERSONAL DEVICES: GIMMICK OR EARLY BREAKTHROUGH?
Humane Ai Pin ($699)	Rabbit R1 ($199)
https://hu.ma.ne/aipin	https://www.rabbit.tech/

It often happens that use cases emerge in a completely unpredictable fashion. For instance, soon after Alcoa (Aluminum Company of America) appointed Paul O’Neill as CEO in 1987, he led the connection of Alcoa’s global offices using the recently innovated computer network. The sole reason for this was his obsession with plant safety, and the sharing of real-time safety data which unwittingly created one of the earliest global corporate emailing systems, giving Alcoa a competitive edge.

AI EXPECTED TO DRIVE S&P 500 EARNINGS

The optimism around ‘next-gen’ technology is illustrated by the consensus year-on-year earnings growth expectations for the S&P 500 Semiconductors Index of 45% and 28% in 2024 and 2025, respectively, compared to the broad S&P 500 Index earnings growth expectations of 7.5% and 12.8%.  Robust demand for GPUs, RAM, and memory follows naturally from the increasing shift towards cloud-based machine learning and AI services. The semiconductor earnings outlook, however, also reflects an expected recovery from the current glut in semiconductor pricing caused by post-pandemic oversupply conditions.

This article will soon become outdated by the blinding pace of developments in the AI space. As it stands, we do not know the capabilities of Open AI’s GTP-5, which has the potential to further disrupt our sense of what AI can do for software, productivity, or devices. What we do know is that the disruption so far has been spectacular, and it is an exciting time to be investing in technology.

REFERENCES

• American Enterprise Institute (AEI): “Computer Prices and Speed: 1970 to 2007”
• everymac.com
• semianalysis.com: “Inference Race To The Bottom – Make It Up On Volume?”
• Odyssey: Pepsi to Apple, Sculley J, Byrne J. (1987)
• The Power of Habit, Duhigg C. (2012)
• laptiomag.com: “Acer Swift Go 14 (Intel Core Ultra) review: Watch out, MacBook Air M2, there’s a new sheriff in town”.
• Bloomberg Terminal (2024)