Channel RVA

RVA's independent media hub — news, video, podcasts, and community content for Richmond, Virginia.

Big Tech's $1.5 Trillion AI Spending Spree Created Small Language Models Now Undercutting It

By Will Melton · August 12, 2026 · 11 min read read

Category: AI

ChatGPT search for, "How do I deploy a small language model?"
People are leveraging large language models to deploy small language models to avoid dependence on frontier AI providers.

Big tech has spent more than $1.5 trillion on AI infrastructure since 2020. The research it published along the way produced small language models that run free on a laptop, and a practical path to using them.

Big tech has spent more than $1.5 trillion on AI infrastructure since 2020, and close to half of that arrived in 2026 alone.

The acceleration is the real story. Big tech spent about $226 billion on AI infrastructure in 2024, about $410 billion in 2025, and is on pace for somewhere between $700 billion and $800 billion in 2026, depending on which tracker and which companies get counted. One widely cited estimate puts 2026's roughly $725 billion above everything spent on AI infrastructure in the several years before it, combined.

Combined revenue across every frontier AI lab remains a fraction of that figure. Anthropic's run rate went from $87 million in January 2024 to roughly $30 billion in April 2026, one of the fastest revenue climbs on record for an enterprise software company, and the gap still has not closed. Sequoia's David Cahn put the annual shortfall between AI infrastructure spending and AI ecosystem revenue at roughly $600 billion in a widely circulated analysis. It has widened since.

That spending also produced something else: a fully documented, publicly funded blueprint for making frontier-grade AI cheap to reproduce at a fraction of the size.

How the Giants Taught Everyone to Build Small

A frontier model needs three things: an architecture that scales, a training corpus large enough to use it, and enough compute to run the training. The major labs solved all three, then published a great deal of how, because the people doing the work are researchers who publish for a living.

Transformers came out of Google, in a 2017 paper called Attention Is All You Need that now underpins essentially every model in this article. Chain-of-thought prompting, scaling laws, reinforcement learning from human feedback, mixture-of-experts routing, and modern quantization schemes moved through arXiv and NeurIPS the way academic research is supposed to move. Meta went further, releasing full model weights with Llama and turning a research literature into a working ecosystem within about 18 months.

The technique that matters most is distillation. A large model can generate enormous volumes of high-quality synthetic training data, and a much smaller model trained on that output inherits a surprising share of the parent's behavior. Microsoft's Phi series demonstrates this cleanly: heavily curated synthetic data, a few billion parameters, benchmark scores that outperform models several times its size. DeepSeek R1 made the same point in January 2025, reaching frontier-grade reasoning at a training cost that did not require a sovereign wealth fund.

By April 2026, DeepSeek V4-Pro scored 80.6% on SWE-bench Verified, two-tenths of a point off Claude Opus 4.6, under an MIT license, at roughly 10 to 13 times lower cost per output token. Qwen, Llama, GLM, Kimi, and Gemma have traced similar curves. Five independent open-weight model families reached the same ceiling in the same window. That degree of convergence is the signature of a shared technique, distillation from frontier output, spreading through the field faster than any single lab anticipated.

The corpora used to train frontier models raise a separate, litigated question: where the data came from. A federal judge gave final approval in July 2026 to Anthropic's $1.5 billion settlement with book authors, the largest copyright payout on record, after the underlying ruling drew a line between two fact patterns: training on lawfully acquired books was fair use, and training on a corpus the company knew was pirated carried liability regardless. More than 35 separate training-data lawsuits were active in major Western courts as of the second quarter of 2026, tracked in BakerHostetler's running AI litigation database. Hachette, Elsevier, and Cengage sued Google over Gemini the same month the Anthropic settlement cleared.

What a Small Language Model Actually Is

Small, here, means under about 10 billion parameters, small enough to run on a laptop or a single consumer GPU without a data center behind it. A 3-billion-parameter model was a curiosity as recently as 2024. In 2026, models at that scale handle production workloads: coding assistants, document question-answering, local agents, on hardware many developers already own.

Quantization does the heavy lifting. Model weights are stored at reduced precision, commonly a scheme called Q4, cutting the memory footprint by roughly a factor of four with a modest quality cost. A 4-billion-parameter model at Q4 needs about 3 GB of memory.

Nine Small Models Worth Knowing in 2026

Each entry lists the size, what it is good at, roughly how much video memory it needs at Q4, and the exact command to pull it once Ollama is installed.

For document-heavy vision work specifically, Qwen3-VL-8B is the strongest small option, scoring 96.1 on DocVQA in about 6 GB of memory.

One warning worth flagging: a model advertising 128K context through simple positional interpolation loses quality well before it reaches that length. Check how the context window was actually trained. The full catalog of what will run locally is at ollama.com/library.

What Hardware You Need to Run One

Eight gigabytes of system memory is the floor. A model under 4B parameters runs on CPU alone at 30 to 70 tokens per second, faster than most people read.

Sixteen gigabytes of unified memory on an Apple Silicon Mac, or an 8 GB GPU such as an RTX 3060 or 4060, runs everything in the list above comfortably except Phi-4.

Twenty-four gigabytes of VRAM, or a Mac with 32 to 64 GB, puts 27B-32B models and multi-model pipelines in reach.

Ollama is the fastest way in. Install it, run a single command such as ollama pull phi4-mini, and a working model answers prompts inside 10 minutes. LM Studio offers the same result with a graphical interface. llama.cpp is the engine underneath both, worth learning if you plan to embed a model in something you ship. All three are free.

Where Frontier Models Still Win

Frontier models remain meaningfully better at five kinds of work:

Which Work to Move to a Small Model Now

Six categories of work already run better locally, on cost, speed, and privacy grounds:

How to Fine-Tune a Small Model on Your Own Data

Full pretraining is out of reach for almost everyone and unnecessary for this. What works is fine-tuning, specifically a method called LoRA, or Low-Rank Adaptation, which freezes the base model and trains a small set of additional weights alongside it. A variant called QLoRA quantizes the base model first, dropping memory requirements far enough that an 8B fine-tune fits on a single consumer GPU.

One hybrid approach covers most real cases: use a frontier model to generate and label training examples, then distill that into a small model that runs on infrastructure someone else does not control. That is the same technique the labs used on themselves, aimed at a narrower problem.

Why the Scale-First Playbook May Not Hold This Time

The strategy behind the spending is a familiar one: spend the most, draw the most attention, raise the most money, repeat until the market has one dominant player. It worked for search, for social networking, for ride-hailing, for streaming video. Each of those bets assumed the spending itself was the moat.

Language modeling looks different. The expensive part of the process produces knowledge. Knowledge compresses. Compression is cheap once someone has shown how, and the labs showed how, in public, with citations attached. One major lab gave its weights away entirely as a move against its competitors.

Frontier labs still matter. They remain the source most distillation techniques pull from today, and someone has to build the first version of anything before it can be compressed. What has changed is the assumption sitting underneath a $1.5 trillion, six-year bet: that raw scale is, by itself, a durable advantage. A growing body of open-weight evidence runs against that assumption, produced largely by the same companies making the bet.

For everyone else, anyone not spending hundreds of billions a year on data centers, the practical upside is real and currently unclaimed. The models are free to download. The tools to run and fine-tune them are free. The hardware is very likely already on the desk.

Install Ollama and pull a model tonight. This guide will still be here tomorrow.