• CO/AI
  • Posts
  • AI Models Stop Improving? 🟢

AI Models Stop Improving? 🟢

Google's Gemini and OpenAI's latest releases mask a troubling trend: big AI models are struggling to improve. Industry data reveals plateauing performance while smaller, efficient models gain traction among businesses.

Today in AI

In partnership with

NEW LAUNCHES

The latest features, products & partnerships in AI

AI AGENTS

Launches, research & more from the AI Agent Report Newsletter

GOVERNMENT

Press releases, regulation, defense & politics

IMPLEMENTATION

Announcements, strategies & case studies

OTHER INTERESTING STORIES

Compelling stories beyond the usual categories

What’s happening in AI right now

AI models hit performance wall

Big tech continues touting AI breakthroughs, but troubling signs of model performance plateaus are emerging. While Google celebrates Gemini 1.5's million-token context window and OpenAI teases GPT-5, users report degraded outputs from current models and experts warn of fundamental limitations ahead.

Signs of stagnation

OpenAI's newest models show only marginal improvements over previous versions - a stark contrast to the dramatic leaps between GPT-3 and GPT-4. Industry data suggests this isn't an isolated case. Major AI platforms are showing increasingly erratic and inaccurate responses, challenging the narrative of inevitable progress through scaling.

The data dilemma

The slowdown stems largely from an emerging crisis in training data. Experts project that by 2026, AI companies will have exhausted most available high-quality public text data. More worryingly, researchers have identified "model collapse" - where systems trained on AI-generated content show degraded performance over time. With OpenAI alone generating approximately 100 billion words per day, the risk of models training on their own increasingly degraded outputs grows.

Big promises, bigger challenges

Yet despite these mounting technical challenges, tech executives continue making aggressive predictions. Industry leaders now forecast artificial general intelligence arriving between 2026-2030, driving unprecedented investment in AI infrastructure. Microsoft alone is spending billions on specialized AI datacenters, while Google and Meta race to match capabilities.

Alternative approaches

The disconnect between bold promises and technical reality has sparked new approaches. Research teams are exploring mathematical data - leveraging patterns and relationships that could provide theoretically infinite training material. Unlike text or images, mathematical structures don't suffer from quality degradation or exhaust their novelty.

The enterprise reality

Meanwhile, enterprises are quietly pivoting to more practical solutions. New data shows 77% of enterprise AI implementations now use models with fewer than 13 billion parameters. These smaller models often match their larger counterparts in specific tasks while offering dramatically lower latency and operating costs.

Small models, big impact

This enterprise shift coincides with growing interest in Small Language Models (SLMs). Rather than chasing raw size, SLMs prioritize efficiency and can run locally on devices without requiring constant internet connectivity or massive computing resources. Companies like IBM and AMD are investing heavily in this approach, betting that practical advantages will outweigh marginal improvements in large model capabilities.

The training data race

The scramble for high-quality training data has meanwhile sparked new competition. Research indicates that AI companies are increasingly paying premium prices for pristine datasets, recognizing that model performance depends heavily on training data quality. Meta reportedly considered acquiring Simon & Schuster to use its publications for training large language models, as revealed in an audio recording shared with The New York Times. Some firms are exploring synthetic data generation, though this approach risks accelerating model collapse if not carefully managed.

As tech giants push forward with massive investments and ambitious timelines, the underlying technical challenges paint a more complex picture. The coming months may reveal whether bigger truly means better in AI development, or if the future belongs to more focused, efficient approaches.

We publish daily research, playbooks, and deep industry data breakdowns. Learn More Here

One of the holy grails of unlocking the potential of AI is to build systems that can reason like humans. By improving AI's, Large Language Models in particular, ability to break down complex problems and apply logical steps.

Bagel’s research team has been exploring this problem. Analyzing LLM building techniques, especially fine-tuning techniques, to allow Large Language Models to evolve from pattern-recognizing prediction agents to true cognitive agents

AI generated art

A look at the art and projects being created with AI.

Midjourney Prompt: A yellow and pink and green Ginkgo trees , with a clear blue sky and lake reflection, in a minimalist style. The composition features a large area of blank space, with a simple layout captured using a wide-angle lens. The delicate texture and cool tones create a static, tranquil atmosphere

How'd you like today's issue?

Have any feedback to help us improve? We'd love to hear it!

Login or Subscribe to participate in polls.

Reply

or to participate.