Outspeaking.com

Programming Languages in the LLM Era: The Feedback Loop

Will LLMs favor old programming languages? Yes, for the first draft. A model writes better code in languages with large, old corpora, and that is a real handicap for low-resource languages and new ones.

Don't mistake an LLM's first draft for the goal. A language has the best shot when it makes the red/green/refactor loop cheap for an AI coding assistant. The further left in the process the checks sit, the better. ("Shift left" means catching a defect at the earliest, cheapest point instead of at human review or after deployment.) A new language cannot win the training-data race, so its best shot is probably the feedback race.

This builds on Theme, Pragmatics, and Purpose in Programming Language Implementation and Why JavaScript and PHP Won the Web: languages win by being good enough at a specific job, and network effects matter more than technical quality. For the economics of picking a language, see The Lemon Market of Programming Language Adoption. The closing section of the Theme article covers what success looks like in the LLM era. This article argues that part of being good enough at the job is now how cheaply a model can check its own work.

Why Do LLMs Write Better Python Than New Languages?

A model learns from what people have already written. That sounds neutral, but it is not. Three effects come from the corpus of training data that gets slurped up when creating the LLM. A fourth comes from how models are trained.

Gravity: more code, better output

More public code means more code in the training set, which means better generated code, which attracts more users, who write more public code. A survey of 111 papers on low-resource and domain-specific languages reports consistently higher scores for Python and Java than for R and Julia, citing a September 2024 leaderboard, so check current numbers. Giagnorio and colleagues studied R and Racket and found no single technique that reliably fixes the gap: fine-tuning helped smaller models, in-context learning helped larger ones, and fine-tuning could even hurt the largest model they tried.

Staleness: popular is not the same as good

The Perl community lived a version of this problem for years. A well-meaning programmer released several web tools collectively known as Matt's Script Archive in the early years, and search engines recommended them for searches such as "web page guestbook" or "form mailer". Even though the quality, security, and utility of these scripts was frozen in time and Perl moved on to better things, the inertia of the existing links, and the later links that spread their authority, kept the perception of Perl low. The community's answer was the NMS project, a set of CGI scripts offered as a replacement for Matt's Script Archive. Better guidance existed too, in chromatic's Modern Perl, Ovid's Beginning Perl, and the perlfaq. The old material still dominated the naive search. (Why Perl Didn't Win has more on why Perl lost ground.)

I cannot show that corpus weighting works like search ranking. I know of no study that measures it as of November 2026. Yet if popular, stale material outranked better material on the web, a model trained on that web may reproduce the weighting. Treat this as a hypothesis, not a finding.

Version blindness: the newest facts are the thinnest

A model's knowledge stops at its cutoff, so the newest facts about a language are the ones it knows least. For example, in September 2026 I asked an LLM to help upgrade a Go project for security. It stopped at the Go release from February 2026, the latest one in its training data, not the current release. A model that can search the web softens the problem, but only if it knows when to search. For security updates, "probably the latest version I remember" is the wrong answer.

Generation bias: a training incentive, not a corpus effect

Models have read the refactoring canon: Martin Fowler, the Pragmatic Programmers, SOLID, Sandi Metz. They know the rule of three, which as I read Fowler's Refactoring he credits to Don Roberts: do a thing once, put up with the duplication the second time, and refactor the third time. Knowing the rule is not following it. A model trained to be helpful is rewarded for answering the request in front of it, which favors producing new code over consolidating what exists.

I have not seen this measured, but in my experience a follow-up prompt to detect and correct exact and near code duplication is a fruitful effort, especially with an explicit goal such as "reduce lines of code", because the goal then sits right next to the request.

On the gripping hand, a naive version of that prompt once made the LLM strip linebreaks from test documents embedded in the source code, because it read "fewer lines of code" as "fewer newline characters in the commit". Models take the metric literally.

I have repeatedly generated complex SQL for data analysis, run it, and asked the model to "make this faster". The first draft is correct but ignores real-world time constraints. Adding "speed" or "responsiveness" as a human-directed metric to the prompt drives the model to balance for "a query that finishes within 30 seconds, not 30 hours". Only a check, here a stopwatch, exposed the problem.

What Does a New Language Need to Compete?

A new language cannot out-publish Python in a span of weeks or months. It has two different ways to work around the corpus problem:

  1. Fit in the context window. If the rules are small enough to hand to the model in a prompt, the model needs less prior exposure, though the research above shows that in-context help is uneven.
  2. Look like something the model knows. Starlark, the configuration language for Bazel, is a small, deterministic dialect of Python. Its spec says it has no user-defined types, no inheritance, no reflection, and no exceptions. In my experience models write it well, especially if they can find extant examples in your codebase.

Is an LLM's First Draft Good Enough?

In my experience, a durable technical task deserves a red/green/refactor cycle to have a hope of meeting quality, time, and scope: write a failing check, make it pass, then clean up (What Is Programming? makes the case for small, testable chunks). Left to a bare prompt, an LLM is very good at the middle step and tends to skip the other two.

Harnesses will run tests when asked, but the check has to exist and be cheap to run. Developers already know not to take the output on faith. In the 2026 Stack Overflow Developer Survey, 80% of respondents used AI at least an hour a day. The largest group, 48%, said they trust the output when they can easily verify it, and only 6.6% trust it for important work decisions. Trust follows verification, which is a good reason to automate the checks and make them cheap.

The language and its tooling are what make the loop cheap or expensive.

What Are the Four Feedback Layers?

LayerWhat it isSpeed and cost
1. The languageType checker, language server, linterSeconds, often as you type, before anything executes.
2. ToolingA test runner and a buildSeconds to minutes, depending on the suite. Cheap enough to run after every change.
3. Harness loopsAn AI coding agent feeds the output of layers 1 and 2 back to the model automatically, such as language server diagnostics or failing testsAs fast as the checks it wraps, with no human relaying the failure.
4. Manual verificationA person reads, runs, and judgesSlowest and most expensive. Use it last.

The earlier you catch a defect, the cheaper it is to fix, and the sooner the model gets the feedback it needs to correct itself. A language with a good type system, a fast compiler, and a built-in test runner gives the model cheap feedback at layers 1 and 2 by default, and makes layer 3 possible. A language with none of them leaves both you and the model to the slowest layer.

Layer 3 is a delivery method. A language server finds the error (layer 1), and the harness is what carries its diagnostics back to the model, so the session corrects itself in seconds without a person pasting anything. A check that lives in layer 1 only helps the model once something delivers it.

That sets up a tension. The feedback argument favors languages with strong static checks, while the corpus argument favors Python. TypeScript and Java combine large corpora with static checking, which is part of why they are hard to dislodge. The ML family (Standard ML, OCaml) and Haskell sit at the other corner: strong type systems that catch whole classes of wrong code before anything runs, but far smaller corpora, so the model has the checker's help and little else to go on. Compile times matter here too: GHC is slow compared to Go's compiler, though its interpreter (ghci) gives Haskell a quick reload-and-test loop.

Go and Rust show the trade-off from the other side. Go has a simple grammar and simple rules, which makes it easy to fit in a prompt, and its compiler is fast, so the loop stays tight. It is verbose even after you refactor, though, so there is more code to generate and verify. Rust gives strong compiler feedback, but the language is complex, and complexity costs both prompt space and correctness. Its compiler is also slower than Go's. In my experience both languages have language servers that work well, which keeps the layer 1 feedback fast.

My judgment, not a measurement: a language with a big corpus but weak checks, or strong checks but a small corpus, pays for the missing half somewhere. A cheap feedback loop is how a small-corpus language, such as OCaml or Haskell, can make up the difference. Adding mypy or another type checker as a linting step in Python from the start seems to help cement thinking in types and catching errors the model can fix on its own.

What Does This Mean for a Team Using LLM Tools?

Enforce checks as far left as you can, so the model gets its feedback quickly and cheaply:

  • Turn on the language's own checks first: types, a language server, and a linter, wired into the editor and into the agent's loop.
  • Keep the test suite fast enough that the model can run it after every change.
  • Automate the loop so failures go back to the model without a person copying them.
  • Save human review for what the cheaper layers cannot catch. When a defect does reach a person, add a check at an earlier layer so it cannot reach them again.

This also counters generation bias. A linter or test that enforces the refactoring step does not depend on the model's good intentions. The language can do some of this too. Go makes errors ordinary return values, and a linter such as errcheck flags the ones you ignore. Rust warns about an unused Result and rejects a match that misses a case. OCaml warns about missing pattern cases by default, and GHC does with -Wall. JavaScript, Python, and Perl leave more of that to opt-in tooling or convention.

What Do I Not Know?

  • Whether stale, popular material outweighs better, newer material in a training corpus. No study I know of measures it.
  • Whether a new language can close the low-resource gap entirely. The research shows the gap and shows partial fixes.
  • Whether the feedback loop decides which language wins more than network effects do. I know of no study which compares the two.

Book links on this page are Amazon affiliate links.

FAQs

Will LLMs kill new programming languages?

They make the cold start harder, because models write better code in languages they saw more of. A new language may be able to partly offset that with rules small enough to fit in a prompt, a familiar syntax, and fast feedback from types, linters, language servers, and tests.

What makes a programming language good for AI coding?

A large corpus helps the first draft. After that, the language needs a cheap red/green/refactor loop: types and a language server, a linter, a fast build, and a built-in test runner.

Which programming language is best for LLM coding?

There is no single winner. Languages with big corpora, such as Python, tend to give the best first drafts, and languages with strong compiler and type feedback, such as OCaml and Haskell, tend to give strong compiler feedback for correcting it. Java and TypeScript combine both strengths, which is part of why they are hard to dislodge.

Why do LLMs often generate more code instead of reusing existing code?

My hypothesis is that models are rewarded for answering the request in front of them, and new code does that more directly than refactoring. A linter or test can enforce the refactor step.

What does shifting left mean for LLM-assisted programming?

It means catching defects at the earliest, cheapest layer: language and editor first, then tests, then automated harness loops, and human review last.