---
title: "Study: LLMs can build their own coding harnesses, imperfectly"
url: https://www.parallelquant.com/posts/study-llms-can-build-their-own-coding-harnesses-imperfectly-6deaeb
source_name: "MarkTechPost"
source_url: https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/
published: 2026-09-11T22:01:08.000Z
topics: ["research", "llms"]
publisher: "Parallel Quant"
---

# Study: LLMs can build their own coding harnesses, imperfectly

*2026-09-11 · Source: [MarkTechPost](https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/)*

Researchers from ByteDance Seed, SUTD, Georgia Tech, M-A-P, and TokenWave.AI built HarnessDev, a benchmark that scores the runnable software harness a model constructs rather than just its final answer. Starting from a blank harness, six creator large language models (LLMs) built and evolved harnesses across 5 benchmarks and 2,207 tasks; the self-built harnesses matched human references on writing and machine-learning experimentation tasks but lagged on coding and search, and only 34 of 64 evolution changes generalized to held-out tasks.

**Why it matters:** This is a concrete data point in the broader push toward self-improving AI agents: it shows current models can bootstrap useful tooling for themselves in some domains, but that their self-modifications don't reliably transfer — a limiting factor for claims of open-ended agent self-improvement.

**Topics:** research, llms

---
Read the original: https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/
Canonical: https://www.parallelquant.com/posts/study-llms-can-build-their-own-coding-harnesses-imperfectly-6deaeb
Published by Parallel Quant — https://www.parallelquant.com