self-evolving harnesses are here. (100% open-source) today you pick...
Akshay 🚀@akshay_pachaar
52 views
Sep 04, 2026
~3 min read
Advertisement
1
self-evolving harnesses are here.
(100% open-source)
today you pick a fixed harness, and every task runs through it. a deep research question and a file-renaming task get the same memory strategy, the same planner, and the same tools exposed at every step.
JIT-Agent is a 27B open-source model that writes the harness instead. you hand it the task and the tool registry, and it emits four Python files plus a prompt config.
that happens once, before the task starts. then a second model, one you don't train or modify, does the actual work by being called step by step from inside those files. it never writes any of the code and never sees it.
each file covers one part of a fixed four-module contract:
→ memory decides what slice of the history that second model sees at the current step.
→ planning turns that view into a directive for the next action.
→ tool policy decides which tools are exposed for that directive.
→ action assembles the prompt from all of it, calls the model, and interprets the reply as either a tool call to run or a final answer.
because the code is written per task, the same generator produces structurally different agents.
here are a few tasks from the JIT-Agent paper, and the harness each one produced:
1) for a request that had to find contact records, build a workbook, and email it, it compiled the requirements into a dependency graph where delivery waits on artifact verification, and stored intermediate artifacts so later nodes consume finished results instead of rebuilding them from the transcript.
2) for a multi-hop identity question, a fixed graph would commit too early to one evidence path, so it synthesized a delegate tool that opens a private research subagent with its own memory and a five-step budget. the returned answer re-enters the parent loop as an ordinary observation.
3) for a task built around comparing numbers and ranking the results, it kept the arithmetic out of the model. the harness computes the numbers by running code, stores the results as fixed values, and hands those same values to every later step, so the model reads figures instead of recomputing them from text.
holding the working model fixed and changing only the harness, JIT-Agent's harnesses matched or beat hand-built runtimes on search and instruction benchmarks while spending well under half the tokens, and roughly a third less per case on average. writing the harness costs tokens too, and that cost sits outside those per-case numbers.
so harness design stops being one choice you commit to and becomes something written fresh for each task. JIT-Agent also keeps the harnesses that beat its current best in an archive, and pulls from it when a later task looks similar. its own weights never change while this happens.
this work is fully open-source.
repo: github.com/bingreeky/JIT
paper: arxiv.org/abs/2608.25593
i have written a detailed article on what a harness actually contains before a model starts generating one.
the article is quoted below.
(100% open-source)
today you pick a fixed harness, and every task runs through it. a deep research question and a file-renaming task get the same memory strategy, the same planner, and the same tools exposed at every step.
JIT-Agent is a 27B open-source model that writes the harness instead. you hand it the task and the tool registry, and it emits four Python files plus a prompt config.
that happens once, before the task starts. then a second model, one you don't train or modify, does the actual work by being called step by step from inside those files. it never writes any of the code and never sees it.
each file covers one part of a fixed four-module contract:
→ memory decides what slice of the history that second model sees at the current step.
→ planning turns that view into a directive for the next action.
→ tool policy decides which tools are exposed for that directive.
→ action assembles the prompt from all of it, calls the model, and interprets the reply as either a tool call to run or a final answer.
because the code is written per task, the same generator produces structurally different agents.
here are a few tasks from the JIT-Agent paper, and the harness each one produced:
1) for a request that had to find contact records, build a workbook, and email it, it compiled the requirements into a dependency graph where delivery waits on artifact verification, and stored intermediate artifacts so later nodes consume finished results instead of rebuilding them from the transcript.
2) for a multi-hop identity question, a fixed graph would commit too early to one evidence path, so it synthesized a delegate tool that opens a private research subagent with its own memory and a five-step budget. the returned answer re-enters the parent loop as an ordinary observation.
3) for a task built around comparing numbers and ranking the results, it kept the arithmetic out of the model. the harness computes the numbers by running code, stores the results as fixed values, and hands those same values to every later step, so the model reads figures instead of recomputing them from text.
holding the working model fixed and changing only the harness, JIT-Agent's harnesses matched or beat hand-built runtimes on search and instruction benchmarks while spending well under half the tokens, and roughly a third less per case on average. writing the harness costs tokens too, and that cost sits outside those per-case numbers.
so harness design stops being one choice you commit to and becomes something written fresh for each task. JIT-Agent also keeps the harnesses that beat its current best in an archive, and pulls from it when a later task looks similar. its own weights never change while this happens.
this work is fully open-source.
repo: github.com/bingreeky/JIT
paper: arxiv.org/abs/2608.25593
i have written a detailed article on what a harness actually contains before a model starts generating one.
the article is quoted below.