Small-Model Distillation — Part 3: DAgger-Style Expert-Correction SFT for a 0.8B SQL Agent

TL;DR

Before this experiment, the best 0.8B SQL agent came from top-k soft-label distillation and reached 55/220 on the fixed eval. That student was already much better than the base model, but it still failed in very agentic ways: wrong submissions, SQL errors, repeated actions, and bad recovery after its own tool calls.

This post tests a different question: what happens if the student generates the states, but GPT 5.5 medium takes over from selected difficult states and finishes the trajectory? This is DAgger-style expert correction SFT: the states come from the student, the teacher gives hard-token continuations, the harness verifies them, and the student trains with supervised fine-tuning. There are no teacher probabilities or logprobs here. Post 04’s token-level probability distillation is a separate method.

The trajectory of results tells the real story. The first correction model moved the top-k soft-label student from 55/220 to 58/220. Mixing verified wrong-submit repair rows with successful-retention rows pushed the best direct repair model to 64/220. Repeating that recipe from the 64/220 parent reached 63/220 and stopped, so the loop did not compound. The final hard-token result, 67/220, came from a retention-heavy anti-forgetting extension: I mixed successful traces from complementary earlier students on tasks the 64/220 parent missed with four copies of parent-success retention, trained with normal hard-token SFT, and selected the best checkpoint.

The useful lesson is not “student-state correction solved it.” It is more specific: hard-token correction helped the 0.8B agent recover from states it actually visits, retention anchors helped with forgetting, and a small amount of complementary successful behavior pushed the best hard-token checkpoint to 67/220, but SQL judgment and loop stability still dominate the failures.

What I Wanted To Test

The hard-token and soft-label experiments before this were both off-policy. The student learned from teacher-created trajectories. That is clean, but it has a limitation: during evaluation, the student does not always follow the teacher’s path. It makes its own mistakes, receives different SQL observations, repeats actions, reaches bad intermediate states, and then has to recover.

The question for this post is:

Can GPT 5.5 medium improve the best 0.8B SQL agent by taking over from the states that the student actually reaches during rollout?

The important constraint is that I want to use hard-token supervision only. I do not use a probability-exporting teacher in this post. GPT 5.5 medium is the correction teacher, and the training target is the canonical action text that the harness already expects.

The SQL-Agent Problem

The task is the same deterministic SQL tool-use harness used for the off-policy hard-token and top-k soft-label experiments.

The action interface is the same BAML layer as the earlier posts; I am not changing it in this post.

This matters because the model is not just writing a final answer; it is controlling a small loop. A model can improve at harness control while still submitting wrong SQL, so success rate and failure decomposition both matter.

Here is the kind of task the agent sees. The user has a bookstore database and a buggy query that tries to find orders where order_id jumps by at least 20% compared with the previous order. The original SQL uses a window function in the wrong direction and then filters in a way that returns too many rows. The agent may first inspect schema, then run a query, then receive an SQLite error such as misuse of aliased window function lg, and only then decide what corrected SQL to submit.

The canonical action shape is intentionally small:

{"draft": "short plan", "output": {"action": "inspect_schema"}}
{"draft": "short plan", "output": {"action": "run_sql_query", "sql": "SELECT ..."}}
{"draft": "short plan", "output": {"action": "submit_sql", "sql": ["SELECT ...;"]}}

The Training Idea

Student-state data collection means the training states come from the current student behavior, not from the teacher. In this experiment, the student acts first, and then GPT 5.5 medium gives hard-token corrections from the states the student actually visited.

This is close to the classic DAgger idea, and the same idea has started to show up in language-agent expert-correction work, where a rollout starts with the student and then switches to an expert model partway through the trajectory.

For this SQL agent, the difference looks like this:

Off-policy:
teacher history -> teacher next action
---
Student-state correction:
student history -> teacher continuation

A clean teacher trace may never contain the student’s weird bad state. But if the student runs a bad SQL query, receives an error, and starts repeating itself, we can ask GPT 5.5 medium to take over from exactly that context.

The verified continuation can then become several SFT rows:

student_bad_history -> GPT action 1
student_bad_history + GPT action 1 + observation -> GPT action 2
student_bad_history + GPT suffix -> GPT submit_sql

This is still supervised fine-tuning on hard tokens; the difference is where the input state comes from and how I verify the teacher output.

There are two methods people often blur together. The first is black-box expert correction: the student creates the state, a teacher writes a better action or suffix, the environment verifies it, and the student trains on those hard tokens. That is the method in this post. The second is token-level probability distillation, which needs teacher probabilities and a probability-matching trainer; that is the post 04 comparison.

Which Student-State Methods Work With Hard Tokens?

Yes, student-state data collection with hard-token teacher correction is possible.

The methods that fit this constraint are:

Method	Uses student rollouts?	Uses GPT 5.5 medium?	Uses probabilities?	Fit for this post
Teacher correction / DAgger-style expert-correction SFT	Yes	Yes	No	Best fit
Recovery expert correction	Yes	Yes	No	Best fit
Verifier-filtered teacher correction	Yes	Yes	No	Good fit
Best-of-N teacher selection	Yes	Yes	No	Possible, but more expensive
Hard-token self-training / self-distillation control	Yes	No	No	Later post
Token-level probability distillation	Yes	Needs probability-exporting teacher	Yes	Post 04 comparison

The main method should be teacher continuation from student-created states, similar in spirit to DAgger and recent expert-correction work for multi-turn LM agents. The student rolls out. I select difficult states. GPT 5.5 medium takes over in the same SQL environment. If the continuation passes the hidden train-task tests, the GPT suffix becomes hard-token SFT data for the 0.8B student.

Recovery distillation is a focused version of the same idea. Instead of labeling every state, we target failure states: repeated actions, SQL/tool errors, wrong-submit contexts, and near-max-turn states. This may be especially useful here because the best current student already knows the basic loop. The remaining problem is often recovery and decision quality.

Verifier-filtered teacher correction is the safety rail. I can ask GPT 5.5 medium for corrections, but I only keep rows that pass structural checks, fit the context limit, produce canonical actions that the harness can parse, and belong to a continuation that eventually passes the hidden train-task tests.

The methods that do not fit this post are probability-based methods and self-distillation; the probability version is the post 04 comparison.

A Small Educational Code Walkthrough

The canonical training row stays close to the earlier hard-token SFT rows. The model input is the full rendered conversation and environment history. The target is one GPT assistant action from a verified continuation.

row = {
    "messages": [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": task_prompt},
        {"role": "assistant", "content": student_action_1},
        {"role": "user", "content": environment_message(sqlite_observation_1)},
        {"role": "assistant", "content": student_action_2},
        {"role": "user", "content": environment_message(sqlite_observation_2)},
    ],
    "target": gpt_5_5_medium_action_from_verified_suffix,
    "source": "student_rollout_teacher_correction",
}

For the bookstore example above, one student prefix looked like this in simplified form:

user: "Why does my lag query return almost every order?"
assistant: inspect_schema
environment: tables include cust_order(...)
assistant: run_sql_query with the student's buggy window-function query
environment: {"ok": false, "error": "misuse of aliased window function lg"}

GPT 5.5 medium then continued from that exact state inside the same SQL environment. It first ran a corrected ascending-lag query, observed the result, and then submitted a final SQL statement using LAG(order_id) OVER (ORDER BY order_id ASC) and order_id >= previous_order_id * 1.2. Because that continuation passed the deterministic train-task scorer, the suffix became training data.

Prompt tokens and previous history tokens are masked. Only the GPT 5.5 medium action target is trained.

labels = input_ids.copy()
labels[:prompt_and_history_token_count] = -100
loss = cross_entropy(student_logits, labels)

This is why student-state hard-token correction is practical with GPT 5.5 medium: I only need the verified action text in the same canonical format the harness already parses.

The training data is still one action at a time. If GPT takes three turns to recover from a student state, that successful suffix becomes three SFT rows, not one giant target. Each row asks the student to imitate the next verified GPT action from the exact history available at that point.

How I Built The Correction Dataset

The data generation pipeline had eight stages:

1. Start from the best top-k soft-label distillation checkpoint.
2. Run it on the train split with the same harness, parser, tools, stop rules, and context budget.
3. Save full student trajectories, not just final submissions.
4. Bucket states by outcome: successful, wrong submit, repeated action, SQL/tool error, parse failure, and max-turn stop.
5. Ask GPT 5.5 medium to continue from selected student-created states.
6. Run the GPT suffix in the real SQL environment.
7. Keep only successful continuations whose final submit_sql passes the deterministic train-task tests.
8. Build a hard-token SFT dataset from the verified GPT suffix rows.

The key design decision was which states to label. Labeling every state is simple but expensive and includes many easy rows. Labeling only failure or near-failure states is cheaper and more targeted, but it may over-focus the training on recovery. For the first pass, I ran the best top-k soft-label student on the full 879-task train split and selected difficult states from that full rollout distribution.

If I kept only the first candidate from each failed task, wrong-submit states dominated because many failed episodes ended with a submitted-but-wrong SQL query. That was useful, but it could hide earlier SQL-error or repeated-action recovery states inside the same episode. For the full correction pass, I used an all-buckets policy: at most one wrong-submit state, one SQL/tool-error state, one repeated-action state, one parse-failure state, and one near-max-turn state per failed task. This cost more GPT calls, but it matched the research question better because I wanted to test recovery from several kinds of student-created bad states.

I also kept the teacher call simple: one selected state got one GPT 5.5 medium continuation attempt. There was no best-of-N search and no retry loop in this experiment. GPT could take multiple tool-use turns inside that one continuation, but if the final submit_sql did not pass the deterministic train-task scorer, that candidate was discarded.

The first full train rollout finished on all 879 train tasks. The warm-started student solved 229 of them before any student-state correction training. From the failed and difficult trajectories, I selected 807 candidate states: wrong submits, SQL/tool-error states, repeated-action states, parse-failure states, and near-max-turn states. GPT 5.5 medium solved 345 of those candidate states under the real train-task scorer, which created 560 SFT rows. The verified rows came from several buckets, not only wrong submits; the exact counts are in the Results section below.

The Method, Step by Step

Every experiment in this post is the same hard-token loop with a different data mix.

Algorithm: Student-state hard-token expert correction

Input: current student checkpoint, fixed 879-task train split, unchanged BAML SQL-agent harness, GPT 5.5 medium correction teacher.

Output: verified hard-token SFT rows and a LoRA adapter, scored on the fixed 220-task eval split.

1. Run the current student on all train tasks and save full trajectories.
2. Select difficult states from failed trajectories: wrong-submit, SQL/tool error, repeated-action, parse-failure, and near-max-turn.
3. Replay each state so the teacher sees exactly what the student reached.
4. Ask GPT 5.5 medium for one continuation, and let the harness execute every action.
5. Keep the continuation only if the final submit_sql passes the deterministic train-task scorer.
6. Turn each accepted teacher action into one hard-token SFT row and train a LoRA adapter from the current checkpoint.

No teacher probabilities or student-token logprobs are used, so this is DAgger-style SFT, not post 04’s probability distillation.

From that one loop, each step changes only the data mix or the parent checkpoint it starts from:

• First correction, 58/220. One correction pass from the 55/220 top-k soft-label student. It made the loop more decisive, with fewer repeated-action and max-turn stops, but it did not remove wrong submissions.
• Correction-only round 2, stalled. Rolling out the 58/220 model and correcting its new states produced more verified rows, but second-round-only training reached 56/220 and the round-1 plus round-2 aggregate only tied 58/220. More correction was not a clean upgrade.
• Repair plus retention, 64/220. Mixing verified wrong-submit repair rows with a 2:1 ratio of successful-retention traces was the first real jump. The retention rows are not rewards or preferences; they are successful hard-token traces that stop the update from forgetting what already worked.
• Iterating repair plus retention, stalled at 63/220. Repeating the 2:1 recipe from the 64/220 parent reached 63/220, so the gated chain stopped. The recipe did not compound on its own.
• Specialist-success anti-forgetting, 67/220. Instead of more corrections, I mixed successful traces from complementary earlier students on parent-missed tasks with four copies of parent-success retention, and checkpoint selection took the best hard-token result to 67/220.

Training Setup

The first correction run starts from the best top-k soft-label checkpoint, not the base model, because the initial question is whether student-state hard-token correction can improve the best student I already have. Later ablations start from the first hard-token correction model. The iterative repair test starts from the 64/220 repair-plus-retention parent, and the final specialist-success anti-forgetting run also starts from that 64/220 parent.

The data split is unchanged: 879 train tasks and 220 held-out eval tasks from the same birdsql/six-gym-sqlite benchmark. For the rollout collection step, the student runs with the same agent harness and the rollout budget in the table below; that larger budget lets the agent carry longer histories at inference time. The first correction run used a 4096-token SFT filtering budget for the rendered training sequence because that matched the earlier LoRA recipe. Later retention runs either kept that filter or explicitly moved the SFT limit to 8192, which I call out where it matters.

Here is the core setup in one place:

Setting	Value
Student	unsloth/Qwen3.5-0.8B
Initialization	Best top-k soft-label checkpoint for the first correction run; hard-token correction adapters for later ablations; 64/220 repair-plus-retention parent for the final anti-forgetting run
Teacher	GPT 5.5 medium
Train / eval split	879 train tasks / 220 held-out eval tasks
Rollout and eval budget	8192 input/history tokens, 512 generated tokens per action
Max tool-use turns	8
Initial correction SFT filter	4096 rendered prompt-plus-target tokens
Later retention SFT filter	4096 or 8192, called out per ablation
Training backend	CUDA LoRA SFT, bf16 when supported, no 4-bit loading
LoRA modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
LoRA rank / alpha	32 / 32
Batch / grad accumulation	1 / 8
Initial correction learning rate	5e-5
Retention repair learning rates	2e-5 and 1e-5
Validation split	5%
Optimizer/scheduler	Trainer defaults, held fixed across comparable SFT runs

The training recipe stayed close to the hard-token and soft-label runs before it (exact values in the table above). The first correction dataset had 560 source rows. Under the 4096-token training budget, 508 rows were kept and 52 long rows were dropped; the final split was 483 train rows and 25 validation rows. The rendered training sequences had p50 2881 tokens, p90 3997, p95 4244, and max 7125 before filtering. Training ran for 183 optimizer steps, which was three epochs over the kept train rows. It took 5941 seconds on the RTX 3090, reached train loss 0.2835, and ended with validation loss 0.4151.

For the later hard-token SFT ablations, I kept the same model family, harness, action schema, eval split, and assistant-token-only SFT contract. The runs are not perfectly single-variable experiments, because some later tests intentionally change the data mix and the continuation setup together. Still, each run has one clear question: second-round correction data, aggregate correction data, wrong-submit repair rows, successful-retention rows, retention ratio, update budget, learning rate, adapter capacity, or one more gated repair-plus-retention iteration.

The retention repair family is still part of the student-state hard-token correction method family. The repair rows are verified hard-token GPT 5.5 medium continuations from student-created wrong-submit states, and the retention rows are the successful-trace anchors described in the progression above. For example, the 64/220 2:1 successful-retention run trained from the first correction adapter on 878 source rows, kept all 878 under an 8192-token SFT limit, with learning rate 2e-5 and one epoch of TRL SFTTrainer.

The final 67/220 checkpoint used the same hard-token SFT contract, but a different data mix. It started from the 64/220 parent and trained on 1260 rows: 1008 parent-success retention rows plus 252 specialist-success rows from earlier complementary students. The SFT sequence limit was 8192, no rows were dropped, the split was 1197 train rows and 63 validation rows, LoRA stayed rank 32 / alpha 32, batch size stayed 1 with gradient accumulation 8, and the learning rate was 5e-6 for one epoch. The final adapter scored 65/220, while checkpoint-100 scored 67/220 and is the checkpoint I treat as the best hard-token correction-family model.

Results As Research Questions

Did hard-token correction beat the top-k soft-label student?

Yes. The top-k soft-label student scored 55/220. After one round of GPT 5.5 medium hard-token correction from student-created states, the same unsloth/Qwen3.5-0.8B student scored 58/220. That first result was only a 3-task gain, so I would not oversell it.

The direct repair-plus-retention result is stronger: wrong-submit repair plus a 2:1 successful-retention mix reached 64/220 on the same fixed 220-task eval. That is a 9-task gain over the top-k soft-label starting point and a 6-task gain over the first correction model. The final anti-forgetting extension pushed the best checkpoint in this hard-token family to 67/220 by adding complementary specialist-success rows with heavy parent retention. It is still not a breakthrough, but it changed my read of the experiment: more correction alone was not enough, while correction plus explicit retention anchors was useful, and complementary successful trajectories could add a few more solved tasks when the parent was strongly anchored.

The important comparison is not against the base model anymore. This experiment starts from the best top-k soft-label checkpoint, so the honest question is whether hard-token correction training improves that warm-started student. The cost is also clear: we had to run students over the full train split, verify candidate trajectories in the environment, train SFT adapters, evaluate checkpoints on the fixed 220-task split, and keep the method labels strict.

What changed inside the loop?

The failure decomposition is more interesting than the headline score. The first hard-token correction model submitted 180 times, up from 171 in the top-k soft-label checkpoint. Repeated-action stops went down from 39 to 33. Max-turn stops went from 3 to 0. Average turns also dropped from 3.39 to 3.20. That is the part of the first result I like: the correction data seems to teach the student to move through the loop more decisively.

The negative side is that SQL/tool errors went up from 12 to 27 in that first correction run. So the model did not simply become better everywhere. It became more willing to act and submit, and some of that turned into solved tasks, but some of it turned into bad SQL.

The 64/220 retention model improved that balance but did not make it clean. It submitted 179 times, had 18 SQL/tool errors, 4 parse failures, 37 repeated-action failures, and no max-turn or runtime failures. Compared with the first correction model, it solved more tasks and made fewer SQL/tool and parse mistakes. Compared with the 61/220 1:1 successful-retention run, it solved three more tasks but had more repeated-action failures.

The 67/220 specialist-success checkpoint had a different shape again. It submitted 174 times, had 8 SQL/tool errors, 5 parse failures, 40 repeated-action failures, one max-turn failure, and no runtime failures. So the score improved, and SQL/tool errors dropped a lot compared with the 64/220 parent. But repeated-action failures rose slightly from 37 to 40. This is the basic tradeoff: better final score does not mean every loop category improved.

Which correction states became training data?

The correction dataset came mostly from wrong-submit states because many failed trajectories eventually submit an incorrect query. GPT solved 193 wrong-submit candidates and those produced 279 SFT rows. SQL-error states were also useful: 76 verified continuations produced 139 rows. Repeated-action states gave 62 verified continuations and 118 rows. Max-turn states were much smaller: 9 verified continuations and 18 rows. Parse-failure states were tiny, with 5 verified continuations and 6 rows.

This does not prove that each bucket caused a marginal eval gain. It only says which buckets produced verified rows that entered the first correction run. Still, the distribution matters because the training signal was not balanced by failure type. The first correction adapter mostly learned from wrong-submit and SQL-error recovery, with some repeated-action correction. The later retention experiments did not change the benchmark or scorer; they changed the hard-token SFT mix to test whether repair rows needed successful behavior anchored beside them.

Was one correction round enough?

One round helped, but it was not enough to change the character of the model. The score moved from 55 to 58, and the loop behavior improved, but wrong submissions still dominated the failures. So I tried the natural next question: if the corrected student rolls out again, can GPT correct the new states and keep improving it?

After the first correction model reached 58/220, I ran it over the full 879-task train split again. It solved 254/879 train tasks and produced 619 candidate failure states: 481 wrong-submit states, 131 repeated-action states, and 7 parse-failure states. GPT 5.5 medium verified 241 of those states and produced 391 second-round SFT rows. Training on only those rows reached 56/220. Aggregating round 1 and round 2 tied the first correction model at 58/220, but did not beat it. Starting again from the top-k soft-label checkpoint with the same aggregate data fell to 47/220. Extra correction rows were real, but they were not a clean upgrade signal by themselves.

The next useful turn was to treat the problem as forgetting repair, not just more correction. Wrong-submit repair alone and targeted replay of forgotten tasks both reached 54/220. But mixing verified wrong-submit repair rows with successful-retention rows from the first correction model finally moved the held-out result: 1:1 successful retention reached 61/220, and a 2:1 successful-retention mix reached 64/220.

This belongs in the hard-token correction family because the training objective is still SFT on hard targets.

The retention curve was not monotonic. Half of the 1:1 retention update reached 59/220. A larger effective batch on the same 1:1 data fell to 54/220. Lowering the learning rate on the 1:1 data tied the first correction model at 58/220. A 1.5:1 retention mix fell to 52/220, while 2:1 retention became the best direct repair-plus-retention run at 64/220. A stepwise learning-rate continuation from the half-budget 1:1 checkpoint reached 60/220: clean repeated-action behavior, but worse SQL/tool and parse behavior than the best run. Merging the 64/220 adapter into the base and training a fresh rank-64 LoRA on the same 2:1 retention data reached 63/220, close but not better. Repeating the 2:1 repair-plus-retention loop from the 64/220 parent also reached 63/220, so the gated chain stopped after one iteration.

The specialist-success anti-forgetting run was the final step in this story, not a separate method bolted on later. It did not ask GPT 5.5 medium for another repair batch. Instead, it mixed successful train trajectories from complementary earlier students on parent-missed tasks with four copies of parent-success retention (the full 1260-row mix and 5e-6 learning rate are in Training Setup). Checkpoint selection did the rest: the final adapter scored 65/220, but checkpoint-100 reached 67/220, the best hard-token correction-family checkpoint before post 04’s probability-distillation experiments.

The earlier ablations still matter as negative evidence. Broad replay and non-submit replay hurt. Critical-state filtering made the model too eager to submit and reached 50/220. Earlier teacher intervention was mechanically clean: GPT took over after the first environment observation, verified 111 of 300 candidates, and produced 178 SFT rows. But the trained model became fast and confidently wrong, reaching 47/220. Adapter soups sometimes reduced one failure mode while creating another. The broader pattern is not “more rows always help”; it is “the data mix and update path decide whether correction becomes learning or forgetting.”

There was one useful extra-compute clue from the first correction checkpoint. The top-k soft-label checkpoint and the first hard-token correction model solved different tasks: 38 overlap, 17 solved only by the soft-label checkpoint, and 20 solved only by the first correction model, an oracle union of 75/220. A simple label-free selector over those two rollouts, using only trace signals like submitted-vs-not, SQL/tool errors, repeated-action stop, parse failure, max-turn stop, and turn count, reached 63/220. But that did not scale cleanly. Across broader sets of weaker runs, the oracle union rose as high as 97/220, while the same simple selector fell back to 55-56/220. So I treat selector/oracle analysis as a research clue, not as a headline model result.

The 64/220 and 67/220 checkpoints also show why the final gain is not just three extra tasks. They solve overlapping but not identical eval tasks: 53 tasks are solved by both, 11 are solved only by the 64/220 parent, 14 are solved only by the 67/220 checkpoint, and 142 are solved by neither. The oracle union is 78/220. I do not use eval complementarity to build training rows, but it is useful for interpretation: the anti-forgetting update added more new wins than it forgot, even though it still moved the agent into a slightly different failure shape.

My read is that the first student-state correction round fixed an obvious behavior problem: the student was getting stuck or repeating itself too often. The second round was harder. It mostly saw wrong-submit states from a model that already had better loop control. Imitating those rescues alone did not make the held-out rollout better. The later retention results changed the answer: one correction round was not enough, but the useful second step was repair plus retention, not correction-only SFT.

Key numbers behind the charts

Model / stage	Success	Submitted	SQL/tool errors	Repeated-action stops	Runtime errors
Best top-k soft-label student	55/220	171	12	39	0
First student-state hard-token correction	58/220	180	27	33	0
Round-2 correction rows only	56/220	174	17	38	0
Aggregate first- and second-round correction rows	58/220	175	25	37	1
Wrong-submit repair only	54/220	179	26	35	0
Wrong-submit repair + 1:1 successful-retention mix	61/220	180	17	30	0
Stepwise learning-rate continuation from half-budget 1:1 retention	60/220	183	24	28	0
Wrong-submit repair + 2:1 successful-retention mix	64/220	179	18	37	0
Merge best into base, fresh rank-64 LoRA on 2:1 retention	63/220	184	23	31	0
Iterative wrong-submit repair + 2:1 retention from 64/220 parent	63/220	179	14	36	0
Specialist success + 4:1 parent-success retention, checkpoint-100	67/220	174	8	40	0
Targeted replay variants	54/220	185-187	28-36	27-31	0
Early intervention	47/220	212	62	1	0

Failure Analysis

The main failure after hard-token correction is still wrong submitted SQL. The final 67/220 checkpoint submitted 174 times and solved 67, which means 107 submitted attempts still failed. That is better than the first correction model’s 122 unsuccessful submissions and the 64/220 parent’s 115 unsuccessful submissions, but it is still the dominant failure mode. The student learned to control parts of the tool loop better, and retention repair preserved more useful behavior, but it still often ended with the wrong final query.

Repeated-action behavior is mixed: it improved from 39 down to 30 across the first correction and 1:1 retention runs, then rose back to 37 for the 64/220 parent and 40 for the final 67/220 checkpoint. That is why I do not read the best score as a clean loop-control fix. It is the best held-out score, not the cleanest trajectory shape.

SQL/tool errors tell the other side: up to 27 after the first correction, back down to 8 by the final checkpoint. My interpretation is that the first correction model became more action-oriented, but not always more careful. Retention and specialist-success consolidation reduced SQL/tool errors, but they did not solve SQL judgment. This is why I do not want to judge agent training only by loss, submission rate, or one failure category. The rollout decomposition tells the real story.

The later ablations made that clearer. The failure was not simply “we need more corrected rows.” Correction-only rows, aggregate rows, replay rows, retention-ratio changes, earlier intervention, learning-rate changes, higher adapter capacity, adapter mixing, and one more gated repair-plus-retention iteration all changed the rollout distribution. Some variants lowered one failure category while making another worse. Early intervention almost eliminated repeated-action stops, but it made the model submit quickly and wrongly. The rank-64 continuation reduced repeated-action failures compared with the 64/220 model, but it made more SQL/tool errors and lost one solved task overall. The extra gated iteration reduced SQL/tool errors to 14, but it still lost one solved task and kept repeated-action failures high at 36. The broad ensemble analysis showed that failed runs contain some unique wins, but visible trace signals were not reliable enough to select those wins across many runs. That is the uncomfortable pattern: the student has pockets of recoverable behavior, but naive extra training moves the agent into new bad states instead of cleanly adding capability.

Hardware And Infrastructure Lessons

This post looked like it should keep paid GPU time low, but the full workflow still needed the GPU in three places: rolling out the current student on all 879 train tasks, training the corrected student, and running the full 220-task eval. The expensive teacher-correction phase was mostly API-bound because GPT 5.5 medium was the teacher. Since there was no probability-exporting teacher in this post, the Mac did not need to load a large local teacher just to prepare soft labels.

The practical workflow was split: use the GPU for local-model rollout and training work, use GPT 5.5 medium to generate verified correction suffixes, then return to the GPU for SFT and evaluation. The train rollout was the slow first stage because every task is an agent episode, not a single forward pass. The full 220 eval was also slow because each eval item can include multiple model/tool turns.

The main infrastructure lesson is that agent experiments are not priced like plain SFT. Rollout and eval can dominate because each task may require several model calls plus SQLite execution. Student-state correction adds another layer: you first pay for the student to create states, then pay the teacher to recover from some of those states, then pay again to train and evaluate the new student.

What I Learned

First, student-state hard-token correction can help, but the first gain was modest. Moving from 55/220 to 58/220 matters because the student was already warm-started from the best top-k soft-label model, but it is not a breakthrough. The cleanest initial improvement was loop behavior: fewer repeated-action stops, no max-turn stops, and slightly fewer turns on average.

Second, verifier filtering is the difference between useful correction data and wishful thinking. GPT 5.5 medium solved 345 out of 807 selected states, which means more than half of the attempted corrections were discarded. I like that. It means the dataset is not “GPT said something plausible.” It is “GPT continued from this student-created state and passed the deterministic train-task scorer.”

Third, retention mattered more than correction-only repetition: repair alone stayed at 54/220, but adding successful-retention rows moved it to 61/220 and then 64/220, and repeating the recipe from the stronger parent did not compound. That makes the method lesson sharper: the student needs corrected actions from bad states, but it also needs enough successful behavior anchored so the update does not forget what already worked.

Fourth, the final improvement came from retention-heavy specialist success, not another correction batch. That keeps it part of the same hard-token SFT family, but it changes the emphasis: after correction teaches recovery, anti-forgetting consolidation decides whether the new behavior survives.

Fifth, better recovery can expose weaker reasoning. The first corrected model submitted more often and repeated less, but it also produced more SQL/tool errors. The follow-up experiments made the same point from different angles: more rows, gentler learning rates, earlier teacher intervention, higher adapter capacity, and adapter mixing did not cleanly add capability. For this SQL agent, preserving loop control while improving final SQL judgment is the hard part.

Conclusion

This experiment answered the narrow question I cared about. Yes, DAgger-style hard-token expert correction from student-created states can improve the best 0.8B SQL agent from the top-k soft-label experiment. The first improvement was small: 55/220 to 58/220. The stronger result came from the same method family plus explicit forgetting repair: wrong-submit repair with a 2:1 successful-retention mix reached 64/220 on the fixed 220-task eval.

The final best result in this post is 67/220, from the retention-heavy specialist-success mix described in Training Setup. The final adapter itself scored 65/220, but checkpoint-100 reached 67/220, so checkpoint selection mattered more than validation loss alone.

Correction-only training did not keep improving the model: second-round rows, aggregate rows, and repair-only rows all stayed at or below the first correction’s 58/220. What worked was correction plus retention, then retention-heavy consolidation of complementary successful behavior.

Off-policy distillation gave the student a strong starting behavior. Soft-label distillation improved it further. DAgger-style expert correction then added recovery ability from states the student actually visits, and anti-forgetting SFT helped preserve behavior that correction-only SFT was prone to losing. Post 04’s token-level probability distillation is a different method, so I report it separately from this hard-token correction family.