Reinforcement Fine-Tuning by OpenAI

OpenAI’s reinforcement fine-tuning guide describes a workflow in which a reasoning model generates an answer, a custom grader scores it, and the training process updates the model toward higher-scoring outputs. The current guide says that the reinforcement fine-tuning platform is winding down, is unavailable to new users, and lists o4-mini-2025-04-16 as the available reasoning model for existing access.

OpenAI introduced the method in this announcement video.

The training contract

OpenAI’s documented workflow has five steps:

1. Prepare training and validation data in JSONL.
2. Define a grader that turns a model response into a score.
3. Upload the files and start an RFT job.
4. Monitor training and validation rewards.
5. Compare the tuned model with a held-out evaluation set.

Each JSONL example contains messages for the conversation and any fields required by the selected grader. For example, a grader may read a reference field such as compliant or explanation. These fields supply grading information; they are not labels for the model’s hidden reasoning. The model’s generated response is what the grader evaluates.

The score range belongs to the grader. A binary string check can return 0 or 1, while a model-based or Python grader can return a different numeric range if that is how the grader is defined. A reward is therefore meaningful only alongside the grader, its inputs, and the evaluation set.

Example: a genetic-disease task

The announcement demo uses patient case reports containing present and absent symptoms and medical history. The task is to rank genes that may explain the case and provide a rationale. The demo shows these figures as accuracy on that genetic-disease benchmark:

These values describe the task, metric, and comparison shown in the announcement. They do not establish a general clinical capability, a diagnosis guarantee, or a general rule that a smaller model will outperform a larger one.

Reinforcement fine-tuning can optimize a model for the behavior measured by a reliable grader. The grader and held-out evaluation set define what the result means.