Skip to content

letter-counting-legacy-rl-v1-0-1

View Markdown

Dependency patch revision of legacy letter-counting RL protocol: first 64 generated tasks train, next 64 held out.

Field Value
Reference nodus/[email protected]
Image nodus/env-letter-counting-legacy-rl:1.0.1
Publisher Open-Thought
Category Reasoning
Readiness Research
Modes Train, Evaluate
Reward Scalar
Held-out measures TrainedTask
Splits 64 train, 64 test (disjoint by canonical identity)
Licenses code Apache-2.0, data Apache-2.0
Source https://github.com/open-thought/reasoning-gym

Completions are graded by the platform, never by the trainer: the trainer submits {taskId, completion} batches and the verdicts come back as task events.

Grader Kind What it checks
score-answer Program Raw completion scored by pinned Reasoning Gym; only full credit is Correct

One line of nodus-env tasks --split test --seed 0; tasks never carry answers.

{
"metadata": {
"family": "letter_counting",
"systemPrompt": "Reply with the final answer only. Do not restate the question, do not show working, and do not add any words, labels or punctuation around it."
},
"prompt": "How many times does the letter \"t\" appear in the text: \"are far more complex than all that In real life every act\"?",
"taskId": "letter-counting-legacy-rl-v1-0-1:test:0:0"
}

Each example is a TrainingJob template. A baseline is shown only where the example was measured by running it.

Example Mode Runtime Model Tasks Baseline Trained Measured on
letter-counting Train nodus/grpo-lora Qwen/Qwen3-1.7B @ 70d244c 64 not measured not measured not measured

From the repository root, preview the generated manifest using your configured API context:

Terminal window
$ nodus create -f examples/training/catalog/letter-counting-legacy-rl-v1-0-1/letter-counting.yaml --dry-run=server -o estimate

Or preview the catalog example with the Python SDK:

import nodus
client = nodus.Client()
nodus.api.set_default_client(client)
job = nodus.TrainingJob.from_example("nodus/letter-counting-legacy-rl-v1-0-1:letter-counting")
print(job.preview().estimate)
client.close()