Skip to content

Rerun a run from its receipt

A run’s receipt names its command and the commit it ran on. Rerun the command on that commit and record the rerun as a check.

Paste this into your agent

Rerun seed 2 from its receipt, on the code it ran on, and check it.

Rerun seed 2 of int8-ptq from its Rabbit Brain receipt, and tell me if it still gives the same numbers.

- Run `rb context --model <your model id>` first.
- Read the run's receipt with `rb show <run id>`: the command, and the commit it ran on.
- Check out that commit in a worktree inside this repository (`git worktree add .worktrees/rerun <commit>`), and run the receipt's command there through `rb journal run`.
- Record the new run as a check of the old one: `--repeats <run id> --again --why "rerun from its receipt"`, with `--commit`.

Finish with `rb show` of the criterion exactly as printed, and what rb says of the repeat.

Since the runs, one commit changed how eval.py builds its output path.

Terminal window
$ rb show top1-drop-budget
Criterion top1-drop-budget on int8-ptq (primary, own): INT8 loses at most 0.5 points of top-1 against FP32
top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
reads: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
result: loss: the interval clear of 0 on the worse side (higher is better)
rule: criterion met: estimate -0.08 ≥ -0.5
timing: declared before the evidence (by production times)
units: 3 seeds from 4 runs, 1 repeat
integrity: ok
· runt6sqax repeats runssvuju: the same number, a check that holds
rung2qpud: difference -0.11 (file) · unit seed=1 · seed=1
runssvuju: difference -0.08 (file) · unit seed=2 · seed=2
runqllqdf: difference -0.05 (file) · unit seed=3 · seed=3
runt6sqax: difference -0.08 (file) · unit seed=2 · repeats runssvuju · seed=2

The rerun is a repeat of seed 2, so the criterion, a rule a result must meet, still rests on 3 seeds: “from 4 runs, 1 repeat”.

  1. See what a rerun needs

    Terminal window
    $ rb show int8-ptq
    Experiment int8-ptq: INT8 post-training quantization · open · under q-int8
    design not confirmed · list it to confirm: rb confirm int8-ptq
    hypothesis or purpose: not recorded
    why: not recorded
    variants: fp32 (baseline) · int8 (candidate)
    varies: precision, calib_images
    spec 85201d31eae8 · frozen 2026-11-10T15:32:35.679+00:00 (85201d31eae8) · note: before the runs · not confirmed
    settings:
    optim.lr 0.1 verified · configs/train.yaml#optim.lr
    optim.momentum 0.9 verified · configs/train.yaml#optim.momentum
    data.batch_size 128 verified · configs/train.yaml#data.batch_size
    eval.batch_size 256 verified · configs/train.yaml#eval.batch_size
    seed (each run) per-run · no source
    fp32.precision fp32 verified · configs/fp32.yaml#precision
    int8.precision int8 verified · configs/int8.yaml#precision
    int8.calib_images 512 verified · configs/int8.yaml#calib_images
    criteria:
    top1-drop-budget INT8 loses at most 0.5 points of top-1 against FP32
    criterion met · loss · change.top1 ≥ -0.5 (n ≥ 3)
    reads: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
    result: loss: the interval clear of 0 on the worse side (higher is better)
    rule: criterion met: estimate -0.08 ≥ -0.5
    timing: declared before the evidence (by production times)
    units: 3 seeds
    integrity: ok
    runs: rung2qpud, runssvuju, runqllqdf
    to run it again:
    code ok commit 12ec61a726fe
    code · commit 12ec61a726fe, which all 3 runs ran at
    your checkout differs in 1 file: eval.py
    show code diff:
    git diff 12ec61a726feda8c90dbba424b07ee7ec0b9d32c -- ":(top)" ":(top,exclude).rb"
    rebuild:
    git worktree add --detach ../int8-ptq-12ec61a 12ec61a726feda8c90dbba424b07ee7ec0b9d32c
    environment ok python 3.14.4
    files ok all 3 files the runs named are here unchanged
  2. Read the run’s receipt

    Terminal window
    $ rb show runssvuju
    Run runssvuju on int8-ptq (file)
    produced 2026-11-10 15:32 UTC (its file's time) · recorded 2026-11-10 15:32 UTC
    fp32.top1 = 91.66
    int8.top1 = 91.58
    per-run: seed=2
    command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-10)
    code that made it: 12ec61a726fe (as given)
    your checkout differs in 1 file: eval.py
    repository when recorded: commit 12ec61a726fe
    config results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
    file results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here
  3. See what changed since

    Terminal window
    $ git diff 12ec61a -- eval.py
    diff --git a/eval.py b/eval.py
    index 35a1cb3..901a798 100644
    --- a/eval.py
    +++ b/eval.py
    @@ -7,7 +7,7 @@ p = argparse.ArgumentParser()
    p.add_argument("--seed", type=int, required=True)
    a = p.parse_args()
    fp32, int8 = TOP1[a.seed]
    -out = f"results/int8-ptq/seed{a.seed}"
    +out = os.path.join("results", "int8-ptq", f"seed{a.seed}")
    os.makedirs(out, exist_ok=True)
    with open(f"{out}/metrics.json", "w") as f:
    json.dump({"fp32": {"top1": fp32}, "int8": {"top1": int8}}, f, indent=2)
  4. Check out the receipt’s commit in a worktree

    Terminal window
    $ git worktree add -q .worktrees/rerun 12ec61a

    Then, in .worktrees/rerun:

    Terminal window
    $ rb context
    # The record: INT8 ResNet-20 on CIFAR-10
    This worktree (no branch · ~/int8-rerun/.worktrees/rerun) has no record of its own: rb writes to the record in main · ~/int8-rerun, and each run keeps this worktree's code.
    … (98 more lines)
  5. Run the receipt’s command

    Terminal window
    $ rb journal run -- python eval.py --seed 2
    {"metrics": {"fp32.top1": 91.66, "int8.top1": 91.58}}
    exit 0. The command, its exit code and what it wrote are in the journal.
    It printed its metrics: fp32.top1=91.66, int8.top1=91.58. rb reconcile proposes no new run: a recorded run of int8-ptq holds these numbers.
    Next: rb reconcile
  6. Record it as a check

    The output matches the first run’s byte for byte. --again --why records it anyway, as a second execution, and --repeats makes it a check of seed 2, not a new seed.

    Terminal window
    $ rb run add int8-ptq --from results/int8-ptq/seed2/metrics.json --set seed=2 --config results/int8-ptq/seed2/config.yaml --command "python eval.py --seed 2" --commit 12ec61a726feda8c90dbba424b07ee7ec0b9d32c --repeats runssvuju --again --why "rerun from its receipt"
    runt6sqax on int8-ptq: fp32.top1=91.66, int8.top1=91.58 · seed=2
    Ran with the spec: 5 of its settings found in .worktrees/rerun/results/int8-ptq/seed2/config.yaml, all matching.
    It repeats runssvuju: a check of that run, in its unit, never a new one.
    Produced: 2026-11-10 15:32:38 (file)
    top1-drop-budget criterion met · loss: change.top1: -0.08 [-0.1545, -0.005476] % · paired difference, t interval, 3 seeds, 95% (rb)
    Next: rb show top1-drop-budget
    Terminal window
    $ rb show runt6sqax
    Run runt6sqax on int8-ptq (file)
    produced 2026-11-10 15:32 UTC (its file's time) · recorded 2026-11-10 15:32 UTC
    fp32.top1 = 91.66
    int8.top1 = 91.58
    per-run: seed=2
    command (as given): python eval.py --seed 2 (rb journal run watched it make these numbers, 2026-11-10)
    code that made it: 12ec61a726fe (as given)
    your checkout has this code
    repository when recorded: commit 12ec61a726fe
    config .worktrees/rerun/results/int8-ptq/seed2/config.yaml: 5 match, 0 differ, 0 absent
    file .worktrees/rerun/results/int8-ptq/seed2/metrics.json sha256 342131bfa92f (72 bytes) · here
    note: recorded again: rerun from its receipt. · not confirmed

Rabbit Brainrabbitbrain.ai · GitHub · PyPI · Privacy · Apache-2.0