AI Skill MartBrowse skillsPromptsProgramsLearnWhat is a Skill?
 
← Back to list

compare — Same-Epoch Training Run Comparison

Compares training runs across wandb, neptune, mlflow, tensorboard, or local result files by aligning them at the same step, and writes experiment reports.

Data & AnalyticsIntermediate★ 438⑂ 38AI score 9/10Last updated: Oct 30, 2026

What it does

  • Auto-detects your tracker in a fixed order (env vars, config keys, mlruns/, runs//lightning_logs/, then local result JSON/CSV).
  • Enforces same-step alignment: if the student run is at epoch 11, it slices the baseline at epoch 11 instead of comparing against its epoch-100 final number, and states when it interpolates or picks the nearest step.
  • Separates proxy metrics (kNN top-1, loss, perplexity) from downstream targets (linear probe, task F1) and refuses to declare a winner from proxy data alone.
  • Goes beyond two runs: multi-run ranking (best overall, best per metric, best per GPU-hour, crashed runs), ablation delta analysis with interaction checks, a structured experiment report template, and cron-based training monitoring with notifications.
  • Bans shortcodes — every report line uses full run names.

Who it's for

  • ML researchers and engineers tracking in-progress runs against a baseline.
  • Anyone assembling results tables or ablation sections for a paper or internal report.
  • Teams already on wandb, neptune, mlflow, or tensorboard.

Example uses

  1. "Is distill-from-vit-l behind the baseline?" → baseline sliced at the student's current epoch, proxy and downstream reported separately, plus a projected catch-up step.
  2. "Summarize this week's 12 runs" → Overview / Results Table / Key Findings / Recommendations report.
  3. "Monitor training and ping me" → a */10 * * * * cron job checking GPU utilization and log freshness that pipes alerts to the plugin notifier.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/fcakyon/labrat/HEAD/plugin/skills/compare/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on AI Skill Mart.
Copy the plugin/skills/compare folder from the GitHub repo fcakyon/labrat into my ~/.claude/skills/fcakyon-compare/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/fcakyon/labrat.git /tmp/labrat && mkdir -p ~/.claude/skills && cp -r /tmp/labrat/plugin/skills/compare ~/.claude/skills/compare

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repo: git clone https://github.com/fcakyon/labrat.git /tmp/labrat
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r /tmp/labrat/plugin/skills/compare ~/.claude/skills/compare
  5. (Recommended) Install the full labrat plugin if you want the notifier script and /labrat:setup, /labrat:debug, /labrat:rebuttal commands.
  6. Authenticate your tracker first: wandb login, or export NEPTUNE_API_TOKEN / MLFLOW_TRACKING_URI.
  7. Restart Claude Code and ask something like "compare run A against the baseline at the same epoch" to trigger the skill.
View source on GitHub ↗License: MIT