Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

compare — Same-Epoch Training Run Comparison

Compares training runs across wandb, neptune, mlflow, or tensorboard by aligning them at the same training step instead of current-vs-final.

Data & AnalyticsIntermediate★ 361⑂ 31AI score 9/10Last updated: Jul 3, 2026

What it does

  • Auto-detects your experiment tracker from env vars and directories (wandb, neptune, mlflow, tensorboard, local JSON results).
  • Pulls the full metric history rather than only the final summary value.
  • Finds the in-progress run's current step and slices the baseline at that same step, killing the classic "epoch 11 vs epoch 100" error.
  • Separates proxy metrics (kNN top-1, loss, perplexity) from downstream targets (linear probe, task F1) and reports them in a compact table.
  • Enforces full run names, flags single-seed comparisons, and warns about metrics logged inconsistently across runs.

Who it's for

  • ML researchers and engineers tracking many concurrent training runs against a baseline
  • Grad students who repeatedly build experiment comparison tables for papers
  • Anyone who gets asked "is this run winning?" mid-training

Examples

  1. "Is my running fastvit experiment catching up to the conv baseline?" → fetches both histories, slices at the current epoch, prints a delta table.
  2. "Rank the 5 runs in this sweep" → aligns all at a common step, ranks proxy and downstream separately, notes single-seed caveats.
  3. "Has my run converged?" → shows the last five epochs of the curve instead of asserting a winner prematurely.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/fcakyon/phd-skills/HEAD/plugin/skills/compare/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the plugin/skills/compare folder from the GitHub repo fcakyon/phd-skills into my ~/.claude/skills/compare/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/fcakyon/phd-skills.git && mkdir -p ~/.claude/skills && cp -r phd-skills/plugin/skills/compare ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repository: git clone https://github.com/fcakyon/phd-skills.git
  3. Create the skills directory: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r phd-skills/plugin/skills/compare ~/.claude/skills/
  5. Restart Claude Code and try a prompt like "compare my wandb run to the baseline".
  6. Prerequisites: set your tracker credentials (WANDB_API_KEY, NEPTUNE_API_TOKEN, MLFLOW_TRACKING_URI) and install the matching Python client before running comparisons.
View source on GitHub ↗License: MIT