Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

Agent Observability

A skill for designing privacy-aware traces, spans, metrics, alerts, and investigation runbooks for AI agents.

AutomationAdvanced★ 161⑂ 32AI score 9/10Last updated: Aug 9, 2026

What it does

Guides end-to-end instrumentation design for AI agent workflows.

  • One trace per user-visible attempt, with spans for model calls, retrieval, tools, handoffs, approvals, retries and final validation
  • Produces a trace/event schema with identifiers, span taxonomy, attributes and redaction rules
  • Defines service indicators: task success, p95 end-to-end latency, tool failure rate, escalation rate, cost per completed task
  • Specifies dashboards, actionable alerts with owners, plus sampling/retention/access/cost plans and an investigation runbook
  • Ships python3 scripts/summarize_traces.py spans.jsonl for a content-free structural check (missing parents, non-single root, cycles, disconnected components) and --strict scanning for likely secret/PII fields

Default is content-free telemetry; prompt/response capture requires explicit authorization, redaction, access control and retention limits.

Who it's for

  • Backend/platform engineers shipping LLM agents to production
  • Teams stuck debugging intermittent tool failures or unexplained cost growth
  • SREs and audit owners standardizing SLOs, alerts and evidence trails

Examples

  1. "Why did the research agent get slower after the release?" → compare version-tagged traces, slice p95 by retrieval/model/browser-tool spans, inspect retries and queue delay, correlate with deploy time
  2. Instrumenting a new multi-tool agent from scratch → get span taxonomy, attribute standards, redaction rules and dashboard drafts in one pass
  3. Run the summarizer on collected spans.jsonl to catch broken parentage and flag field paths that likely carry secrets or personal data

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/seb1n/awesome-ai-agent-skills/HEAD/agent-engineering/agent-observability/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the agent-engineering/agent-observability folder from the GitHub repo seb1n/awesome-ai-agent-skills into my ~/.claude/skills/agent-observability/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/seb1n/awesome-ai-agent-skills.git && mkdir -p ~/.claude/skills && cp -r awesome-ai-agent-skills/agent-engineering/agent-observability ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repo: git clone https://github.com/seb1n/awesome-ai-agent-skills.git
  3. Create the skills folder: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r awesome-ai-agent-skills/agent-engineering/agent-observability ~/.claude/skills/
  5. Confirm references/trace-schema.md and scripts/summarize_traces.py came along.
  6. Restart Claude Code and prompt with something like "design observability for this agent" or "define a trace schema".
  7. Ensure python3 is available for the helper script: python3 --version
View source on GitHub ↗License: MIT