Claude Skill MartBrowse skillsQuick linesLearn by videoTerminal guideWhat is a Skill?
 
← Back to list

AI Data Engineering (RAG & Feature Store Pipelines)

A production playbook for AI data infrastructure: RAG pipelines, embeddings, feature stores, and workflow orchestration.

Data & AnalyticsAdvanced★ 509⑂ 73AI score 9/10Last updated: Dec 11, 2025

What it does

  • Frames every project around a 5-stage RAG architecture: ingestion → indexing → retrieval → generation → evaluation.
  • Gives concrete chunking defaults (512 tokens, 50–100 overlap) plus code for code-aware and semantic splitting.
  • Provides an embedding model decision guide (voyage-3 for production, text-embedding-3-small for dev) with ready-to-paste init code.
  • Documents the four RAGAS metrics with pass thresholds and a runnable evaluation script.
  • Covers Feast feature stores, Dagster/Prefect/Airflow/dbt orchestration, LakeFS data versioning, and FastAPI streaming endpoints.

Who it's for

  • Backend/ML engineers shipping document Q&A or semantic search to production.
  • Teams whose RAG already works but retrieves badly and needs chunking/embedding/re-ranking tuning.
  • Data engineers who want embedding jobs modeled as orchestrated assets with lineage.
  • Intermediate-to-advanced Python developers already inside the LangChain ecosystem.

Example uses

  1. "Build a RAG chatbot over 500 PDF manuals" → sets up a Qdrant collection, 512-token chunks, voyage-3 embeddings, and a streaming endpoint.
  2. "My RAG hallucinates — diagnose it" → measures Faithfulness and Context Precision with RAGAS and tunes chunk size and k against the 0.8/0.6 thresholds.
  3. "Refresh embeddings nightly" → generates a Dagster pipeline with raw_documents → chunked_documents → embedded_documents assets.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/ancoleman/ai-design-components/HEAD/skills/ai-data-engineering/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on Claude Skill Mart.
Copy the skills/ai-data-engineering folder from the GitHub repo ancoleman/ai-design-components into my ~/.claude/skills/ai-data-engineering/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/ancoleman/ai-design-components.git /tmp/ai-design-components && mkdir -p ~/.claude/skills && cp -r /tmp/ai-design-components/skills/ai-data-engineering ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal (Terminal on macOS, Git Bash or WSL on Windows).
  2. Clone the repository: git clone https://github.com/ancoleman/ai-design-components.git
  3. Create the skills directory if it doesn't exist: mkdir -p ~/.claude/skills
  4. Copy just this skill: cp -r ai-design-components/skills/ai-data-engineering ~/.claude/skills/
  5. Install the Python dependencies (a virtualenv is recommended): pip install langchain langchain-openai langchain-voyageai langchain-qdrant qdrant-client ragas
  6. Export your API keys: export OPENAI_API_KEY=... and export VOYAGE_API_KEY=...
  7. Restart Claude Code and ask something like "help me build a RAG pipeline" — the skill should trigger automatically.
View source on GitHub ↗License: MIT