AI Skill MartBrowse skillsPromptsProgramsLearnWhat is a Skill?
 
← Back to list

Vaex Big Data Analytics

A skill that guides Claude to use Vaex for lazy, out-of-core analysis, visualization, and ML on billion-row tabular datasets that exceed RAM.

Data & AnalyticsIntermediate★ 33,030⑂ 3,248AI score 9/10Last updated: Aug 9, 2026

What it does

  • Shows how to open and work with CSV/HDF5/Arrow/Parquet files larger than available memory using Vaex.
  • Provides concrete patterns for virtual columns, lazy evaluation (delay=True), groupby aggregations, and string/datetime ops.
  • Covers big-data visualization (heatmaps, histograms) and ML pipelines with vaex-ml, scikit-learn, XGBoost, CatBoost.
  • Ships topic-based reference docs: core_dataframes, data_processing, performance, visualization, machine_learning, io_operations.
  • Includes a decision guide for choosing vaex vs polars vs dask.

Who it's for

  • Data analysts and engineers handling GB–TB scale logs, financial time series, or scientific datasets on a single machine.
  • Anyone who keeps hitting out-of-memory errors with pandas.
  • ML practitioners building preprocessing and training pipelines over oversized tables.

Example uses

  1. Convert a 50GB CSV into HDF5, then reopen instantly and inspect it with describe().
  2. Batch mean/std/sum computations over a billion rows using delay=True plus vaex.execute().
  3. Plot df.viz.heatmap(...) for millions of points, then chain scaling → PCA → K-means with vaex-ml.

· · · Install guide · · ·

Try it now, no install

Paste this into Claude to use the skill without installing anything.

Read the instructions in this file and follow them to help me:
https://raw.githubusercontent.com/K-Dense-AI/scientific-agent-skills/HEAD/skills/vaex/SKILL.md

What I want: (describe your task here)

If Claude can't open the link, open it yourself and paste the contents instead.

↓ If it works for you, download the ZIP below and install it. Then it runs on its own — no pasting each time.

Install in the Claude app (no terminal)
  1. Download the ZIP with the button below.
  2. In Claude, open Settings → Capabilities and turn on 'Code execution and file creation'. (one time)
  3. Go to Customize → Skills → + → 'Upload a skill' and upload the ZIP.
↓ Download ZIP
Install in Claude Code

Let Claude do it — paste this into Claude Code

Install the skill I found on AI Skill Mart.
Copy the skills/vaex folder from the GitHub repo K-Dense-AI/scientific-agent-skills into my ~/.claude/skills/vaex/.
When it's done, tell me in one line what this skill can do.

Install with a command instead

git clone https://github.com/K-Dense-AI/scientific-agent-skills.git /tmp/scientific-agent-skills && mkdir -p ~/.claude/skills && cp -r /tmp/scientific-agent-skills/skills/vaex ~/.claude/skills/

⚠ This is a third-party skill. Check the source repository before installing.

  1. Open a terminal.
  2. Clone the repo: git clone https://github.com/K-Dense-AI/scientific-agent-skills.git
  3. Create the skills folder: mkdir -p ~/.claude/skills
  4. Copy the skill: cp -r scientific-agent-skills/skills/vaex ~/.claude/skills/
  5. Install Vaex on Python 3.10+: uv pip install vaex (or pip install vaex).
  6. Restart Claude Code and ask something like "analyze this huge CSV with vaex" to trigger the skill.
View source on GitHub ↗License: MIT