AI Agent Skills & perf engineer subagents
zymtrace AI Agent Skills (plus a subagent) let your AI coding assistant profile and optimize the efficiency of your CPU and GPU-accelerated workloads with zymtrace through the MCP: find the bottleneck and apply the fix in your source. They also operate the platform itself, installing the backend and profiler, exposing the gateway, upgrading, and troubleshooting. Describe what you want in plain English and the assistant runs the right workflow, confirming each change with you.
The skills install from a single source into Claude Code, OpenAI Codex, and Cursor. Source and issues: github.com/zystem-io/zymtrace-skills.
Once installed, describe what you want in plain English:
"GPU utilization is 90%+ but tokens/sec is flat. Am I kernel-bound or launch-overhead bound?"
"Is my vLLM server actually GPU-bound, or is host-side tokenization/sampling the bottleneck?"
"My training step has GPU-idle gaps. Find the sync points and D2H copies stalling the pipeline."
"Inference latency regressed after the last deploy. What changed in the hot path?"
"Rank my GPU workloads by wasted GPU time and apply the highest-ROI fix."
Available skills​
| Skill | What it does |
|---|---|
install-zymtrace-backend | Install the backend on Kubernetes (Helm) or Docker Compose. Handles license, databases, and air-gapped registries. |
upgrade-zymtrace-backend | Bump image tags, chart versions, or both, with built-in rollback and verification. |
expose-zymtrace-backend | Expose the gateway via NodePort, AWS ALB, NGINX Ingress, or cloud LoadBalancer, TLS included. |
install-zymtrace-profiler | Install the profiler on Kubernetes, Docker, or bare-metal. CPU + CUDA GPU profiling (CUDA 12.x+). |
troubleshoot-zymtrace-backend | Diagnose "no data appearing", license errors, ingest crashes, slow queries, and storage issues. |
troubleshoot-zymtrace-profiler | Diagnose agent-side failures: CrashLoopBackOff, OOMKilled, NVML missing, PC sampling, license rejected. |
configure-zymtrace-mcp | Connect your assistant to the zymtrace MCP server so you can analyze profiles with natural-language queries. |
optimize-cpu-workloads | Find what's eating CPU on a CPU deployment: rank the top consumers (process / function / container / host / pod), read the flamegraph, name the pattern, and apply the fix in your source. The default when you name an entity with no accelerator hint. |
optimize-gpu-workloads | Analyze a GPU/CUDA workload or inference/training job (vLLM, SGLang, Triton, TensorRT-LLM): pull GPU + matching CPU flamegraphs, find the hot kernel or host-side bottleneck, and apply the fix. |
optimize-memory-allocation | Analyze JVM memory allocation profiles (Java only): find what allocates the most and why GC is busy, then apply the fix. Enables allocation profiling first if it isn't on yet. |
Available subagents​
Invoke the subagent by name from any assistant that supports subagents.
| Subagent | What it does |
|---|---|
zymtrace-perf-engineer | Autonomous, hands-off investigation that fixes, not just analyzes. Ranks the top consumers or drills into a named entity, pulls its metrics, then the CPU flamegraph (and the GPU flamegraph for GPU workloads), recaps, then locates the hot frame in your source and applies the fix (asking for the path if it isn't local), all without stopping to confirm each step. Run several in parallel to triage multiple workloads at once. |
Install the plugin​
- Claude Code
- OpenAI Codex
- Cursor
claude plugin marketplace add zystem-io/zymtrace-skills
claude plugin install zymtrace@zymtrace-skills
Skills become available as /zymtrace:install-zymtrace-backend, /zymtrace:optimize-gpu-workloads, and so on. Run /skills (and /agents for the subagent) to see what loaded.
codex plugin marketplace add zystem-io/zymtrace-skills
Then run /plugins in Codex and install zymtrace from the marketplace. The skills load on your next thread.
- Open Dashboard → Settings → Plugins.
- Under Team Marketplaces, click Import.
- Paste the repository URL
https://github.com/zystem-io/zymtrace-skillsand continue. - Review the parsed zymtrace plugin, set access as you like, and save.
Other tools​
The plugin ships in the Claude Code plugin format, which several other assistants also read. The same marketplace works with them:
| Tool | Install |
|---|---|
| Augment (Auggie CLI) | auggie plugin marketplace add zystem-io/zymtrace-skills → auggie plugin install zymtrace@zymtrace-skills |
| GitHub Copilot CLI | copilot plugin install zystem-io/zymtrace-skills:zymtrace |
For any other MCP-capable assistant (Claude Desktop, Cline, Windsurf, Amazon Q Developer, VS Code + Copilot), you don't need the plugin to analyze profiles. Connect the MCP endpoint directly. If the tool supports skills or rules, you can also copy the Markdown skill files from the repo's zymtrace/skills/ into its skills/rules directory.
Connecting to zymtrace​
The optimize-* skills and the zymtrace-perf-engineer subagent query live profiles through the zymtrace MCP server; configure-zymtrace-mcp automates the connection. This section is the reference.
MCP endpoint​
Your MCP endpoint is your zymtrace gateway URL + /mcp:
https://zymtrace.your-company.com/mcp
zymtrace is self-hosted, so this URL is specific to your deployment. See Configure MCP for the per-tool setup (Claude Code, Codex, Cursor).
Authentication​
| Deployment | What's needed |
|---|---|
| Service-token auth off (default) | Nothing: the connection is open within any network that can reach the gateway. |
| Service-token auth on | A zymtrace service token, passed as a Bearer header. Generate one in the zymtrace UI (Settings → Service Tokens); see Service Tokens. |
The token is always referenced from an environment variable (e.g. $ZYMTRACE_MCP_TOKEN, or ${env:ZYMTRACE_MCP_TOKEN} in Cursor's mcp.json); it is never written to disk in plaintext or pasted into the conversation. For production exposures behind an ALB / Ingress, service-token auth is recommended.