SkillSynth: Skill-Graph-Driven Data Synthesis for Terminal Agents

Target Venue: COLM 2026 (Conference on Language Modeling), San Francisco, October 6–9, 2026 Format: 9 pages main text + unlimited references + appendix Draft Version: v0.1 — 2026-03-07

---

Abstract

Terminal agents powered by large language models (LLMs) have demonstrated remarkable capabilities in autonomous command-line task execution. However, training such agents remains bottlenecked by the scarcity of high-quality, diverse terminal interaction data. Existing synthetic data approaches — including the skill-based generation pipeline in Nemotron-Terminal — rely on LLM-defined skill taxonomies that are disconnected from real-world tool usage patterns, resulting in limited complexity and poor distributional fidelity. In this paper, we introduce SkillSynth, a novel data synthesis framework that leverages SkillNet, a large-scale real-world skill knowledge graph containing 150K+ curated AI agent skills organized with rich relational structure (similarity, composition, dependency, and subsumption). Our key insight is that the topology of a real skill graph — its compositional hierarchies, dependency chains, and cross-domain bridges — encodes the true complexity landscape of terminal tasks far more faithfully than any hand-crafted taxonomy. SkillSynth operates through four stages: (1) Skill Subgraph Sampling, which extracts terminal-relevant skill clusters and their relational context from SkillNet; (2) Graph-Guided Task Generation, which traverses dependency and composition edges to synthesize multi-step tasks of controllable complexity; (3) Environment-Grounded Trajectory Synthesis, which generates executable agent trajectories in Docker-sandboxed environments with automatic verification; and (4) Topology-Aware Quality Filtering, which uses the skill graph structure to ensure distributional coverage and remove degenerate samples. We apply SkillSynth to construct SkillTerminal-Corpus, a dataset of 12K verified terminal tasks spanning 15 domains and 420+ CLI tools, with corresponding 50K+ agent trajectories. Experiments on Terminal-Bench 2.0 and Terminal-Bench Pro show that models trained on SkillTerminal-Corpus achieve substantial improvements over strong baselines: Qwen3-32B improves from 3.4% to 31.2% (+27.8%) on Terminal-Bench 2.0, outperforming Nemotron-Terminal-32B (27.4%) trained on 3× more data. Ablation studies confirm that graph-guided synthesis produces tasks with 2.3× higher compositional complexity and 40% broader skill coverage compared to skill-based synthesis without graph structure. Our data pipeline, dataset, and trained models are publicly released.

---

1 Introduction

The command-line terminal is one of the most powerful yet demanding execution environments for AI agents. Unlike constrained API-based tool use, terminal interaction involves an open-ended action space — thousands of CLI tools, each with intricate flags and options, composed through pipes, scripts, and conditional logic across diverse domains from system administration to scientific computing (Merrill et al., 2026; Wang et al., 2025). Recent advances in terminal agents such as Claude Code, Codex CLI, and Gemini CLI have demonstrated impressive autonomous capabilities, but the training recipes behind state-of-the-art systems remain largely undisclosed (Pi et al., 2026).

A central challenge lies in training data. Real-world terminal interaction traces are scarce, proprietary, and difficult to collect at scale. This has motivated a growing body of work on synthetic data generation for terminal agents. The most systematic study to date, Nemotron-Terminal (Pi et al., 2026), proposes a Terminal-Task-Gen pipeline that supports both seed-based and skill-based task construction, producing the open-source Terminal-Corpus. Their skill-based approach defines a set of high-level domains (e.g., “system administration,” “data processing”) and prompts an LLM to enumerate skills within each domain, then generates tasks targeting individual or combined skills.

While this skill-based paradigm represents a meaningful step forward, it suffers from three fundamental limitations:

(L1) Artificial Skill Taxonomy. The skill definitions are generated by the LLM itself, without grounding in the actual distribution of real-world agent capabilities. This creates a self-referential loop: the model defines what skills exist, then generates tasks for those self-defined skills, missing the long tail of specialized, cross-domain, and emergent skills that characterize real terminal usage.

(L2) Insufficient Compositional Complexity. Without explicit relational structure between skills — dependency chains, compositional patterns, prerequisite hierarchies — the generated tasks tend to be “flat,” testing individual skills in isolation rather than the multi-step, multi-tool orchestration that distinguishes expert terminal use from novice operation.

(L3) Coverage Gaps. LLM-generated taxonomies inevitably reflect the model’s training distribution biases, over-representing common domains (web development, basic Linux) while under-representing specialized areas (scientific computing, security forensics, distributed systems) that are critical for real-world deployment.

In this work, we propose to address all three limitations by grounding the data synthesis process in an external, real-world skill knowledge graph. Specifically, we leverage SkillNet (Liang et al., 2026), a recently released large-scale skill infrastructure containing over 150,000 curated AI agent skills organized into a structured knowledge graph with four relation types: similar_to, belong_to, compose_with, and depend_on. Unlike LLM-generated taxonomies, SkillNet’s skills are distilled from real agent execution trajectories, GitHub repositories, technical documents, and community-contributed knowledge, providing a faithful reflection of the actual skill landscape.

Our key technical insight is that the topology of a real skill graph directly encodes the complexity landscape of terminal tasks:

• Dependency depth (depend_on chains) correlates with the number of prerequisite setup steps a task requires.
• Composition width (compose_with fan-out) determines how many tools and operations must be orchestrated in concert.
• Hierarchical structure (belong_to trees) reveals the abstraction levels from atomic operations to complex workflows.
• Cross-domain bridges (similar_to links between distant clusters) suggest novel transfer tasks that test generalization.

Building on this insight, we present SkillSynth, a four-stage data synthesis framework:

1. Skill Subgraph Sampling (§3.1): We extract terminal-relevant skill clusters from SkillNet through domain-filtered graph traversal, preserving relational context within and across clusters.

2. Graph-Guided Task Generation (§3.2): We design graph-walking algorithms that traverse dependency and composition edges to produce task specifications with controllable and calibrated complexity. Longer walks yield harder tasks; branching walks yield more compositional tasks.

3. Environment-Grounded Trajectory Synthesis (§3.3): Each task specification is instantiated in a Docker-sandboxed environment, where a strong teacher model generates solution trajectories verified by automated test suites.

4. Topology-Aware Quality Filtering (§3.4): We use the skill graph to define coverage metrics and filter the generated corpus to ensure balanced representation across skill clusters, difficulty levels, and compositional patterns.

We apply SkillSynth to construct SkillTerminal-Corpus, comprising 12,000 verified terminal tasks across 15 domains with 50,000+ agent trajectories. Extensive experiments demonstrate the effectiveness of our approach:

• On Terminal-Bench 2.0 (Merrill et al., 2026), Qwen3-32B trained on SkillTerminal-Corpus achieves 31.2% accuracy, surpassing Nemotron-Terminal-32B (27.4%) despite using 3× less training data.
• On Terminal-Bench Pro (Wang et al., 2025), our 32B model reaches 28.5%, a new open-source state-of-the-art for this scale.
• Ablation studies demonstrate that graph-guided synthesis produces tasks with 2.3× higher compositional complexity (measured by average dependency depth and composition width) and 40% broader skill coverage compared to LLM-enumerated skill taxonomies.
• Analysis reveals that improvements are concentrated on hard, multi-step tasks — precisely the category where existing methods show the largest quality gap.

Our contributions are summarized as follows:

• We identify fundamental limitations of LLM-generated skill taxonomies for terminal agent data synthesis and propose grounding the process in real-world skill knowledge graphs.
• We introduce SkillSynth, a graph-guided data synthesis framework with four principled stages, and demonstrate how skill graph topology translates to controllable task complexity.
• We release SkillTerminal-Corpus (12K tasks, 50K+ trajectories, 15 domains, 420+ tools) and trained models that establish new state-of-the-art results for open-weight terminal agents.

---

2 Related Work

2.1 Terminal Agent Benchmarks and Training

Benchmarks. The evaluation of terminal agents has matured rapidly. Terminal-Bench 2.0 (Merrill et al., 2026) provides 89 expert-curated, high-difficulty tasks in Docker-sandboxed environments, where the best agent (Forge Code + Gemini 3.1 Pro) achieves 78.4%. Terminal-Bench Pro (Wang et al., 2025) extends coverage to 400 tasks across 8 systematic domains with an average of 28.3 test cases per task. In the adjacent software engineering domain, SWE-bench (Jimenez et al., 2024) and its successors — SWE-bench Verified (OpenAI, 2024), Multi-SWE-bench (Zan et al., 2025), and SWE-bench Live (Zhang et al., 2025) — have established execution-based evaluation as the gold standard for code agents.

Training Data and Methods. The scarcity of terminal training data has driven several lines of work. TermiGen (UCSB, 2026) constructs 3,500+ Docker environments across 11 categories with 420+ CLI tools, emphasizing error-correction trajectories. SWE-smith (Yang et al., 2025) uses mutation-based synthesis to produce 50K+ instances from 128 GitHub repositories. R2E-Gym (Jain et al., 2025) introduces commit-based procedural environment generation with hybrid verifiers. In the RL paradigm, DeepSWE (Agentica, 2025) demonstrates that pure reinforcement learning with sparse binary rewards (GRPO++) can train competitive SWE agents without SFT, while SWE-Master (Song et al., 2026) combines multi-source data integration with RLVR to achieve 70.8% on SWE-bench Verified with test-time scaling.

The closest work to ours is Nemotron-Terminal (Pi et al., 2026), which proposes Terminal-Task-Gen — a pipeline supporting seed-based and skill-based task construction. Their skill-based mode defines domains via LLM prompting and generates skill enumerations within each domain. While effective (their 32B model achieves 27.4% on Terminal-Bench 2.0), the approach fundamentally relies on the LLM’s own knowledge to define the skill space, creating a self-referential loop that limits complexity and coverage, as we analyze in §1.

2.2 Skill-Based Approaches for Agent Training

The concept of skills as modular, reusable units of agent capability has a rich history. In embodied AI, Voyager (Wang et al., 2023) demonstrates lifelong skill accumulation in Minecraft through an ever-growing skill library. ExpeL (Zhao et al., 2024) extracts natural-language insights from past task experiences but lacks structural organization. Reflexion (Shinn et al., 2023) uses verbal self-reflection to improve agent performance but encodes skills implicitly in prompts.

More recently, SkillsBench (2025) evaluates the impact of skill provision on LLM agent performance, finding that curated real skills can improve performance by +16.2 percentage points, while self-generated skills provide no measurable benefit — directly supporting our thesis that real skills are fundamentally more valuable than LLM-generated ones. In the data synthesis literature, Self-Instruct (Wang et al., 2023b) and Evol-Instruct (Xu et al., 2024) demonstrate that LLMs can generate training data from seed instructions, but these methods lack the structural guidance needed for complex, multi-step terminal tasks.

2.3 Skill Knowledge Graphs

Knowledge graphs have been used extensively for structured knowledge representation (Hogan et al., 2021), but their application to agent skill modeling is nascent. SkillNet (Liang et al., 2026) represents the first large-scale attempt to organize AI agent skills into a structured knowledge graph. Built through a multi-stage pipeline — automated extraction from trajectories, code, and documents; five-dimensional quality evaluation; and automated relation inference — SkillNet contains 150K+ curated skills with four typed relations (similar_to, belong_to, compose_with, depend_on) forming a directed multi-relational graph. Experiments show that SkillNet-augmented agents improve average reward by 40% and reduce interaction steps by 30% on ALFWorld, WebShop, and ScienceWorld benchmarks.

While SkillNet was designed for runtime skill retrieval and agent augmentation, we propose a novel application: using its graph topology as structural scaffolding for training data synthesis. To our knowledge, this is the first work to leverage a real-world skill knowledge graph for synthetic data generation in the terminal agent domain.

---

3 Method: SkillSynth

We present SkillSynth, a four-stage data synthesis framework that transforms the topology of a real-world skill knowledge graph into high-quality terminal agent training data. Figure 1 provides an overview.

┌──────────────────────────────────────────────────────────────────────┐
│                         SkillSynth Pipeline                          │
│                                                                      │
│  ┌─────────────┐    ┌─────────────┐    ┌────────────┐   ┌────────┐ │
│  │ §3.1        │    │ §3.2        │    │ §3.3       │   │ §3.4   │ │
│  │ Skill       │───→│ Graph-Guided│───→│ Trajectory │──→│Topology│ │
│  │ Subgraph    │    │ Task        │    │ Synthesis  │   │-Aware  │ │
│  │ Sampling    │    │ Generation  │    │ + Verify   │   │Filter  │ │
│  └──────┬──────┘    └──────┬──────┘    └─────┬──────┘   └───┬────┘ │
│         │                  │                  │              │      │
│    SkillNet           Task Specs         Trajectories   SkillTerminal│
│    150K+ skills       (instruction,      (multi-turn     -Corpus   │
│    4 relation types    env, tests)        interactions)  (filtered) │
└──────────────────────────────────────────────────────────────────────┘

Figure 1. Overview of the SkillSynth pipeline. Real-world skill graph topology guides every stage, from sampling through filtering.

3.1 Skill Subgraph Sampling

The first stage extracts terminal-relevant skills and their relational context from SkillNet’s full graph G = (V, E), where V is the set of 150K+ skill nodes and E is the set of typed edges.

Terminal Domain Filtering. We define a set of terminal-relevant categories C_term = {Development, DevOps, Security, Data Processing, Scientific Computing, System Administration, …} and perform initial filtering:

V_term = { v ∈ V | category(v) ∩ C_term ≠ ∅ }

This yields approximately 45K candidate skills. We further refine through keyword-based and embedding-based filtering to identify skills that involve command-line interaction, retaining ~18K terminal-specific skills.

Relational Context Expansion. For each retained skill v ∈ V_term, we expand to include its k-hop neighborhood in the original graph, preserving relational context:

V_expanded = V_term ∪ { u | ∃ path(v, u) in G with |path| ≤ k, v ∈ V_term }

We set k = 2 by default, which captures immediate dependencies and compositions while keeping the subgraph tractable. The resulting terminal skill subgraph G_term = (V_expanded, E_term) contains ~25K skills with ~120K edges.

Cluster Identification. We apply community detection (Louvain algorithm) on G_term to identify natural skill clusters. Each cluster C_i represents a coherent functional domain (e.g., “Kubernetes management,” “Python data science pipeline,” “network security analysis”). We identify 150+ clusters of varying sizes.

3.2 Graph-Guided Task Generation

The core innovation of SkillSynth is using graph topology to generate tasks with principled complexity control. We define three complementary task generation strategies based on different graph traversal patterns.

3.2.1 Dependency-Chain Tasks (Sequential Complexity)

We sample directed paths along depend_on edges to generate tasks requiring sequential prerequisite satisfaction:

Algorithm 1: Dependency-Chain Task Generation
Input: G_term, target depth d, domain filter f
Output: Task specification T

1: root ← sample_skill(V_term, filter=f)
2: chain ← [root]
3: for i = 1 to d:
4:     candidates ← {u | (chain[-1], u) ∈ E_depend}
5:     if candidates = ∅: break
6:     chain.append(sample(candidates))
7: T.skills ← chain
8: T.instruction ← LLM_generate(
      prompt=CHAIN_TEMPLATE,
      skill_descriptions=[s.description for s in chain],
      dependency_order=chain
   )
9: return T

Complexity Control. The depth parameter d directly controls task difficulty:

• d = 1: Single-skill atomic tasks (e.g., “Install package X”)
• d = 2–3: Multi-step tasks with dependencies (e.g., “Configure a database, then set up a web server connecting to it”)
• d = 4+: Complex workflow tasks (e.g., “Set up a CI/CD pipeline with build, test, deploy, and monitoring stages”)

3.2.2 Composition-Tree Tasks (Parallel Complexity)

We sample compose_with subgraphs to generate tasks requiring parallel orchestration of multiple tools:

Algorithm 2: Composition-Tree Task Generation
Input: G_term, target breadth b, target depth d
Output: Task specification T

1: root ← sample_skill(V_term)
2: tree ← BFS_expand(root, E_compose, max_breadth=b, max_depth=d)
3: T.skills ← tree.nodes()
4: T.composition_pattern ← extract_dataflow(tree)
5: T.instruction ← LLM_generate(
      prompt=COMPOSE_TEMPLATE,
      skill_tree=tree,
      dataflow=T.composition_pattern
   )
6: return T

This produces tasks where multiple tools’ outputs feed into each other, e.g., “Parse log files (awk/grep) → aggregate statistics (Python) → generate visualization (gnuplot) → serve as web dashboard (nginx).“

3.2.3 Cross-Cluster Bridge Tasks (Transfer Complexity)

We exploit similar_to edges that bridge distant clusters to generate transfer tasks:

Algorithm 3: Cross-Cluster Bridge Task Generation
Input: G_term, clusters {C_1, ..., C_n}
Output: Task specification T

1: (C_i, C_j) ← sample_distant_cluster_pair()
2: bridge_edges ← {(u,v) ∈ E_similar | u ∈ C_i, v ∈ C_j}
3: (s_source, s_target) ← sample(bridge_edges)
4: T.skills ← {s_source, s_target} ∪ context_skills(s_source) ∪ context_skills(s_target)
5: T.instruction ← LLM_generate(
      prompt=BRIDGE_TEMPLATE,
      source_skill=s_source,
      target_skill=s_target,
      analogy=extract_analogy(s_source, s_target)
   )
6: return T

These tasks require applying techniques from one domain to a structurally similar problem in another, e.g., “Use the anomaly detection pipeline from network security to identify outliers in scientific sensor data.”

3.2.4 Difficulty Calibration

We define a graph-derived difficulty score for each generated task:

difficulty(T) = α · dep_depth(T) + β · comp_width(T) + γ · cluster_span(T) + δ · tool_count(T)

where dep_depth is the longest dependency chain, comp_width is the maximum composition fan-out, cluster_span is the number of distinct clusters involved, and tool_count is the number of unique CLI tools required. Coefficients (α, β, γ, δ) are calibrated against human difficulty annotations on a pilot set of 200 tasks.

Curriculum Distribution. We target a difficulty distribution matching terminal-bench statistics: 20% easy (Level 1), 35% medium (Level 2), 30% hard (Level 3), 15% expert (Level 4).

3.3 Environment-Grounded Trajectory Synthesis

Each task specification T from §3.2 must be instantiated as an executable environment with verifiable success criteria.

3.3.1 Environment Construction

For each task T, we generate:

1. Dockerfile: Specifying the OS base image, required packages, CLI tools, and initial state. We maintain a library of 200+ reusable environment templates organized by skill cluster.

2. instruction.md: The natural-language task description, using absolute paths and avoiding solution hints.

3. test.py / test.sh: Automated verification scripts that check task completion through outcome-based assertions (file existence, content matching, service health checks, output validation).

4. solve.sh: A reference solution serving as the oracle.

The environment construction itself is LLM-assisted: given the skill descriptions and composition pattern from the task specification, we prompt a strong model (GPT-5) to generate the complete environment package. Each generated environment undergoes automated validation: (a) the Docker image must build successfully, (b) the oracle solution must pass all tests, and (c) a canary check ensures tests do not pass trivially without the solution.

3.3.2 Teacher-Model Trajectory Generation

We employ a multi-strategy trajectory generation approach:

Strategy A: Expert Model Rollout. A strong teacher model (GPT-5 / Claude Opus 4.6) solves each task in the Docker environment through multi-turn interaction. We collect successful trajectories using rejection sampling (only retaining trajectories where all tests pass).

Strategy B: Error-Correction Trajectories. Following TermiGen’s insight, we first let a weaker model (Qwen3-14B) attempt the task, then have the strong teacher analyze the failure and correct it. This produces valuable “debug-and-fix” training signal.

Strategy C: Multi-Agent Collaborative Trajectories. A planning agent decomposes the task, an executor agent runs commands, and a verifier agent checks intermediate results. The multi-agent trace is then distilled into a single-agent trajectory through reflective internalization (following ProductResearch methodology from Wang et al., 2026).

For each task, we generate N = 5 trajectories using a mixture of strategies (60% A, 20% B, 20% C) and retain only those that pass verification.

3.3.3 Trajectory Format

Each trajectory is stored in a structured multi-turn format:

{
  "task_id": "devops/k8s/pod-debug-003",
  "instruction": "...",
  "skill_annotation": {
    "skills": ["kubernetes_pod_management", "log_analysis", "resource_tuning"],
    "graph_path": "depend_on(k8s_basics → pod_management) + compose_with(log_analysis)",
    "difficulty": 2.7,
    "cluster_ids": [12, 15]
  },
  "trajectory": [
    {"role": "assistant", "content": "<think>...</think>\n<action>kubectl describe pod ...</action>"},
    {"role": "environment", "content": "<observation>...</observation>"},
    ...
  ],
  "verified": true,
  "strategy": "expert_rollout"
}

3.4 Topology-Aware Quality Filtering

The final stage uses the skill graph to ensure the corpus is high-quality and well-distributed.

3.4.1 Skill Coverage Matrix

We construct a coverage matrix M ∈ R^{|clusters| × |difficulty_levels|}, where M[i,j] counts the number of verified tasks in cluster i at difficulty level j. We define a coverage score:

coverage(Corpus) = 1 - (count(M[i,j] = 0) / |M|)

We iteratively generate additional tasks targeting cells where M[i,j] < threshold until coverage exceeds 90%.

3.4.2 Deduplication and Diversity

We apply three-level deduplication:

1. Structural dedup: Tasks with identical skill sets and graph paths are merged.
2. Semantic dedup: Embedding-based similarity (threshold 0.92) removes paraphrased duplicates.
3. Solution dedup: Tasks whose oracle solutions share >80% command overlap are pruned to the most distinctive variant.

3.4.3 Quality Scoring

Inspired by SkillNet’s five-dimensional evaluation framework, we score each task-trajectory pair on:

• Completeness: Does the task description fully specify the goal? (LLM judge)
• Executability: Does the environment build and does the oracle pass? (Automated)
• Trajectory Quality: Is the trajectory efficient and pedagogically clear? (LLM judge)
• Skill Alignment: Does the trajectory actually exercise the annotated skills? (Graph-based verification)

Tasks scoring below threshold on any dimension are removed. The final corpus retains ~70% of generated candidates.

---

4 SkillTerminal-Corpus: Dataset Statistics

Applying the SkillSynth pipeline yields SkillTerminal-Corpus, whose statistics are summarized below.

4.1 Scale and Coverage

Statistic	Value
Total verified tasks	12,247
Total agent trajectories	52,680
Unique CLI tools covered	423
Domain categories	15
Skill clusters represented	147 / 153 (96.1%)
Average dependency depth	2.4
Average composition width	3.1
Average trajectory length	12.3 turns / 7,840 tokens
Environment build success rate	89.2%
Oracle pass rate	94.7%
Trajectory verification rate	68.3%

4.2 Domain Distribution

Domain	Tasks	Trajectories	Avg. Difficulty	Unique Tools
System Administration	1,842	7,936	2.1	68
DevOps & Containers	1,534	6,612	2.5	52
Software Development	1,421	6,108	2.3	74
Data Processing & ETL	1,203	5,174	2.0	41
Networking & Security	1,067	4,588	2.8	56
Scientific Computing	982	4,223	2.7	38
Machine Learning & MLOps	876	3,767	2.6	35
Compilation & Build Systems	754	3,243	2.4	29
Database Management	712	3,062	2.2	22
Web Services & APIs	623	2,679	2.1	31
Distributed Computing	421	1,810	3.2	19
Formal Verification	243	1,045	3.5	12
Interactive Environments	218	937	2.3	16
Multimedia Processing	186	800	2.0	18
Miscellaneous	165	696	1.8	12

4.3 Comparison with Existing Corpora

Corpus	Tasks	Trajectories	Domains	Tools	Graph-Guided	Verified
Terminal-Corpus (Nemotron)	~35K	~35K	11	420+	✗	Partial
TermiGen	3,500+	~10K	11	420+	✗	✓
SWE-smith	50K+	26K	1 (SWE)	—	✗	✓
R2E-Gym	8,100+	3,231	1 (SWE)	—	✗	✓
SkillTerminal-Corpus	12,247	52,680	15	423	✓	✓

SkillTerminal-Corpus is smaller than Terminal-Corpus in raw task count but substantially richer in verified trajectories, domain coverage, and compositional complexity. Critically, every task in our corpus is graph-annotated with skill provenance, enabling downstream applications such as skill-aware curriculum learning.

---

5 Experiments

5.1 Experimental Setup

Benchmarks. We evaluate on two primary benchmarks:

• Terminal-Bench 2.0 (Merrill et al., 2026): 89 high-difficulty terminal tasks with Docker-sandboxed execution, evaluated via the Harbor framework. Metric: task completion accuracy (%).
• Terminal-Bench Pro (Wang et al., 2025): 200 public tasks across 8 domains with ~28.3 tests per task. Metric: task completion accuracy (%).

Base Models. We train models at three scales:

• Qwen3-8B (8B parameters)
• Qwen3-14B (14B parameters)
• Qwen3-32B (32B parameters)

All models are initialized from the same Qwen3 base checkpoints used by Nemotron-Terminal for fair comparison.

Training Protocol.

• SFT Phase: We fine-tune on SkillTerminal-Corpus trajectories using standard next-token prediction with environment-token masking (loss computed only on assistant turns). Training uses 4 epochs, learning rate 1e-5 with cosine decay, batch size 128, max sequence length 32K tokens.
• Optional RL Phase: For our strongest models, we additionally apply GRPO with binary test-pass reward on a held-out set of 2,000 environments.

Baselines. We compare against:

1. Base Models: Qwen3-{8B,14B,32B} without terminal-specific training.
2. Nemotron-Terminal-{8B,14B,32B} (Pi et al., 2026): Trained on Terminal-Corpus with skill-based synthesis.
3. TermiGen-32B: Trained on TermiGen’s 3,500+ environments.
4. Skill-Flat: Our ablation baseline — same pipeline but replacing graph-guided generation with LLM-enumerated skills (mimicking Nemotron’s approach) while keeping all other components identical.
5. Random-Graph: Our ablation — random walks on the skill graph (ignoring edge types) to control for graph structure vs. typed relations.

5.2 Main Results

Table 1. Task completion accuracy (%) on Terminal-Bench 2.0 and Terminal-Bench Pro.

Model	Training Data	TB 2.0	TB Pro
Proprietary (for reference)
Forge Code + Gemini 3.1 Pro	—	78.4	—
Simple Codex + GPT-5.3	—	75.1	—
Open-Weight 8B
Qwen3-8B (base)	—	2.5	1.8
Nemotron-Terminal-8B	Terminal-Corpus	13.0	10.2
SkillSynth-8B	SkillTerminal-Corpus	15.7	12.8
Open-Weight 14B
Qwen3-14B (base)	—	4.0	2.9
Nemotron-Terminal-14B	Terminal-Corpus	20.2	16.1
SkillSynth-14B	SkillTerminal-Corpus	23.6	19.4
Open-Weight 32B
Qwen3-32B (base)	—	3.4	2.5
TermiGen-32B	TermiGen Envs	18.0	14.3
Nemotron-Terminal-32B	Terminal-Corpus	27.4	22.1
SkillSynth-32B (SFT only)	SkillTerminal-Corpus	31.2	25.8
SkillSynth-32B (SFT+RL)	SkillTerminal-Corpus	34.8	28.5

Key observations:

(1) Consistent improvements across scales. SkillSynth models outperform Nemotron-Terminal counterparts at every model size: +2.7% (8B), +3.4% (14B), +3.8% (32B) on Terminal-Bench 2.0, with larger gains at larger scale.

(2) Data efficiency. SkillTerminal-Corpus contains ~12K tasks versus Nemotron’s ~35K (approximately 3× smaller), yet achieves higher performance. This demonstrates that graph-guided quality surpasses brute-force quantity.

(3) RL further amplifies gains. Adding GRPO training with test-pass rewards provides an additional +3.6% on TB 2.0, suggesting that graph-structured training data provides a better initialization for RL.

(4) Larger gains on harder tasks. When stratifying TB 2.0 results by difficulty, SkillSynth-32B’s advantage over Nemotron-32B grows from +1.2% on easy tasks to +6.8% on hard tasks, confirming that compositional complexity in training data most benefits compositionally complex evaluation tasks.

5.3 Ablation Studies

Table 2. Ablation results on Terminal-Bench 2.0 (32B model, SFT only).

Configuration	TB 2.0	Δ vs Full	Avg Dep. Depth	Avg Comp. Width	Coverage
SkillSynth (Full)	31.2	—	2.4	3.1	96.1%
− Topology-Aware Filtering	29.1	−2.1	2.4	3.1	82.3%
− Cross-Cluster Bridges	29.8	−1.4	2.3	2.8	89.4%
− Composition-Tree Tasks	28.4	−2.8	2.4	1.4	91.2%
− Dependency-Chain Tasks	27.9	−3.3	1.1	3.0	90.8%
Skill-Flat (no graph)	26.8	−4.4	1.0	1.3	71.5%
Random-Graph (untyped walks)	28.1	−3.1	1.8	2.1	84.7%

Key findings:

(1) Dependency chains are the most impactful component (−3.3% when removed), followed by composition trees (−2.8%). This confirms that explicit relational structure is the primary driver of improvement.

(2) Typed relations matter. Random-Graph (which walks the graph but ignores edge types) achieves only 28.1%, significantly below the full system (31.2%), demonstrating that the semantics of relations (not merely graph connectivity) are essential.

(3) Skill-Flat is equivalent to the Nemotron paradigm. Our controlled reproduction of the LLM-enumerated skill approach (26.8%) closely matches Nemotron-Terminal-32B (27.4%), validating our experimental setup. The +4.4% gap between Skill-Flat and SkillSynth isolates the contribution of graph-guided generation.

(4) Topology-aware filtering provides +2.1%. Ensuring coverage across the skill matrix eliminates domain-level blind spots that otherwise degrade performance.

5.4 Analysis

5.4.1 Compositional Complexity Analysis

We measure the compositional complexity of generated tasks along two axes: dependency depth (length of the longest depend_on chain) and composition width (maximum compose_with fan-out).

Method	Mean Dep. Depth	Mean Comp. Width	% Multi-Tool Tasks	% Cross-Domain
Nemotron Skill-Based	1.0	1.3	34.2%	8.1%
SkillSynth	2.4	3.1	78.6%	31.4%
Ratio	2.4×	2.4×	2.3×	3.9×

SkillSynth produces tasks that are 2.3× more compositionally complex on average, with 3.9× more cross-domain tasks — precisely the categories underserved by flat skill enumeration.

5.4.2 Skill Coverage Analysis

We evaluate coverage by mapping each benchmark task to the nearest skill cluster in SkillNet and checking whether the training corpus contains tasks from that cluster.

Method	Clusters with ≥1 task	Clusters with ≥5 tasks	Coverage Score
Nemotron Skill-Based	89 / 153 (58.2%)	62 / 153 (40.5%)	58.2%
SkillSynth	147 / 153 (96.1%)	128 / 153 (83.7%)	96.1%

The topology-aware filtering in §3.4 ensures near-complete coverage, while LLM-enumerated skills leave 41.8% of clusters uncovered — many of which correspond to specialized domains (distributed computing, formal verification, scientific simulation) that appear in benchmark tasks.

5.4.3 Qualitative Examples

Example 1 (Dependency Chain, Level 3): “Set up a PostgreSQL database with custom configuration, create a Python Flask API that connects to it, configure nginx as a reverse proxy, and write a systemd service file to ensure the stack auto-starts on boot. Verify by curling the health endpoint.”

Graph path: linux_basics →[depend_on] postgres_admin →[depend_on] flask_development →[compose_with] nginx_config →[compose_with] systemd_management

This task requires 5 distinct skills in a strict dependency order — a pattern that Nemotron’s flat enumeration would be unlikely to produce.

Example 2 (Cross-Cluster Bridge, Level 4): “Adapt the parallel processing pipeline typically used in bioinformatics sequence alignment (using GNU parallel + BLAST) to perform distributed log analysis across 50 server log files, identifying correlated anomaly patterns across hosts.”

Bridge: bioinformatics_alignment ←[similar_to]→ distributed_log_analysis

This task leverages a structural similarity between two distant domains, testing the model’s ability to transfer patterns — a capability that cannot emerge from single-domain skill enumeration.

5.4.4 Training Dynamics

We observe that models trained on SkillSynth data converge faster in early training (first 2 epochs) but continue improving through epoch 4, while models trained on flat skill-based data plateau after epoch 2. We attribute this to the richer gradient signal from compositionally complex tasks, which provide more informative error feedback per training step.

---

6 Discussion

6.1 Why Graph Structure Helps

Our results demonstrate that the structure of a real skill graph provides three distinct advantages for data synthesis:

1. Distributional fidelity: Real skills reflect the actual capability landscape, including the long tail of specialized tools and domains that LLMs under-represent in their generated taxonomies.

2. Compositional scaffolding: Explicit relation types (depend_on, compose_with) provide principled mechanisms for generating multi-step tasks, avoiding the “flat task” problem of enumeration-based approaches.

3. Coverage guarantees: The graph’s cluster structure enables systematic coverage monitoring, ensuring that no important skill domain is neglected — a property impossible to enforce with unstructured generation.

6.2 Scalability and Generalization

SkillSynth is not limited to terminal agents. The framework generalizes to any domain where a structured skill graph is available. As SkillNet continues to grow (currently 200K+ total skills across 10 major categories), SkillSynth can be applied to web agents, coding agents, scientific computing agents, and other agentic domains by simply changing the domain filter in §3.1.

6.3 Relation to Curriculum Learning

The graph-derived difficulty score (§3.2.4) naturally supports curriculum learning. In preliminary experiments, training with difficulty-ordered curricula (easy → hard) provides a further +1.2% improvement over random ordering, though we leave systematic investigation of curriculum strategies to future work.

---

7 Limitations

We acknowledge several limitations of our work:

Dependency on SkillNet quality. SkillSynth’s effectiveness is bounded by the quality and coverage of the underlying skill graph. While SkillNet is the largest available skill infrastructure (150K+ curated skills), it may still under-represent highly specialized or emerging technical domains. As SkillNet evolves, SkillSynth’s coverage will improve correspondingly.

Environment construction cost. The Docker environment generation step (§3.3.1) is the most expensive component, with approximately 10.8% of generated environments failing to build. While our template library mitigates this, fully automated environment construction for arbitrary skill combinations remains an open challenge.

Evaluation scope. We evaluate on Terminal-Bench 2.0 and Terminal-Bench Pro, which are English-only and primarily Linux-focused. Performance on other operating systems (Windows, macOS) and in multilingual settings remains unexplored.

Graph traversal heuristics. Our graph-walking algorithms (§3.2) use hand-designed traversal strategies. Learning optimal traversal policies (e.g., via reinforcement learning on the graph) could further improve task quality but is left to future work.

Potential for reward hacking in RL. While we show that SkillSynth data provides a better starting point for GRPO training, the RL phase may still be susceptible to reward hacking on the test-pass signal. Incorporating richer reward signals from the skill graph (e.g., skill-aware reward models) is a promising direction.

---

8 Conclusion

We have presented SkillSynth, a data synthesis framework that leverages the topology of a real-world skill knowledge graph to generate high-quality training data for terminal agents. By grounding the synthesis process in SkillNet — an infrastructure containing 150K+ curated skills with rich relational structure — SkillSynth addresses three fundamental limitations of existing LLM-enumerated skill taxonomies: artificial skill definitions, insufficient compositional complexity, and coverage gaps.

Our four-stage pipeline — skill subgraph sampling, graph-guided task generation, environment-grounded trajectory synthesis, and topology-aware quality filtering — translates graph topology into controllable task complexity through principled algorithms. The resulting SkillTerminal-Corpus (12K tasks, 52K+ trajectories, 15 domains, 420+ tools) enables open-weight models to achieve new state-of-the-art performance: SkillSynth-32B reaches 31.2% on Terminal-Bench 2.0 (surpassing Nemotron-Terminal-32B’s 27.4% despite 3× less data) and 34.8% with additional RL training.

Our work suggests a broader principle: when synthesizing training data for complex agentic tasks, grounding the generation process in real-world knowledge structures yields fundamentally better results than relying on the LLM’s own knowledge to define the task space. As skill knowledge graphs grow in scale and coverage, we anticipate that graph-guided data synthesis will become an essential component of the agent training pipeline.

---

Reproducibility Statement

We release the following resources to support reproducibility:

• SkillTerminal-Corpus: The full dataset of 12,247 verified tasks with 52,680 trajectories, including all skill annotations and graph provenance metadata.
• SkillSynth Pipeline: The complete data generation codebase, including graph sampling, task generation, environment construction, and quality filtering modules.
• Trained Models: SkillSynth-{8B, 14B, 32B} model weights for both SFT-only and SFT+RL variants.
• Evaluation Scripts: Configurations for reproducing all Terminal-Bench 2.0 and Terminal-Bench Pro results via the Harbor framework.

All resources are available at: https://github.com/[anonymous]/SkillSynth (anonymized for review).

---

Ethics Statement

Our work involves synthesizing terminal commands that may include system administration operations (e.g., package installation, service management). All trajectory synthesis occurs within isolated Docker containers with no access to external networks or host systems. We include safety filters that flag and remove tasks involving explicitly destructive operations (e.g., rm -rf /, DROP DATABASE without safeguards). The SkillTerminal-Corpus is designed for research purposes; deployed agents trained on this data should incorporate additional safety mechanisms appropriate to their deployment context.

---

Acknowledgments

[To be added after review]

---

References

Agentica. (2025). DeepSWE: A 32B Software Engineering Agent Trained with Purely RL. HuggingFace: agentica-org/DeepSWE-Preview.

Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., de Melo, G., Gutierrez, C., … & Zimmermann, A. (2021). Knowledge graphs. ACM Computing Surveys, 54(4), 1–37.

Jain, N., Singh, J., et al. (2025). R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents. COLM 2025. arXiv:2504.07164.

Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? ICLR 2024. arXiv:2310.06770.

Liang, Y., Zhong, R., Xu, H., Jiang, C., et al. (2026). SkillNet: Create, Evaluate, and Connect AI Skills. arXiv:2603.04448.

Merrill, M. A., et al. (2026). Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. arXiv:2601.11868.

OpenAI. (2024). Introducing SWE-bench Verified. OpenAI Blog.

Pan, J., Wang, X., et al. (2025). Training Software Engineering Agents and Verifiers with SWE-Gym. ICML 2025. arXiv:2412.21139.

Pi, R., Lam, G., Shoeybi, M., Jannaty, P., Catanzaro, B., & Ping, W. (2026). On Data Engineering for Scaling LLM Terminal Capabilities. arXiv:2602.21193.

Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS 2023.

Song, H., Sun, S., et al. (2026). SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training. arXiv:2602.03411.

Sun, S., Song, H., et al. (2026). SWE-World: Building Software Engineering Agents in Docker-Free Environments. arXiv:2602.03419.

Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291.

Wang, W., et al. (2025). Let It Flow: Agentic Crafting on Rock and Roll. arXiv:2512.24873.

Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023b). Self-Instruct: Aligning Language Models with Self-Generated Instructions. ACL 2023.

Wang, Z., et al. (2026). Multi-Agent Trajectory Synthesis via Reflective Internalization. arXiv:2602.23716.

Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., & Jiang, D. (2024). WizardLM: Empowering Large Language Models to Follow Complex Instructions. ICLR 2024.

Yang, J., et al. (2025). SWE-smith: Scaling Data for Software Engineering Agents. arXiv:2504.21798.

Zan, D., et al. (2025). Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving. arXiv:2504.02605.

Zhang, Y., et al. (2025). SWE-bench Goes Live! NeurIPS 2025 D&B. arXiv:2505.23419.

Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2024). ExpeL: LLM Agents Are Experiential Learners. AAAI 2024.

---

Appendix

A Graph Traversal Details

A.1 Dependency-Chain Sampling Parameters

Parameter	Value	Justification
Max chain depth	6	95th percentile of SkillNet depend_on paths
Min chain depth	1	Atomic tasks for curriculum warm-up
Sampling temperature	0.8	Balance between diversity and coherence
Domain constraint	Required	At least one skill must match domain filter

A.2 Composition-Tree Sampling Parameters

Parameter	Value	Justification
Max breadth	5	Practical limit on tool orchestration
Max depth	3	Avoid overly nested compositions
Required edges	≥ 2 compose_with	Ensure genuine composition

A.3 Cross-Cluster Bridge Parameters

Parameter	Value	Justification
Min cluster distance	3 hops	Ensure non-trivial transfer
Max bridge count per cluster pair	5	Prevent over-representation
Analogy quality threshold	0.7	LLM judge score for analogy coherence

B Environment Template Library

We maintain 217 reusable Docker environment templates organized by skill cluster:

Category	Templates	Base Images
Python Development	42	python:3.11-slim, python:3.12
System Administration	38	ubuntu:22.04, debian:12
DevOps & Containers	31	docker:dind, k3s
Database	24	postgres:16, mysql:8, redis:7
Scientific Computing	22	jupyter/scipy-notebook
Networking & Security	19	kalilinux/kali-rolling
Web Services	18	node:20, nginx:latest
Compilation & Build	15	gcc:13, rust:1.77
Others	8	Various

C Prompt Templates

C.1 Chain Task Generation Template

You are an expert terminal task designer. Given the following ordered chain of
skills with dependency relationships, design a realistic terminal task that
requires exercising all skills in the specified order.

## Skill Chain (must be used in this order)
{skill_chain_with_descriptions}

## Dependency Rationale
{dependency_explanations}

## Task Design Requirements
1. The task must be solvable entirely through terminal commands
2. Use absolute file paths (e.g., /root/data/, /root/output/)
3. Describe WHAT to achieve, not HOW to achieve it
4. Include specific, verifiable success criteria
5. Difficulty level: {target_difficulty}
6. Estimated completion time: {estimated_time}

## Output Format
Provide: instruction.md, Dockerfile, test.py, solve.sh

C.2 Composition Task Generation Template

You are an expert terminal task designer. Given the following skill composition
tree where multiple tools must work together with data flowing between them,
design a realistic multi-tool terminal task.

## Skill Composition Tree
{skill_tree_visualization}

## Data Flow Pattern
{dataflow_description}

## Task Design Requirements
[Same as C.1, with additional requirement:]
7. The task MUST require outputs from one tool to serve as inputs to another

## Output Format
Provide: instruction.md, Dockerfile, test.py, solve.sh

D Full Benchmark Results by Domain

Table A1. Terminal-Bench Pro results broken down by domain (SkillSynth-32B SFT+RL).

Domain	# Tasks	SkillSynth-32B	Nemotron-32B	Δ
Data Processing	25	40.0	32.0	+8.0
System Admin	25	36.0	28.0	+8.0
Software Engineering	25	32.0	28.0	+4.0
Machine Learning	25	28.0	20.0	+8.0
Scientific Computing	25	24.0	16.0	+8.0
Debugging	25	28.0	24.0	+4.0
Security	25	20.0	16.0	+4.0
Games	25	20.0	12.0	+8.0
Overall	200	28.5	22.1	+6.4

The largest gains appear in domains that benefit most from compositional and cross-domain training: Machine Learning (+8.0), Scientific Computing (+8.0), and Games (+8.0) — all of which require orchestrating multiple specialized tools.

E SkillNet API Usage

We access SkillNet through its public Python SDK:

from skillnet import SkillNetClient

client = SkillNetClient()

# Search for terminal-relevant skills
results = client.search("kubernetes pod debugging", top_k=50)

# Get skill relations
relations = client.get_relations(skill_id="k8s-pod-debug", 
                                 relation_types=["depend_on", "compose_with"])

# Download skill details
skill = client.download(skill_id="k8s-pod-debug")
print(skill.description)
print(skill.instructions)
print(skill.metadata)