arch-design-sft: verified architecture-design SFT data

Name: arch-design-sft: verified architecture-design SFT data
Creator: Neurarch
License: https://opensource.org/licenses/MIT

Supervised fine-tuning data for neural architecture design treated as structured graph editing. Each row pairs a natural-language design spec and a serialized starting graph with a reference action plan, and every row is re-graded by a deterministic verifier before it is written: structural blockers, parameter budgets and bands, required layer families. Ten task families across six design-from-spec and four edit-in-place, where wholesale rebuilds are forbidden so the data teaches surgical edits rather than generation. Tasks are procedurally minted from a seed, so an evaluation split on a different seed has never existed anywhere.

3,010 verified examples MIT Free to use Source repo On Hugging Face

Get it

Chat format (TRL SFTTrainer drop-in)	raw.githubusercontent.com/neurarch-ai/neurarch-arch-bench/main/training/sft-3k.chat.jsonl application/x-ndjson
Raw fields, for custom formatting	raw.githubusercontent.com/neurarch-ai/neurarch-arch-bench/main/training/sft-3k.raw.jsonl application/x-ndjson
Hugging Face mirror (dataset viewer, load_dataset)	huggingface.co/datasets/neurarch-ai/arch-design-sft text/html

curl -sO https://raw.githubusercontent.com/neurarch-ai/neurarch-arch-bench/main/training/sft-3k.chat.jsonl
# or mint a private split of the same distribution:
git clone https://github.com/neurarch-ai/neurarch-arch-bench && cd neurarch-arch-bench/training
node build_sft_dataset.mjs --count=10000 --seed=20260704 --out=arch-design-sft

What is in a row

task family seed design spec starting graph reference action plan verifier grade

What this dataset is not

The targets are machine-checked, not human-preferred. Every assistant turn provably satisfies its task constraints, and nothing in the pipeline claims it is the design an experienced engineer would have chosen. Procedural generation is what makes the split contamination-free and is also what makes it narrower than real design work.

Licence and citation

Released under MIT License. Cite it as:

Neurarch. arch-design-sft: verified architecture-design SFT data. https://neurarch.com/d/arch-design-sft.html

The rest of the set

Neurarch architecture corpus

36 neural network architectures kept as typed graphs rather than diagrams.

36 architectures · CC0-1.0

Neurarch structural check catalogue

The 41 structural checks Neurarch runs on a model graph, as data: id, severity, category, the condition that triggers it, why it costs something, and the fix.

41 checks · CC-BY-4.0

Verifier grounding study (264 graphs)

Clean reference architectures plus systematically corrupted variants (broken attention head divisibility, linear width mismatches, severed connections), each built as a real PyTorch model and run on a GPU.

264 graphs · MIT

Arch-Bench arena results

Frontier language models scored on design-from-spec tasks by a deterministic verifier rather than a human or an LLM judge.

18 model-split results · CC-BY-4.0

Arch-Bench task set

The task definitions behind the benchmark: design-from-spec and repair-and-extend instances for agents that build neural network architectures.

12 curated tasks, 8 fixtures · MIT

Structure, verdict and trained outcome triples

The corpus that pairs what a design looks like with what it did.

80 trained graphs · MIT

Verified architecture-design reasoning traces (Claude)

Spec to reasoning to design triples where the design is re-graded by the same deterministic verifier the benchmark uses, and only passing traces are kept.

306 verified traces · MIT

Verified architecture-design reasoning traces (Grok)

The same verified spec to reasoning to design triples as the Claude split, rejection-sampled from a different frontier model, so the two can be pooled for volume or held apart to see how much of the reasoning style is model-specific.

376 verified traces · MIT