N Neurarch Architectures Checks Data Docs Open the app

Resources / Datasets

Verified architecture-design reasoning traces (Claude)

Spec to reasoning to design triples where the design is re-graded by the same deterministic verifier the benchmark uses, and only passing traces are kept. No LLM judge anywhere. Each row carries the natural-language spec, the starting graph, step-by-step reasoning over the constraints (required layers, shapes, head divisibility, parameter budgets), the structured edits that produce the design, and a chat-format rendering with the reasoning inside reasoning tags. This is verified reasoning data of the shape RLVR and reasoning-model post-training consume, for a domain no code-only corpus contains.

306 verified traces MIT Free to use On Hugging Face

Get it

Hugging Face mirror (dataset viewer, load_dataset)huggingface.co/datasets/neurarch-ai/arch-reasoning-claude
text/html
 
python -c "from datasets import load_dataset; d = load_dataset('neurarch-ai/arch-reasoning-claude'); print(d)"

What is in a row

task id design spec starting graph reasoning actions verified source model

What this dataset is not

Rejection-sampled, so the distribution is what one model got right rather than what the task space looks like: every kept trace passed, and the ones that failed are not here, which makes this useless for measuring how hard the tasks are. The reasoning is the model's own text and was never checked for whether it describes what the actions actually do; only the final graph was verified.

Licence and citation

Released under MIT License. Cite it as:

Neurarch. Verified architecture-design reasoning traces (Claude). https://neurarch.com/d/reasoning-claude.html

The rest of the set

Neurarch architecture corpus
36 neural network architectures kept as typed graphs rather than diagrams.
36 architectures · CC0-1.0
Neurarch structural check catalogue
The 41 structural checks Neurarch runs on a model graph, as data: id, severity, category, the condition that triggers it, why it costs something, and the fix.
41 checks · CC-BY-4.0
Verifier grounding study (264 graphs)
Clean reference architectures plus systematically corrupted variants (broken attention head divisibility, linear width mismatches, severed connections), each built as a real PyTorch model and run on a GPU.
264 graphs · MIT
Arch-Bench arena results
Frontier language models scored on design-from-spec tasks by a deterministic verifier rather than a human or an LLM judge.
18 model-split results · CC-BY-4.0
Arch-Bench task set
The task definitions behind the benchmark: design-from-spec and repair-and-extend instances for agents that build neural network architectures.
12 curated tasks, 8 fixtures · MIT
arch-design-sft: verified architecture-design SFT data
Supervised fine-tuning data for neural architecture design treated as structured graph editing.
3,010 verified examples · MIT
Structure, verdict and trained outcome triples
The corpus that pairs what a design looks like with what it did.
80 trained graphs · MIT
Verified architecture-design reasoning traces (Grok)
The same verified spec to reasoning to design triples as the Claude split, rejection-sampled from a different frontier model, so the two can be pooled for volume or held apart to see how much of the reasoning style is model-specific.
376 verified traces · MIT