Swap a model or checkpoint, a fine-tune or RL recipe, a parallel layout, or a serving config. Before the change takes a GPU or serves traffic, and without running the model, Neurarch answers four questions:
Free to start · no account for the check · derived from the config and calibrated on measured runs, never a claim about accuracy
Train DocLM on 2,048-token windows #11
seq_len: 1024 -> 2048 is one line in a pull request. The batch stays at 32 and the job
still reserves one A10G 24 GB, so on the face of it nothing about the run changes, and the model still runs.
Activations scale with the tokens in a batch. The training peak goes from 20.6 GB to 38.4 GB, +17.8 GB, and with the 1.15x allocator headroom the job needs 44.2 GB of a 24 GB card. None of that is in the diff, and none of it is in review. On the cluster it is an out-of-memory error after the reservation is taken, or a bigger card requested to be safe.
The plan names the change that keeps the run the same: batch 16 is the largest that fits, and 2 gradient-accumulation steps keep the effective batch at 32. The second push does exactly that and the comment goes green, on the card the job already reserves.
Every terminal here is what the command printed, captured by a script and committed. Nothing is typed by hand.
A public model cut its context under everyone who pulls main. Not rare: of the 406 most-downloaded Hugging Face models with a config, 23% changed a structural key after release (Neurarch, from each repository's config.json history, read 2026-10-17).
Both keys are ones the graph reads (max_position_embeddings, rope_scaling), so this is a structural change, not metadata. A job that pulls main picks it up with no change in its own repository, and nothing errors until the next launch. The row is the tracker's own (it reads every tracked repository's history daily); the diff is the Hub's commit, fetched at both revisions. The CLI does not diff two revisions; under them, what it says about a vLLM launch still set to the old length:
$ npx neurarch@0.3.0 serve-check Qwen/Qwen2.5-7B-Instruct --gpu h100 -- vllm serve Qwen/Qwen2.5-7B-Instruct --max-model-len 131072 Qwen/Qwen2.5-7B-Instruct on vLLM TP 1 x PP 1 x DP 1, 1 x h100-sxm (sm90), dtype bf16, weights unquantized, KV cache auto served length 131072 (--max-model-len 131072); the model's derived maximum 32768 (the engine version and the defaults it assumed, and memory per GPU, then:) block Served length 131072 is past the model's 32768-token context The engine refuses to start ("User-specified max_model_len (131072) is greater than the derived max_model_len (32768)") unless VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 is set, and with it set, positions past the trained length are extrapolated. fix: Serve at most 32768, or add rope scaling where the model card supports it, with the card's own original length (e.g. --hf-overrides '{"rope_scaling":{"rope_type":"yarn","factor":F,"original_max_position_embeddings":N}}', with N x F >= 131072; N is the native length the card states, which can be below max_position_embeddings). (two info lines, then:) checked: gpu-count, tp-heads, tp-kv-heads, context, quant-support, weights-vs-checkpoint, memory, memory-fit, chat-template, samplerexit 1
The tracker behind that row follows 471 open-weight models; 410 have a readable history, and it has recorded 586 changes to their config, tokenizer, chat template and generation defaults. A chat template edit is rendered both ways on three fixed conversations: 125 changed the text the model is sent, 40 were cosmetic. It also reads the licence on each card: the licence id itself switched 41 times, across 35 models. The same record as Renovate, OSV and badge feeds →
Real output of neurarch 0.3.0, the release on npm, captured 2026-10-19. config.json before the commit → Every tracked change →
The pull request at the top is this answer on a real change. The memory number behind it, against the GPU and against accelerate: three AdamW steps per run on Modal, torch.cuda.max_memory_allocated as the truth, some runs sized to land near the card on purpose.
Every run that did not run out of memory, 148 points per method
131 times a measured run did not fit a 16, 24, 40 or 80 GB card. How often each method said so before launch:
203 measured runs, 55 of them out of memory (no peak to plot, so they live in the bars). Hover a point for the run. Every row, and how the six factors are fitted →
Qwen3-0.6B in four pipeline stages on four L4s, blocks split evenly. Every divisibility check passes; the last stage carries the LM head and the loss and sets the pace for the other three. The check names a split, and we trained both.
$ npx neurarch@0.3.0 layout Qwen/Qwen3-0.6B --pp 4 --micro 1 --global-batch 8 --seq-len 2048 --gpu l4Parallel layout check: Qwen/Qwen3-0.6B 4 GPUs (l4, 24 GB), 28 blocks, schedule 1f1b, ZeRO 0, 8 microbatches/step, global batch 8 stage blocks extra params FLOPs/tok ms/mb* mem-bound vs mean act/mb in-flight peak/GPU upper fits 0 1-7 (7) embedding 266M 837.1 MFLOP 76 39% 0.77x 0.7 GB 4 7.8 GB 11.2 GB fits 1 8-14 (7) - 110M 837.1 MFLOP 75 38% 0.76x 0.7 GB 3 4.1 GB 5.9 GB fits 2 15-21 (7) - 110M 837.1 MFLOP 75 38% 0.76x 0.7 GB 2 3.5 GB 4.9 GB fits 3 22-28 (7) head+loss 266M 1.77 GFLOP 168 48% 1.70x 0.7 GB 1 9.5 GB 13.6 GB fits Balance: slowest stage 3 at 1.70x the mean stage's predicted time. Predicted step at 8 microbatches (1F1B schedule simulated op by op, a port of Primus's simulator): 1566 ms vs 1082 ms balanced, 44.7% slower than a perfect split (70.3% as microbatches grow). Rebalanced split 9,9,9,1: 1.05x, predicted 40.2% faster per step at 8 microbatches (1117 ms); 62.1% as microbatches grow. Bubble: 27.3% at 8 microbatches/step (17+ for under 15.0%). warn Stage 3 has 70.3% more predicted time than the mean stage (it also carries the LM head and the loss) warn Pipeline bubble is 27.3% of each step info Tied LM head costed on the last stage (detail and fix under each finding, then:) checked: tp-heads, tp-kv-heads, tp-hidden, tp-ffn, tp-node, cp-seq, batch-arithmetic, pp-over-blocks, stage-balance, pipeline-bubble, stage-memory
Forward and backward per microbatch, each stage on its own L4: what the check predicts, and what the A/B measured on the same split.
Measured, not predicted: we trained this exact job (Qwen3-0.6B, 4 L4s, 8 microbatches, 1F1B) at both splits, twice each in separate containers. The step took 1,478 / 1,451 ms at 7,7,7,7 and 1,092 / 1,079 ms at 9,9,9,1: 34.4% and 35.3% faster. On 2 L4s, the split it picks there (18,10) ran 14.4% and 14.8% faster than the even one. One model, one card type, small scale: it says the recommendation held where we timed it, not that it holds on your cluster.
Real output of neurarch 0.3.0, the release on npm, captured 2026-10-19. Why the last stage: in 39.3% of LLM training jobs in ByteDance's straggler study, the last pipeline stage (LM head and loss) caused over half of the slowdown (OSDI 2025, arXiv 2505.05713).
The serving launch against the model it serves (context, tensor parallel against heads, KV cache against the batch it promises, chat template), and the training config against the data it will be fed. Read from the config and the flags, before anything is reserved.
$ npx neurarch@0.3.0 serve-check Qwen/Qwen3-8B --gpu l4 --gpus 4 -- vllm serve Qwen/Qwen3-8B --tensor-parallel-size 4 --max-model-len 65536 --max-num-seqs 256 Qwen/Qwen3-8B on vLLM TP 4 x PP 1 x DP 1, 4 x l4 (sm89), dtype bf16, weights unquantized, KV cache auto served length 65536 (--max-model-len 65536); the model's derived maximum 40960 (the engine version and the defaults it assumed, then:) memory per GPU, predicted: 22.04 GiB x 0.92 = 20.28 GiB - weights 3.81 GiB - activation peak 0.58 GiB (estimate; vLLM measures this) - non-torch (NCCL, CUDA) 0.27 GiB (estimate; vLLM measures this) - CUDA graphs 0.53 GiB (estimate; vLLM measures this) = KV cache 15.08 GiB: 27,459 blocks of 16 tokens (36 full) 36,864 KV bytes per token over the full-length layers; one full-length request 2.25 GiB vLLM would print: GPU KV cache size: 439,344 tokens; Maximum concurrency for 65,536 tokens per request: 6.70x weights: 2 bytes per parameter (16-bit); within 0.0% of the checkpoint's 16.4 GB card total: derived: nvidia-smi 23034 MiB less the 468 MiB gap measured on H100 block Served length 65536 is past the model's 40960-token context The engine refuses to start ("User-specified max_model_len (65536) is greater than the derived max_model_len (40960)") unless VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 is set, and with it set, positions past the trained length are extrapolated. fix: Serve at most 40960, or add rope scaling where the model card supports it, with the card's own original length (e.g. --hf-overrides '{"rope_scaling":{"rope_type":"yarn","factor":F,"original_max_position_embeddings":N}}', with N x F >= 65536; N is the native length the card states, which can be below max_position_embeddings). warn --max-num-seqs 256, but at most 6 requests fit at the full 65536 tokens vLLM's pool would hold 27,459 blocks of 16 tokens; a full-length request takes 4,096. Shorter requests pack more, but once long ones arrive the scheduler preempts or queues past ~6. fix: Set the flag to what fits (6), shorten the max length, use an fp8 KV cache, or add GPUs as DP replicas. (two info lines (reasoning parser, pad equals EOS), then:) checked: gpu-count, tp-heads, tp-kv-heads, tp-ffn, context, quant-support, weights-vs-checkpoint, memory, memory-fit, chat-template, recipe, samplerexit 1 $ npx neurarch@0.3.0 bert-base-uncased --seq-len 1024 --tokenizer-size 30524 (parameter count, layers and structural checks, then:) data fit checked: sequence length, tokenizer size block Sequences of 1024 tokens are longer than the 512-row position table Position ids past 511 index outside a learned table, so the first batch that long raises an IndexError (a device-side assert on GPU) after the reservation is taken. fix: Truncate or chunk to 512 tokens, or pick a model whose context covers 1024. block The tokenizer has 30524 ids and the embedding has 30522 rows Any of the 2 ids past 30521 indexes outside the embedding. The usual cause is special tokens added to the tokenizer without resizing the model. fix: Call model.resize_token_embeddings(len(tokenizer)) before training, and budget for the new rows.exit 1
The first command is a vLLM launch read before it runs: the length is past what the model was trained for, so vLLM refuses to start, and the flag promises 256 sequences where the KV cache holds about 6 at full length. The second is a training config read against the data: two blocks, each of which crashes on the first batch after the reservation is taken. Both exit 1, so a CI step or an admission hook can stop on them.
-tp 8 on Qwen2.5-Omni-7B now blocks on its 28 attention heads.within 0.0% of the checkpoint's 16.4 GB). Past 10% apart it warns and sizes memory on the published bytes.a40, gh200-96g, gh200-144g, rtx-pro-6000 and ascend-910b (memory only), and --gpu-mem <GiB> for any card not in the table.Real output of neurarch 0.3.0, the release on npm, captured 2026-10-19. serve-check reads vLLM, SGLang and llama.cpp launches; it compares configuration, never outputs.
The same four answers at each point a change has to pass before it takes a GPU or serves traffic. Same code behind every one.
pip install neurarch-skypilot
admin_policy: neurarch_skypilot.NeurarchPolicy
SLURM cli_filter (PyPI), then in slurm.conf
pip install neurarch-slurm
CliFilterPlugins=cli_filter/lua
neurarch-slurm --print-lua writes the plugin. It runs in the submitting client, so it tells people; it is not an enforcement boundary.
github.com/apps/neurarch-bot
or a workflow step
uses: neurarch-ai/neurarch-bot@v0
Install the bot →
/plugin marketplace add neurarch-ai/neurarch-mcp
/plugin install neurarch@neurarch-ai
any MCP client
npx -y neurarch-mcp
MCP server →
npx neurarch Qwen/Qwen3-8B --batch 8
npx neurarch layout Qwen/Qwen3-8B --tp 4 --pp 4 --dp 8 --global-batch 512 --seq-len 4096
npx neurarch serve-check Qwen/Qwen3-8B --gpu l4 --gpus 4 -- vllm serve Qwen/Qwen3-8B -tp 4 --max-model-len 32768
Real output above →
data (sequence length, tokenizer size), layout (TP, PP, DP, batch) or serve (the launch command). Arithmetic, no model call.
the four answers
POST /api/v1/check
also
POST /api/v1/plan
POST /api/v1/lint
API reference →
docker run --rm --network none -p 8080:8080 neurarch/verifier:local
The image is not on a public registry yet: we send it, with a script that replaces every network call with one that throws and then answers a real request.
How it runs →
neurarch.com/changes
Model changes →
The same record for Renovate, OSV and badges ↓
Nothing to install: paste a model at /try.
The pre-check and the change record, in formats a dependency bot, a scanner, code scanning and an attestation store take as they are. No account.
.py, notebooks, YAML, shell and Dockerfiles for Hugging Face downloads with no commit pinned: from_pretrained, vllm serve, SGLang, CLI downloads, recipe keys. Aligned with Bandit's B615, plus the forms Bandit does not read.
neurarch 0.3.0
npx neurarch@0.3.0 pin-check .
Warns by default; pin/*: block in .neurarch.yml makes it a gate, --sarif feeds code scanning, --resolve writes the current sha into the fix. On 18 held-out public files: 0 missed, 2 wrong (both info level).
main moves, linking what changed key by key.
renovate.json
{"extends": ["https://www.neurarch.com/changes-feeds/renovate/preset.json"]}
What it matches →
curl -s https://www.neurarch.com/changes-feeds/advisories/x_NXMC-2025-0001.json
The index →
[](https://www.neurarch.com/changes-feeds/models/NousResearch/Hermes-3-Llama-3.1-8B.html)
--attest writes an in-toto statement with in-toto's own test-result predicate; --cyclonedx writes a CycloneDX 1.6 ML-BOM fragment with the same verdicts. The subjects are the model's files hashed the way OpenSSF model signing does it, and the root digest is byte for byte the one model_signing 1.1.1 signs, so a model signature and a pre-check record point at the same bytes. Deterministic: two runs write the same file.
on the main check, layout, serve-check and policy apply, neurarch 0.3.0
npx neurarch@0.3.0 serve-check Qwen/Qwen3-8B --gpu l4 --gpus 4 --attest precheck.intoto.json --cyclonedx precheck.cdx.json -- vllm serve Qwen/Qwen3-8B -tp 4 --max-model-len 32768
Sign it with actions/attest and gh attestation verify finds it. The record's schema →
What it needs, read from what you already have. No account, no cluster access: a CSV of what each job requested, or the error text. Real output, captured once.
$ npx neurarch rightsize jobs.sample.csv# GPU right-sizing: requested vs needed **6 jobs, 174 GPU-hours, $507.85 spent; $328.53 of that was reserved capacity no job needed, claiming nothing on 1 row marked under or unknown.** 4 over, 1 right-sized, 1 under, 0 unknown. (six rows, one per job: drawn below) $ npx neurarch oom error.txt --model FacebookAI/xlm-roberta-large --batch 1 --seq-len 128cause too-big-for-card The job is bigger than this card: the predicted peak is 10.8 GB (12.5 GB with headroom) on a 4.29 GB card. read from - the failing allocation asked for 0.02 GB - the card has 4.29 GB - 3.31 GB was held by live tensors - 0.06 GB was reserved by PyTorch but held no tensor - the plan predicts a 10.8 GB training peak at this batch fixes, least invasive first 1. shard the optimizer state and gradients across cards (FSDP or ZeRO), or use an 8-bit optimizer weights, gradients and optimizer state alone are 10.7 GB, more than half the card 2. train adapters (LoRA) on a frozen base instead of every weight (changes what is trained) the optimizer state then covers the adapters only
Real output of neurarch 0.2.0, captured 2026-10-18 on a 6-job sample CSV. "Reserved, not needed" is capacity at list price over the same hours, not a saving: a smaller card can run longer. Both commands, replayed, with the method →
pip install neurarch-trace
for your own nn.Module: trace a forward pass, get the same plan
"How much CUDA memory is needed to train a new NER tagger?" flairNLP/flair#2364, fine-tuning xlm-roberta-large on a NVIDIA GeForce GTX 1050 Ti (4 GB). The error, verbatim:
RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 4.00 GiB total capacity; 3.08 GiB already allocated; 0 bytes free; 3.14 GiB reserved in total by PyTorch)
too-big-for-card The job is bigger than this card: the predicted peak is 10.8 GB (12.5 GB with headroom) on a 4.29 GB card.
Weights, gradients and AdamW state alone are 10.7 GB on a 4.29 GB card, so no batch size makes this fit. The answer to the question in the issue is a card with at least 12.5 GB, or one of the two fixes the terminal above lists: shard the optimizer state, or train adapters on a frozen base.
The sequence length is the one input assumed (128 tokens; the issue does not state it). What the issue stated, what we assumed, and the run at 512 tokens →
Every answer above is arithmetic from the config. Three places we put it next to the real thing, misses included.
We reconstruct 234 checkpoints people actually download from their config.json alone, no weights read,
and put our parameter count next to the scalars actually in the published weight files.
65 of 81 comparable checkpoints are reproduced within 2% from config alone; the median miss is 0.3%.
| Checkpoint | We derive | In the weights | Delta |
|---|---|---|---|
| Qwen2.5-7B-Instruct | 7.62B | 7.62B | +0.00% |
| t5-small | 60.6M | 60.5M | +0.11% |
| roberta-base | 124.0M | 124.7M | -0.53% |
| vit-base-patch16-224 | 85.8M | 86.6M | -0.89% |
| distilbert-base-uncased | 66.4M | 67.0M | -0.95% |
203 measured training runs on Modal, 55 of them out of memory. Held out by architecture, the median error on runs of 1 GB or more is 6.9%; on the 25 runs measured after the fit, near the edge of a 24 GB card, it is 14.0%. Of the 131 times a run did not fit its card, the estimate said so 113 times; accelerate estimate-memory said so 8 times. The chart · every row →
The last pipeline stage (LM head and loss) against one decoder block, each timed alone on one GPU: measured over what the check predicted. 1.00 is exact.
Closer on every L4 row (worst miss 28.0% before, 16.0% after), and further on both H100 rows (lowest 0.72 before, 0.69 after): on H100 the large head matmul runs well above the 40% of peak the model assumes for long matmuls, so it prices the head too heavy there. Random weights built from each config, 5 runs; the direction, that the head stage is the heavy one, held on every run.
Qwen3-0.6B trained with 1F1B pipeline parallelism on 2 and 4 L4s: how much faster each split's step was than the even split, measured, two runs each.
2 L4, 8 microbatches, against 14,14
2 L4, 16 microbatches, against 14,14
4 L4, 8 microbatches, against 7,7,7,7
4 L4, 16 microbatches, against 7,7,7,7
The recommended split was faster than the even one in every run. The check picks 18,10 on 2 L4s, as it did when these ran. Setup: Qwen3-0.6B built from its config with random weights, bf16, AdamW, torch.distributed.pipelining with 1F1B, sequence 2048, micro batch 1, median of 15 steps after 5 warmup, each split run twice in separate containers. Everything here cost about $1.43 of L4 time. One small model on one card type: it says the recommendation held where we timed it, not that it holds on your cluster.
Priced per managed GPU: a GPU that passed through the admission hook that month. Blocking is never the paywall: the CLI, the PR bot and the hook block on the free tier too. Early-access rates lock in when you subscribe.
cli_filter or SkyPilot admin policyPer managed GPU, billed annually: $180 per GPU a year, minimum 32 GPUs ($5,760 a year). Teams that only use pull requests can pay $20 per active committer a month instead.
Talk to us →Per managed GPU, billed annually: $300 per GPU a year.
Book a call →Neurarch app (design canvas and agent). Free with 15 agent messages a day. Pro, $19/mo: 500 agent messages a month, Hugging Face, arXiv and code import, notebook and model-card export. Pro Plus, $39/mo: 2,000 agent and 200 Claude messages, workspaces, live training dashboard. Existing subscriptions are unchanged.
Where the memory number is weak, from the same table:
An earlier study ranked trained model designs by our static checks; it did not hold up, and we no longer sell it. The study and why it was retired →
.onnx or .safetensors, and export to PyTorch, Keras, ONNX or a training notebook.localStorage, goes straight to the provider, never reaches our servers, and is wiped on sign-out or an account switch. The hosted agent is proxied for signed-in users so per-plan quotas can be enforced; that is the only path where a prompt touches our infrastructure.Does it run the model? No. It reads the config, the launch flags and the graph built from them, and every number is arithmetic calibrated on measured runs. That is why it can answer before a GPU is reserved, and why it says nothing about accuracy.
Does it tell me whether the outputs match the reference? No. It compares configuration: context length, heads against tensor parallel, KV cache against the batch a flag promises, the chat template. Whether outputs match is the job of the model makers' own verifiers.
How is the memory number different from accelerate estimate-memory? accelerate counts the bytes of the parameters and no activations, so on a job near the edge of a card it says "fits" when the run will not. The chart under What it needs puts both against the same measured runs.
What do I point it at? A Hugging Face id, a config.json, a launch command,
a CSV of past jobs, an out-of-memory error, or your own nn.Module traced with
neurarch-trace.
Can I cancel anytime? Yes, from the billing portal. Access runs to the end of the period, and there is a full refund within 7 days of a first payment.
One chapter per answer, each one real output: a context cut from the change feed and the launch it breaks, a training job sized per card and the pull request comment that went red and then green, a pipeline split with the measured A/B behind it, and a vLLM launch and a BERT fine-tune checked against their configs. Then the same checks as a Claude Code hook, a SkyPilot policy, an MCP tool and an API call. Every terminal is what the command printed.
The CLI, the bot, the MCP server and the checks are free, and so is the admission hook under 8 GPUs.
No spam. Answer the question and a person replies. Or just open the app now →
Neurarch plan ·
models/doc_lm.py:DocLMno longer fits A10G 24 GB
Peak training memory: 38.4 GB at batch 32 x 2,048 tokens (weights and optimizer state 2.7 GB, activations 35.7 GB).
fits A10G 24 GB
Peak training memory: 20.6 GB at batch 16 x 2,048 tokens, 2 accumulation steps for an effective batch of 32 (weights and optimizer state 2.7 GB, activations 17.8 GB).
needs 44.2 GB with headroomneeds 23.7 GB with headroom
Structural checks. Will it run: yes. 113 layers, 135.8M params.