Checks / pattern
Linear → Linear with no activation
Check R20. Runs in the editor as you build, in CI through the
GitHub Action, and over the wire at
POST /api/v1/check. Milliseconds, before any GPU is billed.
info
pattern
R20
| Trigger | A linear layer connects directly into another linear layer with no activation between them. |
|---|---|
| Why | Two stacked linear maps collapse into one (W₂·W₁), so the extra layer costs parameters but adds no representational power, usually a forgotten ReLU. Deliberate low-rank / factorized projections (down-proj → up-proj) are the safe exception. |
| Source | Linear-map composition closure; the rank of the product cannot exceed the smaller intermediate dimension. |
Why it is not a lint you can ignore
A structural mistake does not fail at review time and it does not fail at import
time. It fails when the module is constructed on the training node, after the job was queued and
the dataset was downloaded. That is why this runs before the spend and not after it.
Run this check on your own model
Free, no account needed
Every check
41 structural checks: 6 guardrail gates and 35 architecture advisor rules. See the full catalogue.
← R19 Back-to-back activations · R21 Dropout immediately before Output →