Models / resnet

resnet-50: GPU memory to fine-tune

Fine-tuning resnet-50 at batch 1 takes about 0.56 GB of GPU memory for a full fine-tune (fits one T4 16 GB). Estimated from the model's structure, not measured. What fits on which card.

Reconstructed from its own config.json with no weights read. 1.6M downloads on Hugging Face.

Our count against the checkpoint

The left number comes from the graph. The right one is the number of scalars in the published weight files. Nothing on this page was tuned to make them agree.

Derived from structure

25.5M

25,533,314 parameters

In the published checkpoint

25.6M

25,610,152 scalars · safetensors.total, read 2026-09-06

Delta

-0.30%

What it costs to run

Cost is a roofline estimate on the priced GPU for 10 epochs at batch 32 over 50,000 samples (assumed; no dataset attached).

Layers

175

Will it forward-pass

Yes

Priced on

A10G (24GB)

Est. one run

$0.20

Memory: what fits on which card

Largest batch per card. Training is sized at the input shape below per sample.

Full fine-tune, batch 1

0.56 GB

LoRA r=16, batch 1

not sized

Card	Full fine-tune	LoRA r=16
T4 16 GB	batch 193	not sized
L4 24 GB	batch 293	not sized
A100 40 GB	batch 494	not sized
A100 / H100 80 GB	batch 996	not sized
H200 141 GB	batch 1760	not sized

Training: the input shape in the structure table, AdamW with bf16 autocast, SDPA attention (no score matrix stored), 1.15x headroom on every card. Full fine-tune holds weights, gradients and optimizer state (1.2 x 16 bytes per parameter) plus activations. LoRA is r=16 on every linear layer but the head, base frozen in bf16 (2 bytes per parameter), adapters at full AdamW state, same activations. A two-card split is a contiguous layer cut (pipeline parallel), computed only where one card cannot hold batch 1.

These are estimates from the graph, not measurements. Against measured runs, held out by architecture, the estimate flagged 113 of the 131 jobs that did not fit their card and said "fits" on the other 18. Every measured run.

Structure

177 nodes. Output shapes are propagated from the input shape, batch dimension excluded.

	Layer	Type	Output shape
1	Input	Input	3 × 224 × 224
2	Stem_Conv	Conv2D	64 × 112 × 112
3	Stem_BN	BatchNorm	64 × 112 × 112
4	Stem_ReLU	ReLU	64 × 112 × 112
5	Stem_MaxPool	MaxPool2D	64 × 56 × 56
6	Stage1_Block1_Conv1x1_reduce	Conv2D	64 × 56 × 56
7	Stage1_Block1_BN1	BatchNorm	64 × 56 × 56
8	Stage1_Block1_ReLU1	ReLU	64 × 56 × 56
9	Stage1_Block1_Conv3x3	Conv2D	64 × 56 × 56
10	Stage1_Block1_BN2	BatchNorm	64 × 56 × 56
11	Stage1_Block1_ReLU2	ReLU	64 × 56 × 56
12	Stage1_Block1_Conv1x1_expand	Conv2D	256 × 56 × 56
13	Stage1_Block1_BN3	BatchNorm	256 × 56 × 56
14	Stage1_Block1_Downsample	Conv2D	256 × 56 × 56
15	Stage1_Block1_Downsample_BN	BatchNorm	256 × 56 × 56
16	Stage1_Block1_Add	Add	256 × 56 × 56
17	Stage1_Block1_ReLU_out	ReLU	256 × 56 × 56
18	Stage1_Block2_Conv1x1_reduce	Conv2D	64 × 56 × 56
19	Stage1_Block2_BN1	BatchNorm	64 × 56 × 56
20	Stage1_Block2_ReLU1	ReLU	64 × 56 × 56
21	Stage1_Block2_Conv3x3	Conv2D	64 × 56 × 56
22	Stage1_Block2_BN2	BatchNorm	64 × 56 × 56
23	Stage1_Block2_ReLU2	ReLU	64 × 56 × 56
24	Stage1_Block2_Conv1x1_expand	Conv2D	256 × 56 × 56
25	Stage1_Block2_BN3	BatchNorm	256 × 56 × 56
26	Stage1_Block2_Add	Add	256 × 56 × 56
27	Stage1_Block2_ReLU_out	ReLU	256 × 56 × 56
28	Stage1_Block3_Conv1x1_reduce	Conv2D	64 × 56 × 56
29	Stage1_Block3_BN1	BatchNorm	64 × 56 × 56
30	Stage1_Block3_ReLU1	ReLU	64 × 56 × 56
31	Stage1_Block3_Conv3x3	Conv2D	64 × 56 × 56
32	Stage1_Block3_BN2	BatchNorm	64 × 56 × 56
33	Stage1_Block3_ReLU2	ReLU	64 × 56 × 56
34	Stage1_Block3_Conv1x1_expand	Conv2D	256 × 56 × 56
35	Stage1_Block3_BN3	BatchNorm	256 × 56 × 56
36	Stage1_Block3_Add	Add	256 × 56 × 56
37	Stage1_Block3_ReLU_out	ReLU	256 × 56 × 56
38	Stage2_Block1_Conv1x1_reduce	Conv2D	128 × 56 × 56
39	Stage2_Block1_BN1	BatchNorm	128 × 56 × 56
40	Stage2_Block1_ReLU1	ReLU	128 × 56 × 56
41	Stage2_Block1_Conv3x3	Conv2D	128 × 28 × 28
42	Stage2_Block1_BN2	BatchNorm	128 × 28 × 28
43	Stage2_Block1_ReLU2	ReLU	128 × 28 × 28
44	Stage2_Block1_Conv1x1_expand	Conv2D	512 × 28 × 28
45	Stage2_Block1_BN3	BatchNorm	512 × 28 × 28
46	Stage2_Block1_Downsample	Conv2D	512 × 28 × 28
47	Stage2_Block1_Downsample_BN	BatchNorm	512 × 28 × 28
48	Stage2_Block1_Add	Add	512 × 28 × 28
49	Stage2_Block1_ReLU_out	ReLU	512 × 28 × 28
50	Stage2_Block2_Conv1x1_reduce	Conv2D	128 × 28 × 28
51	Stage2_Block2_BN1	BatchNorm	128 × 28 × 28
52	Stage2_Block2_ReLU1	ReLU	128 × 28 × 28
53	Stage2_Block2_Conv3x3	Conv2D	128 × 28 × 28
54	Stage2_Block2_BN2	BatchNorm	128 × 28 × 28
55	Stage2_Block2_ReLU2	ReLU	128 × 28 × 28
56	Stage2_Block2_Conv1x1_expand	Conv2D	512 × 28 × 28
57	Stage2_Block2_BN3	BatchNorm	512 × 28 × 28
58	Stage2_Block2_Add	Add	512 × 28 × 28
59	Stage2_Block2_ReLU_out	ReLU	512 × 28 × 28
60	Stage2_Block3_Conv1x1_reduce	Conv2D	128 × 28 × 28
61	Stage2_Block3_BN1	BatchNorm	128 × 28 × 28
62	Stage2_Block3_ReLU1	ReLU	128 × 28 × 28
63	Stage2_Block3_Conv3x3	Conv2D	128 × 28 × 28
64	Stage2_Block3_BN2	BatchNorm	128 × 28 × 28
65	Stage2_Block3_ReLU2	ReLU	128 × 28 × 28
66	Stage2_Block3_Conv1x1_expand	Conv2D	512 × 28 × 28
67	Stage2_Block3_BN3	BatchNorm	512 × 28 × 28
68	Stage2_Block3_Add	Add	512 × 28 × 28
69	Stage2_Block3_ReLU_out	ReLU	512 × 28 × 28
70	Stage2_Block4_Conv1x1_reduce	Conv2D	128 × 28 × 28
71	Stage2_Block4_BN1	BatchNorm	128 × 28 × 28
72	Stage2_Block4_ReLU1	ReLU	128 × 28 × 28
73	Stage2_Block4_Conv3x3	Conv2D	128 × 28 × 28
74	Stage2_Block4_BN2	BatchNorm	128 × 28 × 28
75	Stage2_Block4_ReLU2	ReLU	128 × 28 × 28
76	Stage2_Block4_Conv1x1_expand	Conv2D	512 × 28 × 28
77	Stage2_Block4_BN3	BatchNorm	512 × 28 × 28
78	Stage2_Block4_Add	Add	512 × 28 × 28
79	Stage2_Block4_ReLU_out	ReLU	512 × 28 × 28
80	Stage3_Block1_Conv1x1_reduce	Conv2D	256 × 28 × 28
81	Stage3_Block1_BN1	BatchNorm	256 × 28 × 28
82	Stage3_Block1_ReLU1	ReLU	256 × 28 × 28
83	Stage3_Block1_Conv3x3	Conv2D	256 × 14 × 14
84	Stage3_Block1_BN2	BatchNorm	256 × 14 × 14
85	Stage3_Block1_ReLU2	ReLU	256 × 14 × 14
86	Stage3_Block1_Conv1x1_expand	Conv2D	1024 × 14 × 14
87	Stage3_Block1_BN3	BatchNorm	1024 × 14 × 14
88	Stage3_Block1_Downsample	Conv2D	1024 × 14 × 14
89	Stage3_Block1_Downsample_BN	BatchNorm	1024 × 14 × 14
90	Stage3_Block1_Add	Add	1024 × 14 × 14
91	Stage3_Block1_ReLU_out	ReLU	1024 × 14 × 14
92	Stage3_Block2_Conv1x1_reduce	Conv2D	256 × 14 × 14
93	Stage3_Block2_BN1	BatchNorm	256 × 14 × 14
94	Stage3_Block2_ReLU1	ReLU	256 × 14 × 14
95	Stage3_Block2_Conv3x3	Conv2D	256 × 14 × 14
96	Stage3_Block2_BN2	BatchNorm	256 × 14 × 14
97	Stage3_Block2_ReLU2	ReLU	256 × 14 × 14
98	Stage3_Block2_Conv1x1_expand	Conv2D	1024 × 14 × 14
99	Stage3_Block2_BN3	BatchNorm	1024 × 14 × 14
100	Stage3_Block2_Add	Add	1024 × 14 × 14
101	Stage3_Block2_ReLU_out	ReLU	1024 × 14 × 14
102	Stage3_Block3_Conv1x1_reduce	Conv2D	256 × 14 × 14
103	Stage3_Block3_BN1	BatchNorm	256 × 14 × 14
104	Stage3_Block3_ReLU1	ReLU	256 × 14 × 14
105	Stage3_Block3_Conv3x3	Conv2D	256 × 14 × 14
106	Stage3_Block3_BN2	BatchNorm	256 × 14 × 14
107	Stage3_Block3_ReLU2	ReLU	256 × 14 × 14
108	Stage3_Block3_Conv1x1_expand	Conv2D	1024 × 14 × 14
109	Stage3_Block3_BN3	BatchNorm	1024 × 14 × 14
110	Stage3_Block3_Add	Add	1024 × 14 × 14
111	Stage3_Block3_ReLU_out	ReLU	1024 × 14 × 14
112	Stage3_Block4_Conv1x1_reduce	Conv2D	256 × 14 × 14
113	Stage3_Block4_BN1	BatchNorm	256 × 14 × 14
114	Stage3_Block4_ReLU1	ReLU	256 × 14 × 14
115	Stage3_Block4_Conv3x3	Conv2D	256 × 14 × 14
116	Stage3_Block4_BN2	BatchNorm	256 × 14 × 14
117	Stage3_Block4_ReLU2	ReLU	256 × 14 × 14
118	Stage3_Block4_Conv1x1_expand	Conv2D	1024 × 14 × 14
119	Stage3_Block4_BN3	BatchNorm	1024 × 14 × 14
120	Stage3_Block4_Add	Add	1024 × 14 × 14
121	Stage3_Block4_ReLU_out	ReLU	1024 × 14 × 14
122	Stage3_Block5_Conv1x1_reduce	Conv2D	256 × 14 × 14
123	Stage3_Block5_BN1	BatchNorm	256 × 14 × 14
124	Stage3_Block5_ReLU1	ReLU	256 × 14 × 14
125	Stage3_Block5_Conv3x3	Conv2D	256 × 14 × 14
126	Stage3_Block5_BN2	BatchNorm	256 × 14 × 14
127	Stage3_Block5_ReLU2	ReLU	256 × 14 × 14
128	Stage3_Block5_Conv1x1_expand	Conv2D	1024 × 14 × 14
129	Stage3_Block5_BN3	BatchNorm	1024 × 14 × 14
130	Stage3_Block5_Add	Add	1024 × 14 × 14
131	Stage3_Block5_ReLU_out	ReLU	1024 × 14 × 14
132	Stage3_Block6_Conv1x1_reduce	Conv2D	256 × 14 × 14
133	Stage3_Block6_BN1	BatchNorm	256 × 14 × 14
134	Stage3_Block6_ReLU1	ReLU	256 × 14 × 14
135	Stage3_Block6_Conv3x3	Conv2D	256 × 14 × 14
136	Stage3_Block6_BN2	BatchNorm	256 × 14 × 14
137	Stage3_Block6_ReLU2	ReLU	256 × 14 × 14
138	Stage3_Block6_Conv1x1_expand	Conv2D	1024 × 14 × 14
139	Stage3_Block6_BN3	BatchNorm	1024 × 14 × 14
140	Stage3_Block6_Add	Add	1024 × 14 × 14
141	Stage3_Block6_ReLU_out	ReLU	1024 × 14 × 14
142	Stage4_Block1_Conv1x1_reduce	Conv2D	512 × 14 × 14
143	Stage4_Block1_BN1	BatchNorm	512 × 14 × 14
144	Stage4_Block1_ReLU1	ReLU	512 × 14 × 14
145	Stage4_Block1_Conv3x3	Conv2D	512 × 7 × 7
146	Stage4_Block1_BN2	BatchNorm	512 × 7 × 7
147	Stage4_Block1_ReLU2	ReLU	512 × 7 × 7
148	Stage4_Block1_Conv1x1_expand	Conv2D	2048 × 7 × 7
149	Stage4_Block1_BN3	BatchNorm	2048 × 7 × 7
150	Stage4_Block1_Downsample	Conv2D	2048 × 7 × 7
151	Stage4_Block1_Downsample_BN	BatchNorm	2048 × 7 × 7
152	Stage4_Block1_Add	Add	2048 × 7 × 7
153	Stage4_Block1_ReLU_out	ReLU	2048 × 7 × 7
154	Stage4_Block2_Conv1x1_reduce	Conv2D	512 × 7 × 7
155	Stage4_Block2_BN1	BatchNorm	512 × 7 × 7
156	Stage4_Block2_ReLU1	ReLU	512 × 7 × 7
157	Stage4_Block2_Conv3x3	Conv2D	512 × 7 × 7
158	Stage4_Block2_BN2	BatchNorm	512 × 7 × 7
159	Stage4_Block2_ReLU2	ReLU	512 × 7 × 7
160	Stage4_Block2_Conv1x1_expand	Conv2D	2048 × 7 × 7
161	Stage4_Block2_BN3	BatchNorm	2048 × 7 × 7
162	Stage4_Block2_Add	Add	2048 × 7 × 7
163	Stage4_Block2_ReLU_out	ReLU	2048 × 7 × 7
164	Stage4_Block3_Conv1x1_reduce	Conv2D	512 × 7 × 7
165	Stage4_Block3_BN1	BatchNorm	512 × 7 × 7
166	Stage4_Block3_ReLU1	ReLU	512 × 7 × 7
167	Stage4_Block3_Conv3x3	Conv2D	512 × 7 × 7
168	Stage4_Block3_BN2	BatchNorm	512 × 7 × 7
169	Stage4_Block3_ReLU2	ReLU	512 × 7 × 7
170	Stage4_Block3_Conv1x1_expand	Conv2D	2048 × 7 × 7
171	Stage4_Block3_BN3	BatchNorm	2048 × 7 × 7
172	Stage4_Block3_Add	Add	2048 × 7 × 7
173	Stage4_Block3_ReLU_out	ReLU	2048 × 7 × 7
174	GlobalAvgPool	GlobalAvgPool2D	2048
175	Flatten	Flatten	2048
176	Classifier	Linear	1000
177	Output	Output	1000

What the verifier says

No finding on the reconstructed graph. See the checks.

What changed upstream

The change tracker follows microsoft/resnet-50: every revision it has read, what changed in the config, tokenizer, chat template or weights between them, and an OSV advisory per change.

Revision history and changes Badge JSON OSV advisories All tracked changes

Do this to your own model

Same numbers, on a model in your repo, in one command. No account.

pip install neurarch-trace
neurarch-trace microsoft/resnet-50 --plan --share

Paste a model instead Markdown twin Index JSON POST /api/v1/plan