Comparisons / ViT-B/16 vs DiT-XL/2

ViT-B/16 vs DiT-XL/2

A vision transformer against a diffusion transformer.

DiT-XL/2 has 662M more parameters than ViT-B/16: 192 layers added, 1 removed, 9 changed.

Baseline

ViT-B/16

Layers: 11
Parameters: 8.4M
Input: 3 × 224 × 224
Output: 196 × 1000
Forward-passes: yes
Est. train cost: $0.105

Compared

DiT-XL/2

Layers: 201
Parameters: 671M
Input: 4 × 32 × 32
Output: 256 × 32
Forward-passes: yes
Est. train cost: $4.41

The deltas

Every number is DiT-XL/2 relative to ViT-B/16.

Parameters

+662M (79× the size)

Layers

+190

Added

192

Removed

Changed

Unchanged

Which GPUs each one fits

Each side is measured at its own declared input (3 × 224 × 224 against 4 × 32 × 32). Both columns are right about their own model; the difference between them is not a fact about the designs.

GPU	ViT-B/16	DiT-XL/2
T4 16GB	fits	no
A100 40GB	fits	fits
H100 80GB	fits	fits

Layer by layer

Aligned in topological order. 3 of 205 rows are the same layer with the same parameters.

Show all 205 rows

		ViT-B/16			DiT-XL/2
		Layer	Params	Output	Layer	Params	Output
1	added	—			noisy_latent Input		4 × 32 × 32
2	changed shape	image Input		3 × 224 × 224	timestep_+_class Input		1 × 2
3	changed imgSize, patchSize, inChans, embedDim	patch_embed Patch Embed	591K	196 × 768	patchify_2x2 Patch Embed	20K	256 × 1152
4	added	—			cond_embed Embedding	1.2M	1 × 2 × 1152
5	changed maxLen, embedDim	pos_embed Positional Encoding		196 × 768	pos_embed Positional Encoding		256 × 1152
6	removed	dropout Dropout		196 × 768	—
7	added	—			adaLN_1 Linear	8.0M	1 × 2 × 6912
8	added	—			adaLN_2 Linear	8.0M	1 × 2 × 6912
9	added	—			adaLN_3 Linear	8.0M	1 × 2 × 6912
10	added	—			adaLN_4 Linear	8.0M	1 × 2 × 6912
11	added	—			adaLN_5 Linear	8.0M	1 × 2 × 6912
12	added	—			adaLN_6 Linear	8.0M	1 × 2 × 6912
13	added	—			adaLN_7 Linear	8.0M	1 × 2 × 6912
14	added	—			adaLN_8 Linear	8.0M	1 × 2 × 6912
15	added	—			adaLN_9 Linear	8.0M	1 × 2 × 6912
16	added	—			adaLN_10 Linear	8.0M	1 × 2 × 6912
17	added	—			adaLN_11 Linear	8.0M	1 × 2 × 6912
18	added	—			adaLN_12 Linear	8.0M	1 × 2 × 6912
19	added	—			adaLN_13 Linear	8.0M	1 × 2 × 6912
20	added	—			adaLN_14 Linear	8.0M	1 × 2 × 6912
21	added	—			adaLN_15 Linear	8.0M	1 × 2 × 6912
22	added	—			adaLN_16 Linear	8.0M	1 × 2 × 6912
23	added	—			adaLN_17 Linear	8.0M	1 × 2 × 6912
24	added	—			adaLN_18 Linear	8.0M	1 × 2 × 6912
25	added	—			adaLN_19 Linear	8.0M	1 × 2 × 6912
26	added	—			adaLN_20 Linear	8.0M	1 × 2 × 6912
27	added	—			adaLN_21 Linear	8.0M	1 × 2 × 6912
28	added	—			adaLN_22 Linear	8.0M	1 × 2 × 6912
29	added	—			adaLN_23 Linear	8.0M	1 × 2 × 6912
30	added	—			adaLN_24 Linear	8.0M	1 × 2 × 6912
31	added	—			adaLN_25 Linear	8.0M	1 × 2 × 6912
32	added	—			adaLN_26 Linear	8.0M	1 × 2 × 6912
33	added	—			adaLN_27 Linear	8.0M	1 × 2 × 6912
34	added	—			adaLN_28 Linear	8.0M	1 × 2 × 6912
35	added	—			norm1_1 Layer Norm	2.3K	256 × 1152
36	added	—			self_attn_1 Multi Head Attention	5.3M	256 × 1152
37	added	—			residual1_1 Add		256 × 1152
38	added	—			norm2_1 Layer Norm	2.3K	256 × 1152
39	added	—			mlp_1 Feed Forward	11M	256 × 1152
40	added	—			residual2_1 Add		256 × 1152
41	added	—			norm1_2 Layer Norm	2.3K	256 × 1152
42	added	—			self_attn_2 Multi Head Attention	5.3M	256 × 1152
43	added	—			residual1_2 Add		256 × 1152
44	added	—			norm2_2 Layer Norm	2.3K	256 × 1152
45	added	—			mlp_2 Feed Forward	11M	256 × 1152
46	added	—			residual2_2 Add		256 × 1152
47	added	—			norm1_3 Layer Norm	2.3K	256 × 1152
48	added	—			self_attn_3 Multi Head Attention	5.3M	256 × 1152
49	added	—			residual1_3 Add		256 × 1152
50	added	—			norm2_3 Layer Norm	2.3K	256 × 1152
51	added	—			mlp_3 Feed Forward	11M	256 × 1152
52	added	—			residual2_3 Add		256 × 1152
53	added	—			norm1_4 Layer Norm	2.3K	256 × 1152
54	added	—			self_attn_4 Multi Head Attention	5.3M	256 × 1152
55	added	—			residual1_4 Add		256 × 1152
56	added	—			norm2_4 Layer Norm	2.3K	256 × 1152
57	added	—			mlp_4 Feed Forward	11M	256 × 1152
58	added	—			residual2_4 Add		256 × 1152
59	added	—			norm1_5 Layer Norm	2.3K	256 × 1152
60	added	—			self_attn_5 Multi Head Attention	5.3M	256 × 1152
61	added	—			residual1_5 Add		256 × 1152
62	added	—			norm2_5 Layer Norm	2.3K	256 × 1152
63	added	—			mlp_5 Feed Forward	11M	256 × 1152
64	added	—			residual2_5 Add		256 × 1152
65	added	—			norm1_6 Layer Norm	2.3K	256 × 1152
66	added	—			self_attn_6 Multi Head Attention	5.3M	256 × 1152
67	added	—			residual1_6 Add		256 × 1152
68	added	—			norm2_6 Layer Norm	2.3K	256 × 1152
69	added	—			mlp_6 Feed Forward	11M	256 × 1152
70	added	—			residual2_6 Add		256 × 1152
71	added	—			norm1_7 Layer Norm	2.3K	256 × 1152
72	added	—			self_attn_7 Multi Head Attention	5.3M	256 × 1152
73	added	—			residual1_7 Add		256 × 1152
74	added	—			norm2_7 Layer Norm	2.3K	256 × 1152
75	added	—			mlp_7 Feed Forward	11M	256 × 1152
76	added	—			residual2_7 Add		256 × 1152
77	added	—			norm1_8 Layer Norm	2.3K	256 × 1152
78	added	—			self_attn_8 Multi Head Attention	5.3M	256 × 1152
79	added	—			residual1_8 Add		256 × 1152
80	added	—			norm2_8 Layer Norm	2.3K	256 × 1152
81	added	—			mlp_8 Feed Forward	11M	256 × 1152
82	added	—			residual2_8 Add		256 × 1152
83	added	—			norm1_9 Layer Norm	2.3K	256 × 1152
84	added	—			self_attn_9 Multi Head Attention	5.3M	256 × 1152
85	added	—			residual1_9 Add		256 × 1152
86	added	—			norm2_9 Layer Norm	2.3K	256 × 1152
87	added	—			mlp_9 Feed Forward	11M	256 × 1152
88	added	—			residual2_9 Add		256 × 1152
89	added	—			norm1_10 Layer Norm	2.3K	256 × 1152
90	added	—			self_attn_10 Multi Head Attention	5.3M	256 × 1152
91	added	—			residual1_10 Add		256 × 1152
92	added	—			norm2_10 Layer Norm	2.3K	256 × 1152
93	added	—			mlp_10 Feed Forward	11M	256 × 1152
94	added	—			residual2_10 Add		256 × 1152
95	added	—			norm1_11 Layer Norm	2.3K	256 × 1152
96	added	—			self_attn_11 Multi Head Attention	5.3M	256 × 1152
97	added	—			residual1_11 Add		256 × 1152
98	added	—			norm2_11 Layer Norm	2.3K	256 × 1152
99	added	—			mlp_11 Feed Forward	11M	256 × 1152
100	added	—			residual2_11 Add		256 × 1152
101	added	—			norm1_12 Layer Norm	2.3K	256 × 1152
102	added	—			self_attn_12 Multi Head Attention	5.3M	256 × 1152
103	added	—			residual1_12 Add		256 × 1152
104	added	—			norm2_12 Layer Norm	2.3K	256 × 1152
105	added	—			mlp_12 Feed Forward	11M	256 × 1152
106	added	—			residual2_12 Add		256 × 1152
107	added	—			norm1_13 Layer Norm	2.3K	256 × 1152
108	added	—			self_attn_13 Multi Head Attention	5.3M	256 × 1152
109	added	—			residual1_13 Add		256 × 1152
110	added	—			norm2_13 Layer Norm	2.3K	256 × 1152
111	added	—			mlp_13 Feed Forward	11M	256 × 1152
112	added	—			residual2_13 Add		256 × 1152
113	added	—			norm1_14 Layer Norm	2.3K	256 × 1152
114	added	—			self_attn_14 Multi Head Attention	5.3M	256 × 1152
115	added	—			residual1_14 Add		256 × 1152
116	added	—			norm2_14 Layer Norm	2.3K	256 × 1152
117	added	—			mlp_14 Feed Forward	11M	256 × 1152
118	added	—			residual2_14 Add		256 × 1152
119	added	—			norm1_15 Layer Norm	2.3K	256 × 1152
120	added	—			self_attn_15 Multi Head Attention	5.3M	256 × 1152
121	added	—			residual1_15 Add		256 × 1152
122	added	—			norm2_15 Layer Norm	2.3K	256 × 1152
123	added	—			mlp_15 Feed Forward	11M	256 × 1152
124	added	—			residual2_15 Add		256 × 1152
125	added	—			norm1_16 Layer Norm	2.3K	256 × 1152
126	added	—			self_attn_16 Multi Head Attention	5.3M	256 × 1152
127	added	—			residual1_16 Add		256 × 1152
128	added	—			norm2_16 Layer Norm	2.3K	256 × 1152
129	added	—			mlp_16 Feed Forward	11M	256 × 1152
130	added	—			residual2_16 Add		256 × 1152
131	added	—			norm1_17 Layer Norm	2.3K	256 × 1152
132	added	—			self_attn_17 Multi Head Attention	5.3M	256 × 1152
133	added	—			residual1_17 Add		256 × 1152
134	added	—			norm2_17 Layer Norm	2.3K	256 × 1152
135	added	—			mlp_17 Feed Forward	11M	256 × 1152
136	added	—			residual2_17 Add		256 × 1152
137	added	—			norm1_18 Layer Norm	2.3K	256 × 1152
138	added	—			self_attn_18 Multi Head Attention	5.3M	256 × 1152
139	added	—			residual1_18 Add		256 × 1152
140	added	—			norm2_18 Layer Norm	2.3K	256 × 1152
141	added	—			mlp_18 Feed Forward	11M	256 × 1152
142	added	—			residual2_18 Add		256 × 1152
143	added	—			norm1_19 Layer Norm	2.3K	256 × 1152
144	added	—			self_attn_19 Multi Head Attention	5.3M	256 × 1152
145	added	—			residual1_19 Add		256 × 1152
146	added	—			norm2_19 Layer Norm	2.3K	256 × 1152
147	added	—			mlp_19 Feed Forward	11M	256 × 1152
148	added	—			residual2_19 Add		256 × 1152
149	added	—			norm1_20 Layer Norm	2.3K	256 × 1152
150	added	—			self_attn_20 Multi Head Attention	5.3M	256 × 1152
151	added	—			residual1_20 Add		256 × 1152
152	added	—			norm2_20 Layer Norm	2.3K	256 × 1152
153	added	—			mlp_20 Feed Forward	11M	256 × 1152
154	added	—			residual2_20 Add		256 × 1152
155	added	—			norm1_21 Layer Norm	2.3K	256 × 1152
156	added	—			self_attn_21 Multi Head Attention	5.3M	256 × 1152
157	added	—			residual1_21 Add		256 × 1152
158	added	—			norm2_21 Layer Norm	2.3K	256 × 1152
159	added	—			mlp_21 Feed Forward	11M	256 × 1152
160	added	—			residual2_21 Add		256 × 1152
161	added	—			norm1_22 Layer Norm	2.3K	256 × 1152
162	added	—			self_attn_22 Multi Head Attention	5.3M	256 × 1152
163	added	—			residual1_22 Add		256 × 1152
164	added	—			norm2_22 Layer Norm	2.3K	256 × 1152
165	added	—			mlp_22 Feed Forward	11M	256 × 1152
166	added	—			residual2_22 Add		256 × 1152
167	added	—			norm1_23 Layer Norm	2.3K	256 × 1152
168	added	—			self_attn_23 Multi Head Attention	5.3M	256 × 1152
169	added	—			residual1_23 Add		256 × 1152
170	added	—			norm2_23 Layer Norm	2.3K	256 × 1152
171	added	—			mlp_23 Feed Forward	11M	256 × 1152
172	added	—			residual2_23 Add		256 × 1152
173	added	—			norm1_24 Layer Norm	2.3K	256 × 1152
174	added	—			self_attn_24 Multi Head Attention	5.3M	256 × 1152
175	added	—			residual1_24 Add		256 × 1152
176	added	—			norm2_24 Layer Norm	2.3K	256 × 1152
177	added	—			mlp_24 Feed Forward	11M	256 × 1152
178	added	—			residual2_24 Add		256 × 1152
179	added	—			norm1_25 Layer Norm	2.3K	256 × 1152
180	added	—			self_attn_25 Multi Head Attention	5.3M	256 × 1152
181	added	—			residual1_25 Add		256 × 1152
182	added	—			norm2_25 Layer Norm	2.3K	256 × 1152
183	added	—			mlp_25 Feed Forward	11M	256 × 1152
184	added	—			residual2_25 Add		256 × 1152
185	added	—			norm1_26 Layer Norm	2.3K	256 × 1152
186	added	—			self_attn_26 Multi Head Attention	5.3M	256 × 1152
187	added	—			residual1_26 Add		256 × 1152
188	added	—			norm2_26 Layer Norm	2.3K	256 × 1152
189	added	—			mlp_26 Feed Forward	11M	256 × 1152
190	added	—			residual2_26 Add		256 × 1152
191	added	—			norm1_27 Layer Norm	2.3K	256 × 1152
192	added	—			self_attn_27 Multi Head Attention	5.3M	256 × 1152
193	added	—			residual1_27 Add		256 × 1152
194	added	—			norm2_27 Layer Norm	2.3K	256 × 1152
195	added	—			mlp_27 Feed Forward	11M	256 × 1152
196	added	—			residual2_27 Add		256 × 1152
197	changed normalizedShape	norm_1 Layer Norm	1.5K	196 × 768	norm1_28 Layer Norm	2.3K	256 × 1152
198	changed embedDim, numHeads	attn Multi Head Attention	2.4M	196 × 768	self_attn_28 Multi Head Attention	5.3M	256 × 1152
199	same	residual_1 Add		196 × 768	residual1_28 Add		256 × 1152
200	changed normalizedShape	norm_2 Layer Norm	1.5K	196 × 768	norm2_28 Layer Norm	2.3K	256 × 1152
201	changed hiddenDim, ffDim, embedDim	mlp Feed Forward	4.7M	196 × 768	mlp_28 Feed Forward	11M	256 × 1152
202	same	residual_2 Add		196 × 768	residual2_28 Add		256 × 1152
203	changed normalizedShape	norm_final Layer Norm	1.5K	196 × 768	final_norm Layer Norm	2.3K	256 × 1152
204	changed outFeatures, inFeatures	head Linear		196 × 1000	unpatchify Linear	37K	256 × 32
205	same	class_logits Output		196 × 1000	predicted_noise Output		256 × 32

What this is not

The two are priced at different declared inputs (3 × 224 × 224 against 4 × 32 × 32), so memory, cost and GPU fit are each right about their own model and are not a comparison between them. The layer and parameter deltas are unaffected.
Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.

Take it further

Open either graph in the editor, change it, and check it again: ViT-B/16 · DiT-XL/2

Compare any two models of your own, including anything on Hugging Face: the comparison tool.

Machine-readable: this page as markdown · the pair index · POST https://www.neurarch.com/api/v1/plan for a graph of your own.

Related comparisons

ResNet Block vs ViT-B/16

Convolution against attention for images.

ViT-B/16 vs Swin-Tiny

A flat vision transformer against a hierarchical one.

BERT Base vs ViT-B/16

The same transformer applied to text and to images.

Diffusion UNet vs DiT-XL/2

Convolutional against transformer backbones for diffusion.