N Neurarch Try it See it Changes Bot Pricing API Open app

Comparisons / LLaVA-1.5 vs CLIP ViT-B/32

LLaVA-1.5 vs CLIP ViT-B/32

A vision-language model against the encoder it is built on.

CLIP ViT-B/32 has 6.9B fewer parameters than LLaVA-1.5: 7 layers added, 196 removed, 32 changed.

Baseline

LLaVA-1.5

Layers
226
Parameters
7.1B
Input
3 × 336 × 336
Output
1 × 2624 × 32000
Forward-passes
yes
Est. train cost
$690.06
Compared

CLIP ViT-B/32

Layers
37
Parameters
151M
Input
3 × 224 × 224
Output
512
Forward-passes
yes
Est. train cost
$0.313

The deltas

Every number is CLIP ViT-B/32 relative to LLaVA-1.5.

Parameters
-6.9B (-97.9%)
Layers
-189
Added
7
Removed
196
Changed
32
Unchanged
1

Which GPUs each one fits

Each side is measured at its own declared input (3 × 336 × 336 against 3 × 224 × 224). Both columns are right about their own model; the difference between them is not a fact about the designs.

GPULLaVA-1.5CLIP ViT-B/32
T4 16GBnofits
A100 40GBnofits
H100 80GBnofits

Layer by layer

Aligned in topological order. 1 of 236 rows are the same layer with the same parameters.

Show all 236 rows
LLaVA-1.5CLIP ViT-B/32
LayerParamsOutputLayerParamsOutput
1changed
shape
image
Input
3 × 336 × 336image
Input
3 × 224 × 224
2changed
shape
text_tokens
Input
1 × 2048tokens
Input
77
3changed
imgSize, patchSize, embedDim
clip_patch_14
Patch Embed
603K576 × 1024patch_embed
Patch Embed
2.4M49 × 768
4changed
numEmbeddings, embeddingDim
token_embed
Embedding
131M1 × 2048 × 4096tok_embed
Embedding
25M77 × 512
5added—vis_pos
Positional Encoding
49 × 768
6changed
maxLen, embedDim
vis_pos
Positional Encoding
576 × 1024txt_pos
Positional Encoding
77 × 512
7changed
embedDim, numHeads, ffDim
clip_block_1
Transformer Block
13M576 × 1024vis_block_1
Transformer Block
7.1M49 × 768
8changed
embedDim, numHeads, ffDim
clip_block_2
Transformer Block
13M576 × 1024txt_block_1
Transformer Block
3.2M77 × 512
9changed
embedDim, numHeads, ffDim
clip_block_3
Transformer Block
13M576 × 1024vis_block_2
Transformer Block
7.1M49 × 768
10changed
embedDim, numHeads, ffDim
clip_block_4
Transformer Block
13M576 × 1024txt_block_2
Transformer Block
3.2M77 × 512
11changed
embedDim, numHeads, ffDim
clip_block_5
Transformer Block
13M576 × 1024vis_block_3
Transformer Block
7.1M49 × 768
12changed
embedDim, numHeads, ffDim
clip_block_6
Transformer Block
13M576 × 1024txt_block_3
Transformer Block
3.2M77 × 512
13changed
embedDim, numHeads, ffDim
clip_block_7
Transformer Block
13M576 × 1024vis_block_4
Transformer Block
7.1M49 × 768
14changed
embedDim, numHeads, ffDim
clip_block_8
Transformer Block
13M576 × 1024txt_block_4
Transformer Block
3.2M77 × 512
15changed
embedDim, numHeads, ffDim
clip_block_9
Transformer Block
13M576 × 1024vis_block_5
Transformer Block
7.1M49 × 768
16changed
embedDim, numHeads, ffDim
clip_block_10
Transformer Block
13M576 × 1024txt_block_5
Transformer Block
3.2M77 × 512
17changed
embedDim, numHeads, ffDim
clip_block_11
Transformer Block
13M576 × 1024vis_block_6
Transformer Block
7.1M49 × 768
18changed
embedDim, numHeads, ffDim
clip_block_12
Transformer Block
13M576 × 1024txt_block_6
Transformer Block
3.2M77 × 512
19changed
embedDim, numHeads, ffDim
clip_block_13
Transformer Block
13M576 × 1024vis_block_7
Transformer Block
7.1M49 × 768
20changed
embedDim, numHeads, ffDim
clip_block_14
Transformer Block
13M576 × 1024txt_block_7
Transformer Block
3.2M77 × 512
21changed
embedDim, numHeads, ffDim
clip_block_15
Transformer Block
13M576 × 1024vis_block_8
Transformer Block
7.1M49 × 768
22changed
embedDim, numHeads, ffDim
clip_block_16
Transformer Block
13M576 × 1024txt_block_8
Transformer Block
3.2M77 × 512
23changed
embedDim, numHeads, ffDim
clip_block_17
Transformer Block
13M576 × 1024vis_block_9
Transformer Block
7.1M49 × 768
24changed
embedDim, numHeads, ffDim
clip_block_18
Transformer Block
13M576 × 1024txt_block_9
Transformer Block
3.2M77 × 512
25changed
embedDim, numHeads, ffDim
clip_block_19
Transformer Block
13M576 × 1024vis_block_10
Transformer Block
7.1M49 × 768
26changed
embedDim, numHeads, ffDim
clip_block_20
Transformer Block
13M576 × 1024txt_block_10
Transformer Block
3.2M77 × 512
27changed
embedDim, numHeads, ffDim
clip_block_21
Transformer Block
13M576 × 1024vis_block_11
Transformer Block
7.1M49 × 768
28changed
embedDim, numHeads, ffDim
clip_block_22
Transformer Block
13M576 × 1024txt_block_11
Transformer Block
3.2M77 × 512
29changed
embedDim, numHeads, ffDim
clip_block_23
Transformer Block
13M576 × 1024vis_block_12
Transformer Block
7.1M49 × 768
30changed
embedDim, numHeads, ffDim
clip_block_24
Transformer Block
13M576 × 1024txt_block_12
Transformer Block
3.2M77 × 512
31removedprojector_1
Linear
4.2M576 × 4096—
32removedprojector_gelu
Gelu
576 × 4096—
33removedprojector_2
Linear
17M576 × 4096—
34removedvision_tokens
Reshape
1 × 576 × 4096—
35removedimage+text_tokens
Concatenate
1 × 2624 × 4096—
36removedllm_attn_norm_1
Rms Norm
4.1K1 × 2624 × 4096—
37removedllm_attn_1
Grouped Query Attention
67M1 × 2624 × 4096—
38removedllm_res1_1
Add
1 × 2624 × 4096—
39removedllm_ffn_norm_1
Rms Norm
4.1K1 × 2624 × 4096—
40removedllm_ffn_1
Swiglu
135M1 × 2624 × 4096—
41removedllm_res2_1
Add
1 × 2624 × 4096—
42removedllm_attn_norm_2
Rms Norm
4.1K1 × 2624 × 4096—
43removedllm_attn_2
Grouped Query Attention
67M1 × 2624 × 4096—
44removedllm_res1_2
Add
1 × 2624 × 4096—
45removedllm_ffn_norm_2
Rms Norm
4.1K1 × 2624 × 4096—
46removedllm_ffn_2
Swiglu
135M1 × 2624 × 4096—
47removedllm_res2_2
Add
1 × 2624 × 4096—
48removedllm_attn_norm_3
Rms Norm
4.1K1 × 2624 × 4096—
49removedllm_attn_3
Grouped Query Attention
67M1 × 2624 × 4096—
50removedllm_res1_3
Add
1 × 2624 × 4096—
51removedllm_ffn_norm_3
Rms Norm
4.1K1 × 2624 × 4096—
52removedllm_ffn_3
Swiglu
135M1 × 2624 × 4096—
53removedllm_res2_3
Add
1 × 2624 × 4096—
54removedllm_attn_norm_4
Rms Norm
4.1K1 × 2624 × 4096—
55removedllm_attn_4
Grouped Query Attention
67M1 × 2624 × 4096—
56removedllm_res1_4
Add
1 × 2624 × 4096—
57removedllm_ffn_norm_4
Rms Norm
4.1K1 × 2624 × 4096—
58removedllm_ffn_4
Swiglu
135M1 × 2624 × 4096—
59removedllm_res2_4
Add
1 × 2624 × 4096—
60removedllm_attn_norm_5
Rms Norm
4.1K1 × 2624 × 4096—
61removedllm_attn_5
Grouped Query Attention
67M1 × 2624 × 4096—
62removedllm_res1_5
Add
1 × 2624 × 4096—
63removedllm_ffn_norm_5
Rms Norm
4.1K1 × 2624 × 4096—
64removedllm_ffn_5
Swiglu
135M1 × 2624 × 4096—
65removedllm_res2_5
Add
1 × 2624 × 4096—
66removedllm_attn_norm_6
Rms Norm
4.1K1 × 2624 × 4096—
67removedllm_attn_6
Grouped Query Attention
67M1 × 2624 × 4096—
68removedllm_res1_6
Add
1 × 2624 × 4096—
69removedllm_ffn_norm_6
Rms Norm
4.1K1 × 2624 × 4096—
70removedllm_ffn_6
Swiglu
135M1 × 2624 × 4096—
71removedllm_res2_6
Add
1 × 2624 × 4096—
72removedllm_attn_norm_7
Rms Norm
4.1K1 × 2624 × 4096—
73removedllm_attn_7
Grouped Query Attention
67M1 × 2624 × 4096—
74removedllm_res1_7
Add
1 × 2624 × 4096—
75removedllm_ffn_norm_7
Rms Norm
4.1K1 × 2624 × 4096—
76removedllm_ffn_7
Swiglu
135M1 × 2624 × 4096—
77removedllm_res2_7
Add
1 × 2624 × 4096—
78removedllm_attn_norm_8
Rms Norm
4.1K1 × 2624 × 4096—
79removedllm_attn_8
Grouped Query Attention
67M1 × 2624 × 4096—
80removedllm_res1_8
Add
1 × 2624 × 4096—
81removedllm_ffn_norm_8
Rms Norm
4.1K1 × 2624 × 4096—
82removedllm_ffn_8
Swiglu
135M1 × 2624 × 4096—
83removedllm_res2_8
Add
1 × 2624 × 4096—
84removedllm_attn_norm_9
Rms Norm
4.1K1 × 2624 × 4096—
85removedllm_attn_9
Grouped Query Attention
67M1 × 2624 × 4096—
86removedllm_res1_9
Add
1 × 2624 × 4096—
87removedllm_ffn_norm_9
Rms Norm
4.1K1 × 2624 × 4096—
88removedllm_ffn_9
Swiglu
135M1 × 2624 × 4096—
89removedllm_res2_9
Add
1 × 2624 × 4096—
90removedllm_attn_norm_10
Rms Norm
4.1K1 × 2624 × 4096—
91removedllm_attn_10
Grouped Query Attention
67M1 × 2624 × 4096—
92removedllm_res1_10
Add
1 × 2624 × 4096—
93removedllm_ffn_norm_10
Rms Norm
4.1K1 × 2624 × 4096—
94removedllm_ffn_10
Swiglu
135M1 × 2624 × 4096—
95removedllm_res2_10
Add
1 × 2624 × 4096—
96removedllm_attn_norm_11
Rms Norm
4.1K1 × 2624 × 4096—
97removedllm_attn_11
Grouped Query Attention
67M1 × 2624 × 4096—
98removedllm_res1_11
Add
1 × 2624 × 4096—
99removedllm_ffn_norm_11
Rms Norm
4.1K1 × 2624 × 4096—
100removedllm_ffn_11
Swiglu
135M1 × 2624 × 4096—
101removedllm_res2_11
Add
1 × 2624 × 4096—
102removedllm_attn_norm_12
Rms Norm
4.1K1 × 2624 × 4096—
103removedllm_attn_12
Grouped Query Attention
67M1 × 2624 × 4096—
104removedllm_res1_12
Add
1 × 2624 × 4096—
105removedllm_ffn_norm_12
Rms Norm
4.1K1 × 2624 × 4096—
106removedllm_ffn_12
Swiglu
135M1 × 2624 × 4096—
107removedllm_res2_12
Add
1 × 2624 × 4096—
108removedllm_attn_norm_13
Rms Norm
4.1K1 × 2624 × 4096—
109removedllm_attn_13
Grouped Query Attention
67M1 × 2624 × 4096—
110removedllm_res1_13
Add
1 × 2624 × 4096—
111removedllm_ffn_norm_13
Rms Norm
4.1K1 × 2624 × 4096—
112removedllm_ffn_13
Swiglu
135M1 × 2624 × 4096—
113removedllm_res2_13
Add
1 × 2624 × 4096—
114removedllm_attn_norm_14
Rms Norm
4.1K1 × 2624 × 4096—
115removedllm_attn_14
Grouped Query Attention
67M1 × 2624 × 4096—
116removedllm_res1_14
Add
1 × 2624 × 4096—
117removedllm_ffn_norm_14
Rms Norm
4.1K1 × 2624 × 4096—
118removedllm_ffn_14
Swiglu
135M1 × 2624 × 4096—
119removedllm_res2_14
Add
1 × 2624 × 4096—
120removedllm_attn_norm_15
Rms Norm
4.1K1 × 2624 × 4096—
121removedllm_attn_15
Grouped Query Attention
67M1 × 2624 × 4096—
122removedllm_res1_15
Add
1 × 2624 × 4096—
123removedllm_ffn_norm_15
Rms Norm
4.1K1 × 2624 × 4096—
124removedllm_ffn_15
Swiglu
135M1 × 2624 × 4096—
125removedllm_res2_15
Add
1 × 2624 × 4096—
126removedllm_attn_norm_16
Rms Norm
4.1K1 × 2624 × 4096—
127removedllm_attn_16
Grouped Query Attention
67M1 × 2624 × 4096—
128removedllm_res1_16
Add
1 × 2624 × 4096—
129removedllm_ffn_norm_16
Rms Norm
4.1K1 × 2624 × 4096—
130removedllm_ffn_16
Swiglu
135M1 × 2624 × 4096—
131removedllm_res2_16
Add
1 × 2624 × 4096—
132removedllm_attn_norm_17
Rms Norm
4.1K1 × 2624 × 4096—
133removedllm_attn_17
Grouped Query Attention
67M1 × 2624 × 4096—
134removedllm_res1_17
Add
1 × 2624 × 4096—
135removedllm_ffn_norm_17
Rms Norm
4.1K1 × 2624 × 4096—
136removedllm_ffn_17
Swiglu
135M1 × 2624 × 4096—
137removedllm_res2_17
Add
1 × 2624 × 4096—
138removedllm_attn_norm_18
Rms Norm
4.1K1 × 2624 × 4096—
139removedllm_attn_18
Grouped Query Attention
67M1 × 2624 × 4096—
140removedllm_res1_18
Add
1 × 2624 × 4096—
141removedllm_ffn_norm_18
Rms Norm
4.1K1 × 2624 × 4096—
142removedllm_ffn_18
Swiglu
135M1 × 2624 × 4096—
143removedllm_res2_18
Add
1 × 2624 × 4096—
144removedllm_attn_norm_19
Rms Norm
4.1K1 × 2624 × 4096—
145removedllm_attn_19
Grouped Query Attention
67M1 × 2624 × 4096—
146removedllm_res1_19
Add
1 × 2624 × 4096—
147removedllm_ffn_norm_19
Rms Norm
4.1K1 × 2624 × 4096—
148removedllm_ffn_19
Swiglu
135M1 × 2624 × 4096—
149removedllm_res2_19
Add
1 × 2624 × 4096—
150removedllm_attn_norm_20
Rms Norm
4.1K1 × 2624 × 4096—
151removedllm_attn_20
Grouped Query Attention
67M1 × 2624 × 4096—
152removedllm_res1_20
Add
1 × 2624 × 4096—
153removedllm_ffn_norm_20
Rms Norm
4.1K1 × 2624 × 4096—
154removedllm_ffn_20
Swiglu
135M1 × 2624 × 4096—
155removedllm_res2_20
Add
1 × 2624 × 4096—
156removedllm_attn_norm_21
Rms Norm
4.1K1 × 2624 × 4096—
157removedllm_attn_21
Grouped Query Attention
67M1 × 2624 × 4096—
158removedllm_res1_21
Add
1 × 2624 × 4096—
159removedllm_ffn_norm_21
Rms Norm
4.1K1 × 2624 × 4096—
160removedllm_ffn_21
Swiglu
135M1 × 2624 × 4096—
161removedllm_res2_21
Add
1 × 2624 × 4096—
162removedllm_attn_norm_22
Rms Norm
4.1K1 × 2624 × 4096—
163removedllm_attn_22
Grouped Query Attention
67M1 × 2624 × 4096—
164removedllm_res1_22
Add
1 × 2624 × 4096—
165removedllm_ffn_norm_22
Rms Norm
4.1K1 × 2624 × 4096—
166removedllm_ffn_22
Swiglu
135M1 × 2624 × 4096—
167removedllm_res2_22
Add
1 × 2624 × 4096—
168removedllm_attn_norm_23
Rms Norm
4.1K1 × 2624 × 4096—
169removedllm_attn_23
Grouped Query Attention
67M1 × 2624 × 4096—
170removedllm_res1_23
Add
1 × 2624 × 4096—
171removedllm_ffn_norm_23
Rms Norm
4.1K1 × 2624 × 4096—
172removedllm_ffn_23
Swiglu
135M1 × 2624 × 4096—
173removedllm_res2_23
Add
1 × 2624 × 4096—
174removedllm_attn_norm_24
Rms Norm
4.1K1 × 2624 × 4096—
175removedllm_attn_24
Grouped Query Attention
67M1 × 2624 × 4096—
176removedllm_res1_24
Add
1 × 2624 × 4096—
177removedllm_ffn_norm_24
Rms Norm
4.1K1 × 2624 × 4096—
178removedllm_ffn_24
Swiglu
135M1 × 2624 × 4096—
179removedllm_res2_24
Add
1 × 2624 × 4096—
180removedllm_attn_norm_25
Rms Norm
4.1K1 × 2624 × 4096—
181removedllm_attn_25
Grouped Query Attention
67M1 × 2624 × 4096—
182removedllm_res1_25
Add
1 × 2624 × 4096—
183removedllm_ffn_norm_25
Rms Norm
4.1K1 × 2624 × 4096—
184removedllm_ffn_25
Swiglu
135M1 × 2624 × 4096—
185removedllm_res2_25
Add
1 × 2624 × 4096—
186removedllm_attn_norm_26
Rms Norm
4.1K1 × 2624 × 4096—
187removedllm_attn_26
Grouped Query Attention
67M1 × 2624 × 4096—
188removedllm_res1_26
Add
1 × 2624 × 4096—
189removedllm_ffn_norm_26
Rms Norm
4.1K1 × 2624 × 4096—
190removedllm_ffn_26
Swiglu
135M1 × 2624 × 4096—
191removedllm_res2_26
Add
1 × 2624 × 4096—
192removedllm_attn_norm_27
Rms Norm
4.1K1 × 2624 × 4096—
193removedllm_attn_27
Grouped Query Attention
67M1 × 2624 × 4096—
194removedllm_res1_27
Add
1 × 2624 × 4096—
195removedllm_ffn_norm_27
Rms Norm
4.1K1 × 2624 × 4096—
196removedllm_ffn_27
Swiglu
135M1 × 2624 × 4096—
197removedllm_res2_27
Add
1 × 2624 × 4096—
198removedllm_attn_norm_28
Rms Norm
4.1K1 × 2624 × 4096—
199removedllm_attn_28
Grouped Query Attention
67M1 × 2624 × 4096—
200removedllm_res1_28
Add
1 × 2624 × 4096—
201removedllm_ffn_norm_28
Rms Norm
4.1K1 × 2624 × 4096—
202removedllm_ffn_28
Swiglu
135M1 × 2624 × 4096—
203removedllm_res2_28
Add
1 × 2624 × 4096—
204removedllm_attn_norm_29
Rms Norm
4.1K1 × 2624 × 4096—
205removedllm_attn_29
Grouped Query Attention
67M1 × 2624 × 4096—
206removedllm_res1_29
Add
1 × 2624 × 4096—
207removedllm_ffn_norm_29
Rms Norm
4.1K1 × 2624 × 4096—
208removedllm_ffn_29
Swiglu
135M1 × 2624 × 4096—
209removedllm_res2_29
Add
1 × 2624 × 4096—
210removedllm_attn_norm_30
Rms Norm
4.1K1 × 2624 × 4096—
211removedllm_attn_30
Grouped Query Attention
67M1 × 2624 × 4096—
212removedllm_res1_30
Add
1 × 2624 × 4096—
213removedllm_ffn_norm_30
Rms Norm
4.1K1 × 2624 × 4096—
214removedllm_ffn_30
Swiglu
135M1 × 2624 × 4096—
215removedllm_res2_30
Add
1 × 2624 × 4096—
216removedllm_attn_norm_31
Rms Norm
4.1K1 × 2624 × 4096—
217removedllm_attn_31
Grouped Query Attention
67M1 × 2624 × 4096—
218removedllm_res1_31
Add
1 × 2624 × 4096—
219removedllm_ffn_norm_31
Rms Norm
4.1K1 × 2624 × 4096—
220removedllm_ffn_31
Swiglu
135M1 × 2624 × 4096—
221removedllm_res2_31
Add
1 × 2624 × 4096—
222removedllm_attn_norm_32
Rms Norm
4.1K1 × 2624 × 4096—
223removedllm_attn_32
Grouped Query Attention
67M1 × 2624 × 4096—
224removedllm_res1_32
Add
1 × 2624 × 4096—
225changed
type, normalizedShape
llm_ffn_norm_32
Rms Norm
4.1K1 × 2624 × 4096vis_norm
Layer Norm
1.5K49 × 768
226removedllm_ffn_32
Swiglu
135M1 × 2624 × 4096—
227removedllm_res2_32
Add
1 × 2624 × 4096—
228changed
type, normalizedShape
final_norm
Rms Norm
4.1K1 × 2624 × 4096txt_norm
Layer Norm
1.0K77 × 512
229added—to_channels_vis
Permute
768 × 49
230added—to_channels_txt
Permute
512 × 77
231added—vis_pool
Global Avg Pool1d
768
232added—txt_pool
Global Avg Pool1d
512
233added—img_proj
Linear
394K512
234changed
inFeatures, outFeatures, bias
lm_head
Linear
131M1 × 2624 × 32000txt_proj
Linear
263K512
235added—similarity
Matmul
512
236samelogits
Output
1 × 2624 × 32000logits
Output
512

What this is not

Take it further

Open either graph in the editor, change it, and check it again: LLaVA-1.5 · CLIP ViT-B/32

Compare any two models of your own, including anything on Hugging Face: the comparison tool.

Machine-readable: this page as markdown · the pair index · POST https://www.neurarch.com/api/v1/plan for a graph of your own.