# ESMC-6B

Hugging Face: `biohub/ESMC-6B`
Reconstructed from `config.json` alone. No weights were read.

| | |
|---|---|
| Derived from structure | 3,358,597,120 |
| In the published checkpoint | 6,352,005,184 (safetensors.total, 2026-09-06) |
| Delta | -47.1% |
| Comparability | comparable |
| Layers | 322 |
| Priced on | A10G (24GB) |
| Est. one run | $66.36 |

## Structure

| # | Layer | Type | Output shape |
|---|---|---|---|
| 1 | Input | Input | 1 × 512 |
| 2 | Embedding | Embedding | 1 × 512 × 2560 |
| 3 | Positional_Embedding | Learned Pos Embed | 1 × 512 × 2560 |
| 4 | Attention_1 | Multi-Head Attention | 1 × 512 × 2560 |
| 5 | Add_1 | Add | 1 × 512 × 2560 |
| 6 | LayerNorm_1_1 | LayerNorm | 1 × 512 × 2560 |
| 7 | FFN_1 | Feed Forward | 1 × 512 × 2560 |
| 8 | Attention_2 | Multi-Head Attention | 1 × 512 × 2560 |
| 9 | Add_2 | Add | 1 × 512 × 2560 |
| 10 | LayerNorm_2_1 | LayerNorm | 1 × 512 × 2560 |
| 11 | FFN_2 | Feed Forward | 1 × 512 × 2560 |
| 12 | Attention_3 | Multi-Head Attention | 1 × 512 × 2560 |
| 13 | Add_3 | Add | 1 × 512 × 2560 |
| 14 | LayerNorm_3_1 | LayerNorm | 1 × 512 × 2560 |
| 15 | FFN_3 | Feed Forward | 1 × 512 × 2560 |
| 16 | Attention_4 | Multi-Head Attention | 1 × 512 × 2560 |
| 17 | Add_4 | Add | 1 × 512 × 2560 |
| 18 | LayerNorm_4_1 | LayerNorm | 1 × 512 × 2560 |
| 19 | FFN_4 | Feed Forward | 1 × 512 × 2560 |
| 20 | Attention_5 | Multi-Head Attention | 1 × 512 × 2560 |
| 21 | Add_5 | Add | 1 × 512 × 2560 |
| 22 | LayerNorm_5_1 | LayerNorm | 1 × 512 × 2560 |
| 23 | FFN_5 | Feed Forward | 1 × 512 × 2560 |
| 24 | Attention_6 | Multi-Head Attention | 1 × 512 × 2560 |
| 25 | Add_6 | Add | 1 × 512 × 2560 |
| 26 | LayerNorm_6_1 | LayerNorm | 1 × 512 × 2560 |
| 27 | FFN_6 | Feed Forward | 1 × 512 × 2560 |
| 28 | Attention_7 | Multi-Head Attention | 1 × 512 × 2560 |
| 29 | Add_7 | Add | 1 × 512 × 2560 |
| 30 | LayerNorm_7_1 | LayerNorm | 1 × 512 × 2560 |
| 31 | FFN_7 | Feed Forward | 1 × 512 × 2560 |
| 32 | Attention_8 | Multi-Head Attention | 1 × 512 × 2560 |
| 33 | Add_8 | Add | 1 × 512 × 2560 |
| 34 | LayerNorm_8_1 | LayerNorm | 1 × 512 × 2560 |
| 35 | FFN_8 | Feed Forward | 1 × 512 × 2560 |
| 36 | Attention_9 | Multi-Head Attention | 1 × 512 × 2560 |
| 37 | Add_9 | Add | 1 × 512 × 2560 |
| 38 | LayerNorm_9_1 | LayerNorm | 1 × 512 × 2560 |
| 39 | FFN_9 | Feed Forward | 1 × 512 × 2560 |
| 40 | Attention_10 | Multi-Head Attention | 1 × 512 × 2560 |
| 41 | Add_10 | Add | 1 × 512 × 2560 |
| 42 | LayerNorm_10_1 | LayerNorm | 1 × 512 × 2560 |
| 43 | FFN_10 | Feed Forward | 1 × 512 × 2560 |
| 44 | Attention_11 | Multi-Head Attention | 1 × 512 × 2560 |
| 45 | Add_11 | Add | 1 × 512 × 2560 |
| 46 | LayerNorm_11_1 | LayerNorm | 1 × 512 × 2560 |
| 47 | FFN_11 | Feed Forward | 1 × 512 × 2560 |
| 48 | Attention_12 | Multi-Head Attention | 1 × 512 × 2560 |
| 49 | Add_12 | Add | 1 × 512 × 2560 |
| 50 | LayerNorm_12_1 | LayerNorm | 1 × 512 × 2560 |
| 51 | FFN_12 | Feed Forward | 1 × 512 × 2560 |
| 52 | Attention_13 | Multi-Head Attention | 1 × 512 × 2560 |
| 53 | Add_13 | Add | 1 × 512 × 2560 |
| 54 | LayerNorm_13_1 | LayerNorm | 1 × 512 × 2560 |
| 55 | FFN_13 | Feed Forward | 1 × 512 × 2560 |
| 56 | Attention_14 | Multi-Head Attention | 1 × 512 × 2560 |
| 57 | Add_14 | Add | 1 × 512 × 2560 |
| 58 | LayerNorm_14_1 | LayerNorm | 1 × 512 × 2560 |
| 59 | FFN_14 | Feed Forward | 1 × 512 × 2560 |
| 60 | Attention_15 | Multi-Head Attention | 1 × 512 × 2560 |
| 61 | Add_15 | Add | 1 × 512 × 2560 |
| 62 | LayerNorm_15_1 | LayerNorm | 1 × 512 × 2560 |
| 63 | FFN_15 | Feed Forward | 1 × 512 × 2560 |
| 64 | Attention_16 | Multi-Head Attention | 1 × 512 × 2560 |
| 65 | Add_16 | Add | 1 × 512 × 2560 |
| 66 | LayerNorm_16_1 | LayerNorm | 1 × 512 × 2560 |
| 67 | FFN_16 | Feed Forward | 1 × 512 × 2560 |
| 68 | Attention_17 | Multi-Head Attention | 1 × 512 × 2560 |
| 69 | Add_17 | Add | 1 × 512 × 2560 |
| 70 | LayerNorm_17_1 | LayerNorm | 1 × 512 × 2560 |
| 71 | FFN_17 | Feed Forward | 1 × 512 × 2560 |
| 72 | Attention_18 | Multi-Head Attention | 1 × 512 × 2560 |
| 73 | Add_18 | Add | 1 × 512 × 2560 |
| 74 | LayerNorm_18_1 | LayerNorm | 1 × 512 × 2560 |
| 75 | FFN_18 | Feed Forward | 1 × 512 × 2560 |
| 76 | Attention_19 | Multi-Head Attention | 1 × 512 × 2560 |
| 77 | Add_19 | Add | 1 × 512 × 2560 |
| 78 | LayerNorm_19_1 | LayerNorm | 1 × 512 × 2560 |
| 79 | FFN_19 | Feed Forward | 1 × 512 × 2560 |
| 80 | Attention_20 | Multi-Head Attention | 1 × 512 × 2560 |
| 81 | Add_20 | Add | 1 × 512 × 2560 |
| 82 | LayerNorm_20_1 | LayerNorm | 1 × 512 × 2560 |
| 83 | FFN_20 | Feed Forward | 1 × 512 × 2560 |
| 84 | Attention_21 | Multi-Head Attention | 1 × 512 × 2560 |
| 85 | Add_21 | Add | 1 × 512 × 2560 |
| 86 | LayerNorm_21_1 | LayerNorm | 1 × 512 × 2560 |
| 87 | FFN_21 | Feed Forward | 1 × 512 × 2560 |
| 88 | Attention_22 | Multi-Head Attention | 1 × 512 × 2560 |
| 89 | Add_22 | Add | 1 × 512 × 2560 |
| 90 | LayerNorm_22_1 | LayerNorm | 1 × 512 × 2560 |
| 91 | FFN_22 | Feed Forward | 1 × 512 × 2560 |
| 92 | Attention_23 | Multi-Head Attention | 1 × 512 × 2560 |
| 93 | Add_23 | Add | 1 × 512 × 2560 |
| 94 | LayerNorm_23_1 | LayerNorm | 1 × 512 × 2560 |
| 95 | FFN_23 | Feed Forward | 1 × 512 × 2560 |
| 96 | Attention_24 | Multi-Head Attention | 1 × 512 × 2560 |
| 97 | Add_24 | Add | 1 × 512 × 2560 |
| 98 | LayerNorm_24_1 | LayerNorm | 1 × 512 × 2560 |
| 99 | FFN_24 | Feed Forward | 1 × 512 × 2560 |
| 100 | Attention_25 | Multi-Head Attention | 1 × 512 × 2560 |
| 101 | Add_25 | Add | 1 × 512 × 2560 |
| 102 | LayerNorm_25_1 | LayerNorm | 1 × 512 × 2560 |
| 103 | FFN_25 | Feed Forward | 1 × 512 × 2560 |
| 104 | Attention_26 | Multi-Head Attention | 1 × 512 × 2560 |
| 105 | Add_26 | Add | 1 × 512 × 2560 |
| 106 | LayerNorm_26_1 | LayerNorm | 1 × 512 × 2560 |
| 107 | FFN_26 | Feed Forward | 1 × 512 × 2560 |
| 108 | Attention_27 | Multi-Head Attention | 1 × 512 × 2560 |
| 109 | Add_27 | Add | 1 × 512 × 2560 |
| 110 | LayerNorm_27_1 | LayerNorm | 1 × 512 × 2560 |
| 111 | FFN_27 | Feed Forward | 1 × 512 × 2560 |
| 112 | Attention_28 | Multi-Head Attention | 1 × 512 × 2560 |
| 113 | Add_28 | Add | 1 × 512 × 2560 |
| 114 | LayerNorm_28_1 | LayerNorm | 1 × 512 × 2560 |
| 115 | FFN_28 | Feed Forward | 1 × 512 × 2560 |
| 116 | Attention_29 | Multi-Head Attention | 1 × 512 × 2560 |
| 117 | Add_29 | Add | 1 × 512 × 2560 |
| 118 | LayerNorm_29_1 | LayerNorm | 1 × 512 × 2560 |
| 119 | FFN_29 | Feed Forward | 1 × 512 × 2560 |
| 120 | Attention_30 | Multi-Head Attention | 1 × 512 × 2560 |
| 121 | Add_30 | Add | 1 × 512 × 2560 |
| 122 | LayerNorm_30_1 | LayerNorm | 1 × 512 × 2560 |
| 123 | FFN_30 | Feed Forward | 1 × 512 × 2560 |
| 124 | Attention_31 | Multi-Head Attention | 1 × 512 × 2560 |
| 125 | Add_31 | Add | 1 × 512 × 2560 |
| 126 | LayerNorm_31_1 | LayerNorm | 1 × 512 × 2560 |
| 127 | FFN_31 | Feed Forward | 1 × 512 × 2560 |
| 128 | Attention_32 | Multi-Head Attention | 1 × 512 × 2560 |
| 129 | Add_32 | Add | 1 × 512 × 2560 |
| 130 | LayerNorm_32_1 | LayerNorm | 1 × 512 × 2560 |
| 131 | FFN_32 | Feed Forward | 1 × 512 × 2560 |
| 132 | Attention_33 | Multi-Head Attention | 1 × 512 × 2560 |
| 133 | Add_33 | Add | 1 × 512 × 2560 |
| 134 | LayerNorm_33_1 | LayerNorm | 1 × 512 × 2560 |
| 135 | FFN_33 | Feed Forward | 1 × 512 × 2560 |
| 136 | Attention_34 | Multi-Head Attention | 1 × 512 × 2560 |
| 137 | Add_34 | Add | 1 × 512 × 2560 |
| 138 | LayerNorm_34_1 | LayerNorm | 1 × 512 × 2560 |
| 139 | FFN_34 | Feed Forward | 1 × 512 × 2560 |
| 140 | Attention_35 | Multi-Head Attention | 1 × 512 × 2560 |
| 141 | Add_35 | Add | 1 × 512 × 2560 |
| 142 | LayerNorm_35_1 | LayerNorm | 1 × 512 × 2560 |
| 143 | FFN_35 | Feed Forward | 1 × 512 × 2560 |
| 144 | Attention_36 | Multi-Head Attention | 1 × 512 × 2560 |
| 145 | Add_36 | Add | 1 × 512 × 2560 |
| 146 | LayerNorm_36_1 | LayerNorm | 1 × 512 × 2560 |
| 147 | FFN_36 | Feed Forward | 1 × 512 × 2560 |
| 148 | Attention_37 | Multi-Head Attention | 1 × 512 × 2560 |
| 149 | Add_37 | Add | 1 × 512 × 2560 |
| 150 | LayerNorm_37_1 | LayerNorm | 1 × 512 × 2560 |
| 151 | FFN_37 | Feed Forward | 1 × 512 × 2560 |
| 152 | Attention_38 | Multi-Head Attention | 1 × 512 × 2560 |
| 153 | Add_38 | Add | 1 × 512 × 2560 |
| 154 | LayerNorm_38_1 | LayerNorm | 1 × 512 × 2560 |
| 155 | FFN_38 | Feed Forward | 1 × 512 × 2560 |
| 156 | Attention_39 | Multi-Head Attention | 1 × 512 × 2560 |
| 157 | Add_39 | Add | 1 × 512 × 2560 |
| 158 | LayerNorm_39_1 | LayerNorm | 1 × 512 × 2560 |
| 159 | FFN_39 | Feed Forward | 1 × 512 × 2560 |
| 160 | Attention_40 | Multi-Head Attention | 1 × 512 × 2560 |
| 161 | Add_40 | Add | 1 × 512 × 2560 |
| 162 | LayerNorm_40_1 | LayerNorm | 1 × 512 × 2560 |
| 163 | FFN_40 | Feed Forward | 1 × 512 × 2560 |
| 164 | Attention_41 | Multi-Head Attention | 1 × 512 × 2560 |
| 165 | Add_41 | Add | 1 × 512 × 2560 |
| 166 | LayerNorm_41_1 | LayerNorm | 1 × 512 × 2560 |
| 167 | FFN_41 | Feed Forward | 1 × 512 × 2560 |
| 168 | Attention_42 | Multi-Head Attention | 1 × 512 × 2560 |
| 169 | Add_42 | Add | 1 × 512 × 2560 |
| 170 | LayerNorm_42_1 | LayerNorm | 1 × 512 × 2560 |
| 171 | FFN_42 | Feed Forward | 1 × 512 × 2560 |
| 172 | Attention_43 | Multi-Head Attention | 1 × 512 × 2560 |
| 173 | Add_43 | Add | 1 × 512 × 2560 |
| 174 | LayerNorm_43_1 | LayerNorm | 1 × 512 × 2560 |
| 175 | FFN_43 | Feed Forward | 1 × 512 × 2560 |
| 176 | Attention_44 | Multi-Head Attention | 1 × 512 × 2560 |
| 177 | Add_44 | Add | 1 × 512 × 2560 |
| 178 | LayerNorm_44_1 | LayerNorm | 1 × 512 × 2560 |
| 179 | FFN_44 | Feed Forward | 1 × 512 × 2560 |
| 180 | Attention_45 | Multi-Head Attention | 1 × 512 × 2560 |
| 181 | Add_45 | Add | 1 × 512 × 2560 |
| 182 | LayerNorm_45_1 | LayerNorm | 1 × 512 × 2560 |
| 183 | FFN_45 | Feed Forward | 1 × 512 × 2560 |
| 184 | Attention_46 | Multi-Head Attention | 1 × 512 × 2560 |
| 185 | Add_46 | Add | 1 × 512 × 2560 |
| 186 | LayerNorm_46_1 | LayerNorm | 1 × 512 × 2560 |
| 187 | FFN_46 | Feed Forward | 1 × 512 × 2560 |
| 188 | Attention_47 | Multi-Head Attention | 1 × 512 × 2560 |
| 189 | Add_47 | Add | 1 × 512 × 2560 |
| 190 | LayerNorm_47_1 | LayerNorm | 1 × 512 × 2560 |
| 191 | FFN_47 | Feed Forward | 1 × 512 × 2560 |
| 192 | Attention_48 | Multi-Head Attention | 1 × 512 × 2560 |
| 193 | Add_48 | Add | 1 × 512 × 2560 |
| 194 | LayerNorm_48_1 | LayerNorm | 1 × 512 × 2560 |
| 195 | FFN_48 | Feed Forward | 1 × 512 × 2560 |
| 196 | Attention_49 | Multi-Head Attention | 1 × 512 × 2560 |
| 197 | Add_49 | Add | 1 × 512 × 2560 |
| 198 | LayerNorm_49_1 | LayerNorm | 1 × 512 × 2560 |
| 199 | FFN_49 | Feed Forward | 1 × 512 × 2560 |
| 200 | Attention_50 | Multi-Head Attention | 1 × 512 × 2560 |
| 201 | Add_50 | Add | 1 × 512 × 2560 |
| 202 | LayerNorm_50_1 | LayerNorm | 1 × 512 × 2560 |
| 203 | FFN_50 | Feed Forward | 1 × 512 × 2560 |
| 204 | Attention_51 | Multi-Head Attention | 1 × 512 × 2560 |
| 205 | Add_51 | Add | 1 × 512 × 2560 |
| 206 | LayerNorm_51_1 | LayerNorm | 1 × 512 × 2560 |
| 207 | FFN_51 | Feed Forward | 1 × 512 × 2560 |
| 208 | Attention_52 | Multi-Head Attention | 1 × 512 × 2560 |
| 209 | Add_52 | Add | 1 × 512 × 2560 |
| 210 | LayerNorm_52_1 | LayerNorm | 1 × 512 × 2560 |
| 211 | FFN_52 | Feed Forward | 1 × 512 × 2560 |
| 212 | Attention_53 | Multi-Head Attention | 1 × 512 × 2560 |
| 213 | Add_53 | Add | 1 × 512 × 2560 |
| 214 | LayerNorm_53_1 | LayerNorm | 1 × 512 × 2560 |
| 215 | FFN_53 | Feed Forward | 1 × 512 × 2560 |
| 216 | Attention_54 | Multi-Head Attention | 1 × 512 × 2560 |
| 217 | Add_54 | Add | 1 × 512 × 2560 |
| 218 | LayerNorm_54_1 | LayerNorm | 1 × 512 × 2560 |
| 219 | FFN_54 | Feed Forward | 1 × 512 × 2560 |
| 220 | Attention_55 | Multi-Head Attention | 1 × 512 × 2560 |
| 221 | Add_55 | Add | 1 × 512 × 2560 |
| 222 | LayerNorm_55_1 | LayerNorm | 1 × 512 × 2560 |
| 223 | FFN_55 | Feed Forward | 1 × 512 × 2560 |
| 224 | Attention_56 | Multi-Head Attention | 1 × 512 × 2560 |
| 225 | Add_56 | Add | 1 × 512 × 2560 |
| 226 | LayerNorm_56_1 | LayerNorm | 1 × 512 × 2560 |
| 227 | FFN_56 | Feed Forward | 1 × 512 × 2560 |
| 228 | Attention_57 | Multi-Head Attention | 1 × 512 × 2560 |
| 229 | Add_57 | Add | 1 × 512 × 2560 |
| 230 | LayerNorm_57_1 | LayerNorm | 1 × 512 × 2560 |
| 231 | FFN_57 | Feed Forward | 1 × 512 × 2560 |
| 232 | Attention_58 | Multi-Head Attention | 1 × 512 × 2560 |
| 233 | Add_58 | Add | 1 × 512 × 2560 |
| 234 | LayerNorm_58_1 | LayerNorm | 1 × 512 × 2560 |
| 235 | FFN_58 | Feed Forward | 1 × 512 × 2560 |
| 236 | Attention_59 | Multi-Head Attention | 1 × 512 × 2560 |
| 237 | Add_59 | Add | 1 × 512 × 2560 |
| 238 | LayerNorm_59_1 | LayerNorm | 1 × 512 × 2560 |
| 239 | FFN_59 | Feed Forward | 1 × 512 × 2560 |
| 240 | Attention_60 | Multi-Head Attention | 1 × 512 × 2560 |
| 241 | Add_60 | Add | 1 × 512 × 2560 |
| 242 | LayerNorm_60_1 | LayerNorm | 1 × 512 × 2560 |
| 243 | FFN_60 | Feed Forward | 1 × 512 × 2560 |
| 244 | Attention_61 | Multi-Head Attention | 1 × 512 × 2560 |
| 245 | Add_61 | Add | 1 × 512 × 2560 |
| 246 | LayerNorm_61_1 | LayerNorm | 1 × 512 × 2560 |
| 247 | FFN_61 | Feed Forward | 1 × 512 × 2560 |
| 248 | Attention_62 | Multi-Head Attention | 1 × 512 × 2560 |
| 249 | Add_62 | Add | 1 × 512 × 2560 |
| 250 | LayerNorm_62_1 | LayerNorm | 1 × 512 × 2560 |
| 251 | FFN_62 | Feed Forward | 1 × 512 × 2560 |
| 252 | Attention_63 | Multi-Head Attention | 1 × 512 × 2560 |
| 253 | Add_63 | Add | 1 × 512 × 2560 |
| 254 | LayerNorm_63_1 | LayerNorm | 1 × 512 × 2560 |
| 255 | FFN_63 | Feed Forward | 1 × 512 × 2560 |
| 256 | Attention_64 | Multi-Head Attention | 1 × 512 × 2560 |
| 257 | Add_64 | Add | 1 × 512 × 2560 |
| 258 | LayerNorm_64_1 | LayerNorm | 1 × 512 × 2560 |
| 259 | FFN_64 | Feed Forward | 1 × 512 × 2560 |
| 260 | Attention_65 | Multi-Head Attention | 1 × 512 × 2560 |
| 261 | Add_65 | Add | 1 × 512 × 2560 |
| 262 | LayerNorm_65_1 | LayerNorm | 1 × 512 × 2560 |
| 263 | FFN_65 | Feed Forward | 1 × 512 × 2560 |
| 264 | Attention_66 | Multi-Head Attention | 1 × 512 × 2560 |
| 265 | Add_66 | Add | 1 × 512 × 2560 |
| 266 | LayerNorm_66_1 | LayerNorm | 1 × 512 × 2560 |
| 267 | FFN_66 | Feed Forward | 1 × 512 × 2560 |
| 268 | Attention_67 | Multi-Head Attention | 1 × 512 × 2560 |
| 269 | Add_67 | Add | 1 × 512 × 2560 |
| 270 | LayerNorm_67_1 | LayerNorm | 1 × 512 × 2560 |
| 271 | FFN_67 | Feed Forward | 1 × 512 × 2560 |
| 272 | Attention_68 | Multi-Head Attention | 1 × 512 × 2560 |
| 273 | Add_68 | Add | 1 × 512 × 2560 |
| 274 | LayerNorm_68_1 | LayerNorm | 1 × 512 × 2560 |
| 275 | FFN_68 | Feed Forward | 1 × 512 × 2560 |
| 276 | Attention_69 | Multi-Head Attention | 1 × 512 × 2560 |
| 277 | Add_69 | Add | 1 × 512 × 2560 |
| 278 | LayerNorm_69_1 | LayerNorm | 1 × 512 × 2560 |
| 279 | FFN_69 | Feed Forward | 1 × 512 × 2560 |
| 280 | Attention_70 | Multi-Head Attention | 1 × 512 × 2560 |
| 281 | Add_70 | Add | 1 × 512 × 2560 |
| 282 | LayerNorm_70_1 | LayerNorm | 1 × 512 × 2560 |
| 283 | FFN_70 | Feed Forward | 1 × 512 × 2560 |
| 284 | Attention_71 | Multi-Head Attention | 1 × 512 × 2560 |
| 285 | Add_71 | Add | 1 × 512 × 2560 |
| 286 | LayerNorm_71_1 | LayerNorm | 1 × 512 × 2560 |
| 287 | FFN_71 | Feed Forward | 1 × 512 × 2560 |
| 288 | Attention_72 | Multi-Head Attention | 1 × 512 × 2560 |
| 289 | Add_72 | Add | 1 × 512 × 2560 |
| 290 | LayerNorm_72_1 | LayerNorm | 1 × 512 × 2560 |
| 291 | FFN_72 | Feed Forward | 1 × 512 × 2560 |
| 292 | Attention_73 | Multi-Head Attention | 1 × 512 × 2560 |
| 293 | Add_73 | Add | 1 × 512 × 2560 |
| 294 | LayerNorm_73_1 | LayerNorm | 1 × 512 × 2560 |
| 295 | FFN_73 | Feed Forward | 1 × 512 × 2560 |
| 296 | Attention_74 | Multi-Head Attention | 1 × 512 × 2560 |
| 297 | Add_74 | Add | 1 × 512 × 2560 |
| 298 | LayerNorm_74_1 | LayerNorm | 1 × 512 × 2560 |
| 299 | FFN_74 | Feed Forward | 1 × 512 × 2560 |
| 300 | Attention_75 | Multi-Head Attention | 1 × 512 × 2560 |
| 301 | Add_75 | Add | 1 × 512 × 2560 |
| 302 | LayerNorm_75_1 | LayerNorm | 1 × 512 × 2560 |
| 303 | FFN_75 | Feed Forward | 1 × 512 × 2560 |
| 304 | Attention_76 | Multi-Head Attention | 1 × 512 × 2560 |
| 305 | Add_76 | Add | 1 × 512 × 2560 |
| 306 | LayerNorm_76_1 | LayerNorm | 1 × 512 × 2560 |
| 307 | FFN_76 | Feed Forward | 1 × 512 × 2560 |
| 308 | Attention_77 | Multi-Head Attention | 1 × 512 × 2560 |
| 309 | Add_77 | Add | 1 × 512 × 2560 |
| 310 | LayerNorm_77_1 | LayerNorm | 1 × 512 × 2560 |
| 311 | FFN_77 | Feed Forward | 1 × 512 × 2560 |
| 312 | Attention_78 | Multi-Head Attention | 1 × 512 × 2560 |
| 313 | Add_78 | Add | 1 × 512 × 2560 |
| 314 | LayerNorm_78_1 | LayerNorm | 1 × 512 × 2560 |
| 315 | FFN_78 | Feed Forward | 1 × 512 × 2560 |
| 316 | Attention_79 | Multi-Head Attention | 1 × 512 × 2560 |
| 317 | Add_79 | Add | 1 × 512 × 2560 |
| 318 | LayerNorm_79_1 | LayerNorm | 1 × 512 × 2560 |
| 319 | FFN_79 | Feed Forward | 1 × 512 × 2560 |
| 320 | Attention_80 | Multi-Head Attention | 1 × 512 × 2560 |
| 321 | Add_80 | Add | 1 × 512 × 2560 |
| 322 | LayerNorm_80_1 | LayerNorm | 1 × 512 × 2560 |
| 323 | FFN_80 | Feed Forward | 1 × 512 × 2560 |
| 324 | Output | Output | 1 × 512 × 2560 |

## Do this to your own model

```
pip install neurarch-trace
neurarch-trace biohub/ESMC-6B --plan --share
```

- Page: https://neurarch.com/m/esmc-6b.html
- All models: https://neurarch.com/m/index.json
- The checks: https://neurarch.com/r/index.json