# EXAONE-3.5-7.8B-Instruct-AWQ

Hugging Face: `LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-AWQ`
Reconstructed from `config.json` alone. No weights were read.

| | |
|---|---|
| Derived from structure | 5,940,359,168 |
| In the published checkpoint | 7,818,448,896 (safetensors.total, 2026-02-06) |
| Delta | -24.0% |
| Comparability | quantized |
| Layers | 196 |
| Priced on | A10G (24GB) |
| Est. one run | $12037.47 |

> This checkpoint is stored quantized (awq). The tensor count in the file counts stored elements under a packing scheme, not logical parameters, so the two numbers below are not measuring the same thing in either direction.

## Structure

| # | Layer | Type | Output shape |
|---|---|---|---|
| 1 | Input | Input | 1 × 32768 |
| 2 | Embedding | Embedding | 1 × 32768 × 4096 |
| 3 | RoPE | RoPE | 1 × 32768 × 4096 |
| 4 | LayerNorm_1_1 | LayerNorm | 1 × 32768 × 4096 |
| 5 | Attention_1 | Grouped Query Attn | 1 × 32768 × 4096 |
| 6 | Add_1_attn | Add | 1 × 32768 × 4096 |
| 7 | LayerNorm_1_2 | LayerNorm | 1 × 32768 × 4096 |
| 8 | FFN_1 | Feed Forward | 1 × 32768 × 4096 |
| 9 | Add_1_ffn | Add | 1 × 32768 × 4096 |
| 10 | LayerNorm_2_1 | LayerNorm | 1 × 32768 × 4096 |
| 11 | Attention_2 | Grouped Query Attn | 1 × 32768 × 4096 |
| 12 | Add_2_attn | Add | 1 × 32768 × 4096 |
| 13 | LayerNorm_2_2 | LayerNorm | 1 × 32768 × 4096 |
| 14 | FFN_2 | Feed Forward | 1 × 32768 × 4096 |
| 15 | Add_2_ffn | Add | 1 × 32768 × 4096 |
| 16 | LayerNorm_3_1 | LayerNorm | 1 × 32768 × 4096 |
| 17 | Attention_3 | Grouped Query Attn | 1 × 32768 × 4096 |
| 18 | Add_3_attn | Add | 1 × 32768 × 4096 |
| 19 | LayerNorm_3_2 | LayerNorm | 1 × 32768 × 4096 |
| 20 | FFN_3 | Feed Forward | 1 × 32768 × 4096 |
| 21 | Add_3_ffn | Add | 1 × 32768 × 4096 |
| 22 | LayerNorm_4_1 | LayerNorm | 1 × 32768 × 4096 |
| 23 | Attention_4 | Grouped Query Attn | 1 × 32768 × 4096 |
| 24 | Add_4_attn | Add | 1 × 32768 × 4096 |
| 25 | LayerNorm_4_2 | LayerNorm | 1 × 32768 × 4096 |
| 26 | FFN_4 | Feed Forward | 1 × 32768 × 4096 |
| 27 | Add_4_ffn | Add | 1 × 32768 × 4096 |
| 28 | LayerNorm_5_1 | LayerNorm | 1 × 32768 × 4096 |
| 29 | Attention_5 | Grouped Query Attn | 1 × 32768 × 4096 |
| 30 | Add_5_attn | Add | 1 × 32768 × 4096 |
| 31 | LayerNorm_5_2 | LayerNorm | 1 × 32768 × 4096 |
| 32 | FFN_5 | Feed Forward | 1 × 32768 × 4096 |
| 33 | Add_5_ffn | Add | 1 × 32768 × 4096 |
| 34 | LayerNorm_6_1 | LayerNorm | 1 × 32768 × 4096 |
| 35 | Attention_6 | Grouped Query Attn | 1 × 32768 × 4096 |
| 36 | Add_6_attn | Add | 1 × 32768 × 4096 |
| 37 | LayerNorm_6_2 | LayerNorm | 1 × 32768 × 4096 |
| 38 | FFN_6 | Feed Forward | 1 × 32768 × 4096 |
| 39 | Add_6_ffn | Add | 1 × 32768 × 4096 |
| 40 | LayerNorm_7_1 | LayerNorm | 1 × 32768 × 4096 |
| 41 | Attention_7 | Grouped Query Attn | 1 × 32768 × 4096 |
| 42 | Add_7_attn | Add | 1 × 32768 × 4096 |
| 43 | LayerNorm_7_2 | LayerNorm | 1 × 32768 × 4096 |
| 44 | FFN_7 | Feed Forward | 1 × 32768 × 4096 |
| 45 | Add_7_ffn | Add | 1 × 32768 × 4096 |
| 46 | LayerNorm_8_1 | LayerNorm | 1 × 32768 × 4096 |
| 47 | Attention_8 | Grouped Query Attn | 1 × 32768 × 4096 |
| 48 | Add_8_attn | Add | 1 × 32768 × 4096 |
| 49 | LayerNorm_8_2 | LayerNorm | 1 × 32768 × 4096 |
| 50 | FFN_8 | Feed Forward | 1 × 32768 × 4096 |
| 51 | Add_8_ffn | Add | 1 × 32768 × 4096 |
| 52 | LayerNorm_9_1 | LayerNorm | 1 × 32768 × 4096 |
| 53 | Attention_9 | Grouped Query Attn | 1 × 32768 × 4096 |
| 54 | Add_9_attn | Add | 1 × 32768 × 4096 |
| 55 | LayerNorm_9_2 | LayerNorm | 1 × 32768 × 4096 |
| 56 | FFN_9 | Feed Forward | 1 × 32768 × 4096 |
| 57 | Add_9_ffn | Add | 1 × 32768 × 4096 |
| 58 | LayerNorm_10_1 | LayerNorm | 1 × 32768 × 4096 |
| 59 | Attention_10 | Grouped Query Attn | 1 × 32768 × 4096 |
| 60 | Add_10_attn | Add | 1 × 32768 × 4096 |
| 61 | LayerNorm_10_2 | LayerNorm | 1 × 32768 × 4096 |
| 62 | FFN_10 | Feed Forward | 1 × 32768 × 4096 |
| 63 | Add_10_ffn | Add | 1 × 32768 × 4096 |
| 64 | LayerNorm_11_1 | LayerNorm | 1 × 32768 × 4096 |
| 65 | Attention_11 | Grouped Query Attn | 1 × 32768 × 4096 |
| 66 | Add_11_attn | Add | 1 × 32768 × 4096 |
| 67 | LayerNorm_11_2 | LayerNorm | 1 × 32768 × 4096 |
| 68 | FFN_11 | Feed Forward | 1 × 32768 × 4096 |
| 69 | Add_11_ffn | Add | 1 × 32768 × 4096 |
| 70 | LayerNorm_12_1 | LayerNorm | 1 × 32768 × 4096 |
| 71 | Attention_12 | Grouped Query Attn | 1 × 32768 × 4096 |
| 72 | Add_12_attn | Add | 1 × 32768 × 4096 |
| 73 | LayerNorm_12_2 | LayerNorm | 1 × 32768 × 4096 |
| 74 | FFN_12 | Feed Forward | 1 × 32768 × 4096 |
| 75 | Add_12_ffn | Add | 1 × 32768 × 4096 |
| 76 | LayerNorm_13_1 | LayerNorm | 1 × 32768 × 4096 |
| 77 | Attention_13 | Grouped Query Attn | 1 × 32768 × 4096 |
| 78 | Add_13_attn | Add | 1 × 32768 × 4096 |
| 79 | LayerNorm_13_2 | LayerNorm | 1 × 32768 × 4096 |
| 80 | FFN_13 | Feed Forward | 1 × 32768 × 4096 |
| 81 | Add_13_ffn | Add | 1 × 32768 × 4096 |
| 82 | LayerNorm_14_1 | LayerNorm | 1 × 32768 × 4096 |
| 83 | Attention_14 | Grouped Query Attn | 1 × 32768 × 4096 |
| 84 | Add_14_attn | Add | 1 × 32768 × 4096 |
| 85 | LayerNorm_14_2 | LayerNorm | 1 × 32768 × 4096 |
| 86 | FFN_14 | Feed Forward | 1 × 32768 × 4096 |
| 87 | Add_14_ffn | Add | 1 × 32768 × 4096 |
| 88 | LayerNorm_15_1 | LayerNorm | 1 × 32768 × 4096 |
| 89 | Attention_15 | Grouped Query Attn | 1 × 32768 × 4096 |
| 90 | Add_15_attn | Add | 1 × 32768 × 4096 |
| 91 | LayerNorm_15_2 | LayerNorm | 1 × 32768 × 4096 |
| 92 | FFN_15 | Feed Forward | 1 × 32768 × 4096 |
| 93 | Add_15_ffn | Add | 1 × 32768 × 4096 |
| 94 | LayerNorm_16_1 | LayerNorm | 1 × 32768 × 4096 |
| 95 | Attention_16 | Grouped Query Attn | 1 × 32768 × 4096 |
| 96 | Add_16_attn | Add | 1 × 32768 × 4096 |
| 97 | LayerNorm_16_2 | LayerNorm | 1 × 32768 × 4096 |
| 98 | FFN_16 | Feed Forward | 1 × 32768 × 4096 |
| 99 | Add_16_ffn | Add | 1 × 32768 × 4096 |
| 100 | LayerNorm_17_1 | LayerNorm | 1 × 32768 × 4096 |
| 101 | Attention_17 | Grouped Query Attn | 1 × 32768 × 4096 |
| 102 | Add_17_attn | Add | 1 × 32768 × 4096 |
| 103 | LayerNorm_17_2 | LayerNorm | 1 × 32768 × 4096 |
| 104 | FFN_17 | Feed Forward | 1 × 32768 × 4096 |
| 105 | Add_17_ffn | Add | 1 × 32768 × 4096 |
| 106 | LayerNorm_18_1 | LayerNorm | 1 × 32768 × 4096 |
| 107 | Attention_18 | Grouped Query Attn | 1 × 32768 × 4096 |
| 108 | Add_18_attn | Add | 1 × 32768 × 4096 |
| 109 | LayerNorm_18_2 | LayerNorm | 1 × 32768 × 4096 |
| 110 | FFN_18 | Feed Forward | 1 × 32768 × 4096 |
| 111 | Add_18_ffn | Add | 1 × 32768 × 4096 |
| 112 | LayerNorm_19_1 | LayerNorm | 1 × 32768 × 4096 |
| 113 | Attention_19 | Grouped Query Attn | 1 × 32768 × 4096 |
| 114 | Add_19_attn | Add | 1 × 32768 × 4096 |
| 115 | LayerNorm_19_2 | LayerNorm | 1 × 32768 × 4096 |
| 116 | FFN_19 | Feed Forward | 1 × 32768 × 4096 |
| 117 | Add_19_ffn | Add | 1 × 32768 × 4096 |
| 118 | LayerNorm_20_1 | LayerNorm | 1 × 32768 × 4096 |
| 119 | Attention_20 | Grouped Query Attn | 1 × 32768 × 4096 |
| 120 | Add_20_attn | Add | 1 × 32768 × 4096 |
| 121 | LayerNorm_20_2 | LayerNorm | 1 × 32768 × 4096 |
| 122 | FFN_20 | Feed Forward | 1 × 32768 × 4096 |
| 123 | Add_20_ffn | Add | 1 × 32768 × 4096 |
| 124 | LayerNorm_21_1 | LayerNorm | 1 × 32768 × 4096 |
| 125 | Attention_21 | Grouped Query Attn | 1 × 32768 × 4096 |
| 126 | Add_21_attn | Add | 1 × 32768 × 4096 |
| 127 | LayerNorm_21_2 | LayerNorm | 1 × 32768 × 4096 |
| 128 | FFN_21 | Feed Forward | 1 × 32768 × 4096 |
| 129 | Add_21_ffn | Add | 1 × 32768 × 4096 |
| 130 | LayerNorm_22_1 | LayerNorm | 1 × 32768 × 4096 |
| 131 | Attention_22 | Grouped Query Attn | 1 × 32768 × 4096 |
| 132 | Add_22_attn | Add | 1 × 32768 × 4096 |
| 133 | LayerNorm_22_2 | LayerNorm | 1 × 32768 × 4096 |
| 134 | FFN_22 | Feed Forward | 1 × 32768 × 4096 |
| 135 | Add_22_ffn | Add | 1 × 32768 × 4096 |
| 136 | LayerNorm_23_1 | LayerNorm | 1 × 32768 × 4096 |
| 137 | Attention_23 | Grouped Query Attn | 1 × 32768 × 4096 |
| 138 | Add_23_attn | Add | 1 × 32768 × 4096 |
| 139 | LayerNorm_23_2 | LayerNorm | 1 × 32768 × 4096 |
| 140 | FFN_23 | Feed Forward | 1 × 32768 × 4096 |
| 141 | Add_23_ffn | Add | 1 × 32768 × 4096 |
| 142 | LayerNorm_24_1 | LayerNorm | 1 × 32768 × 4096 |
| 143 | Attention_24 | Grouped Query Attn | 1 × 32768 × 4096 |
| 144 | Add_24_attn | Add | 1 × 32768 × 4096 |
| 145 | LayerNorm_24_2 | LayerNorm | 1 × 32768 × 4096 |
| 146 | FFN_24 | Feed Forward | 1 × 32768 × 4096 |
| 147 | Add_24_ffn | Add | 1 × 32768 × 4096 |
| 148 | LayerNorm_25_1 | LayerNorm | 1 × 32768 × 4096 |
| 149 | Attention_25 | Grouped Query Attn | 1 × 32768 × 4096 |
| 150 | Add_25_attn | Add | 1 × 32768 × 4096 |
| 151 | LayerNorm_25_2 | LayerNorm | 1 × 32768 × 4096 |
| 152 | FFN_25 | Feed Forward | 1 × 32768 × 4096 |
| 153 | Add_25_ffn | Add | 1 × 32768 × 4096 |
| 154 | LayerNorm_26_1 | LayerNorm | 1 × 32768 × 4096 |
| 155 | Attention_26 | Grouped Query Attn | 1 × 32768 × 4096 |
| 156 | Add_26_attn | Add | 1 × 32768 × 4096 |
| 157 | LayerNorm_26_2 | LayerNorm | 1 × 32768 × 4096 |
| 158 | FFN_26 | Feed Forward | 1 × 32768 × 4096 |
| 159 | Add_26_ffn | Add | 1 × 32768 × 4096 |
| 160 | LayerNorm_27_1 | LayerNorm | 1 × 32768 × 4096 |
| 161 | Attention_27 | Grouped Query Attn | 1 × 32768 × 4096 |
| 162 | Add_27_attn | Add | 1 × 32768 × 4096 |
| 163 | LayerNorm_27_2 | LayerNorm | 1 × 32768 × 4096 |
| 164 | FFN_27 | Feed Forward | 1 × 32768 × 4096 |
| 165 | Add_27_ffn | Add | 1 × 32768 × 4096 |
| 166 | LayerNorm_28_1 | LayerNorm | 1 × 32768 × 4096 |
| 167 | Attention_28 | Grouped Query Attn | 1 × 32768 × 4096 |
| 168 | Add_28_attn | Add | 1 × 32768 × 4096 |
| 169 | LayerNorm_28_2 | LayerNorm | 1 × 32768 × 4096 |
| 170 | FFN_28 | Feed Forward | 1 × 32768 × 4096 |
| 171 | Add_28_ffn | Add | 1 × 32768 × 4096 |
| 172 | LayerNorm_29_1 | LayerNorm | 1 × 32768 × 4096 |
| 173 | Attention_29 | Grouped Query Attn | 1 × 32768 × 4096 |
| 174 | Add_29_attn | Add | 1 × 32768 × 4096 |
| 175 | LayerNorm_29_2 | LayerNorm | 1 × 32768 × 4096 |
| 176 | FFN_29 | Feed Forward | 1 × 32768 × 4096 |
| 177 | Add_29_ffn | Add | 1 × 32768 × 4096 |
| 178 | LayerNorm_30_1 | LayerNorm | 1 × 32768 × 4096 |
| 179 | Attention_30 | Grouped Query Attn | 1 × 32768 × 4096 |
| 180 | Add_30_attn | Add | 1 × 32768 × 4096 |
| 181 | LayerNorm_30_2 | LayerNorm | 1 × 32768 × 4096 |
| 182 | FFN_30 | Feed Forward | 1 × 32768 × 4096 |
| 183 | Add_30_ffn | Add | 1 × 32768 × 4096 |
| 184 | LayerNorm_31_1 | LayerNorm | 1 × 32768 × 4096 |
| 185 | Attention_31 | Grouped Query Attn | 1 × 32768 × 4096 |
| 186 | Add_31_attn | Add | 1 × 32768 × 4096 |
| 187 | LayerNorm_31_2 | LayerNorm | 1 × 32768 × 4096 |
| 188 | FFN_31 | Feed Forward | 1 × 32768 × 4096 |
| 189 | Add_31_ffn | Add | 1 × 32768 × 4096 |
| 190 | LayerNorm_32_1 | LayerNorm | 1 × 32768 × 4096 |
| 191 | Attention_32 | Grouped Query Attn | 1 × 32768 × 4096 |
| 192 | Add_32_attn | Add | 1 × 32768 × 4096 |
| 193 | LayerNorm_32_2 | LayerNorm | 1 × 32768 × 4096 |
| 194 | FFN_32 | Feed Forward | 1 × 32768 × 4096 |
| 195 | Add_32_ffn | Add | 1 × 32768 × 4096 |
| 196 | Final_LayerNorm | LayerNorm | 1 × 32768 × 4096 |
| 197 | LM_Head | Linear | 1 × 32768 × 102400 |
| 198 | Output | Output | 1 × 32768 × 102400 |

## Do this to your own model

```
pip install neurarch-trace
neurarch-trace LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct-AWQ --plan --share
```

- Page: https://neurarch.com/m/exaone-3-5-7-8b-instruct-awq.html
- All models: https://neurarch.com/m/index.json
- The checks: https://neurarch.com/r/index.json