# Qwen3.8-27B-DSpark

Hugging Face: `RadixArk/Qwen3.8-27B-DSpark`
Reconstructed from `config.json` alone. No weights were read.

| | |
|---|---|
| Derived from structure | 4,142,179,840 |
| In the published checkpoint | 1,857,358,337 (safetensors.total, 2026-08-29) |
| Delta | +123% |
| Comparability | custom-code |
| Layers | 34 |
| Priced on | A10G (24GB) |
| Est. one run | $80838.69 |

> This repository ships its own modeling code (`auto_map`, e.g. `dspark.py`), so `config.json` names a class in the repo rather than an architecture `transformers` defines. The graph below is what those config keys mean under `transformers` semantics, which is not necessarily what the repo's own file builds. A gap here is a statement about what we read, not about the checkpoint.

## Structure

| # | Layer | Type | Output shape |
|---|---|---|---|
| 1 | Input | Input | 1 × 262144 |
| 2 | Embedding | Embedding | 1 × 262144 × 5120 |
| 3 | RoPE | RoPE | 1 × 262144 × 5120 |
| 4 | RMSNorm_1_1 | RMSNorm | 1 × 262144 × 5120 |
| 5 | Attention_1 | Grouped Query Attn | 1 × 262144 × 5120 |
| 6 | Add_1_attn | Add | 1 × 262144 × 5120 |
| 7 | RMSNorm_1_2 | RMSNorm | 1 × 262144 × 5120 |
| 8 | FFN_1 | SwiGLU | 1 × 262144 × 5120 |
| 9 | Add_1_ffn | Add | 1 × 262144 × 5120 |
| 10 | RMSNorm_2_1 | RMSNorm | 1 × 262144 × 5120 |
| 11 | Attention_2 | Grouped Query Attn | 1 × 262144 × 5120 |
| 12 | Add_2_attn | Add | 1 × 262144 × 5120 |
| 13 | RMSNorm_2_2 | RMSNorm | 1 × 262144 × 5120 |
| 14 | FFN_2 | SwiGLU | 1 × 262144 × 5120 |
| 15 | Add_2_ffn | Add | 1 × 262144 × 5120 |
| 16 | RMSNorm_3_1 | RMSNorm | 1 × 262144 × 5120 |
| 17 | Attention_3 | Grouped Query Attn | 1 × 262144 × 5120 |
| 18 | Add_3_attn | Add | 1 × 262144 × 5120 |
| 19 | RMSNorm_3_2 | RMSNorm | 1 × 262144 × 5120 |
| 20 | FFN_3 | SwiGLU | 1 × 262144 × 5120 |
| 21 | Add_3_ffn | Add | 1 × 262144 × 5120 |
| 22 | RMSNorm_4_1 | RMSNorm | 1 × 262144 × 5120 |
| 23 | Attention_4 | Grouped Query Attn | 1 × 262144 × 5120 |
| 24 | Add_4_attn | Add | 1 × 262144 × 5120 |
| 25 | RMSNorm_4_2 | RMSNorm | 1 × 262144 × 5120 |
| 26 | FFN_4 | SwiGLU | 1 × 262144 × 5120 |
| 27 | Add_4_ffn | Add | 1 × 262144 × 5120 |
| 28 | RMSNorm_5_1 | RMSNorm | 1 × 262144 × 5120 |
| 29 | Attention_5 | Grouped Query Attn | 1 × 262144 × 5120 |
| 30 | Add_5_attn | Add | 1 × 262144 × 5120 |
| 31 | RMSNorm_5_2 | RMSNorm | 1 × 262144 × 5120 |
| 32 | FFN_5 | SwiGLU | 1 × 262144 × 5120 |
| 33 | Add_5_ffn | Add | 1 × 262144 × 5120 |
| 34 | Final_RMSNorm | RMSNorm | 1 × 262144 × 5120 |
| 35 | LM_Head | Linear | 1 × 262144 × 248320 |
| 36 | Output | Output | 1 × 262144 × 248320 |

## Do this to your own model

```
pip install neurarch-trace
neurarch-trace RadixArk/Qwen3.8-27B-DSpark --plan --share
```

- Page: https://neurarch.com/m/qwen3-8-27b-dspark.html
- All models: https://neurarch.com/m/index.json
- The checks: https://neurarch.com/r/index.json