# EEG Conformer

> Conv stem + Transformer encoder — SOTA for high-channel motor imagery EEG (Song 2023)

Pick when you have ≥16 channels and ~400+ trials per subject. Best published accuracy on BCI IV-2a/2b; expect tricky regularization.

- Category: Biosignal
- Layers: 23
- Parameters: 78.6K
- Input shape (batchless): 1 × 22 × 1000
- Output shape: 4
- Verifier verdict: warn
- Graph JSON: https://neurarch.com/templates/eeg-conformer/model.json
- Open on the canvas: https://neurarch.com/?template=eeg-conformer

## Structure

| # | Layer | Type | Parameters | Output shape |
| --- | --- | --- | --- | --- |
| 1 | eeg_window | Input | shape=[1, 22, 1000] | 1 × 22 × 1000 |
| 2 | temporal_conv | Conv2D | outChannels=40, kernelSize=[1, 25], stride=1 | 40 × 22 × 976 |
| 3 | spatial_conv | Conv2D | outChannels=40, kernelSize=[22, 1], stride=1 | 40 × 1 × 976 |
| 4 | norm | BatchNorm | normalizedShape=40 | 40 × 1 × 976 |
| 5 | act | ELU |  | 40 × 1 × 976 |
| 6 | pool | AvgPool2D | kernelSize=[1, 75], stride=[1, 15] | 40 × 1 × 61 |
| 7 | patch_proj | Conv2D | outChannels=40, kernelSize=[1, 1], stride=1 | 40 × 1 × 61 |
| 8 | to_tokens | Permute |  | 61 × 1 × 40 |
| 9 | norm | LayerNorm | normalizedShape=40 | 61 × 1 × 40 |
| 10 | self_attn | Multi-Head Attention | embedDim=40, numHeads=10 | 61 × 1 × 40 |
| 11 | residual | Add |  | 61 × 1 × 40 |
| 12 | norm | LayerNorm | normalizedShape=40 | 61 × 1 × 40 |
| 13 | dense | Feed Forward | embedDim=40, ffDim=160 | 61 × 1 × 40 |
| 14 | residual | Add |  | 61 × 1 × 40 |
| 15 | norm | LayerNorm | normalizedShape=40 | 61 × 1 × 40 |
| 16 | self_attn | Multi-Head Attention | embedDim=40, numHeads=10 | 61 × 1 × 40 |
| 17 | residual | Add |  | 61 × 1 × 40 |
| 18 | norm | LayerNorm | normalizedShape=40 | 61 × 1 × 40 |
| 19 | dense | Feed Forward | embedDim=40, ffDim=160 | 61 × 1 × 40 |
| 20 | residual | Add |  | 61 × 1 × 40 |
| 21 | flatten | Flatten |  | 2440 |
| 22 | classifier | Linear | outFeatures=4 | 4 |
| 23 | logits | Output |  | 4 |

## Verifier findings

- **warn** `attention-no-pe` at `self_attn`: 2 attention layer(s) present but no positional encoding found. Attention is permutation-invariant, without position information the model cannot distinguish token order. Fix: Add a PositionalEncoding (sinusoidal) or RoPE layer before the first attention layer.

## Exported PyTorch (first 46 lines)

```python
# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)
#
# WARNING: 1 layer(s) below are not yet supported by the PyTorch
# exporter and pass their input through UNCHANGED in forward():
#   - to_tokens (permute)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class EEGConformer(nn.Module):
    def __init__(self):
        super().__init__()

        self.conv2d_1 = nn.Conv2d(1, 40, kernel_size=(1, 25), stride=1, padding=0)
        self.conv2d_2 = nn.Conv2d(40, 40, kernel_size=(22, 1), stride=1, padding=0)
        self.batchNorm_1 = nn.BatchNorm2d(40)
        self.elu_1 = nn.ELU(alpha=1)
        self.avgpool2d_1 = nn.AvgPool2d(kernel_size=(1, 75), stride=(1, 15), padding=0)
        self.conv2d_3 = nn.Conv2d(40, 40, kernel_size=(1, 1), stride=1, padding=0)
        self.layerNorm_1 = nn.LayerNorm(40)
        self.multiHeadAttention_1 = nn.MultiheadAttention(embed_dim=40, num_heads=10, batch_first=True)
        self.layerNorm_2 = nn.LayerNorm(40)
        self.feedForward_1 = nn.Sequential(
            nn.Linear(40, 160),
            nn.ReLU(),
            nn.Linear(160, 40)
        )
        self.layerNorm_3 = nn.LayerNorm(40)
        self.multiHeadAttention_2 = nn.MultiheadAttention(embed_dim=40, num_heads=10, batch_first=True)
        self.layerNorm_4 = nn.LayerNorm(40)
        self.feedForward_2 = nn.Sequential(
            nn.Linear(40, 160),
            nn.ReLU(),
            nn.Linear(160, 40)
        )
        self.linear_1 = nn.Linear(2440, 4)

    def forward(self, x):
        # eeg_window shape: [1,22,1000]
        conv2d_mporal = self.conv2d_1(x)
        conv2d_patial = self.conv2d_2(conv2d_mporal)
        batch_norm_bn_1 = self.batchNorm_1(conv2d_patial)
```

## Machine access

- Every architecture: https://neurarch.com/a/index.json
- Verify a graph of your own: `POST https://www.neurarch.com/api/v1/check` (see https://neurarch.com/developer.html)
- MCP server, so an agent edits the graph with the checks in the loop: https://neurarch.com/docs/mcp.md