# EEGNet vs EEG Conformer

A compact EEG convnet against a convolution-plus-transformer one.

**EEG Conformer has 76K more parameters than EEGNet: 12 layers added, 5 removed, 6 changed.**

Source: https://neurarch.com/diff/eegnet-vs-eeg-conformer.html

## Sides

| | EEGNet | EEG Conformer |
|---|---|---|
| Layers | 14 | 21 |
| Parameters | 2.7K | 79K |
| Input | 1 × 22 × 1000 | 1 × 22 × 1000 |
| Output | 4 | 4 |
| Forward-passes | yes | yes |
| Est. train cost | $0.044 | $0.045 |
| T4 16GB | fits | fits |
| A100 40GB | fits | fits |
| H100 80GB | fits | fits |

## Deltas (EEG Conformer relative to EEGNet)

- Parameters: +76K (29× the size)
- Layers: +7
- Added 12, removed 5, changed 6, unchanged 5

## Layer by layer

| # | Status | EEGNet | Params | Output | EEG Conformer | Params | Output |
|---|---|---|---|---|---|---|---|
| 1 | same | eeg_window (Input) |  | 1 × 22 × 1000 | eeg_window (Input) |  | 1 × 22 × 1000 |
| 2 | changed (outChannels, kernelSize, padding) | temporal_conv (Conv2d) | 80 | 8 × 22 × 1001 | temporal_conv (Conv2d) | 400 | 40 × 22 × 976 |
| 3 | removed | norm (Batch Norm) | 16 | 8 × 22 × 1001 |  | |  |
| 4 | changed (type, outChannels, depthMultiplier) | spatial_depthwise (Depthwise Conv2d) | 20 | 16 × 1 × 1001 | spatial_conv (Conv2d) | 400 | 40 × 1 × 976 |
| 5 | changed (normalizedShape) | norm (Batch Norm) | 32 | 16 × 1 × 1001 | norm (Batch Norm) | 80 | 40 × 1 × 976 |
| 6 | same | act (Elu) |  | 16 × 1 × 1001 | act (Elu) |  | 40 × 1 × 976 |
| 7 | changed (kernelSize, stride) | pool (Avgpool2d) |  | 16 × 1 × 250 | pool (Avgpool2d) |  | 40 × 1 × 61 |
| 8 | removed | drop (Dropout) |  | 16 × 1 × 250 |  | |  |
| 9 | changed (type, outChannels, kernelSize, padding) | separable_conv (Separable Conv2d) | 41 | 16 × 1 × 251 | patch_proj (Conv2d) | 400 | 40 × 1 × 61 |
| 10 | added |  | |  | to_tokens (Reshape) |  | 61 × 40 |
| 11 | changed (type, normalizedShape) | norm (Batch Norm) | 32 | 16 × 1 × 251 | norm (Layer Norm) | 80 | 61 × 40 |
| 12 | removed | act (Elu) |  | 16 × 1 × 251 |  | |  |
| 13 | removed | pool (Avgpool2d) |  | 16 × 1 × 31 |  | |  |
| 14 | removed | drop (Dropout) |  | 16 × 1 × 31 |  | |  |
| 15 | added |  | |  | self_attn (Multi Head Attention) | 6.6K | 61 × 40 |
| 16 | added |  | |  | residual (Add) |  | 61 × 40 |
| 17 | added |  | |  | norm (Layer Norm) | 80 | 61 × 40 |
| 18 | added |  | |  | dense (Feed Forward) | 13K | 61 × 40 |
| 19 | added |  | |  | residual (Add) |  | 61 × 40 |
| 20 | added |  | |  | norm (Layer Norm) | 80 | 61 × 40 |
| 21 | added |  | |  | self_attn (Multi Head Attention) | 6.6K | 61 × 40 |
| 22 | added |  | |  | residual (Add) |  | 61 × 40 |
| 23 | added |  | |  | norm (Layer Norm) | 80 | 61 × 40 |
| 24 | added |  | |  | dense (Feed Forward) | 13K | 61 × 40 |
| 25 | added |  | |  | residual (Add) |  | 61 × 40 |
| 26-28 | same | 3 unchanged layers | | | | | |

## What this is not

- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.

## Graphs

- EEGNet: https://neurarch.com/templates/eegnet/model.json
- EEG Conformer: https://neurarch.com/templates/eeg-conformer/model.json
- Check a graph of your own: `POST https://www.neurarch.com/api/v1/plan`