# Whisper Small

> Whisper speech encoder-decoder — conv1d audio stem + transformer encoder/decoder (384D)

Pick for ASR or as a pretrained audio encoder — drop the decoder + add a head for audio classification (UrbanSound, ESC-50).

- Category: Audio
- Layers: 18
- Parameters: 46.99M
- Input shape (batchless): 1 × 80 × 3000
- Output shape: 1 × 448 × 51865
- Verifier verdict: pass
- Graph JSON: https://neurarch.com/templates/whisper-small/model.json
- Open on the canvas: https://neurarch.com/?template=whisper-small

## Structure

| # | Layer | Type | Parameters | Output shape |
| --- | --- | --- | --- | --- |
| 1 | mel_features | Input | shape=[1, 80, 3000] | 1 × 80 × 3000 |
| 2 | conv1 | Audio Conv | outChannels=384, kernelSize=3, stride=1 | 1 × 384 × 3000 |
| 3 | gelu_1 | GELU |  | 1 × 384 × 3000 |
| 4 | conv2 | Audio Conv | outChannels=384, kernelSize=3, stride=2 | 1 × 384 × 1500 |
| 5 | gelu_2 | GELU |  | 1 × 384 × 1500 |
| 6 | to_tokens | Permute |  | 1 × 1500 × 384 |
| 7 | enc_pos_emb | Positional Encoding | embedDim=384, maxLen=1500 | 1 × 1500 × 384 |
| 8 | enc_block_1 | Transformer Block | embedDim=384, numHeads=6, ffDim=1536 | 1 × 1500 × 384 |
| 9 | enc_block_2 | Transformer Block | embedDim=384, numHeads=6, ffDim=1536 | 1 × 1500 × 384 |
| 10 | enc_norm | LayerNorm | normalizedShape=384 | 1 × 1500 × 384 |
| 11 | decoder_tokens | Input | shape=[1, 448] | 1 × 448 |
| 12 | token_embed | Embedding |  | 1 × 448 × 384 |
| 13 | dec_pos_emb | Positional Encoding | embedDim=384, maxLen=448 | 1 × 448 × 384 |
| 14 | dec_block_1 | Transformer Block | embedDim=384, numHeads=6, ffDim=1536 | 1 × 448 × 384 |
| 15 | dec_block_2 | Transformer Block | embedDim=384, numHeads=6, ffDim=1536 | 1 × 448 × 384 |
| 16 | dec_norm | LayerNorm | normalizedShape=384 | 1 × 448 × 384 |
| 17 | lm_head | Linear | outFeatures=51865 | 1 × 448 × 51865 |
| 18 | token_logits | Output |  | 1 × 448 × 51865 |

## Verifier findings

No finding. Shapes propagate end to end and no advisory rule fires.

## Exported PyTorch (first 46 lines)

```python
# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)
# Audio: pip install torchaudio
#
# WARNING: 1 layer(s) below are not yet supported by the PyTorch
# exporter and pass their input through UNCHANGED in forward():
#   - to_tokens (permute)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple
import torchaudio

class WhisperSmall(nn.Module):
    def __init__(self):
        super().__init__()

        self.audioConv_1 = nn.Conv1d(1, 384, kernel_size=3, stride=1, padding=1)
        self.gelu_1 = nn.GELU()
        self.audioConv_2 = nn.Conv1d(1, 384, kernel_size=3, stride=2, padding=1)
        self.gelu_2 = nn.GELU()
        self.transformerBlock_1 = nn.TransformerEncoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.transformerBlock_2 = nn.TransformerEncoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.layerNorm_1 = nn.LayerNorm(384)
        self.embedding_1 = nn.Embedding(51865, 384)
        self.transformerBlock_3 = nn.TransformerDecoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.transformerBlock_4 = nn.TransformerDecoderLayer(d_model=384, nhead=6, dim_feedforward=1536, batch_first=True)
        self.layerNorm_2 = nn.LayerNorm(384)
        self.linear_1 = nn.Linear(384, 51865)

    def forward(self, src, tgt=None):
        # mel_features shape: [1,80,3000]
        # decoder_tokens shape: [1,448]
        audio_conv_Conv_1 = self.audioConv_1(src)
        gelu_gelu_1 = self.gelu_1(audio_conv_Conv_1)
        audio_conv_Conv_2 = self.audioConv_2(gelu_gelu_1)
        gelu_gelu_2 = self.gelu_2(audio_conv_Conv_2)
        # TODO: layer 'to_tokens' (permute) is not yet supported by the exporter; passing through unchanged
        # positionalEncoding: add positional encoding externally (e.g. sinusoidal or learned PE)
        transformer_block__enc_1 = self.transformerBlock_1(gelu_gelu_2)
        transformer_block__enc_2 = self.transformerBlock_2(transformer_block__enc_1)
        layer_norm_rm_enc = self.layerNorm_1(transformer_block__enc_2)
        embedding_ng_dec = self.embedding_1(tgt)
        # positionalEncoding: add positional encoding externally (e.g. sinusoidal or learned PE)
```

## Machine access

- Every architecture: https://neurarch.com/a/index.json
- Verify a graph of your own: `POST https://www.neurarch.com/api/v1/check` (see https://neurarch.com/developer.html)
- MCP server, so an agent edits the graph with the checks in the loop: https://neurarch.com/docs/mcp.md