# Latent Video Diffusion Transformer

> The shape every 2024-2025 video generator shares: a causal 3D VAE compresses the clip, the latent is patchified into space-time tokens, a DiT stack denoises them.

Pick as the starting point for text-to-video or video-to-video work. Check the token count first: space-time patches grow with frames as well as resolution.

- Category: Generative
- Layers: 15
- Parameters: 99.31M
- Input shape (batchless): 3 × 16 × 256 × 256
- Output shape: 1024 × 64
- Verifier verdict: pass
- Graph JSON: https://neurarch.com/templates/video-dit/model.json
- Open on the canvas: https://neurarch.com/?template=video-dit

## Structure

| # | Layer | Type | Parameters | Output shape |
| --- | --- | --- | --- | --- |
| 1 | Video clip (16 frames, 256x256) | Input | shape=[3, 16, 256, 256] | 3 × 16 × 256 × 256 |
| 2 | Causal VAE Down 128 | Causal Conv3D | outChannels=128, inChannels=3, kernelSize=3 | 128 × 16 × 128 × 128 |
| 3 | Causal VAE Down 256 | Causal Conv3D | outChannels=256, inChannels=128, kernelSize=3 | 256 × 8 × 64 × 64 |
| 4 | Causal VAE to latent 16 | Causal Conv3D | outChannels=16, inChannels=256, kernelSize=3 | 16 × 4 × 32 × 32 |
| 5 | Patchify 1x2x2 | Tubelet Embedding (3D Patch) | embedDim=1152 | 1024 × 1152 |
| 6 | Pos Encoding | Positional Encoding | maxLen=1024 | 1024 × 1152 |
| 7 | Diffusion timestep | Input | shape=[1] | 1 |
| 8 | Timestep Embed | Time Embedding |  | 1152 |
| 9 | DiT Block 1 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 10 | DiT Block 2 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 11 | DiT Block 3 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 12 | DiT Block 4 (AdaLN-Zero) | DiT Block (AdaLN-Zero) | numHeads=16 | 1024 × 1152 |
| 13 | Final LayerNorm | LayerNorm | normalizedShape=1152 | 1024 × 1152 |
| 14 | Unpatchify to latent | Linear | outFeatures=64, inFeatures=1152 | 1024 × 64 |
| 15 | predicted latent velocity | Output |  | 1024 × 64 |

## Verifier findings

No finding. Shapes propagate end to end and no advisory rule fires.

## Exported PyTorch (first 46 lines)

```python
# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class TimestepEmbedding(nn.Module):
    """Sinusoidal diffusion-timestep features, then an MLP. Takes a scalar per
    sample and returns one conditioning vector per sample."""

    def __init__(self, dim: int = 256):
        super().__init__()
        self.dim = dim
        self.mlp = nn.Sequential(nn.Linear(dim, dim), nn.SiLU(), nn.Linear(dim, dim))

    def forward(self, t: torch.Tensor) -> torch.Tensor:
        t = t.reshape(t.size(0), -1)[:, 0].float()
        half = self.dim // 2
        freqs = torch.exp(
            -torch.arange(half, device=t.device, dtype=t.dtype) * (9.2103403719762 / max(1, half - 1)))
        ang = t[:, None] * freqs[None]
        emb = torch.cat([ang.cos(), ang.sin()], dim=-1)
        if emb.size(-1) < self.dim:
            emb = F.pad(emb, (0, self.dim - emb.size(-1)))
        return self.mlp(emb)


class DiTBlock(nn.Module):
    """DiT block with adaLN-Zero: the conditioning vector produces the scale,
    shift and gate for both sublayers, and the gates start at zero so a fresh
    block is the identity."""

    def __init__(self, hidden_dim: int = 1152, num_heads: int = 16, cond_dim: int = 1152):
        super().__init__()
        self.norm1 = nn.LayerNorm(hidden_dim, elementwise_affine=False)
        self.attn = nn.MultiheadAttention(hidden_dim, num_heads, batch_first=True)
        self.norm2 = nn.LayerNorm(hidden_dim, elementwise_affine=False)
        self.mlp = nn.Sequential(
            nn.Linear(hidden_dim, 4 * hidden_dim), nn.GELU(), nn.Linear(4 * hidden_dim, hidden_dim))
        self.ada = nn.Sequential(nn.SiLU(), nn.Linear(cond_dim, 6 * hidden_dim))
        nn.init.zeros_(self.ada[1].weight)
        nn.init.zeros_(self.ada[1].bias)
```

## Machine access

- Every architecture: https://neurarch.com/a/index.json
- Verify a graph of your own: `POST https://www.neurarch.com/api/v1/check` (see https://neurarch.com/developer.html)
- MCP server, so an agent edits the graph with the checks in the loop: https://neurarch.com/docs/mcp.md