# Neural Collaborative Filtering

> He et al. 2017 NeuMF — GMF + MLP fused for recommendation

Pick for pure user/item collaborative filtering when you have no side features. Fused GMF+MLP variant; concat-only variant is `ncf`.

- Category: Recommendation
- Layers: 16
- Parameters: 105.61M
- Input shape (batchless): 1
- Output shape: 1 × 1
- Verifier verdict: pass
- Graph JSON: https://neurarch.com/templates/neumf/model.json
- Open on the canvas: https://neurarch.com/?template=neumf

## Structure

| # | Layer | Type | Parameters | Output shape |
| --- | --- | --- | --- | --- |
| 1 | User ID | Input | shape=[1] | 1 |
| 2 | Item ID | Input | shape=[1] | 1 |
| 3 | User MF Emb | Embedding | vocabSize=100000 | 1 × 32 |
| 4 | Item MF Emb | Embedding | vocabSize=1000000 | 1 × 32 |
| 5 | User MLP Emb | Embedding | vocabSize=100000 | 1 × 64 |
| 6 | Item MLP Emb | Embedding | vocabSize=1000000 | 1 × 64 |
| 7 | GMF (⊙) | Multiply |  | 1 × 32 |
| 8 | Concat | Concatenate |  | 1 × 128 |
| 9 | MLP FC 1 | Linear | outFeatures=64, inFeatures=128 | 1 × 64 |
| 10 | ReLU | ReLU |  | 1 × 64 |
| 11 | MLP FC 2 | Linear | outFeatures=32, inFeatures=64 | 1 × 32 |
| 12 | ReLU | ReLU |  | 1 × 32 |
| 13 | Fuse GMF+MLP | Concatenate |  | 1 × 64 |
| 14 | Predict | Linear | outFeatures=1, inFeatures=64 | 1 × 1 |
| 15 | Sigmoid | Sigmoid |  | 1 × 1 |
| 16 | P(rating) | Output |  | 1 × 1 |

## Verifier findings

- **info** `output-activation` at `Sigmoid`: "Sigmoid" feeds directly into Output. PyTorch's nn.CrossEntropyLoss already applies log-softmax internally, an explicit Softmax causes double-application and degrades training stability. Fix: Remove Softmax/Sigmoid for training. Restore it in a separate inference wrapper or ONNX export.
- **info** `vanishing-gradient` at `Sigmoid`: Sigmoid saturates to [0,1] / [-1,1], and its gradient approaches zero for large inputs. In networks deeper than 5 layers, this halts learning in early layers. Fix: Use ReLU, GELU, or SiLU for hidden layers. Keep Sigmoid only at binary classification outputs; Tanh in specific contexts (GAN generators, LSTM gates).
- **info** `deep-no-norm`: 13 layers with no BatchNorm, LayerNorm, or GroupNorm. Without normalization, activations can explode or vanish across layers, causing slow or unstable training. Fix: Add BatchNorm after Conv2d (CV tasks), LayerNorm after attention/FFN (NLP/LLM), or GroupNorm for small batch sizes.
- **info** `init-activation-mismatch` at `Predict`: PyTorch initializes Linear/Conv with Kaiming (He) init, which is derived for ReLU-family activations. Feeding a saturating activation (sigmoid/tanh) from a He-initialized layer starts training in the saturated tails, shrinking early gradients. Fix: Initialize these layers with Xavier instead: nn.init.xavier_uniform_(w, gain=nn.init.calculate_gain("sigmoid"|"tanh")), or switch the activation to a ReLU-family one.

## Exported PyTorch (first 46 lines)

```python
# Architecture designed with Neurarch: https://neurarch.com
# PyTorch: compatible with Python 3.8+ and torch>=1.12
# Colab: pip install torch torchvision  (usually pre-installed)

import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple

class NeuralCollaborativeFilteringNeuMF(nn.Module):
    def __init__(self):
        super().__init__()

        self.embedding_1 = nn.Embedding(100000, 32)
        self.embedding_2 = nn.Embedding(1000000, 32)
        self.embedding_3 = nn.Embedding(100000, 64)
        self.embedding_4 = nn.Embedding(1000000, 64)
        self.linear_1 = nn.Linear(128, 64)
        self.linear_2 = nn.Linear(64, 32)
        self.linear_3 = nn.Linear(64, 1)

    def forward(self, src, tgt=None):
        # User ID shape: [1]
        # Item ID shape: [1]
        embedding_mf_emb = self.embedding_1(src)
        embedding_mf_emb = self.embedding_2(tgt)
        embedding_lp_emb = self.embedding_3(src)
        embedding_lp_emb = self.embedding_4(tgt)
        multiply_mf_mul = embedding_mf_emb * embedding_mf_emb
        concatenate_lp_cat = torch.cat([embedding_lp_emb, embedding_lp_emb], dim=-1)
        linear_lp_fc1 = self.linear_1(concatenate_lp_cat)
        relu__relu1 = F.relu(linear_lp_fc1)
        linear_lp_fc2 = self.linear_2(relu__relu1)
        relu__relu2 = F.relu(linear_lp_fc2)
        concatenate_fuse = torch.cat([multiply_mf_mul, relu__relu2], dim=-1)
        linear_head = self.linear_3(concatenate_fuse)
        sigmoid_sig = torch.sigmoid(linear_head)
        # Output
        return sigmoid_sig


if __name__ == '__main__':
    model = NeuralCollaborativeFilteringNeuMF()
    model.eval()

    src = torch.randint(0, 1, (1))  # (batch, src_seq_len)
```

## Machine access

- Every architecture: https://neurarch.com/a/index.json
- Verify a graph of your own: `POST https://www.neurarch.com/api/v1/check` (see https://neurarch.com/developer.html)
- MCP server, so an agent edits the graph with the checks in the loop: https://neurarch.com/docs/mcp.md