# Wide & Deep vs Neural Collaborative Filtering

The memorisation-plus-generalisation recommender against a purely neural one.

**Neural Collaborative Filtering has 32M more parameters than Wide & Deep: 2 layers added, 3 removed, 5 changed.**

Source: https://neurarch.com/diff/wide-and-deep-vs-ncf.html

## Sides

| | Wide & Deep | Neural Collaborative Filtering |
|---|---|---|
| Layers | 10 | 9 |
| Parameters | 3.7M | 35M |
| Input | 10000 | 1 |
| Output | 1 | 1 × 1 |
| Forward-passes | yes | yes |
| Est. train cost | $0.043 | $0.053 |
| T4 16GB | fits | fits |
| A100 40GB | fits | fits |
| H100 80GB | fits | fits |

## Deltas (Neural Collaborative Filtering relative to Wide & Deep)

- Parameters: +32M (9.6× the size)
- Layers: -1
- Added 2, removed 3, changed 5, unchanged 5

## Layer by layer

| # | Status | Wide & Deep | Params | Output | Neural Collaborative Filtering | Params | Output |
|---|---|---|---|---|---|---|---|
| 1 | changed (shape) | Wide Input (cross feats) (Input) |  | 10000 | User ID (Input) |  | 1 |
| 2 | changed (shape) | Deep Input (sparse cat) (Input) |  | 50 | Item ID (Input) |  | 1 |
| 3 | removed | Wide Linear (Linear) | 10K | 1 |  | |  |
| 4 | same | Embeddings (Embedding) | 3.2M | 50 × 32 | User Embedding (Embedding) | 3.2M | 1 × 32 |
| 5 | removed | Flatten (Flatten) |  | 1600 |  | |  |
| 6 | added |  | |  | Item Embedding (Embedding) | 32M | 1 × 32 |
| 7 | added |  | |  | Concat [u; i] (Concatenate) |  | 1 × 64 |
| 8 | changed (inFeatures, outFeatures) | Deep FC 1 (Linear) | 410K | 256 | MLP 1 (Linear) | 4.2K | 1 × 64 |
| 9 | same | ReLU 1 (Relu) |  | 256 | ReLU 1 (Relu) |  | 1 × 64 |
| 10 | changed (inFeatures, outFeatures) | Deep FC 2 (Linear) | 33K | 128 | MLP 2 (Linear) | 2.1K | 1 × 32 |
| 11 | same | ReLU 2 (Relu) |  | 128 | ReLU 2 (Relu) |  | 1 × 32 |
| 12 | changed (inFeatures) | Deep Out (Linear) | 129 | 1 | Score Head (Linear) | 33 | 1 × 1 |
| 13 | removed | Wide + Deep (Add) |  | 1 |  | |  |
| 14 | same | Sigmoid CTR (Sigmoid) |  | 1 | Sigmoid (Sigmoid) |  | 1 × 1 |
| 15 | same | P(click) (Output) |  | 1 | P(click) (Output) |  | 1 × 1 |

## What this is not

- The two are priced at different declared inputs (10000 against 1), so memory, cost and GPU fit are each right about their own model and are not a comparison between them. The layer and parameter deltas are unaffected.
- Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
- Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.

## Graphs

- Wide & Deep: https://neurarch.com/templates/wide-and-deep/model.json
- Neural Collaborative Filtering: https://neurarch.com/templates/ncf/model.json
- Check a graph of your own: `POST https://www.neurarch.com/api/v1/plan`