Comparisons / 1D CNN + LSTM vs PatchTST

1D CNN + LSTM vs PatchTST

Convolution-plus-recurrence against patched attention for time series.

PatchTST has 84K more parameters than 1D CNN + LSTM: 14 layers added, 9 removed, 4 changed.

Baseline

1D CNN + LSTM

Layers: 12
Parameters: 312K
Input: 12 × 5000
Output: 5
Forward-passes: yes
Est. train cost: $0.059

Compared

PatchTST

Layers: 17
Parameters: 395K
Input: 1 × 22 × 1000
Output: 4
Forward-passes: yes
Est. train cost: $0.043

The deltas

Every number is PatchTST relative to 1D CNN + LSTM.

Parameters

+84K (+26.8%)

Layers

Added

Removed

Changed

Unchanged

Which GPUs each one fits

Each side is measured at its own declared input (12 × 5000 against 1 × 22 × 1000). Both columns are right about their own model; the difference between them is not a fact about the designs.

GPU	1D CNN + LSTM	PatchTST
T4 16GB	fits	fits
A100 40GB	fits	fits
H100 80GB	fits	fits

Layer by layer

Aligned in topological order. 1 of 28 rows are the same layer with the same parameters.

Hide all 28 rows

		1D CNN + LSTM			PatchTST
		Layer	Params	Output	Layer	Params	Output
1	changed shape	ts_window Input		12 × 5000	ts_window Input		1 × 22 × 1000
2	removed	conv1 Conv1d	512	64 × 5000	—
3	added	—			patch_embed Patch Embed	98K	62 × 128
4	added	—			pos_embed Positional Encoding		62 × 128
5	changed type, normalizedShape	bn Batch Norm	128	64 × 5000	norm Layer Norm	256	62 × 128
6	removed	act Relu		64 × 5000	—
7	removed	pool Maxpool1d		64 × 2500	—
8	removed	conv2 Conv1d	768	128 × 2500	—
9	added	—			self_attn Multi Head Attention	66K	62 × 128
10	added	—			residual Add		62 × 128
11	changed type	bn Batch Norm	256	128 × 2500	norm Layer Norm	256	62 × 128
12	removed	act Relu		128 × 2500	—
13	removed	pool Maxpool1d		128 × 1250	—
14	removed	to_timesteps Permute		1250 × 128	—
15	removed	lstm Lstm	199K	128	—
16	removed	drop Dropout		128	—
17	added	—			dense Feed Forward	66K	62 × 128
18	added	—			residual Add		62 × 128
19	added	—			norm Layer Norm	256	62 × 128
20	added	—			self_attn Multi Head Attention	66K	62 × 128
21	added	—			residual Add		62 × 128
22	added	—			norm Layer Norm	256	62 × 128
23	added	—			dense Feed Forward	66K	62 × 128
24	added	—			residual Add		62 × 128
25	added	—			norm Layer Norm	256	62 × 128
26	added	—			flatten Flatten		7936
27	changed outFeatures	classifier Linear		5	classifier Linear		4
28	same	logits Output		5	logits Output		4

What this is not

The two are priced at different declared inputs (12 × 5000 against 1 × 22 × 1000), so memory, cost and GPU fit are each right about their own model and are not a comparison between them. The layer and parameter deltas are unaffected.
Parameter counts are derived from the graph, not read from a checkpoint. They are exact for a graph that is fully specified and approximate for one that is not.
Cost and GPU fit are estimates from the graph under one set of assumptions, not measurements of a run.

Take it further

Open either graph in the editor, change it, and check it again: 1D CNN + LSTM · PatchTST

Compare any two models of your own, including anything on Hugging Face: the comparison tool.

Machine-readable: this page as markdown · the pair index · POST https://www.neurarch.com/api/v1/plan for a graph of your own.