Models / florence2
Florence-2-base-ft: GPU memory to fine-tune
Fine-tuning Florence-2-base-ft at batch 1 and 1024 tokens takes about 1.75 GB of GPU memory for a full fine-tune (fits one T4 16 GB), and 0.36 GB with LoRA r=16 on a bf16 base (fits one T4 16 GB). Estimated from the model's structure, reading the config under transformers semantics, not measured. What fits on which card.
Reconstructed from its own config.json
with no weights read. 225K downloads on Hugging Face.
Our count against the checkpoint
The left number comes from the graph. The right one is the number of scalars in the published weight files. Nothing on this page was tuned to make them agree.
custom-code This repository ships its own modeling code (`auto_map`, e.g. `configuration_florence2.py`), so `config.json` names a class in the repo rather than an architecture `transformers` defines. The graph below is what those config keys mean under `transformers` semantics, which is not necessarily what the repo's own file builds. A gap here is a statement about what we read, not about the checkpoint.
What it costs to run
Cost is a roofline estimate on the priced GPU for 10 epochs at batch 32, 1024 tokens over 50,000 samples (assumed; no dataset attached).
Memory: what fits on which card
Largest batch per card. Training is sized at 1024 tokens per sample, serving at 1024 tokens per sequence.
| Card | Full fine-tune | LoRA r=16 | Serve bf16, 1024 tokens |
|---|---|---|---|
| T4 16 GB | batch 74 | batch 82 | batch 728 |
| L4 24 GB | batch 116 | batch 124 | batch 1096 |
| A100 40 GB | batch 200 | batch 208 | batch 1834 |
| A100 / H100 80 GB | batch 410 | batch 418 | batch 3676 |
| H200 141 GB | batch 730 | batch 738 | batch 6487 |
Training: 1024 tokens per sample (2048, or the model's context when shorter), AdamW with bf16 autocast, SDPA attention (no score matrix stored), 1.15x headroom on every card. Full fine-tune holds weights, gradients and optimizer state (1.2 x 16 bytes per parameter) plus activations. LoRA is r=16 on every linear layer but the head, base frozen in bf16 (2 bytes per parameter), adapters at full AdamW state, same activations. A two-card split is a contiguous layer cut (pipeline parallel), computed only where one card cannot hold batch 1. Serving: bf16 weights plus the KV cache of 1024 tokens per sequence; prefill working set, CUDA context and framework reserve not counted.
These are estimates from the graph, not measurements. Against measured runs, held out by architecture, the estimate flagged 113 of the 131 jobs that did not fit their card and said "fits" on the other 18. Every measured run.
Structure
28 nodes. Output shapes are propagated from the input shape, batch dimension excluded, at 1024 tokens: the config's maximum context.
| Layer | Type | Output shape | |
|---|---|---|---|
| 1 | Input | Input | 1 × 1024 |
| 2 | Embedding | Embedding | 1 × 1024 × 768 |
| 3 | Positional_Embedding | Learned Pos Embed | 1 × 1024 × 768 |
| 4 | Attention_1 | Multi-Head Attention | 1 × 1024 × 768 |
| 5 | Add_1 | Add | 1 × 1024 × 768 |
| 6 | LayerNorm_1_1 | LayerNorm | 1 × 1024 × 768 |
| 7 | FFN_1 | Feed Forward | 1 × 1024 × 768 |
| 8 | Attention_2 | Multi-Head Attention | 1 × 1024 × 768 |
| 9 | Add_2 | Add | 1 × 1024 × 768 |
| 10 | LayerNorm_2_1 | LayerNorm | 1 × 1024 × 768 |
| 11 | FFN_2 | Feed Forward | 1 × 1024 × 768 |
| 12 | Attention_3 | Multi-Head Attention | 1 × 1024 × 768 |
| 13 | Add_3 | Add | 1 × 1024 × 768 |
| 14 | LayerNorm_3_1 | LayerNorm | 1 × 1024 × 768 |
| 15 | FFN_3 | Feed Forward | 1 × 1024 × 768 |
| 16 | Attention_4 | Multi-Head Attention | 1 × 1024 × 768 |
| 17 | Add_4 | Add | 1 × 1024 × 768 |
| 18 | LayerNorm_4_1 | LayerNorm | 1 × 1024 × 768 |
| 19 | FFN_4 | Feed Forward | 1 × 1024 × 768 |
| 20 | Attention_5 | Multi-Head Attention | 1 × 1024 × 768 |
| 21 | Add_5 | Add | 1 × 1024 × 768 |
| 22 | LayerNorm_5_1 | LayerNorm | 1 × 1024 × 768 |
| 23 | FFN_5 | Feed Forward | 1 × 1024 × 768 |
| 24 | Attention_6 | Multi-Head Attention | 1 × 1024 × 768 |
| 25 | Add_6 | Add | 1 × 1024 × 768 |
| 26 | LayerNorm_6_1 | LayerNorm | 1 × 1024 × 768 |
| 27 | FFN_6 | Feed Forward | 1 × 1024 × 768 |
| 28 | Output | Output | 1 × 1024 × 768 |
What the verifier says
No finding on the reconstructed graph. See the checks.
What changed upstream
The change tracker follows microsoft/Florence-2-base-ft: every revision it has read, what changed in the config, tokenizer, chat template or weights between them, and an OSV advisory per change.
Do this to your own model
Same numbers, on a model in your repo, in one command. No account.
pip install neurarch-trace
neurarch-trace microsoft/Florence-2-base-ft --plan --share