Data / Research
A pinned revision does not pin the code: 17 Hugging Face models load it from another repository
Neurarch research, 2026-11-17. Counts are dated readings; method and limits at the end.
Summary. When a Hugging Face model's config.json maps its classes to code in a different repository ("AutoModel": "other/repo--modeling.Model"), passing revision=<commit> with trust_remote_code=True pins the weights and config but not the code. transformers loads that code from the other repository's main unless you also pass code_revision=. Of the 608 open-weight models we track, 150 run code from a repository and 17 load it from another one. Those 17 were downloaded 19.3 million times in the last 30 days, and for 11 of them (18.3 million downloads) the code repository has changed its Python since the model started pointing at it.
This is known to maintainers (see "Prior reports" below). What this report adds is the measured exposure, two reproducible cases, and a scanner that finds the pattern in your own code.
The mechanism
In transformers/dynamic_module_utils.py (get_class_from_dynamic_module, v4.57.1 lines 595 to 602):
if "--" in class_reference:
repo_id, class_reference = class_reference.split("--")
else:
repo_id = pretrained_model_name_or_path
...
if code_revision is None and pretrained_model_name_or_path == repo_id:
code_revision = revision
revision is carried over to the code only when the code lives in the model's own repository. For a cross-repository reference, code_revision stays None and the file is fetched from the other repository's default branch.
Reproduction (run 2026-11-17, transformers 4.57.1 and 5.23.0, no GPU, no torch needed)
from transformers import AutoConfig
AutoConfig.from_pretrained("nomic-ai/nomic-embed-text-v1.5",
revision="b0753ae76394dd36bcfb912a46018088bca48be0",
trust_remote_code=True)
The Hugging Face cache afterwards holds the model at b0753ae (2024-06-05) and nomic-ai/nomic-bert-2048 at 7710840 (2025-04-29), its main today. transformers prints "A new version of the following files was downloaded". Adding code_revision="1fe3d3fc459a66fb1bd71ddbf08250250521b268" loads the code at that commit instead.
Two cases
1. A public benchmark's pin. MTEB's model registry pins nomic-ai/nomic-embed-text-v1.5 at b0753ae with trust_remote_code=True and loads it through sentence-transformers. The code that runs is nomic-bert-2048 at its 2025-04-29 head, 23 Python changes after that pin, including commits titled "fix: dtype" and "fix: mask". These read as ordinary bug fixes by the model's own authors; we found nothing malicious. The point is that the pin does not decide which code runs, so a result reproduced "at the pinned revision" may have run different code.
2. A pin that no longer loads. Pinned at its first commit with an auto_map (e178409, 2024-02-10), nomic-ai/nomic-embed-text-v1.5 pointed at nomic-ai/nomic-embed-text-v1-unsupervised, whose main no longer has configuration_hf_nomic_bert.py. Loading that pin today fails on both transformers versions with OSError: ... does not appear to have a file named configuration_hf_nomic_bert.py.
Exposure
Hub API, 2026-11-17. "Code changes" counts commits to the code repository that changed a .py file after the model first pointed at it.
| Model | Code lives in | Points there since | Downloads, last 30 days | Code changes since |
|---|
11 18325761 19348395
| nomic-ai/nomic-embed-text-v1.5 | nomic-ai/nomic-bert-2048 | 2024-05-02 | 11,551,104 | 29 |
|---|---|---|---|---|
| jinaai/jina-embeddings-v3 | jinaai/xlm-roberta-flash-implementation | 2024-09-05 | 1,373,350 | 15 |
| jinaai/jina-embeddings-v2-small-en | jinaai/jina-bert-implementation | 2023-10-24 | 1,031,834 | 3 |
| nomic-ai/nomic-embed-text-v1 | nomic-ai/nomic-bert-2048 | 2024-01-31 | 1,014,213 | 39 |
| Alibaba-NLP/gte-multilingual-base | Alibaba-NLP/new-impl | 2024-07-25 | 857,796 | 0 |
| nomic-ai/nomic-embed-text-v2-moe | nomic-ai/nomic-bert-2048 | 2025-02-07 | 610,353 | 9 |
| Alibaba-NLP/gte-base-en-v1.5 | Alibaba-NLP/new-impl | 2024-04-20 | 434,681 | 3 |
| jinaai/jina-embeddings-v2-base-en | jinaai/jina-bert-implementation | 2023-10-24 | 355,451 | 3 |
| jinaai/jina-clip-v1 | jinaai/jina-clip-implementation | 2024-05-25 | 226,938 | 39 |
| jinaai/jina-embeddings-v2-base-code | jinaai/jina-bert-v2-qk-post-norm | 2024-01-17 | 122,908 | 4 |
| dangvantuan/vietnamese-document-embedding | dangvantuan/Vietnamese_impl | 2024-08-15 | 93,002 | 0 |
| jinaai/jina-clip-v2 | jinaai/jina-clip-implementation | 2024-10-08 | 54,209 | 14 |
| Alibaba-NLP/gte-multilingual-reranker-base | Alibaba-NLP/new-impl | 2024-07-25 | 41,523 | 0 |
| aipib/Florence-2-VQAJP2 | microsoft/Florence-2-base-ft | 2025-02-27 | 20,391 | 0 |
| RedHatAI/DeepSeek-Coder-V2-Lite-Instruct-FP8 | deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct | 2024-07-17 | 6,333 | 0 |
| TIGER-Lab/VLM2Vec-Full | microsoft/Phi-3.5-vision-instruct | 2024-10-08 | 3,589 | 0 |
In public code
We searched GitHub for Python files that name one of the 17 models together with trust_remote_code and read all 353 results (282 repositories). 167 load through sentence-transformers. 17 files pin a commit; 5 mention code_revision (one is sentence-transformers itself). Most code does not pin at all, which is a separate and better-known problem (an unpinned load runs whatever main holds in both repositories). The pinned minority is the group this report is about: they did the recommended thing and are still not pinned.
Why it matters
- Reproducibility. A pinned evaluation or a pinned production model can run different code than it ran on the day it was pinned.
- Trust boundary. With trust_remote_code=True, whoever can push to the code repository decides what runs in a pinned deployment. Unit 42 (Palo Alto Networks, 2025) showed that a deleted Hugging Face namespace can be re-registered by someone else; a pin is the recommended defence, and here it does not cover the code.
- Engines inherit it. vLLM's CVE-2026-47155 (GHSA-3ww4-5jv9-j5gm, fixed in 0.22.0) is the engine-side version: a deployment that set --revision or --code-revision could still load dynamic code, processors and some weights from an unpinned revision.
What to do
- Pass code_revision=<commit of the code repository> alongside revision=. In sentence-transformers, put it in model_kwargs and config_kwargs.
- Or vendor the module and drop trust_remote_code.
- Or run npx neurarch scan on your repository: it lists every Hugging Face model your code loads, pinned or not, and flags a cross-repository import that is not pinned by code_revision.
Prior reports
- sentence-transformers issue #2613 (opened 2024-04-25, still open): a model saved for an offline image, nomic-ai/nomic-embed-text-v1 with a pinned revision, still fetched its code from the nomic code repository. The maintainer confirmed the cause, gave a workaround, and judged a fix possibly too niche because "very few models use third party repositories to host their code". On the popular models we track it is 17 of 150 that run repository code, 19.3 million downloads a month.
- transformers issue #49272 (closed 2026-10-27): code_revision passed through pipeline(model_kwargs=...) did not reach the load.
- vLLM CVE-2026-47155 (GHSA-3ww4-5jv9-j5gm): pins not applied to every artifact vLLM loads, dynamic code included.
Method and limits
- The 608 tracked models are Neurarch's list of popular open-weight repositories, not a random sample of the Hub; 150 and 17 are counts on that list.
- "Code changes" compares .py blob ids between consecutive commits of the code repository. It counts changes, not their effect; we did not judge any of them harmful.
- The GitHub sample is what code search returns, which favours public, indexed, recently changed files.
- Static reading cannot see a revision held in a variable, so neurarch scan misses pins like MTEB's, which pass the revision from metadata. We read MTEB's source by hand.