N Neurarch Changes CLI and API Open app

Data / Research

A pinned revision does not pin the code: 17 Hugging Face models load it from another repository

Neurarch research, 2026-11-17. Counts are dated readings; method and limits at the end.

Summary. When a Hugging Face model's config.json maps its classes to code in a different repository ("AutoModel": "other/repo--modeling.Model"), passing revision=<commit> with trust_remote_code=True pins the weights and config but not the code. transformers loads that code from the other repository's main unless you also pass code_revision=. Of the 608 open-weight models we track, 150 run code from a repository and 17 load it from another one. Those 17 were downloaded 19.3 million times in the last 30 days, and for 11 of them (18.3 million downloads) the code repository has changed its Python since the model started pointing at it.

This is known to maintainers (see "Prior reports" below). What this report adds is the measured exposure, two reproducible cases, and a scanner that finds the pattern in your own code.

The mechanism

In transformers/dynamic_module_utils.py (get_class_from_dynamic_module, v4.57.1 lines 595 to 602):

if "--" in class_reference:
    repo_id, class_reference = class_reference.split("--")
else:
    repo_id = pretrained_model_name_or_path
...
if code_revision is None and pretrained_model_name_or_path == repo_id:
    code_revision = revision

revision is carried over to the code only when the code lives in the model's own repository. For a cross-repository reference, code_revision stays None and the file is fetched from the other repository's default branch.

Reproduction (run 2026-11-17, transformers 4.57.1 and 5.23.0, no GPU, no torch needed)

from transformers import AutoConfig
AutoConfig.from_pretrained("nomic-ai/nomic-embed-text-v1.5",
                           revision="b0753ae76394dd36bcfb912a46018088bca48be0",
                           trust_remote_code=True)

The Hugging Face cache afterwards holds the model at b0753ae (2024-06-05) and nomic-ai/nomic-bert-2048 at 7710840 (2025-04-29), its main today. transformers prints "A new version of the following files was downloaded". Adding code_revision="1fe3d3fc459a66fb1bd71ddbf08250250521b268" loads the code at that commit instead.

Two cases

1. A public benchmark's pin. MTEB's model registry pins nomic-ai/nomic-embed-text-v1.5 at b0753ae with trust_remote_code=True and loads it through sentence-transformers. The code that runs is nomic-bert-2048 at its 2025-04-29 head, 23 Python changes after that pin, including commits titled "fix: dtype" and "fix: mask". These read as ordinary bug fixes by the model's own authors; we found nothing malicious. The point is that the pin does not decide which code runs, so a result reproduced "at the pinned revision" may have run different code.

2. A pin that no longer loads. Pinned at its first commit with an auto_map (e178409, 2024-02-10), nomic-ai/nomic-embed-text-v1.5 pointed at nomic-ai/nomic-embed-text-v1-unsupervised, whose main no longer has configuration_hf_nomic_bert.py. Loading that pin today fails on both transformers versions with OSError: ... does not appear to have a file named configuration_hf_nomic_bert.py.

Exposure

Hub API, 2026-11-17. "Code changes" counts commits to the code repository that changed a .py file after the model first pointed at it.

ModelCode lives inPoints there sinceDownloads, last 30 daysCode changes since

11 18325761 19348395

nomic-ai/nomic-embed-text-v1.5nomic-ai/nomic-bert-20482024-05-0211,551,10429
jinaai/jina-embeddings-v3jinaai/xlm-roberta-flash-implementation2024-09-051,373,35015
jinaai/jina-embeddings-v2-small-enjinaai/jina-bert-implementation2023-10-241,031,8343
nomic-ai/nomic-embed-text-v1nomic-ai/nomic-bert-20482024-01-311,014,21339
Alibaba-NLP/gte-multilingual-baseAlibaba-NLP/new-impl2024-07-25857,7960
nomic-ai/nomic-embed-text-v2-moenomic-ai/nomic-bert-20482025-02-07610,3539
Alibaba-NLP/gte-base-en-v1.5Alibaba-NLP/new-impl2024-04-20434,6813
jinaai/jina-embeddings-v2-base-enjinaai/jina-bert-implementation2023-10-24355,4513
jinaai/jina-clip-v1jinaai/jina-clip-implementation2024-05-25226,93839
jinaai/jina-embeddings-v2-base-codejinaai/jina-bert-v2-qk-post-norm2024-01-17122,9084
dangvantuan/vietnamese-document-embeddingdangvantuan/Vietnamese_impl2024-08-1593,0020
jinaai/jina-clip-v2jinaai/jina-clip-implementation2024-10-0854,20914
Alibaba-NLP/gte-multilingual-reranker-baseAlibaba-NLP/new-impl2024-07-2541,5230
aipib/Florence-2-VQAJP2microsoft/Florence-2-base-ft2025-02-2720,3910
RedHatAI/DeepSeek-Coder-V2-Lite-Instruct-FP8deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct2024-07-176,3330
TIGER-Lab/VLM2Vec-Fullmicrosoft/Phi-3.5-vision-instruct2024-10-083,5890

In public code

We searched GitHub for Python files that name one of the 17 models together with trust_remote_code and read all 353 results (282 repositories). 167 load through sentence-transformers. 17 files pin a commit; 5 mention code_revision (one is sentence-transformers itself). Most code does not pin at all, which is a separate and better-known problem (an unpinned load runs whatever main holds in both repositories). The pinned minority is the group this report is about: they did the recommended thing and are still not pinned.

Why it matters

What to do

Prior reports

Method and limits