CSIPE

Published

- 15 min read

Repository Code Belongs Behind a Loader Boundary


Books by the author

Compare all 5

As an Amazon Associate I earn from qualifying purchases. Buying through these links costs you nothing extra and helps pay for the blog.

On 29 October 2026, Pluto Security documented a Hugging Face model copy uploaded on 25 October that contained a cryptominer and a reverse shell. BleepingComputer independently examined the copy and confirmed both payloads.

The dangerous part was ordinary Python supplied beside the model. The loader was allowed to trust that code, so loading the repository crossed the same boundary as installing and running somebody else’s software. A familiar model name, a plausible card, and a pile of weights did not make that boundary smaller.

Teams that evaluate public models need to treat the loader as a package installer with access to a GPU. Find the places where repository code can run. Pin what you reviewed. Give the evaluation process a disposable machine, empty credentials, and a short list of permitted network destinations. If the malicious copy already ran, stop debugging model quality and start incident response.

What happened on 25 October

Pluto Security’s report followed repository-supplied Python in that copy to the two independently confirmed payloads.

Those payloads matter for different reasons. A miner consumes compute for someone else’s benefit. A reverse shell gives a remote operator an interactive route back into the machine. Finding one should make you look for the other because the confirmed repository carried both. A GPU running hot is an observable symptom; remote access is the part that changes the incident from wasted compute to lost trust.

That route sets the scope for response. According to both reports, repository-supplied Python executed when the 25 October copy was loaded. Pickle scanning covers deserialization risks within serialized files, whereas review of this incident has to follow the ordinary Python that the loader had permission to run beside the weights.

Start the inventory with loading behavior: applications that pass trust_remote_code=True, internal wrappers that set it for callers, notebooks that still run copied examples, and evaluation workers that can reach secrets, internal services, or the public internet. A model-file inventory alone can miss the Python path that executed.

This is a supply-chain incident in a shape AI teams recognise too late. The repository looked like a model input, while the runtime treated part of it as a software dependency. Loader configuration determines the runtime authority granted to that repository code.

trust_remote_code means Python from the repository

Hugging Face’s Transformers documentation explains the mechanism directly. A custom model can extend the library’s configuration and modeling classes, but “the modeling code is not from Transformers.” The same page says: “Take extra precaution when loading a custom model”, then shows trust_remote_code=True as the switch that loads it.

That flag has a legitimate job. Some model architectures need code that the installed Transformers release does not contain. Without a custom-code path, users would have to wait for library support or install the model implementation another way. The engineering mistake is letting a legitimate compatibility feature inherit the trust level of a data file.

Consider what the process can see at load time. It may have the developer’s Hugging Face token in an environment variable, cloud credentials supplied to the notebook, mounted source code, an SSH agent, internal package credentials, or network access to services that trust the host. Evaluation on a normal development machine inherits that surrounding context even when the quality test only needs model input and output. The convenience of reusing the machine quietly grants repository code access to credentials, mounted code, local agents, and network routes.

Repository code inherits the process context: imports, functions, filesystem permissions, environment variables, and sockets. The operator’s belief that they were “just loading weights” places no restriction on the Python interpreter. The model abstraction disappears at the security boundary.

Warnings and code review reduce the chance that untrusted code runs, but copied evaluation snippets, automated flows, and internal helpers can bypass them. Put the loader in an environment where an unreviewed line reaches very little. Review addresses what should execute; containment limits the cost when review misses something.

Hugging Face also recommends loading custom model code from a specific revision identified by a commit hash. That closes a real gap: review and execution refer to the same stable input. A model name or branch can point at different code tomorrow.

Stable identity makes a commit pin reproducible; safety still depends on the pinned code. A malicious revision abc123 will run faithfully every time. The pin lets review, testing, logging, and execution refer to that exact code.

Match each scanner to its mechanism

Hugging Face documents more than one scanner because model repositories can carry more than one kind of risk. Its malware-scanning page says every repository file smaller than 2 GB is sent through a malware scanner at each commit. The page also explains that a file with no ok or infected badge may still be scanning, waiting, or in an error state.

The pickle-scanning documentation covers a different mechanism. Pickle can execute Python during deserialization, so the Hub extracts and displays referenced imports without executing the file. The documentation says the scanner combines ClamAV scans with pickle-import scans and explicitly says its best-effort import lists provide no guarantee.

Treat each scan result as evidence about its named mechanism. Malware scanning looks for patterns the scanner recognises. Pickle scanning reports what a serialized object may import or call when loaded. Custom-Python review examines ordinary source code that the loader is explicitly allowed to run. An approval record should identify which of these mechanisms it checked.

The October reports traced execution through repository-supplied Python. A “pickle scan passed” approval ticket would therefore record evidence about a different boundary, because a pickle result does not describe what ordinary source code will do after the loader imports it. Security receipts are useful only when they name the mechanism they checked.

Platform upload scanning supplies useful pre-runtime evidence by catching known patterns, raising friction for attackers, and exposing findings to users. Internal model-admission policy must also cover the point where the runtime combines a repository with its library version, local settings, credentials, and network reach.

The runtime owner makes the final decision. Hub scanners have no visibility into the loader’s remote-code setting, the process’s production token, or the host’s route to an internal control plane. The team running the code must set the boundary around those deployment facts.

Admit custom code before the benchmark

Model evaluation usually starts with answer quality, memory use, latency, and performance against the current baseline. When loading can execute repository code, admission comes first because the benchmark begins only after Python has already crossed the boundary.

Suppose an engineer submits org/model at commit abc123 to the shared evaluator. Its admission record names that exact repository and revision, then answers whether the load requests custom code. The reviewer reads the code diff at abc123 and follows its local imports; a short loader can transfer control to a second module with one import. The approval scope is all code reached from that immutable revision.

The same record describes the worker that will run it. Include mounted paths, environment variables, identity credentials, permitted network destinations, and maximum compute allocation. Another engineer can then check whether “sandboxed” means an empty secret set and blocked outbound traffic or merely a container on a credentialed host. Runtime authority is part of the admission decision because it determines what a missed line of code can reach.

Put one person on the decision. The named owner can approve a load that avoids repository code, accept the pinned custom code for a limited evaluation, or reject it. For an exception, record the approved revision and an expiry date. When either changes, the old decision no longer admits the job.

Keep this material with the model registry, where it survives a disposable notebook. Six months later, the service image may still contain the model after the experiment has vanished. Its registry entry needs the custom-code determination, approved commit, code-review location, owner, expiry, and the authority granted to the evaluation worker. That is enough for an operator to reconstruct why execution was allowed.

Test enforcement with a deliberate mismatch by submitting an unapproved revision to the shared loader and confirming that the worker refuses it before importing repository Python. In a test record, change abc123 to another commit and expect the same refusal; then submit the reviewed revision and verify that its execution receipt records abc123. Together, these runs show that the loader enforces the registry decision.

Updates to Transformers, the internal wrapper, or the evaluation image trigger this test again. A helper can change loading behavior while every application still calls the same function. CI checks whether the run requested repository code and whether this worker has approval for that exact revision. Either failed answer stops the job before quality testing. This regression gate binds the reviewed code, tested loader behavior, and recorded runtime authority to one admission.

Build a loader boundary that assumes review can fail

Start with a disposable evaluation worker. It can be a short-lived virtual machine or container, provided the isolation actually matches what you are protecting. The worker should be replaceable from a known image, accept the pinned repository revision, emit its results and logs, then disappear. Reusing a developer workstation defeats the point because the workstation contains the long-lived context an attacker wants.

Give that worker no production credentials. Do not mount a home directory, SSH agent, cloud configuration folder, source tree, or general secrets bundle. If a download needs authentication, issue a narrowly scoped credential for that repository and evaluation window. Destroy it when the run ends.

Block outbound traffic by default. Model evaluation often needs a way in to fetch approved artifacts, but a reverse shell needs a way out. Put the fetch step in a controlled stage, cache the pinned artifact, then run the loader without general internet access. When runtime access is genuinely needed, allow named destinations through a proxy and keep the request log.

Separate download from execution. The first stage resolves the repository and commit, saves the artifact, records hashes, and does not import it. The second stage reviews or scans the frozen copy. The final stage runs the loader in the restricted worker. This division makes each decision visible and stops a friendly from_pretrained() call from quietly doing discovery, download, and execution in one opaque moment.

Cap compute as well. A cryptominer turns an uncapped evaluation box into somebody else’s revenue source. CPU, GPU, memory, wall-clock, and job-count limits constrain mining impact and make abnormal use easier to detect while leaving arbitrary malicious code free to act within the worker’s authority. A model evaluation that exceeds its expected deadline must fail closed.

Keep enough evidence to explain a failed run. Record the repository revision, container or image digest, loader command, installed package lock, network decisions, process tree, and output artifact locations. Exclude secret values from the logs while preserving this execution receipt.

The boundary should be cheap enough that engineers use it. A ten-page exception process sends experiments back to laptops. A small command that submits a pinned revision to a disposable, credential-empty worker is safer because it becomes the easy path. Security that loses to copy and paste will lose often.

Find every loader and map its authority

Search for trust_remote_code=True across application code, notebooks, deployment templates, and shared ML utilities. Include generated configuration and wrapper functions that pass keyword arguments through to from_pretrained(). The setting may sit one level below the project that owns the risk.

Instrument model-loading jobs as well. Capture whether custom repository code was requested, the resolved revision, and the worker that executed it. Runtime evidence catches dynamic settings and libraries that hide the switch, and it answers the incident question: where did this revision run?

For every loader, map the secrets and routes available to its process. A notebook with a read-only model token and no outbound route has a different blast radius from a shared training host carrying cloud deployment credentials, so it receives a different response priority.

Include research notebooks, evaluation scripts, CI jobs, and one-off benchmark runners alongside production repositories. These paths often have thinner controls because teams label the work experimental, a label that does not reduce attacker interest.

Mark unknown authority explicitly and assign the work needed to resolve it. The inventory exists to find code-executing loaders and decide which ones still deserve that authority.

Enforce the decision at the common entry point. A shared wrapper can reject unpinned custom code, require an approved revision, and submit the evaluation to the restricted worker. Make that path obvious so engineers do not have to remember five security flags for every comparison.

If the October copy ran, treat the host as compromised

A process that launched a reverse shell has crossed the line from suspicious package to host incident. Isolate the machine from the network while preserving the evidence your incident team needs. Do not keep the host online merely to watch whether GPU usage settles down. The confirmed payload included an operator route.

Record the repository, revision, timestamps, command, user, and worker identity. Preserve relevant process, network, authentication, and job logs before rebuilding. If the machine is ephemeral, save the evidence deliberately; automatic deletion can erase the only account of what happened.

Scope the incident to every credential source the loader process could access: environment variables, mounted files, metadata services, local agents, and cached login sessions. Revoke or rotate those credentials from a clean system regardless of observed payload use. Secret theft may leave no helpful line in the application log.

Review reachable systems for follow-on activity. The host’s identity and network position may have allowed access that an external attacker would not normally get. Look from the compromised worker outward: package registries, source hosts, object stores, experiment trackers, internal APIs, and deployment systems. Prioritise the routes backed by credentials that were present.

Because the machine executed untrusted Python and opened remote access, rebuild it from a known image. Deleting the repository or miner process leaves persistence uncertain. Continue the wider incident response from the clean replacement.

Close the loader path that allowed the execution. Remove remote code where the model no longer needs it. Where custom code remains necessary, pin the reviewed commit and move the load into the restricted evaluation boundary. Incident response without that repair leaves the same doorway ready for the next copy.

Admission workflow for an unfamiliar model

Run this sequence before quality testing because the first quality test may execute repository code.

  1. Resolve one immutable revision. Record the repository name and exact commit hash. Refuse a branch or moving tag for any custom-code load.

  2. Determine whether repository code will execute. Inspect the loader options and the model’s requirements. Record trust_remote_code explicitly, including when a wrapper sets it.

  3. Freeze the artifact before execution. Download the pinned revision in a controlled stage, record hashes, and keep discovery separate from the loader run.

  4. Review the custom Python and its local imports. Read the code that the loader will import at that revision. Reject unexplained subprocess, network, credential, persistence, or shell behavior.

  5. Run in a disposable worker with empty credentials. Mount only the frozen artifact and a writeable result directory. Set CPU, GPU, memory, and wall-clock limits.

  6. Deny general outbound traffic. Add only the named destination the evaluation can justify, through a logged route. Most inference checks need no internet after the artifact is present.

  7. Keep an execution receipt. Save the revision, image digest, package lock, loader command, network policy, process evidence, and result location.

  8. Promote the reviewed immutable object. If the model passes quality and security review, move that same pinned artifact forward. A later revision starts a new admission.

Authority decides the blast radius

The October reports traced the copy’s execution path through repository-supplied Python. That narrow finding points to the operational question for every evaluator: what authority does the loader grant when it imports custom code from a repository?

Dependency controls answer the question in stages. An immutable revision fixes the object under review. Code review follows the imports that object can trigger, and a disposable worker limits the credentials and network routes available if the review misses something. Scanner results stay attached to the mechanism each scanner actually inspected. The execution receipt then connects the approved revision to the environment that ran it.

Chapter 16 of The AI Native Engineer: Stay relevant as a Software Engineer places this supply-chain risk in a wider security map: external content needs layered controls, and the runtime boundary has to survive one missed warning. Here, the practical test is whether an unfamiliar revision can reach anything beyond the files, compute, and named destinations needed for its evaluation. A low-authority worker sets filesystem permissions and compute quotas, then restricts network policy to those named destinations.

Search for trust_remote_code=True, including the wrappers and notebooks around it. Route each first run through the low-authority evaluation boundary, and make that path easier than loading an unfamiliar repository on a laptop.

For one practical security lesson each month, join the Cyber Security in Plain English newsletter. One email per month.

Sources