Published
- 20 min read
A Green Padlock Did Not Authenticate This Software Update
Books by the author
Compare all 5-
The Digital Fortress
Anyone protecting their own money, accounts and family online
Buy on Amazon Buy the book now What's in it, and who it's for -
The Anonymity Playbook
Journalists, sources, activists and anyone whose privacy has stakes
Buy on Amazon Buy the book now What's in it, and who it's for -
Secure Software Development
Developers shipping production code under delivery pressure
Buy on Amazon Buy the book now What's in it, and who it's for -
The Secure Harness
Teams running AI coding agents in repositories others depend on
Buy on Amazon Buy the book now What's in it, and who it's for -
The AI Native Engineer
Engineers taking AI features from a working demo to production
Buy on Amazon Buy the book now What's in it, and who it's for
As an Amazon Associate I earn from qualifying purchases. Buying through these links costs you nothing extra and helps pay for the blog.
A server asked its usual vendor for an update. The address looked right. The connection carried a valid TLS certificate, so the familiar secure-connection checks passed. The server downloaded a package and ran it with the authority needed to manage virtual machines.
The package came from an attacker.
Between 28 and 30 August 2026, someone diverted internet traffic meant for part of Softaculous’s infrastructure. The affected address range included the update service for Virtualizor, a control panel used to manage virtual private servers. The attacker also obtained a valid certificate for the vendor’s domains. A small number of installations received a malicious update, according to the vendor’s incident report published on 31 August (Virtualizor; Softaculous).
This incident matters beyond one hosting product. It joined three systems that engineers often treat as separate: internet routing, encrypted connections, and software updates. Each system did its assigned job under the information it had. Together, they still delivered attacker-controlled code as root because the update client did not cryptographically verify the package itself.
The practical lesson is narrow and useful. TLS protects a journey to whoever currently controls the destination. It does not prove that the file at the end was released by your software vendor. An update needs its own identity, checked independently of the route and the download server.
The road to the update server changed
The first part of the incident happened below the application layer. No one had to alter the domain name typed into a browser or configured in an update client. The internet’s directions to the correct address changed instead.
Border Gateway Protocol, usually shortened to BGP, is how large networks tell one another which blocks of internet addresses they can reach. A route announcement is much like a road sign saying, “send traffic for these addresses through me.” Routers compare the available signs and choose a path. More specific directions usually win over broad ones because they claim to know a narrower destination.
Softaculous normally relied on Hetzner to announce the larger address range 162.55.0.0/16. At about 20:57 UTC on 28 August, an unauthorised route for the smaller 162.55.80.0/24 range began propagating through the internet. That smaller range contains 256 addresses and included Softaculous systems. Since the new direction was more specific, networks that accepted it sent relevant traffic toward the attacker-controlled server rather than the legitimate host (Virtualizor; Ars Technica).
The diversion came in two waves. The first ran from roughly 20:57 UTC on 28 August until 08:50 UTC on 29 August. A corrective announcement then produced about eleven hours with little or no observed diversion. The bad route returned around 20:00 UTC and remained broadly visible until about 06:10 UTC on 30 August. Softaculous reconstructed a total incident window of about 33 hours, with the unauthorised route active intermittently for roughly 22 of them.
Those numbers need careful reading. Softaculous sampled 368 peers in the RIPE Routing Information Service, which collects routing views from participating networks. All 368 saw the hijacked route at some point. During sustained periods, a median of 266 peers carried it, about 72 percent of the complete sample. That is a view of route propagation, not proof that 72 percent of all customer bytes were intercepted. The route also flapped, meaning it appeared and disappeared repeatedly, so any one server’s path could change during the window.
A Virtualizor installation was exposed only if its update request crossed a network carrying the bad route at that moment, reached the impostor, and completed. This explains how a route with broad visibility could produce a small number of confirmed malicious downloads. Exposure was conditional. It was still real.
The vendor could not identify every affected server from its own download logs because requests diverted to the attacker never reached the legitimate service. That is the first operational trap in this story. Clean origin logs cannot account for traffic that was stolen before it arrived. When control fails upstream, absence from the expected log is not evidence of absence.
The certificate proved control of the detour
Most engineers have been taught to look for an encrypted HTTPS connection. That remains good advice. It protects data against many ordinary interception attempts and lets a client verify that the server presents a certificate valid for the requested name.
During this incident, however, the attacker obtained a technically valid Let’s Encrypt certificate covering affected Softaculous and Virtualizor domains. Certificate authorities issue domain-validated certificates after checking that the requester can answer a challenge for the domain. Since traffic for the relevant addresses had been diverted, the validation checks also reached the attacker. From the certificate authority’s point of view, the requester controlled the destination it was told to check (Softaculous; Ars Technica).
The result looked normal to a client. The hostname matched. The certificate chain was valid. The encrypted session terminated at the attacker’s server, but there was no certificate warning to reveal that fact. The padlock could truthfully say that the client had an encrypted connection to a server authorised through the domain-validation process. It could not say that the internet had led the client to the vendor’s intended machine.
This distinction often gets flattened into “HTTPS means secure.” HTTPS answers important questions: is this connection encrypted, and does the presented certificate satisfy the browser or client’s identity checks? It does not answer every question about the content travelling through that connection. If the wrong party can satisfy the identity check, or if the authorised service itself is compromised, HTTPS can carry malicious content perfectly securely.
The incident also shows why independent checks can share a hidden dependency. The update client and the certificate authority appeared to be two separate parties. Both relied on internet routing to reach the right address. Divert the route widely enough and the attacker can influence the file download plus the evidence used to bless the connection.
Ars Technica reported that the forged route retained Hetzner’s autonomous-system number at the apparent origin and fit within the range permitted by the existing Route Origin Authorisation. That made the route appear valid under Resource Public Key Infrastructure route-origin validation, commonly called RPKI ROV. The unauthorised path claimed a more specific prefix while preserving the expected origin at the end of the path. Routers that rejected invalid origins therefore had no invalid origin to reject.
RPKI is valuable. It stops many route leaks and hijacks from spreading. This event does not make it pointless any more than one stolen password makes authentication pointless. It shows the edge of the check that was configured. Origin validation can confirm that an authorised network appears as the origin and that the prefix length is allowed. It does not establish that every network named earlier in the path had permission to announce that route.
The useful engineering response is to stop asking whether a control is “strong” in the abstract. Ask what fact it establishes and which dependencies it shares with the system it is meant to protect. TLS established an encrypted session to the holder of a valid certificate. Route-origin validation accepted an announcement compatible with the published authorisation. Neither established who created the package.
The update file had no separate identity
A software update is one of the most privileged objects in an infrastructure stack. Administrators want it to replace binaries, restart services, change databases, and sometimes modify the operating system. An update system therefore converts a downloaded file into authority.
Virtualizor said its update clients did not cryptographically verify update packages during the August incident. A modified package delivered over the diverted connection was not rejected on that basis. Once the attacker had the route and a certificate, the last gate before execution depended on the same download channel that had already been captured (Virtualizor).
Package signing breaks that dependency. The vendor signs a release digest with a private signing key. The client carries or obtains a trusted public key through a separate, controlled process. Before installation, it verifies that the package bytes match the signed digest and that the signature traces to an allowed release identity. A copied, truncated, or attacker-built package fails even when it arrives from the expected hostname over valid HTTPS.
That final clause is the point. A signature makes the authenticity decision about the artifact rather than the road used to fetch it. The download server can be mirrored, cached, or temporarily misrouted without gaining permission to invent releases. Transport still matters for confidentiality, availability, and protection against downgrade tricks, but it no longer holds the only vote on what code may run.
Signing has to be part of an enforced release path, not a decorative .sig file placed beside the archive. If the update client downloads the package and signature from the same impostor but never anchors verification in a trusted key, the attacker can replace both. If a failed check produces a warning that unattended automation ignores, the mechanism fails open. If the client accepts any key supplied by the update endpoint, it asks the captured channel to vouch for itself.
The verifier also needs to bind more than a checksum. A bare checksum downloaded from the same server detects accidental corruption but gives an attacker no difficulty; they can calculate a checksum for their own package. A signed manifest can bind the package digest to a product, version, release channel, architecture, and expiry or freshness policy. That makes it harder to replay an old but valid package or move a signed component into a context where it was never meant to run.
Good update verification therefore asks several concrete questions. Was this artifact signed by a currently authorised release key? Is it intended for this product and platform? Does its version move forward under the organisation’s downgrade policy? Has the key or release been revoked? Does the complete dependency set match what was approved, rather than only the outer installer?
Key custody decides whether this architecture deserves trust. A signing key left on the public update server would have fallen inside the same incident boundary. A useful design keeps release authority away from content delivery. Signing happens in a constrained release system after tests and approvals, preferably with hardware-backed or short-lived credentials. Distribution systems receive signed artifacts but not the power to create new ones.
Clients need a safe key-transition process too. “Fetch the new public key from the update server and trust it” restores the original circular dependency. Key rotation should be authorised by an existing trusted key, an offline root, or another channel with a deliberately different failure mode. Recovery from a lost or compromised key should be designed before the emergency, because improvised trust replacement is where strong systems quietly become optional.
Softaculous said on 31 August that it would put code signing in place for all packages. It also released Virtualizor 3.2.9.9 with a mitigation tool for known indicators. Those steps address different problems. Signing protects future acceptance decisions. A scanner helps find traces of an event that may already have crossed the old boundary. One cannot substitute for the other.
Installing the fixed release does not clear the server
The vendor identified /etc/systemd/system/java-jre-update.service as a known indicator of compromise. Operators who find it are told not to delete it immediately and to contact support so evidence can be preserved. The service name matters, but its absence cannot prove a clean system when the complete behaviour and scope of the malicious package remain uncertain as of 3 September 2026.
At that point, ordinary patch management turns into incident response. Updating to a repaired client closes a known route for the next package. It does not reverse commands already executed as root. Code with that authority could add an SSH key, create a user, schedule another task, change an existing binary, capture API credentials, or open an outbound connection. Deleting one service file would remove one clue while leaving the trust question unanswered.
Virtualizor manages hypervisors. That makes the surrounding authority unusually broad. A compromised management server may hold API keys, control virtual machines, reach storage or backup systems, and influence customer workloads. The correct scope comes from the actual deployment, not from the name of the affected package.
Consider two installations. The first sits in an isolated management network, uses narrowly restricted API credentials, and cannot reach guest data or external administration systems. The second has reusable root keys, broad network access, backup credentials, and an integration with the customer portal. Both may show the same malicious service. They do not have the same incident.
The vendor recommends resetting Virtualizor API keys, limiting API access to trusted addresses, checking for unknown SSH keys and accounts, reviewing scheduled tasks and unexpected outbound connections, and regenerating client-area API keys. Softaculous also advises anyone who logged into its client area during the diversion windows to reset that password and change it anywhere it was reused (Virtualizor; The Register).
These are sensible first actions, but sequence matters. Preserve enough evidence to understand what happened before making broad changes that erase timestamps, running processes, temporary files, and network state. Contain paths the attacker could still use. Rotate credentials from a known-clean system, not from the possibly compromised host. Then rebuild trust in layers.
A server that installed the malicious package should not earn a clean bill of health because a vendor scanner prints no current finding. Detection tools recognise known traces. They cannot prove the absence of every change made under root authority. Where the hypervisor controls sensitive workloads, rebuilding from trusted media and restoring reviewed configuration is usually more defensible than trying to remove every possible alteration in place.
Rebuild does not mean copying the old server wholesale. That can preserve the attacker’s changes. Start from a known image, install packages from verified sources, restore the smallest necessary configuration, issue fresh credentials, and compare behaviour against expected infrastructure definitions. Guest systems and artifacts produced during the questionable period may need separate review according to what the management plane could modify.
The same caution applies to backups. A backup taken after compromise is useful evidence and may contain irreplaceable data. It is not automatically a trusted system image. Keep recovery data, forensic evidence, and known-clean baselines as separate concepts. A single “latest backup” cannot serve all three roles safely.
What Virtualizor operators should do now
Every Virtualizor server belongs in the initial check because the vendor cannot derive a complete victim list from logs that diverted traffic never touched. Treating a server as in scope means investigating it. It does not mean declaring every installation compromised.
A practical response can move quickly without destroying the evidence needed for a sound decision.
-
Record the system’s current state and ownership. Identify the Virtualizor version, update configuration, host role, administrator, network segment, and workloads under management. Preserve relevant logs, process information, service definitions, recent file metadata, outbound connection records, and infrastructure snapshots according to your incident procedure.
-
Check the vendor’s named indicator. Look for the
java-jre-update.serviceunit at the exact path published by Virtualizor and determine whether a matching service is enabled or running. If present, follow the vendor’s instruction to contact support rather than deleting it immediately. Preserve the file and associated metadata in a controlled evidence location. -
Establish whether an update could have completed during the two diversion waves. Compare system update records, scheduled jobs, proxy logs, resolver data, firewall records, and package timestamps with 28 August 20:57 to 29 August 08:50 UTC and 29 August roughly 20:00 to 30 August 06:10 UTC. Remember that missing vendor-side download logs do not clear the host.
-
Contain according to the server’s reach. Restrict management access and unnecessary outbound traffic while keeping evidence available. If there are compromise indicators, isolate the host from workloads and control planes using your tested procedure. Avoid an impulsive power-off unless your incident lead judges the active risk greater than the evidence loss.
-
Rotate reachable credentials from a clean machine. Reset Virtualizor API keys, remove unknown keys, restrict accepted source addresses, regenerate relevant Softaculous client API keys, and review SSH trust. Include backup, storage, cloud, DNS, billing, and automation credentials if the affected root account could read or use them.
-
Inspect persistence and downstream effects. Review users, SSH keys, systemd units, scheduled tasks, startup scripts, package changes, privileged containers, unexpected listeners, and outbound destinations. Check management actions against virtual machines and templates during and after the possible installation time. Let the host’s real authority map drive the search.
-
Choose cleanup or rebuild based on assurance required. A false positive with no update evidence may close after documented review. A confirmed malicious service on a low-value lab host may still be quickest to rebuild. A production hypervisor with root-level execution should return to service only after you can explain why its operating system, credentials, control data, and managed workloads are trustworthy again.
-
Install the current vendor release through a verified path. Obtain current instructions directly from Virtualizor, verify the release through whatever authenticity mechanism the vendor now provides, and confirm the running build after maintenance. Do not use a saved link or package copied during the incident window.
-
Watch for delayed use of stolen authority. Fresh keys and rebuilt systems deserve focused monitoring. Look for use of retired credentials, unusual management actions, new administrative sessions, changes to templates, and outbound connections that do not fit the service’s role. A quiet first day does not prove that nothing was copied.
The decisive step is number seven. Teams often perform the visible tasks, remove the indicator, rotate one key, patch the software, and call the incident closed. Closure should instead answer a harder question: what evidence supports trusting this management plane again?
That answer may be different for each operator. The standard can remain consistent. Root execution on an update server is an authority event, and the response must cover everything that authority could change or impersonate.
Software vendors need two independent release boundaries
The customer actions address the current incident. Vendors and platform teams should also examine why a network detour could become an authorised installation. The release design needs separate boundaries for creating an update and delivering it.
The creation boundary decides what counts as a release. It covers source control, builds, tests, human approvals, signing, provenance, and key custody. The delivery boundary makes an approved artifact available through web servers, mirrors, caches, content networks, and update endpoints. Delivery may be highly distributed and internet-facing. It should not possess release authority.
When those boundaries are independent, capturing delivery lets an attacker delay, replay, or deny updates, all serious problems. It does not let them mint a new accepted package. When the client trusts any bytes received over a valid HTTPS session, delivery and release collapse into one boundary. Control the route or server and you control the software.
Package verification should fail closed before extraction or execution. The client should report an authentication failure in terms operators can investigate, with the expected product, version, signing identity, and reason. It should not silently fall back to an unsigned path because the signed channel is unavailable. Availability pressure is predictable, so teams must decide in advance how emergency releases work without bypassing the identity check.
The updater itself deserves hostile testing. Point DNS at an impostor. Serve a valid certificate from a controlled test domain. Replace the package, signature, manifest, version metadata, and redirect target one at a time. Replay an old signed release. Present a package signed by a retired key. Interrupt the transfer. The expected outcome is no installation unless the artifact and release policy both verify.
Logs should make those tests and real incidents visible. Record the requested version, received digest, signing key identifier, verification result, final source, redirect chain, and installation outcome. Keep enough information to identify clients that accepted a specific release without logging credentials or sensitive customer data. Vendor telemetry cannot see traffic that never arrives, so client-side records and customer-exportable evidence matter.
Network controls still belong in the design. Route Origin Authorisations should grant only the prefix lengths and origins actually needed. Networks should monitor for unexpected more-specific announcements and changes in route visibility. Certificate Authority Authorization records can restrict which certificate authorities and validation methods may issue for a domain. Ars Technica reported that Let’s Encrypt said account binding in CAA would have made this incident materially harder.
None of those controls removes the need for package signatures. Their failures are different. Tight routing authorisation reduces the chance of diversion. Route monitoring shortens detection time. Certificate restrictions reduce fraudulent issuance paths. Signed artifacts stop a captured delivery channel from defining a new release. Layering works when each layer checks a different fact.
Vendor incident plans should assume the origin logs may be incomplete. Keep a way to contact customers outside the affected service, publish signed advisories where possible, and provide precise UTC windows plus stable indicators. Tell operators what a finding proves, what a clean result cannot prove, and when evidence preservation should come before deletion. Softaculous’s disclosure did several of these things well, including publishing the route timeline and explicitly refusing to treat every in-scope server as infected.
A tabletop exercise can expose awkward dependencies before an attack. Suppose your update hostname resolves correctly but its address range is diverted. Could an attacker obtain a certificate? Could the client accept a package they built? Can you revoke the signing identity without using the captured server? Can customers distinguish an authentic emergency notice? Can you identify installed release digests from their side when your logs show nothing?
If all answers depend on the same domain, network, account, or operations team, the controls are numerous but the boundary is singular.
Trust the artifact, then verify the machine
The Virtualizor incident was not simply a BGP story. Routing provided the detour. Certificate validation made the detour look ordinary. Missing package verification turned ordinary-looking delivery into root execution. Removing any one of those links would have changed the outcome.
As of 3 September 2026, the vendor says a small number of Virtualizor installations received the malicious package, but it cannot publish a definitive list. It has not identified a malicious package for its other products, and that investigation remains open. Operators should keep those limits intact: check every Virtualizor server, do not claim every server was infected, and do not extend the confirmed package compromise to unrelated products without evidence.
The strongest fix is equally precise. Authenticate software at the object that will run. Use HTTPS because transport security matters. Protect routes because internet infrastructure matters. Restrict certificate issuance because domain identity matters. Then require a signature rooted in release authority before an update receives the machine’s authority.
After suspected installation, reverse the logic. A newly signed update can secure the next transition, but it cannot authenticate the state left by earlier root code. Preserve evidence, map what the server could reach, rotate that authority from clean systems, and rebuild wherever the required assurance exceeds what an in-place cleanup can prove.
The green padlock did its limited job. The mistake was asking it to sign the release.
If you want practical security and AI engineering explained without the daily noise, subscribe to the newsletter. It is one email per month.
Sources
- Virtualizor: Security Incident, BGP Hijacking, accessed 2026-09-03
- Softaculous: Security Incident, BGP Hijacking, Update, accessed 2026-09-03
- Ars Technica: BGP hijack infecting networks caused by a comedy of errors that is not funny at all, accessed 2026-09-03
- The Register: 33-hour BGP hijack of Softaculous traffic prompts security scramble, accessed 2026-09-03