Google has addressed a security flaw in the Python SDK for Vertex AI after researchers demonstrated that attackers could potentially intercept machine learning model uploads and substitute them with malicious files.
The issue was identified by researchers from Palo Alto Networks' Unit 42 team, who disclosed the findings through Google's bug bounty program. According to the researchers, the vulnerability could be exploited without compromising a target organization's cloud environment, stealing credentials, or tricking users through phishing campaigns. Instead, the attack relied on weaknesses in how the SDK handled temporary storage locations during model uploads.
Researchers referred to the technique as "Pickle in the Middle." They reported no evidence that the flaw had been exploited outside of controlled testing environments. Google has since released security updates, and organizations using Vertex AI are advised to upgrade to version 1.148.0 or newer.
Predictable Storage Names Created an Opening
The vulnerability originated from the SDK's automatic staging process.
When developers uploaded a machine learning model without manually specifying a Cloud Storage bucket, the SDK generated a temporary bucket name based on information such as the Google Cloud project identifier and deployment region.
The problem was not that the bucket name could be predicted. The problem was that the SDK only checked whether the bucket existed. It did not verify whether that bucket belonged to the project performing the upload.
Because Cloud Storage bucket names are globally unique across Google Cloud, an attacker could create the expected bucket before the victim did. If that happened, model files uploaded by the victim could be redirected into infrastructure controlled by the attacker.
In practical terms, a developer could believe a model was being uploaded to their own cloud environment while the files were actually being delivered elsewhere.
Attackers Could Replace Models Before Deployment
After receiving the uploaded files, an attacker could modify or replace the model before Vertex AI retrieved it for deployment.
This becomes particularly important because many machine learning workflows rely on serialization formats such as Pickle and Joblib. These formats are commonly used to save trained models, but they also contain functionality capable of executing instructions when the file is loaded.
As a result, a manipulated model may do more than generate predictions. It can potentially run arbitrary code inside the environment responsible for serving the model.
Unit 42 researchers demonstrated that this behavior could be abused to execute attacker-controlled code inside Vertex AI's serving infrastructure.
Researchers Exploited a Narrow Timing Window
The attack required the malicious file replacement to occur very quickly.
During testing, researchers observed that Vertex AI typically retrieved uploaded files roughly 2.5 seconds after the upload process completed.
To exploit this short interval, they created an automated Cloud Function that monitored the attacker-controlled bucket and immediately replaced newly uploaded files. The replacement process took approximately 1.4 seconds, allowing the malicious model to be swapped before Vertex AI accessed it.
This timing-based attack demonstrated that the vulnerability was practical under the right conditions rather than being a purely theoretical risk.
Proof-of-Concept Reached Beyond a Single Model
After achieving code execution, researchers tested what level of access could be obtained from the serving environment.
Their proof-of-concept extracted an OAuth token from the container's metadata service and used it to interact with resources available within Google's managed infrastructure.
According to the report, the token provided visibility into additional machine learning assets, model artifacts, TensorFlow files, BigQuery metadata, access control information, system logs, Kubernetes cluster identifiers, and internal infrastructure references.
The findings suggested that a successful compromise could potentially expose information beyond the originally targeted model deployment.
Exploitation Required Specific Conditions
The vulnerability was not universally exploitable.
Researchers noted that two requirements had to be met before the attack could succeed.
First, the expected default staging bucket could not already exist in the chosen deployment region. Second, the developer needed to rely on the SDK's default bucket-generation behavior rather than specifying a storage bucket manually.
The researchers noted that newly created Vertex AI projects often satisfy the first condition because the default bucket may not yet have been created.
Google Introduced Multiple Fixes
Unit 42 reported the issue to Google on March 5, 2026.
Google's initial response introduced additional randomness into bucket names by appending a UUID value, making bucket prediction substantially more difficult.
The company later strengthened the mitigation by implementing ownership validation checks. These checks ensure that automatically selected buckets belong to the project initiating the upload, preventing bucket-squatting attacks from succeeding.
The ownership verification mechanism was included in Vertex AI SDK version 1.148.0.
At the time the researchers published their findings, neither Google's Vertex AI security advisories nor the research report listed a CVE identifier for the vulnerability.
Recommendations for Organizations
Security teams using Vertex AI should verify that all environments are running updated versions of the google-cloud-aiplatform package. This includes development notebooks, machine learning pipelines, automated build systems, testing environments, and production deployments.
Researchers also recommend explicitly defining a staging bucket owned by the organization instead of relying on SDK defaults. This reduces the risk of storage misconfigurations and provides greater visibility into where machine learning artifacts are stored during deployment.
The disclosure is the latest example of how weaknesses in supporting cloud infrastructure can affect AI systems. As organizations continue moving model development and deployment into managed cloud platforms, security reviews must extend beyond the model itself to include storage, deployment pipelines, permissions, and the services that support the AI lifecycle.
.webp "North Korean Hackers Target Softwares that Support Online Services")
.webp)
