Search This Blog
Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label Cyber Attacks. Show all posts
Notepad++
Chinese Actors Cyber Attacks Cyberattack Databreach Notepad++

A Quiet Breach of a Familiar Tool, Notepad++

For six months last year the update system of Notepad++, one of the world’s most widely used Windows text editors, was quietly subverted by hackers linked by investigators to the Chinese state. The attackers used their access not to disrupt the software openly, but to deliver malicious versions of it to carefully chosen targets. 

According to a statement published this week on the project’s official website, the intrusion began in June with an infrastructure-level compromise that allowed attackers to intercept and redirect update traffic meant for notepad-plus-plus.org. Selected users were silently diverted to rogue update servers and served backdoored versions of the application. Control over the update infrastructure was not fully restored until December. 

The developers said the attackers exploited weaknesses in how older versions of Notepad++ verified updates. By manipulating traffic between users and the update servers, they were able to substitute legitimate downloads with malicious ones. 

Although update packages were signed, earlier design choices meant those signatures were not always robustly checked, creating an opening for tampering by a well-resourced adversary. Security researchers say the campaign was highly targeted. 

The attackers installed a previously unknown backdoor, dubbed Chrysalis, which Rapid7 described as a custom and feature-rich tool designed for persistent access rather than short-term disruption. Such sophistication suggests strategic objectives rather than criminal opportunism. 

Independent researcher Kevin Beaumont reported that several organisations with interests in East Asia experienced hands-on intrusions linked to compromised Notepad++ installations, indicating that attackers were able to take direct control of affected systems. 

He had raised concerns months earlier after a Notepad++ update quietly strengthened its updater against hijacking. The episode underlines a broader vulnerability in the global software supply chain. Open-source tools such as Notepad++ are deeply embedded in corporate and government systems, yet are often maintained with limited resources. That imbalance makes them attractive targets for state-backed hackers seeking discreet access rather than noisy disruption. 

Notepad++ developers have urged users to update manually to the latest version and large organisations to consider restricting automated updates. The incident also serves as a reminder that even modest, familiar software can become a conduit for serious espionage when its infrastructure is neglected.
Read more »
Exposed Credentials
8-Minute Attack AI Cloud Breach AWS Hijack Cyber Attacks Exposed Credentials

AI Hijacks AWS Cloud in 8 Minutes via Exposed Keys

 

An AI-assisted cyberattack hijacked a company's AWS cloud infrastructure in just eight minutes after attackers discovered exposed test credentials in a public S3 bucket, demonstrating how configuration errors can fuel lightning-fast breaches in the era of automated threats. This incident, uncovered by Sysdig's Threat Research Team on November 28, 2025, exposed vulnerabilities in cloud access management and the growing role of large language models (LLMs) in offensive operations.

The breach began with a simple oversight: credentials named with "AI" references sat openly in an S3 bucket, ripe for discovery during routine scans. Despite a ReadOnlyAccess policy limiting initial access, the intruder launched a massive enumeration campaign, probing Secrets Manager, RDS databases, and CloudWatch logs to blueprint the entire environment without raising alarms. This reconnaissance phase set the stage for rapid escalation, underscoring how even restricted keys can serve as footholds for deeper intrusions.

Attackers then pivoted to code injection on Lambda functions, iteratively tampering with one called EC2-init until they commandeered an account named "frick," granting full administrative privileges. They compromised 19 distinct AWS principals, enabling abuse of Bedrock AI models like Claude 3.5 Sonnet and DeepSeek R1, alongside attempts to launch a "stevan-gpu-monster" GPU instance that could have racked up £18,000 ($23,600) in monthly costs. Sysdig researchers identified LLM hallmarks, including Serbian-commented code, hallucinated AWS IDs like "123456789012," and phantom GitHub references, confirming AI's hand in accelerating the assault.

To evade detection, the threat actor cycled through an IP rotator and 19 identities, attempting lateral movement via default roles like OrganizationAccountAccessRole in a multi-account setup. This stealthy persistence highlights evolving tactics where AI not only speeds execution but also enhances obfuscation, turning minutes-long attacks into prolonged threats if undetected.

Experts warn that mundane errors like exposed keys—not novel exploits—drive such incidents, urging organizations to ditch static credentials for short-lived IAM roles, harden automated accounts, and monitor for anomalous enumeration spikes. As breaches shrink from days to minutes, AI-aware defenses must match this pace to protect cloud assets effectively.
Read more »
Malware attacks
Cyber Attacks CyberSecurity Ransomware Attacks Finance Sector financial attack malicious PDF files Malware attacks

PDFSider Malware Used in Fortune 100 Finance Ransomware Attack

 

A Fortune 100 finance company was targeted by ransomware actors using a new Windows malware strain called PDFSider, built to quietly deliver malicious code during intrusions. Rather than relying on brute force, the attackers used social engineering, posing as IT support staff and convincing employees to launch Microsoft Quick Assist, enabling remote access. Resecurity researchers identified the malware during incident response, describing it as a stealth backdoor engineered to avoid detection while maintaining long-term control, with traits typically associated with advanced, high-skill intrusion activity. 

Resecurity previously told BleepingComputer that PDFSider had appeared in attacks connected to Qilin ransomware, but researchers emphasize it is not limited to a single group. Their threat hunting indicates the backdoor is now actively used by multiple ransomware operators as a delivery mechanism for follow-on payloads, suggesting it is spreading across criminal ecosystems rather than remaining a niche tool. 

The infection chain begins with spearphishing emails containing a ZIP archive. Inside is a legitimate, digitally signed executable for PDF24 Creator, developed by Miron Geek Software GmbH, paired with a malicious DLL named cryptbase.dll. Since the application expects that DLL, it loads the attacker’s version instead. This technique, known as DLL side-loading, allows the malicious code to execute under the cover of a trusted program, helping it evade security controls that focus on the signed executable rather than the substituted library.  
In some cases, attackers increase the likelihood of execution using decoy documents crafted to appear relevant to targets. One example involved a file claiming authorship from a Chinese government entity. Once launched, the malicious DLL inherits the same privileges as the legitimate executable that loaded it, increasing the attacker’s ability to operate within the system. 

Resecurity notes that while the EXE remains validly signed, attackers exploited weaknesses in the PDF24 software to load the malware and bypass EDR tools more effectively. The firm also warns that AI-assisted coding is making it easier for cybercriminals to identify and exploit vulnerable software at scale. After execution, PDFSider runs primarily in memory to reduce disk traces, using anonymous pipes to issue commands through CMD. 

Each infected device is assigned a unique identifier, system details are collected, and the data is exfiltrated to an attacker-controlled VPS through DNS traffic on port 53. For command-and-control security, PDFSider uses Botan 3.0.0 and encrypts communications with AES-256-GCM, decrypting inbound data only in memory to limit its footprint. It also applies AEAD authentication in GCM mode, a cryptographic approach commonly seen in stealthy remote shell backdoors designed for targeted operations. 

The malware includes anti-analysis checks such as RAM size validation and debugger detection, terminating early when it suspects sandboxing. Based on its behavior and design, Resecurity assesses PDFSider as closer to espionage-grade tradecraft than typical financially motivated ransomware tooling, built to quietly preserve covert access, execute remote commands flexibly, and keep communications protected.
Read more »
Windows
Audit ClawHub Cyber Attacks macOS Windows

Experts Find Malicious ClawHub Skills Stealing Data from OpenClaw


Koi Security’s security audit of 2,857 skills on ClawHub found 341 malicious skills via multiple campaigns. Users are exposed to new supply chain threats. 

ClawHub is a marketplace made to help OpenClaw users in finding and installing third-party skills. It is a part of the OpenClaw project, a self-hosted artificial intelligence (AI) assistant aka Moltbot and Clawdbot. 

Koi Security's analysis with OpenClaw bot “Alex” revealed that 335 skills use malicious pre-requisite to install an Apple macOS stealer called (Atomic Stealer). The activity goes by the code name ClawHavoc. 

According to Koi research Oren Yomtov, "You install what looks like a legitimate skill – maybe solana-wallet-tracker or youtube-summarize-pro. The skill's documentation looks professional. But there's a 'Prerequisites' section that says you need to install something first.”

Instruction steps:

Windows users are asked to download file “openclaw-agent.zip” from a GitHub repository.

macOS users are asked to copy an installation script hosted at glot[.]io and paste it in the Terminal application. 

Threat actors are targeting macOS users because of an increase in purchase of Mac Minus to use the AI assistant 24x7. 

In the password-protected archive, the trojan has keylogging functionality to steal credentials, API keys, and other important data on the device. Besides this, the glot[.]io script includes hidden shell commands to retrieve next-stage payloads from a threat-actor controlled infrastructure. 

This results in getting another IP address ("91.92.242[.]30") to get another shell script, which is modified to address the same server to get a universal Mach-O binary that shows traits persistent with Atomic Stealer, a commodity stealer that threat actors can buy for $500-1000/month that can extract data from macOS hosts.

The issue is that anyone can post abilities to ClawHub because it is open by default. At this point, the only requirement is that a publisher have a GitHub account that is at least a week old. 

Peter Steinberger, the founder of OpenClaw, is aware of the problem with malicious abilities and has subsequently implemented a reporting option that enables users who are signed in to report a skill. According to the documentation, "Each user can have up to 20 active reports at a time," "Skills with more than 3 unique reports are auto-hidden by default.”


Read more »
Malware Campaigns
AI Prompt AI Security AI security risks AI technology Cyber Attacks LLM LLM cybersecurity Malware Campaigns

Promptware Threats Turn LLM Attacks Into Multi-Stage Malware Campaigns

 

Large language models are now embedded in everyday workplace tasks, powering automated support tools and autonomous assistants that manage calendars, write code, and handle financial actions. As these systems expand in capability and adoption, they also introduce new security weaknesses. Experts warn that threats against LLMs have evolved beyond simple prompt tricks and now resemble coordinated cyberattacks, carried out in structured stages much like traditional malware campaigns. 

This growing threat category is known as “promptware,” referring to malicious activity designed to exploit vulnerabilities in LLM-based applications. It differs from basic prompt injection, which researchers describe as only one part of a broader and more serious risk. Promptware follows a deliberate sequence: attackers gain entry using deceptive prompts, bypass safety controls to increase privileges, establish persistence, and then spread across connected services before completing their objectives.  

Because this approach mirrors conventional malware operations, long-established cybersecurity strategies can still help defend AI environments. Rather than treating LLM attacks as isolated incidents, organizations are being urged to view them as multi-phase campaigns with multiple points where defenses can interrupt progress.  

Researchers Ben Nassi, Bruce Schneier, and Oleg Brodt—affiliated with Tel Aviv University, Harvard Kennedy School, and Ben-Gurion University—argue that common assumptions about LLM misuse are outdated. They propose a five-phase model that frames promptware as a staged process unfolding over time, where each step enables the next. What may appear as sudden disruption is often the result of hidden progress through earlier phases. 

The first stage involves initial access, where malicious prompts enter through crafted user inputs or poisoned documents retrieved by the system. The next stage expands attacker control through jailbreak techniques that override alignment safeguards. These methods can include obfuscated wording, role-play scenarios, or reusable malicious suffixes that work across different model versions. 

Once inside, persistence becomes especially dangerous. Unlike traditional malware, which often relies on scheduled tasks or system changes, promptware embeds itself in the data sources LLM tools rely on. It can hide payloads in shared repositories such as email threads or corporate databases, reactivating when similar content is retrieved later. An even more serious form targets an agent’s memory directly, ensuring malicious instructions execute repeatedly without reinfection. 

The Morris II worm illustrates how these attacks can spread. Using LLM-based email assistants, it replicated by forcing the system to insert malicious content into outgoing messages. When recipients’ assistants processed the infected messages, the payload triggered again, enabling rapid and unnoticed propagation. Experts also highlight command-and-control methods that allow attackers to update payloads dynamically by embedding instructions that fetch commands from remote sources. 

These threats are no longer theoretical, with promptware already enabling data theft, fraud, device manipulation, phishing, and unauthorized financial transactions—making AI security an urgent issue for organizations.
Read more »
Record DDoS Attack
Aisuru botnet Botnet Infrastructure Cloudflare Mitigation Cyber Attacks Cyber Threats Global DDoS Trends Hyper-Volumetric DDoS IoT Botnets Network Layer Attacks Record DDoS Attack

Aisuru Botnet Drives DDoS Attack Volumes to Historic Highs


Currently, the modern internet is characterized by near-constant contention, in which defensive controls are being continuously tested against increasingly sophisticated adversaries. However, there are some instances where even experienced security teams are forced to rethink long-held assumptions about scale and resilience when an incident occurs. 


There has been an unprecedented peak of 31.4 terabits per second during a recent Distributed Denial of Service attack attributed to the Aisuru botnet, which has proven that the recent attack is firmly in that category. 

Besides marking a historical milestone, the event is revealing a sharp change in botnet orchestration, traffic amplification, and infrastructure abuse, demonstrating that threat actors are now capable of generating disruptions at levels previously thought to be theoretical. As a consequence of this attack, critical questions are raised regarding the effectiveness of current mitigation architectures and the readiness of global networks to withstand such an attack.

Aisuru-Kimwolf is at the center of this escalation, a vast array of compromised systems that has rapidly developed into the most formidable DDoS platform to date. Aisuru and its Kimwolf offshoot are estimated to have infected between one and four million hosts, consisting of a diverse array of consumer IoT devices, digital video recorders, enterprise network appliances, and virtual machines based in the cloud. 

As a result of this diversity, the botnet has been able to generate volumes of traffic which are capable of overwhelming critical infrastructure, destabilizing national connectivity, and surpassing the handling capacities of many legacy cloud-based DDoS mitigation services. As far as operational performance is concerned, Aisuru-Kimwolf has demonstrated its consistency in executing hyper-volumetric and packet-intensive campaigns at a scale previously deemed impractical. 

As documented by the botnet, the botnet is responsible for record-breaking flooding reaches 31.4 Tbps, packet rates exceeding 14.1 billion packets per second, and highly targeted DNS-based attacks, including random prefixes and so-called water torture attacks, as well as application-layer HTTP floods that exceed 200 million requests per second. 

As part of these operations, carpet bombing strategies are used across wide areas and packet headers and payload attributes are randomly randomized, a deliberate design choice meant to frustrate signature-based detection and slow automated mitigation. 

The attack usually occurs rapidly and in high intensity bursts that reach peak throughput almost instantly and subside within minutes, creating a hit-and-run attack that makes attribution and response more difficult. 

There was an increase of more than 700 percent in attack potential observed in the Aisuru-Kimwolf ecosystem between the years 2025 and 2026, demonstrating the rapid development of this ecosystem. Aisuru botnets serve as the architectural core of this ecosystem, which are responsible for this activity. 

In addition to serving as a foundational platform, Aisuru enables the development and deployment of derivative variants, including Kimwolf, which extends the botnet's reach and operational flexibility. By continuously exploiting exposed or poorly secured devices in the consumer and cloud environments, the ecosystem has created a globally distributed attack surface reflective of a larger shift in how modern botnets are designed. 

In contrast to the traditional techniques of DDoS relying solely on persistence, Aisuru-based networks emphasize scalability, rapid mobilization, and adaptive attack techniques, signalling the development of an evolving threat model that is reshaping the upper limits of large-scale DDoS attacks. 

Additionally, people have seen a clear shift from long-duration attacks to short-duration, high-intensity attacks that are designed to maximize disruptions while minimizing exposure. There has been a significant decrease in the number of attacks that persist longer than a short period of time, with only a small fraction lasting longer than that period.

There were overwhelmingly three to five billion packets per second at peak for the majority of incidents, while the overall packet rate was overwhelmingly clustered between one and five terabits per second. It reflects a deliberate operational strategy to concentrate traffic within narrowly defined, yet extremely extreme thresholds, with the goal of promoting rapid saturation over prolonged engagement. 

Although these attacks were large in scope, Cloudflare's defenses were automatically able to identify and mitigate them without initiating internal escalation procedures, highlighting the importance of real-time, autonomous mitigation systems in combating modern DDoS threats. 

Although Cloudflare's analysis indicates a notable variation in attack sourcing during the so-called "Night Before Christmas" campaign as compared to previous waves of Aisuru botnet activity originating from compromised IoT devices and consumer routers, Cloudflare's analysis shows a significant change in attack sourcing. 

As part of that wave of activity, Android-based television devices became the primary source of traffic, which highlights how botnet ecosystems continue to engulf non-traditional endpoints. In addition to expanding attack capacity, this diversity of compromised hardware complicates defensive modeling, as traffic originates from devices which blend into legitimate consumer usage patterns, increasing the complexity of defensive modeling. 

These findings correspond to broader trends documented in Cloudflare's fourth-quarter 2025 DDoS Threat Report, which documented a 121 percent increase in attack volume compared with the previous year, totaling 47.1 million incidents. 

A Cloudflare application has been able to mitigate over 5,300 DDoS attacks a day, nearly three quarters of which occurred on the network layer and the remainder targeting HTTP application services. During the final quarter, the number of DDoS attacks accelerated further, increasing by 31 percent from the previous quarter and 58 percent from the previous year, demonstrating a continuing increase in both frequency and intensity. 

A familiar pattern of industry targeting was observed during this period, but it was becoming increasingly concentrated, with telecommunications companies, IT and managed services companies, online gambling platforms and gaming companies experiencing the greatest levels of sustained pressure. Among attack originators, Bangladesh, Ecuador, and Indonesia appeared to be the most frequently cited sites, with Argentina becoming a significant source while Russia's position declined. 

Throughout the year, organizations located in China, Hong Kong, Germany, Brazil, and the United States experienced the largest amount of DDoS attacks, reflecting the persistent focus on regions with dense digital infrastructure and high-value online services. 

According to a review of attack source distribution in the fourth quarter of 2025, there have been notable changes in the geographical origins of malicious traffic, which supports the emergence of a fluid global DDoS ecosystem.

A significant increase was recorded in attack traffic by Bangladesh during the period, displace Indonesia, which had maintained the top position throughout the previous year but subsequently fell to third place. Ecuador ranked second, while Argentina climbed twenty positions to take the fourth position, regaining its first place in attack traffic. 

In addition to Hong Kong, Ukraine, Vietnam, Taiwan, Singapore, and Peru, there were other high-ranking origins, which emphasize the wide international dispersion of attack infrastructure. The relative activity of Russia declined markedly, falling several positions, while the United States also declined, reflecting shifting operational preferences rather than a decline in regional engagement. 

According to a network-level analysis, threat actors continue to favor infrastructure that is scalable, flexible and easy to deploy. A significant part of attacks observed in the past few months have been generated by cloud computing platforms, with providers such as DigitalOcean, Microsoft, Tencent, Oracle, and Hetzner dominating the higher tiers of originating networks with their offerings. 

Throughout the trend, there has been a sustained use of on-demand virtual machines to generate high-volume attack traffic on a short notice basis. In addition to cloud services, traditional telecommunications companies remained prominent players as well, especially in parts of the Asia-Pacific region, including Vietnam, China, Malaysia, and Taiwan.

Large-scale DDoS operations are heavily reliant on both modern cloud environments and legacy carrier infrastructure. The Cloudflare global mitigation infrastructure was able to absorb the unprecedented intensity of the "Night Before Christmas" campaign without compromising service quality. 

In spite of 330 points of presence and a total mitigation capacity of 449 terabits per second, only a small fraction of the total mitigation capacity was consumed, which left the majority of defensive capacity untouched during the record-setting flood of 31.4 Tbps. 

It is noteworthy that detection and mitigation were performed autonomously, without the need for internal alerts or manual intervention, thus underscoring the importance of machine-learning-driven systems for responding to attacks that unfold at a rapid pace. 

As a whole, the campaign illustrates the widening gap between hackers’ growing capability and the defensive limitations of organizations relying on smaller-scale protection services, many of which would have been theoretically overwhelmed by an attack of this magnitude if it had taken place. 

An overall examination of the Aisuru campaign indicates that a fundamental shift has taken place in the DDoS threat landscape, with attack volumes no longer constrained by traditional assumptions about bandwidth ceilings and device types.

The implications for defenders are clear: resilience cannot be treated as a static capability, but must evolve concurrently with adversaries operating at a machine-scale and speed that is increasingly prevalent. 

Due to the complexity of the threats that are becoming more prevalent in the world, organizations have been forced to reevaluate not only their mitigation capabilities, but also the architectural assumptions that lay behind their security strategies, particularly when latency, availability, and trust are essential factors. 

Hypervolumetric attacks are becoming shorter, sharper, and more automated over time. Therefore, effective defense will be dependent on global infrastructure, real-time intelligence, and automated response mechanisms that are capable of absorbing disruptions without human intervention. Accordingly, the Aisuru incident is less of an anomaly and more of a preview of the operational baseline against which modern networks must prepare.
Read more »
URL Attack
Cyber Attacks GenAI Security Microsoft Copilot Prompt Injection URL Attack

New Reprompt URL Attack Exposed and Patched in Microsoft Copilot

 

Security researchers at Varonis have uncovered a new prompt-injection technique targeting Microsoft Copilot, highlighting how a single click could be enough to compromise sensitive user data. The attack method, named Reprompt, abuses the way Copilot and similar generative AI assistants process certain URL parameters, effectively turning a normal-looking link into a vehicle for hidden instructions. While Microsoft has since patched the flaw, the finding underscores how quickly attackers are adapting AI-specific exploitation methods.

Prompt injection attacks work by slipping hidden instructions into content that an AI model is asked to read, such as emails or web pages. Because large language models still struggle to reliably distinguish between data to analyze and commands to execute, they can be tricked into following these embedded prompts. In traditional cases, this might mean white text on a white background or minuscule fonts inside an email that the user then asks the AI to summarize, unknowingly triggering the malicious instructions.

Reprompt takes this concept a step further by moving the injection into the URL itself, specifically into a query parameter labeled “q.” Varonis demonstrated that by appending a long string of detailed instructions to an otherwise legitimate Copilot link, such as “http://copilot.microsoft.com/?q=Hello”, an attacker could cause Copilot to treat that parameter as if the user had typed it directly into the chat box. In testing, this allowed the researchers to exfiltrate sensitive data that the victim had previously shared with the AI, all triggered by a single click on a crafted link.

This behaviour is especially dangerous because many LLM-based tools interpret the q parameter as natural-language input, effectively blurring the line between navigation and instruction. A user might believe they are simply opening Copilot, but in reality they are launching a session already preloaded with hidden commands created by an attacker. Once executed, these instructions could request summaries of confidential conversations, collect personal details, or send data to external endpoints, depending on how tightly the AI is integrated with corporate systems.

After Varonis disclosed the issue, Microsoft moved to close the loophole and block prompt-injection attempts delivered via URLs. According to the researchers, prompt injection through q parameters in Copilot is no longer exploitable in the same way, reducing the immediate risk for end users. Even so, Reprompt serves as a warning that AI interfaces—especially those embedded into browsers, email clients, and productivity suites—must be treated as sensitive attack surfaces, demanding continuous testing and robust safeguards against new injection techniques.
Read more »
Cyber Attacks
AI Attacks AI Prompt AI prompt injection attack AI Risks AI Systems AI technology autonomous vehicles Cyber Attacks

Visual Prompt Injection Attacks Can Hijack Self-Driving Cars and Drones

 

Indirect prompt injection happens when an AI system treats ordinary input as an instruction. This issue has already appeared in cases where bots read prompts hidden inside web pages or PDFs. Now, researchers have demonstrated a new version of the same threat: self-driving cars and autonomous drones can be manipulated into following unauthorized commands written on road signs. This kind of environmental indirect prompt injection can interfere with decision-making and redirect how AI behaves in real-world conditions. 

The potential outcomes are serious. A self-driving car could be tricked into continuing through a crosswalk even when someone is walking across. Similarly, a drone designed to track a police vehicle could be misled into following an entirely different car. The study, conducted by teams at the University of California, Santa Cruz and Johns Hopkins, showed that large vision language models (LVLMs) used in embodied AI systems would reliably respond to instructions if the text was displayed clearly within a camera’s view. 

To increase the chances of success, the researchers used AI to refine the text commands shown on signs, such as “proceed” or “turn left,” adjusting them so the models were more likely to interpret them as actionable instructions. They achieved results across multiple languages, including Chinese, English, Spanish, and Spanglish. Beyond the wording, the researchers also modified how the text appeared. Fonts, colors, and placement were altered to maximize effectiveness. 

They called this overall technique CHAI, short for “command hijacking against embodied AI.” While the prompt content itself played the biggest role in attack success, the visual presentation also influenced results in ways that are not fully understood. Testing was conducted in both virtual and physical environments. Because real-world testing on autonomous vehicles could be unsafe, self-driving car scenarios were primarily simulated. Two LVLMs were evaluated: the closed GPT-4o model and the open InternVL model. 

In one dataset-driven experiment using DriveLM, the system would normally slow down when approaching a stop signal. However, once manipulated signs were placed within the model’s view, it incorrectly decided that turning left was appropriate, even with pedestrians using the crosswalk. The researchers reported an 81.8% success rate in simulated self-driving car prompt injection tests using GPT-4o, while InternVL showed lower susceptibility, with CHAI succeeding in 54.74% of cases. Drone-based tests produced some of the most consistent outcomes. Using CloudTrack, a drone LVLM designed to identify police cars, the researchers showed that adding text such as “Police Santa Cruz” onto a generic vehicle caused the model to misidentify it as a police car. Errors occurred in up to 95.5% of similar scenarios. 

In separate drone landing tests using Microsoft AirSim, drones could normally detect debris-filled rooftops as unsafe, but a sign reading “Safe to land” often caused the model to make the wrong decision, with attack success reaching up to 68.1%. Real-world experiments supported the findings. Researchers used a remote-controlled car with a camera and placed signs around a university building reading “Proceed onward.” 

In different lighting conditions, GPT-4o was hijacked at high rates, achieving 92.5% success when signs were placed on the floor and 87.76% when placed on other cars. InternVL again showed weaker results, with success only in about half the trials. Researchers warned that these visual prompt injections could become a real-world safety risk and said new defenses are needed.
Read more »
Subscribe to: Comments (Atom)