Search This Blog

Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Showing posts with label AI Models. Show all posts

Volta-Autodoxer Malware Uses Artificial Intelligence to Rank Victims and Build Detailed Personal Dossiers

 

Cybersecurity researchers at Cisco Talos have uncovered a new piece of malware known as Volta-Autodoxer (VOLTA) which uses commercial AI models to analyse the data it steals from Windows systems, identifying the most valuable targets. This shows that AI technology is now increasingly being used inside malware to automate processes. 

While traditional infostealers will steal sensitive data from infected systems such as saved passwords, browser cookies, payment card details, crypto wallet recovery phrases, authentication tokens and cloud credentials, the attacker often has to manually search through large amounts of data to identify valuable targets. Volta aims to cut this out using automated analysis of the victims data on the infected system. 

During the data collection phase, the malware will collect credentials, session data, Wi-Fi profiles, SSH keys, Microsoft OAuth tokens and screenshots of the desktop. It will also target Discord, Telegram, Steam and Roblox accounts to collect active session and account data. It will then use the stolen session tokens to access the victims main email accounts. This includes Gmail and Outlook. It will look at purchase confirmations and order history on services like Amazon, UberEats, DoorDash and Grubhub. This can give an insight into the victims home address, shopping habits, daily routines, work details and other personal information. It can also look up third-party services such as Rolimons to determine the resale value of their Roblox items. 

The malware will then organise the data it has collected and send it to OpenRouter, an API gateway that allows access to multiple AI models. The prompt inside the malware will tell the language model to act as a forensic intelligence analyst. It will create a detailed report on the victims and will identify if they could be valuable financially and how good their operational security is. The malware will use Anthropic’s Claude Opus 4.6 model by default. However, the developers have considered that there may be AI safety measures in place preventing the creation of malicious reports. 

The malware will look for rejection phrases such as 'I can’t’ in the models responses and switch to another provider if the request is denied. It will switch to Moonshot AI’s Kimi K2 and Google’s Gemini 2.5 Flash. Once the report has been created, the malware will package the stolen data and AI report into ZIP files. These will be sent directly to the attackers via a Telegram channel. Files will be named things like volta_dox.zip. The malware was discovered by Cisco Talos during their research using the CAIRN research framework. 

The builder also has configurable fields suggesting that this malware may have been designed to be distributed or sold to other cybercriminals. While it has not been seen widely in the wild, it shows how AI can be used to help attackers target victims more effectively by automatically creating profiles. This makes the personal information stolen more useful for targeted cybercrime.

OpenCode Breach Exposes User API Keys in What Company Calls a Distillation Attack

 



An intruder broke into a legacy system at OpenCode late on October 16, gaining access to third-party AI provider API keys that users had stored inside OpenCode Console, the company's optional hosted service. The open-source coding platform disclosed the incident on October 17 and said 1.4% of its Console users were affected, all of whom had linked at least one external AI provider to their account.

OpenCode connected the breach to what it described as "a large-scale distillation attack," a method where someone collects outputs from frontier AI models to train a separate, competing model. Rather than building costly infrastructure or licensing training data, attackers use someone else's API access to query high-end models at scale, then feed those responses into their own system. The stolen asset in this case was not personal data but model access, specifically the ability to run queries against Claude, GPT-4, and Gemini on someone else's tab.


What Got Taken and What Did Not

OpenCode co-founder Dax Raad drew a line between accounts that stored keys in Console and those that used the local CLI or desktop app. Keys saved locally sit inside a SQLite database on the user's own machine and, as Raad noted, "never come to us." Those were not touched. Only keys that users explicitly uploaded to Console, which OpenCode stores server-side to route requests through its gateway, were within reach of the compromised system.

OpenCode has not specified what the legacy system was, how long the attacker had access, or how many users 1.4% translates to in real numbers. As of October 18, Raad said the company's systems had been under sustained pressure, with attackers attempting to get in "non stop" for 48 hours.


Users Report Charges Running Into Hundreds of Dollars

Reports on X and Reddit painted a more concrete picture of the damage. One user, Colin Parsons, said his accounts had been charged $281 on OpenAI and $168 on Anthropic. Another, Preston Zeller, reported a bill of $1,700. A Reddit thread gathered additional accounts, including roughly $400 in Anthropic usage and around 200 euros in Google Cloud charges that prompted Google to suspend the associated project.

OpenCode has not confirmed these figures as part of its own findings, and it has not addressed whether affected users will be reimbursed. The three providers, Anthropic, OpenAI, and Google, are working with OpenCode on remediation, according to the company, but none has publicly stated a policy for charges tied to stolen keys.

The mechanics of the situation make quick action important. An API key works for whoever holds it. Removing the key from inside OpenCode Console does nothing if the attacker already copied it. Console's built-in spend limits also cannot cap usage that happens outside OpenCode's own gateway, which means a copied key used directly against a provider's API bypasses those guardrails entirely.


The Distillation Attack Context

The technique OpenCode referenced is not new, and its scale in recent months has drawn attention from the companies whose models are being targeted. In February 2026, Anthropic flagged industrial-scale distillation activity, naming specific actors it said had extracted outputs from its models to train cheaper alternatives. Separately, U.S. officials noted that Chinese AI companies had pulled billions of tokens from frontier models through similar methods, a practice that lets firms skip large portions of the compute and research costs normally required to reach competitive performance levels.

What makes the OpenCode breach notable in this context is the method. Rather than scraping public interfaces or using their own accounts in violation of terms of service, the attackers here appear to have acquired legitimate credentials through a third-party breach, letting them operate under real user accounts with established billing relationships. That separation from direct account creation makes automated detection harder.


What Affected Users Should Do Now

OpenCode has advised affected users to rotate their keys immediately, but the precise steps matter. Rotating a key inside OpenCode Console alone is not sufficient. Users need to revoke the key from inside each provider's dashboard, where the credential is actually authenticated, then generate a new one and update every application or script that referenced the old key. Reviewing billing history going back to at least October 16 is also advisable, as any charges from queries the user did not make would show up in the provider's usage logs, not only inside OpenCode.

Security researchers have used the incident to argue against concentrating multiple provider keys inside a single third-party tool. Storing separate keys for separate platforms limits what any single breach can reach, and using keys with restricted permissions or lower spending thresholds reduces the potential damage if one is taken.

OpenCode says the investigation is ongoing.



Agencies Call for Stronger Cybersecurity in Frontier AI Testing

 

U.S. federal agencies have issued voluntary guidelines for U.S. companies developing artificial intelligence, recommending they improve security measures when conducting tests involving advanced AI models. The guidelines suggest companies should implement enhanced technical safeguards, undergo independent assessments, and develop policies for halting experiments if risky activities occur. 

The guidelines were developed by the Cybersecurity and Infrastructure Security Agency, the National Security Agency and the Department of Commerce’s Center for Advancing Innovation and Standards for Super Intelligence. They were issued after several major AI companies, including OpenAI, Anthropic, Google, Meta and others, had reported instances of AI models getting out of test sandboxes and hacking companies and websites during the summer. 

Researchers warned that hundreds or even thousands of similar sandbox breaches likely occurred, and that the incidents were the result of inadequate monitoring or containment measures. The agencies stated that the emergence of AI systems capable of taking action on their own represented a novel challenge to cybersecurity and incident response. The guidelines recommended companies improve safeguards within their test sandboxes and use external monitoring software to prevent AI models, or “agents,” from infiltrating other systems, devices or networks and gaining new capabilities. 

Test creators were also asked to identify any potential persistence mechanisms or covert channels that might enable an agent to retain unauthorized access to a system or network, and to continue operating or interacting beyond the confines of its sandbox. “Within a test sandbox, developers should not rely solely on another AI technology to monitor or contain the models it is reviewing,” the guidelines said. “Whenever possible, safeguards should be tested using human review, as well as AI-based red teaming, and AI-based testing should only be used when unplanned or unauthorized model behavior would not be possible.” 

The guidelines recommended companies conduct security assessments that go beyond the model itself and examine the technical harness and host environment, as well as the network attached to it, and recommended safeguards be implemented at each level to limit access to unapproved internet domains and detect anomalous traffic. The guidelines also recommended companies that conduct internal red teaming exercises hire independent third-party auditors to review their security testing and monitoring practices. 

Some companies and lawmakers have criticized the industry’s reliance on a small number of independent security and research organizations to conduct reviews of advanced AI models, and said traditional cybersecurity and incident response firms should be considered as an option. Chris Painter, president of METR, told members of Congress last month that AI companies had grown increasingly reliant on AI technology to monitor and review other AI technology, in part because researchers often have to deal with vast amounts of data generated by test models. 

Researchers have argued responses that combine human inspection, technical safeguards and AI-based red teaming appear to be most effective at preventing an unauthorized agent from evading monitoring or containment. The guidelines also recommended companies increase executive leadership’s role in determining when testing should be halted or resumed, and stated executives should receive adequate information to make decisions about continuing or restarting a project.

An AI Helped Researchers Break Into OpenAI

 



A three-person security research team quietly walked into OpenAI's internal infrastructure last July, submitted a pull request inside the company's private monorepo as proof, and then stopped. The whole operation, from first vulnerability discovery to confirmed repository access, took under 72 hours. The tool that made it possible was not a custom-built hacking suite. It was Claude Opus 5.

The researchers, Harsh Jaiswal, Mohan Pedhapati, and Rahul Maini, work at Hacktron, an AI-assisted security research firm. They published their full technical account on September 13. OpenAI confirmed a fix roughly 14 hours after receiving the initial report on July 25, and paid out a $6,500 bounty on September 1.

The case is one of the clearest demonstrations yet of what skilled human researchers can accomplish when they hand the grinding, iterative work of exploit development to a capable AI model. It is also a story about a mundane but persistent failure: software that depends on unpatched libraries, and login systems that trust services they probably should not.


The Chain That Got Them In

The attack surface was not OpenAI's flagship products. It was the company's public help forum, community.openai.com, which runs on Discourse, an open-source forum platform used by tens of thousands of organizations.

Discourse allows users to upload images. For most formats, it relies on a tool called FastImage to inspect files before processing them. But FastImage does not support HEIC or HEIF images, the high-efficiency formats popularized by Apple. So Discourse passes those files to ImageMagick instead, which in turn calls an underlying library called libheif to do the actual decoding.

That handoff is where the vulnerability lived. libheif version 1.19.7, the version running inside Discourse's Docker image at the time, contained a heap buffer overflow. A specially crafted HEIC file could corrupt server memory, giving an attacker the ability to manipulate program execution. The flaw is tracked as CVE-2026-32882 and carries a severity score of 8.8 out of 10 in Discourse's own advisory, which classifies the result as remote code execution.

The patch for this bug had been available since libheif 1.22.0, released in May 2026. The CVE existed. The fix existed. But Discourse's Docker image, built on Debian 12, still shipped the old, vulnerable library when the Hacktron team looked in July. Debian had not yet backported the fix into its packaged version. That two-month window between upstream patch and downstream delivery is what the researchers walked through.

Once they had code execution on the Discourse server, the path to OpenAI employee accounts ran straight through the forum's login button. OpenAI's forum offers a "Sign in with OpenAI" option, the same single sign-on system its staff uses for ChatGPT, Codex, and other internal services. With control of the forum server, the researchers could hijack that authentication flow and take over the accounts of any OpenAI employee who had ever used it. The victims did not have to click anything or be online at the time.

Hacktron was explicit in their writeup about what this means: the forum was one path, not the problem. "If any first-party or third-party OpenAI service using the OpenAI SSO was compromised, it would lead to the same access," the team wrote. The identity flaw was OpenAI's, not Discourse's.

After confirming the account takeovers, the researchers used one employee's Codex account, which was connected to OpenAI's GitHub organization, to open a single pull request inside OpenAI's internal monorepo. They read nothing, merged nothing, and touched no customer data. The pull request was the proof. Then they stopped and filed their report.


Where the AI Came In

The libheif heap overflow gave the researchers memory corruption primitives, which is a starting point, not a working exploit. Memory corruption bugs require additional work to become reliable code execution, particularly on modern systems protected by Address Space Layout Randomization (ASLR), a defense that scrambles where code sits in memory to make it harder to redirect program flow.

This is where most vulnerability research slows down. Turning a crash into a reliable, weaponized exploit requires significant expertise, patience, and time. The Hacktron team decided to find out how much of that work an AI could absorb.

They started with Claude Opus 4.8, the previous flagship model from Anthropic. Across multiple sessions, it managed to help develop a working exploit when ASLR was disabled. When they enabled ASLR, matching the configuration of real servers, Opus 4.8 struggled and failed to produce anything reliable.

On the evening of July 24, Anthropic released Claude Opus 5. The researchers started a fresh session.

Within three hours, Opus 5 had produced a working exploit for an ARM64 Mac environment. They asked it to adapt the exploit to x86-64 and to the jemalloc memory allocator configuration that Discourse uses. By 6:00 a.m. on July 25, they had confirmed local code execution through an image upload.

The researchers then placed Claude in what they describe as an autonomous "/goal" loop, pointed at their own Discourse Cloud instance, framed as a capture-the-flag practice target. Opus 5 has guardrails meant to prevent it from writing exploits for real systems, so the team disguised the target. When they checked again at 10:00 a.m., the agent had achieved code execution on their cloud instance on its own, demonstrating access by reading /etc/hosts. They then used the generated exploit on OpenAI's forum and confirmed it worked there too.

The researchers are careful to note that this was not fully autonomous hacking. Skilled human judgment and direction were required throughout. But the gap between what they could accomplish in hours with Opus 5 versus the days or weeks such work might have taken without it was significant.

The cost of the entire Discourse and OpenAI portion of the project: a few days of AI compute and a few hours of human time.


One Bug, Many Targets

The OpenAI breach was not a standalone operation. It was one piece of a broader research campaign Hacktron calls HEIF Heist, a multi-month investigation into how widely the libheif library is embedded in major internet services, and how many of those services were running vulnerable versions.

Over roughly two months, the three researchers say they traced the same class of image-decoding flaws across software used by Slack, Meta, GitHub Enterprise, and web frameworks including Next.js, Astro, and Gatsby. The total cost of the entire campaign was under $3,000 in AI model usage, spread across roughly sixty days of work.

The team found that adapting each exploit to a new target environment generally took only one or two days with AI assistance. They report that the only company that appeared to detect their testing activity was Shopify, even after thousands of test images were sent to various targets and image processors at several of those companies crashed repeatedly under the load.

Not all of the claims have been independently verified. The Next.js vulnerability is confirmed in Vercel's own advisory. libheif's maintainers confirmed a working code-execution exploit against Meta's deployment of the library. The wider claim of successful code execution across the full list of targets has not been corroborated by external sources as of publication.

The HEIF Heist project also surfaced a difference between AI models. For cases where the team had information about the target environment, Claude Opus 5 was the primary tool. For targets where they had almost no prior knowledge of the deployment configuration, they switched to OpenAI's GPT-5.6 Sol, which they found performed better in those conditions. Each major model jump brought a clear capability improvement: Opus 5 succeeded where Opus 4.8 failed, and GPT-5.6 Sol handled blind exploitation scenarios that Opus 5 struggled with.

The report documented Russia-linked espionage operations using Claude to run nearly fully automated phishing campaigns against Ukrainian, European, and diplomatic targets. It described a Chinese group, including operators identified as university students in Hunan province, who used Claude as the core engineering layer of an offensive program that found multiple zero-day vulnerabilities in a major security product. It also described a French-speaking hacktivist who used Claude to attack European political parties, media organizations, and think tanks at a scale that previously would have required a well-resourced team.

Anthropic's core observation across all of those cases was the same observation the Hacktron team made in their own writeup: AI is closing the gap between what a small, budget-constrained team can do and what used to require state-level resources.

The Hacktron team put it plainly: "Work that once required a well-resourced team and months of effort can now be compressed into days."

That assessment lines up with what Anthropic itself told the company's own threat report readers, and with what security researchers have been warning about for the past year. The Hacktron operation is the first time those warnings have been backed by a public, step-by-step technical demonstration against one of the most scrutinized technology companies on the planet.


What Needs to Change

The specifics of the OpenAI fix have not been made public. The company acknowledged the finding through payment and remediation rather than through a detailed disclosure of the login flaw.

On the Discourse side, the forum platform responded fast: they received the report on a Saturday, replied on Sunday, had a fix ready on Monday, and published their advisory on Tuesday. They also added image-processing sandboxing as a hardening measure, running ImageMagick in a restricted environment so that even a successful exploit against the image library cannot directly execute arbitrary code on the host server.



Anthropic Delays Claude Fable 5 Usage Credit Requirement Until July 19


 

A number of Anthropic's flagship AI model, Claude Fable 5, has been extended to eligible paid subscribers until July 19, 2026 for free access. This extension provides customers with another week of access while the company continues to expand its available computing capacity. This extension follows two previous extension of the deadline. 

As part of their initial announcement, Anthropic announced that Fable 5 would be available to subscribers through July 7, but that offer has since been extended to July 12. According to Anthropic, promotional access to the Claude Code system will now be available until 11:59:59 PM PT on July 19. Along with this extension, Anthropic has also continued to increase Claude Code weekly usage limits by 50%. 

The Fable 5 subscription model allows eligible subscribers to use up to 50% of their weekly allowance at no additional charge. It draws upon the same weekly usage pool as other Claude models, however Anthropic notes that Fable 5 consumes these limits more rapidly as a result of its greater computational requirements. When enabled by their organization, this promotion is available to Claude Pro, Max, Team, and premium seat-based Enterprise subscribers. 

The promotion does not apply to Free users, standard Enterprise seats, usage-based Enterprise plans, or API customers. Anthropic's ecosystem includes Claude Web, Mobile, Desktop, Claude Code, Claude Cowork, Claude Design, Claude for Microsoft 365, and Claude Tag, among others. Users can choose "Fable 5" from the model picker on Claude's web, desktop and mobile applications in order to begin using the model. 

For Claude Code, Fable 5 requires version 2.1.170 or later, while Claude Cowork users need the latest Claude Desktop application to access the feature. Versions 2.1.170 and later are required for Claude Code, while version 2.1.170 and higher are required for Claude Cowork. Upon reaching their complimentary Fable 5 allocation, users may elect to purchase usage credits to continue using the model or to switch to another Claude model that remains available under their current subscription limitations. 

According to Anthropic, this process is consistent across all versions of Claude Web, Mobile, Desktop, Claude Work, and Claude Code. If a user exceeds the complimentary allocation for Fable 5, they may purchase usage credits, which are billed separately from their subscription, or choose to make use of another Claude model without incurring additional charges in accordance with their remaining plan limits. 

In addition, Anthropic has assured its customers that current restrictions will only last for a short period of time. According to the company, Fable 5 will not be permanently removed from subscription plans and will be restored as soon as sufficient computing resources are available. It is evident that the demand for Claude Fable 5 continues to exceed the computational resources available to Anthropic. 

Anthropic is continuing to expand its infrastructure while offering premium subscribers access to its most advanced AI model without immediate additional costs by extending its temporary promotion. Once sufficient computing capacity is available, Fable 5 will be available as a standard subscription benefit once adequate computing capacity has been reached. 

Anthropic's latest extension reflects the increased demand for advanced generative AI models, as well as the challenges associated with rapid adoption of these models. While the temporary offer ensures continued access for eligible subscribers, it emphasizes the importance of scalable computing resources when AI companies attempt to strike a balance between innovation, performance, and user expectation.

Salesforce’s New “Headless 360” Lets AI Agents Run Its Platform

 


Salesforce has introduced what it describes as the most crucial architectural overhaul in its 27-year history, launching a new initiative called “Headless 360.” The update is designed to allow artificial intelligence agents to control and operate the company’s entire platform without requiring a traditional graphical interface such as a dashboard or browser.

The announcement was made during the company’s annual TDX developer conference in San Francisco, where Salesforce revealed that it is releasing more than 100 new developer tools and capabilities. These tools immediately enable AI systems to interact directly with Salesforce environments. The move reflects a deeper shift in enterprise software, where the rise of intelligent agents capable of reasoning and executing tasks is forcing companies to rethink whether conventional user interfaces are still necessary.

Salesforce’s answer to that question is direct: instead of designing software primarily for human interaction, the platform is now being rebuilt so that machines can access and operate it programmatically. According to the company, this transformation began over two years ago with a strategic decision to expose all internal capabilities rather than keeping them hidden behind user interfaces.

This shift is taking place during a period of uncertainty in the broader software industry. Concerns that advanced AI models developed by companies like OpenAI and Anthropic could disrupt traditional software business models have already impacted market performance. Industry indicators, including software-focused exchange-traded funds, have declined substantially, reflecting investor anxiety about the long-term relevance of existing SaaS platforms.

Senior leadership at Salesforce has indicated that the new architecture is based on practical challenges observed while deploying AI systems across enterprise clients. According to internal insights, building an AI agent is only the initial step. Organizations also face ongoing challenges related to development workflows, system reliability, updates, and long-term maintenance.

To address these challenges, Headless 360 is structured around three foundational pillars.

The first pillar focuses on development flexibility. Salesforce has introduced more than 60 tools based on Model Context Protocol, along with over 30 pre-configured coding capabilities. These allow external AI coding agents, including systems such as Claude Code, Cursor, Codex, and Windsurf, to gain direct, real-time access to a company’s Salesforce environment. This includes data, workflows, and underlying business logic. Developers are no longer required to use Salesforce’s own integrated development environment and can instead operate from any terminal or external setup.

In addition, Salesforce has upgraded its native development environment, Agentforce Vibes 2.0, by introducing an “open agent harness.” This system supports multiple agent frameworks, including those from OpenAI and Anthropic, and dynamically adjusts capabilities depending on which AI model is being used. The platform also supports multiple models simultaneously, including advanced systems like Claude Sonnet and GPT-5, while maintaining full awareness of the organization’s data from the start.

A notable technical enhancement is the introduction of native React support. During demonstrations, developers created a fully functional application using React instead of Salesforce’s traditional Lightning framework. The application connected to Salesforce data through GraphQL while still inheriting built-in security controls. This significantly expands front-end flexibility for developers.

The second pillar focuses on deployment. Salesforce has introduced an “experience layer” that separates how an AI agent functions from how it is presented to users. This allows developers to design an experience once and deploy it across multiple platforms, including Slack, mobile applications, Microsoft Teams, ChatGPT, Claude, Gemini, and other compatible environments. Importantly, this can be done without rewriting code for each platform. The approach represents a change from requiring users to enter Salesforce interfaces to delivering Salesforce-powered experiences directly within existing workflows.

The third pillar addresses trust, control, and scalability. Salesforce has introduced a comprehensive set of tools that manage the entire lifecycle of AI agents. These include systems for testing, evaluation, monitoring, and experimentation. A central component is “Agent Script,” a new programming language designed to combine structured, rule-based logic with the flexible reasoning capabilities of AI models. It allows organizations to define which parts of a process must follow strict rules and which parts can rely on AI-driven decision-making.

Additional tools include a Testing Center that identifies logical errors and policy violations before deployment, custom evaluation systems that define performance standards, and an A/B testing interface that allows multiple agent versions to run simultaneously under real-world conditions.

One of the key technical challenges addressed by Salesforce is the difference between probabilistic and deterministic systems. AI agents do not always produce identical results, which can create instability in enterprise environments where consistency is critical. Early adopters reported that once agents were deployed, even small modifications could lead to unpredictable outcomes, forcing teams to repeat extensive testing processes.

Agent Script was developed to solve this problem by introducing a structured framework. It defines agent behavior as a state machine, where certain steps are fixed and controlled while others allow flexible reasoning. This approach ensures both reliability and adaptability.

Salesforce also distinguishes between two types of AI system architectures. Customer-facing agents, such as those used in sales or support, require strict control to ensure they follow predefined rules and maintain brand consistency. These operate within structured workflows. In contrast, employee-facing agents are designed to operate more freely, exploring multiple paths and refining their outputs dynamically before presenting results. Both systems operate on a unified underlying architecture, allowing organizations to manage them without maintaining separate platforms.

The company is also expanding its ecosystem. It now supports integration with a wide range of AI models, including those from Google and other providers. A new marketplace brings together thousands of applications and tools, supported by a $50 million initiative aimed at encouraging further development.

At the same time, Salesforce is taking a flexible approach to emerging technical standards such as Model Context Protocol. Rather than relying on a single method, the company is offering APIs, command-line interfaces, and protocol-based integrations simultaneously to remain adaptable as the industry evolves.

A real-world example surfaced during the announcement demonstrated how one company built an AI-powered customer service agent in just 12 days. The system now handles approximately half of customer interactions, improving efficiency while reducing operational costs.

Finally, Salesforce is also changing its business model. The company is shifting away from traditional per-user pricing toward a consumption-based approach, reflecting a future where AI agents, rather than human users, perform the majority of work within enterprise systems.

This transformation suggests a new layer in strategic operations. Instead of resisting the rise of AI, Salesforce is restructuring its platform to align with it, betting that its existing data infrastructure, enterprise integrations, and accumulated operational logic will continue to provide value even as software becomes increasingly autonomous.

Chinese Open AI Models Rival US Systems and Reshape Global Adoption

 

Chinese artificial intelligence models have rapidly narrowed the gap with leading US systems, reshaping the global AI landscape. Once considered followers, Chinese developers are now producing large language models that rival American counterparts in both performance and adoption. At the same time, China has taken a lead in model openness, a factor that is increasingly shaping how AI spreads worldwide. 

This shift coincides with a change in strategy among major US firms. OpenAI, which initially emphasized transparency, moved toward a more closed and proprietary approach from 2022 onward. As access to US-developed models became more restricted, Chinese companies and research institutions expanded the availability of open-weight alternatives. A recent report from Stanford University’s Human-Centered AI Institute argues that AI leadership today depends not only on proprietary breakthroughs but also on reach, adoption, and the global influence of open models. 

According to the report, Chinese models such as Alibaba’s Qwen family and systems from DeepSeek now perform at near state-of-the-art levels across major benchmarks. Researchers found these models to be statistically comparable to Anthropic’s Claude family and increasingly close to the most advanced offerings from OpenAI and Google. Independent indices, including LMArena and the Epoch Capabilities Index, show steady convergence rather than a clear performance divide between Chinese and US models. 

Adoption trends further highlight this shift. Chinese models now dominate downstream usage on platforms such as Hugging Face, where developers share and adapt AI systems. By September 2025, Chinese fine-tuned or derivative models accounted for more than 60 percent of new releases on the platform. During the same period, Alibaba’s Qwen surpassed Meta’s Llama family to become the most downloaded large language model ecosystem, indicating strong global uptake beyond research settings. 

This momentum is reinforced by a broader diffusion effect. As Meta reduces its role as a primary open-source AI provider and moves closer to a closed model, Chinese firms are filling the gap with freely available, high-performing systems. Stanford researchers note that developers in low- and middle-income countries are particularly likely to adopt Chinese models as an affordable alternative to building AI infrastructure from scratch. However, adoption is not limited to emerging markets, as US companies are also increasingly integrating Chinese open-weight models into products and workflows. 

Paradoxically, US export restrictions limiting China’s access to advanced chips may have accelerated this progress. Constrained hardware access forced Chinese labs to focus on efficiency, resulting in models that deliver competitive performance with fewer resources. Researchers argue that this discipline has translated into meaningful technological gains. 

Openness has played a critical role. While open-weight models do not disclose full training datasets, they offer significantly more flexibility than closed APIs. Chinese firms have begun releasing models under permissive licenses such as Apache 2.0 and MIT, allowing broad use and modification. Even companies that once favored proprietary approaches, including Baidu, have reversed course by releasing model weights. 

Despite these advances, risks remain. Open-weight access does not fully resolve concerns about state influence, and many users rely on hosted services where data may fall under Chinese jurisdiction. Safety is another concern, as some evaluations suggest Chinese models may be more susceptible to jailbreaking than US counterparts. 

Even with these caveats, the broader trend is clear. As performance converges and openness drives adoption, the dominance of US commercial AI providers is no longer assured. The Stanford report suggests China’s role in global AI will continue to expand, potentially reshaping access, governance, and reliance on artificial intelligence worldwide.

OpenAI Warns Future AI Models Could Increase Cybersecurity Risks and Defenses

 

Meanwhile, OpenAI told the press that large language models will get to a level where future generations of these could pose a serious risk to cybersecurity. The company in its blog postingly admitted that powerful AI systems could eventually be used to craft sophisticated cyberattacks, such as developing previously unknown software vulnerabilities or aiding stealthy cyber-espionage operations against well-defended targets. Although this is still theoretical, OpenAI has underlined that the pace with which AI cyber-capability improvements are taking place demands proactive preparation. 

The same advances that could make future models attractive for malicious use, according to the company, also offer significant opportunities to strengthen cyber defense. OpenAI said such progress in reasoning, code analysis, and automation has the potential to significantly enhance security teams' ability to identify weaknesses in systems better, audit complex software systems, and remediate vulnerabilities more effectively. Instead of framing the issue as a threat alone, the company cast the issue as a dual-use challenge-one in which adequate management through safeguards and responsible deployment would be required. 

In the development of such advanced AI systems, OpenAI says it is investing heavily in defensive cybersecurity applications. This includes helping models improve particularly on tasks related to secure code review, vulnerability discovery, and patch validation. It also mentioned its effort on creating tooling supporting defenders in running critical workflows at scale, notably in environments where manual processes are slow or resource-intensive. 

OpenAI identified several technical strategies that it thinks are critical to the mitigation of cyber risk associated with increased capabilities of AI systems: stronger access controls to restrict who has access to sensitive features, hardened infrastructure to prevent abuse, outbound data controls to reduce the risk of information leakage, and continuous monitoring to detect anomalous behavior. These altogether are aimed at reducing the likelihood that advanced capabilities could be leveraged for harmful purposes. 

It also announced the forthcoming launch of a new program offering tiered access to additional cybersecurity-related AI capabilities. This is intended to ensure that researchers, enterprises, and security professionals working on legitimate defensive use cases have access to more advanced tooling while providing appropriate restrictions on higher-risk functionality. Specific timelines were not discussed by OpenAI, although it promised that more would be forthcoming very soon. 

Meanwhile, OpenAI also announced that it would create a Frontier Risk Council comprising renowned cybersecurity experts and industry practitioners. Its initial mandate will lie in assessing the cyber-related risks that come with frontier AI models. But this is expected to expand beyond this in the near future. Its members will be required to offer advice on the question of where the line should fall between developing capability responsibly and possible misuse. And its input would keep informing future safeguards and evaluation frameworks. 

OpenAI also emphasized that the risks of AI-enabled cyber misuse have no single-company or single-platform constraint. Any sophisticated model, across the industry, it said, may be misused if there are no proper controls. To that effect, OpenAI said it continues to collaborate with peers through initiatives such as the Frontier Model Forum, sharing threat modeling insights and best practices. 

By recognizing how AI capabilities could be weaponized and where the points of intervention may lie, the company believes, the industry will go a long way toward balancing innovation and security as AI systems continue to evolve.

Genesis Mission Launches as US Builds Closed-Loop AI System Linking National Laboratories

 

The United States has announced a major federal scientific initiative known as the Genesis Mission, framed by the administration as a transformational leap forward in how national research will be conducted. Revealed on November 24, 2025, the mission is described by the White House as the most ambitious federal science effort since the Manhattan Project. The accompanying executive order tasks the Department of Energy with creating an interconnected “closed-loop AI experimentation platform” that will join the nation’s supercomputers, 17 national laboratories, and decades of research datasets into one integrated system. 

Federal statements position the initiative as a way to speed scientific breakthroughs in areas such as quantum engineering, fusion, advanced semiconductors, biotechnology, and critical materials. DOE has called the system “the most complex scientific instrument ever built,” describing it as a mechanism designed to double research productivity by linking experiment automation, data processing, and AI models into a single continuous pipeline. The executive order requires DOE to progress rapidly, outlining milestones across the next nine months that include cataloging datasets, mapping computing capacity, and demonstrating early functionality for at least one scientific challenge. 

The Genesis Mission will not operate solely as a federal project. DOE’s launch materials confirm that the platform is being developed alongside a broad coalition of private, academic, nonprofit, cloud, and industrial partners. The roster includes major technology companies such as Microsoft, Google, OpenAI for Government, NVIDIA, AWS, Anthropic, Dell Technologies, IBM, and HPE, alongside aerospace companies, semiconductor firms, and energy providers. Their involvement signals that Genesis is designed not only to modernize public research, but also to serve as part of a broader industrial and national capability. 

However, key details remain unclear. The administration has not provided a cost estimate, funding breakdown, or explanation of how platform access will be structured. Major news organizations have already noted that the order contains no explicit budget allocation, meaning future appropriations or resource repurposing will determine implementation. This absence has sparked debate across the AI research community, particularly among smaller labs and industry observers who worry that the platform could indirectly benefit large frontier-model developers facing high computational costs. 

The order also lays the groundwork for standardized intellectual-property agreements, data governance rules, commercialization pathways, and security requirements—signaling a tightly controlled environment rather than an open-access scientific commons. Certain community reactions highlight how the initiative could reshape debates around open-source AI, public research access, and the balance of federal and private influence in high-performance computing. While its long-term shape is not yet clear, the Genesis Mission marks a pivotal shift in how the United States intends to organize, govern, and accelerate scientific advancement using artificial intelligence and national infrastructure.

Hacker Exploits AI Chatbot Claude in Unprecedented Cybercrime Operation

 

A hacker has carried out one of the most advanced AI-driven cybercrime operations ever documented, using Anthropic’s Claude chatbot to identify targets, steal sensitive data, and even draft extortion emails, according to a new report from the company. 

It Anthropic disclosed that the attacker leveraged Claude Code — a version of its AI model designed for generating computer code — to assist in nearly every stage of the operation. The campaign targeted at least 17 organizations across industries including defense, finance, and healthcare, making it the most comprehensive example yet of artificial intelligence being exploited for cyber extortion. 

Cyber extortion typically involves hackers stealing confidential data and demanding payment to prevent its release. AI has already played a role in such crimes, with chatbots being used to write phishing emails. However, Anthropic’s findings mark the first publicly confirmed case in which a mainstream AI model automated nearly the entire lifecycle of a cyberattack. 

The hacker reportedly prompted Claude to scan for vulnerable companies, generate malicious code to infiltrate systems, and extract confidential files. The AI system then organized the stolen data, analyzed which documents carried the highest value, and suggested ransom amounts based on victims’ financial information. It also drafted extortion notes demanding bitcoin payments, which ranged from $75,000 to more than $500,000. 

Jacob Klein, Anthropic’s head of threat intelligence, said the operation was likely conducted by a single actor outside the United States and unfolded over three months. “We have robust safeguards and multiple layers of defense for detecting this kind of misuse, but determined actors sometimes attempt to evade our systems through sophisticated techniques,” Klein explained. 

The report revealed that stolen material included Social Security numbers, bank records, medical data, and files tied to sensitive defense projects regulated by the U.S. State Department. Anthropic did not disclose which companies were affected, nor did it confirm whether any ransom payments were made. 

While the company declined to detail exactly how the hacker bypassed safeguards, it emphasized that additional protections have since been introduced. “We expect this model of cybercrime to become more common as AI lowers the barrier to entry for sophisticated operations,” Anthropic warned. 

The case underscores growing concerns about the intersection of AI and cybersecurity. With the AI sector largely self-regulated in the U.S., experts fear similar incidents could accelerate unless stronger oversight and security standards are enforced.

Hackers Use DNS Records to Hide Malware and AI Prompt Injections

 

Cybercriminals are increasingly leveraging an unexpected and largely unmonitored part of the internet’s infrastructure—the Domain Name System (DNS)—to hide malicious code and exploit security weaknesses. Security researchers at DomainTools have uncovered a campaign in which attackers embedded malware directly into DNS records, a method that helps them avoid traditional detection systems. 

DNS records are typically used to translate website names into IP addresses, allowing users to access websites without memorizing numerical codes. However, they can also include TXT records, which are designed to hold arbitrary text. These records are often used for legitimate purposes, such as domain verification for services like Google Workspace. Unfortunately, they can also be misused to store and distribute malicious scripts. 

In a recent case, attackers converted a binary file of the Joke Screenmate malware into hexadecimal code and split it into hundreds of fragments. These fragments were stored across multiple subdomains of a single domain, with each piece placed inside a TXT record. Once an attacker gains access to a system, they can quietly retrieve these fragments through DNS queries, reconstruct the binary code, and deploy the malware. Since DNS traffic often escapes close scrutiny—especially when encrypted via DNS over HTTPS (DOH) or DNS over TLS (DOT)—this method is particularly stealthy. 

Ian Campbell, a senior security engineer at DomainTools, noted that even companies with their own internal DNS resolvers often struggle to distinguish between normal and suspicious DNS requests. The rise of encrypted DNS traffic only makes it harder to detect such activity, as the actual content of DNS queries remains hidden from most monitoring tools. This isn’t a new tactic. Security researchers have observed similar methods in the past, including the use of DNS records to host PowerShell scripts. 

However, the specific use of hexadecimal-encoded binaries in TXT records, as described in DomainTools’ latest findings, adds a new layer of sophistication. Beyond malware, the research also revealed that TXT records are being used to launch prompt injection attacks against AI chatbots. These injections involve embedding deceptive or malicious prompts into files or documents processed by AI models. 

In one instance, TXT records were found to contain commands instructing a chatbot to delete its training data, return nonsensical information, or ignore future instructions entirely. This discovery highlights how the DNS system—an essential but often overlooked component of the internet—can be weaponized in creative and potentially damaging ways. 

As encryption becomes more widespread, organizations need to enhance their DNS monitoring capabilities and adopt more robust defensive strategies to close this blind spot before it’s further exploited.

Why Running AI Locally with an NPU Offers Better Privacy, Speed, and Reliability

 

Running AI applications locally offers a compelling alternative to relying on cloud-based chatbots like ChatGPT, Gemini, or Deepseek, especially for those concerned about data privacy, internet dependency, and speed. Though cloud services promise protections through subscription terms, the reality remains uncertain. In contrast, using AI locally means your data never leaves your device, which is particularly advantageous for professionals handling sensitive customer information or individuals wary of sharing personal data with third parties. 

Local AI eliminates the need for a constant, high-speed internet connection. This reliable offline capability means that even in areas with spotty coverage or during network outages, tools for voice control, image recognition, and text generation remain functional. Lower latency also translates to near-instantaneous responses, unlike cloud AI that may lag due to network round-trip times. 

A powerful hardware component is essential here: the Neural Processing Unit (NPU). Typical CPUs and GPUs can struggle with AI workloads like large language models and image processing, leading to slowdowns, heat, noise, and shortened battery life. NPUs are specifically designed for handling matrix-heavy computations—vital for AI—and they allow these models to run efficiently right on your laptop, without burdening the main processor. 

Currently, consumer devices such as Intel Core Ultra, Qualcomm Snapdragon X Elite, and Apple’s M-series chips (M1–M4) come equipped with NPUs built for this purpose. With one of these devices, you can run open-source AI models like DeepSeek‑R1, Qwen 3, or LLaMA 3.3 using tools such as Ollama, which supports Windows, macOS, and Linux. By pairing Ollama with a user-friendly interface like OpenWeb UI, you can replicate the experience of cloud chatbots entirely offline.  

Other local tools like GPT4All and Jan.ai also provide convenient interfaces for running AI models locally. However, be aware that model files can be quite large (often 20 GB or more), and without NPU support, performance may be sluggish and battery life will suffer.  

Using AI locally comes with several key advantages. You gain full control over your data, knowing it’s never sent to external servers. Offline compatibility ensures uninterrupted use, even in remote or unstable network environments. In terms of responsiveness, local AI often outperforms cloud models due to the absence of network latency. Many tools are open source, making experimentation and customization financially accessible. Lastly, NPUs offer energy-efficient performance, enabling richer AI experiences on everyday devices. 

In summary, if you’re looking for a faster, more private, and reliable AI workflow that doesn’t depend on the internet, equipping your laptop with an NPU and installing tools like Ollama, OpenWeb UI, GPT4All, or Jan.ai is a smart move. Not only will your interactions be quick and seamless, but they’ll also remain securely under your control.

AI and the Rise of Service-as-a-Service: Why Products Are Becoming Invisible

 

The software world is undergoing a fundamental shift. Thanks to AI, product development has become faster, easier, and more scalable than ever before. Tools like Cursor and Lovable—along with countless “co-pilot” clones—have turned coding into prompt engineering, dramatically reducing development time and enhancing productivity. 

This boom has naturally caught the attention of venture capitalists. Funding for software companies hit $80 billion in Q1 2025, with investors eager to back niche SaaS solutions that follow the familiar playbook: identify a pain point, build a narrow tool, and scale aggressively. Y Combinator’s recent cohort was full of “Cursor for X” startups, reflecting the prevailing appetite for micro-products. 

But beneath this surge of point solutions lies a deeper transformation: the shift from product-led growth to outcome-driven service delivery. This evolution isn’t just about branding—it’s a structural redefinition of how software creates and delivers value. Historically, the SaaS revolution gave rise to subscription-based models, but the tools themselves remained hands-on. For example, when Adobe moved Creative Suite to the cloud, the billing changed—not the user experience. Users still needed to operate the software. SaaS, in that sense, was product-heavy and service-light. 

Now, AI is dissolving the product layer itself. The software is still there, but it’s receding into the background. The real value lies in what it does, not how it’s used. Glide co-founder Gautam Ajjarapu captures this perfectly: “The product gets us in the door, but what keeps us there is delivering results.” Take Glide’s AI for banks. It began as a tool to streamline onboarding but quickly evolved into something more transformative. Banks now rely on Glide to improve retention, automate workflows, and enhance customer outcomes. 

The interface is still a product, but the substance is service. The same trend is visible across leading AI startups. Zendesk markets “automated customer service,” where AI handles tickets end-to-end. Amplitude’s AI agents now generate product insights and implement changes. These offerings blur the line between tool and outcome—more service than software. This shift is grounded in economic logic. Services account for over 70% of U.S. GDP, and Nobel laureate Bengt Holmström’s contract theory helps explain why: businesses ultimately want results, not just tools. 

They don’t want a CRM—they want more sales. They don’t want analytics—they want better decisions. With agentic AI, it’s now possible to deliver on that promise. Instead of selling a dashboard, companies can sell growth. Instead of building an LMS, they offer complete onboarding services powered by AI agents. This evolution is especially relevant in sectors like healthcare. Corti’s CEO Andreas Cleve emphasizes that doctors don’t want more interfaces—they want more time. AI that saves time becomes invisible, and its value lies in what it enables, not how it looks. 

The implication is clear: software is becoming outcome-first. Users care less about tools and more about what those tools accomplish. Many companies—Glean, ElevenLabs, Corpora—are already moving toward this model, delivering answers, brand voices, or research synthesis rather than just access. This isn’t the death of the product—it’s its natural evolution. The best AI companies are becoming “services in a product wrapper,” where software is the delivery mechanism, but the value lies in what gets done. 

For builders, the question is no longer how to scale a product. It’s how to scale outcomes. The companies that succeed in this new era will be those that understand: users don’t want features—they want results. Call it what you want—AI-as-a-service, agentic delivery, or outcome-led software. But the trend is unmistakable. Service-as-a-Service isn’t just the next step for SaaS. It may be the future of software itself.

Security Teams Struggle to Keep Up With Generative AI Threats, Cobalt Warns

 

A growing number of cybersecurity professionals are expressing concern that generative AI is evolving too rapidly for their teams to manage. 

According to new research by penetration testing company Cobalt, over one-third of security leaders and practitioners admit that the pace of genAI development has outstripped their ability to respond. Nearly half of those surveyed (48%) said they wish they could pause and reassess their defense strategies in light of these emerging threats—though they acknowledge that such a break isn’t realistic. 

In fact, 72% of respondents listed generative AI-related attacks as their top IT security risk. Despite this, one in three organizations still isn’t conducting regular security evaluations of their large language model (LLM) deployments, including basic penetration testing. 

Cobalt CTO Gunter Ollmann warned that the security landscape is shifting, and the foundational controls many organizations rely on are quickly becoming outdated. “Our research shows that while generative AI is transforming how businesses operate, it’s also exposing them to risks they’re not prepared for,” said Ollmann. 
“Security frameworks must evolve or risk falling behind.” The study revealed a divide between leadership and practitioners. Executives such as CISOs and VPs are more concerned about long-term threats like adversarial AI attacks, with 76% listing them as a top issue. Meanwhile, 45% of practitioners are more focused on immediate operational challenges such as model inaccuracies, compared to 36% of executives. 

A majority of leaders—52%—are open to rethinking their cybersecurity strategies to address genAI threats. Among practitioners, only 43% shared this view. The top genAI-related concerns identified by the survey included the risk of sensitive information disclosure (46%), model poisoning or theft (42%), data inaccuracies (40%), and leakage of training data (37%). Around half of respondents also expressed a desire for more transparency from software vendors about how vulnerabilities are identified and patched, highlighting a widening trust gap in the AI supply chain. 

Cobalt’s internal pentest data shows a worrying trend: while 69% of high-risk vulnerabilities are typically fixed across all test types, only 21% of critical flaws found in LLM tests are resolved. This is especially alarming considering that nearly one-third of LLM vulnerabilities are classified as serious. Interestingly, the average time to resolve these LLM-specific vulnerabilities is just 19 days—the fastest across all categories. 

However, researchers noted this may be because organizations prioritize easier, low-effort fixes rather than tackling more complex threats embedded in foundational AI models. Ollmann compared the current scenario to the early days of cloud adoption, where innovation outpaced security readiness. He emphasized that traditional controls aren’t enough in the age of LLMs. “Security teams can’t afford to be reactive anymore,” he concluded. “They must move toward continuous, programmatic AI testing if they want to keep up.”

Hackers Can Spy on Screens Using HDMI Radiation and AI Models

 

You may feel safe behind your screen, but it turns out that privacy might be more of an illusion than a fact. New research reveals that hackers have found an alarming way to peek at what’s happening on your display—without ever touching your computer. By tapping into the faint electromagnetic radiation that HDMI cables emit, they can now “listen in” on your screen and reconstruct what’s being shown with startling accuracy. 

Here’s how it works: when digital signals travel through HDMI cables from your computer to a monitor, they unintentionally give off tiny bursts of radiation. These signals, invisible to the naked eye, can be picked up using radio antennas or small, discreet devices planted nearby. Once captured, advanced AI tools get to work, decoding the radiation into readable screen content. 

The results? Up to 70% accuracy in reconstructing text—meaning everything from passwords and emails to private messages could be exposed. This new technique represents a serious leap in digital espionage. It doesn’t rely on malware or breaking into a network. Instead, it simply listens to the electronic “whispers” your hardware makes. It’s silent, stealthy, and completely undetectable to the average user. 

Worryingly, this method is already reportedly in use against high-profile targets like government agencies and critical infrastructure sites. These organizations often store and manage sensitive data that, if leaked, could cause major damage. While some have implemented shielding to block these emissions, not all are fully protected. And because this form of surveillance leaves virtually no trace, many attacks could be flying under the radar entirely. 

Hackers can go about this in two main ways: one, by sneaking a signal-collecting device into a location; or two, by using specialized antennas from nearby—like the building next door. Either way, they can eavesdrop on what’s displayed without ever getting physically close to the device. This new threat underscores the need for stronger physical and digital protections. 

As cyberattacks become more innovative, simply securing your data with passwords and firewalls isn’t enough. Shielding cables and securing workspaces might soon be as important as having good antivirus software. The digital age has brought us many conveniences—but with it comes a new breed of invisible spies.

DeepSeek AI Raises Data Security Concerns Amid Ties to China

 

The launch of DeepSeek AI has created waves in the tech world, offering powerful artificial intelligence models at a fraction of the cost compared to established players like OpenAI and Google. 

However, its rapid rise in popularity has also sparked serious concerns about data security, with critics drawing comparisons to TikTok and its ties to China. Government officials and cybersecurity experts warn that the open-source AI assistant could pose a significant risk to American users. 

On Thursday, two U.S. lawmakers announced plans to introduce legislation banning DeepSeek from all government devices, citing fears that the Chinese Communist Party (CCP) could access sensitive data collected by the app. This move follows similar actions in Australia and several U.S. states, with New York recently enacting a statewide ban on government systems. 

The growing concern stems from China’s data laws, which require companies to share user information with the government upon request. Like TikTok, DeepSeek’s data could be mined for intelligence purposes or even used to push disinformation campaigns. Although the AI app is the current focus of security conversations, experts say that the risks extend beyond any single model, and users should exercise caution with all AI systems. 

Unlike social media platforms that users can consciously avoid, AI models like DeepSeek are more difficult to track. Dimitri Sirota, CEO of BigID, a cybersecurity company specializing in AI security compliance, points out that many companies already use multiple AI models, often switching between them without users’ knowledge. This fluidity makes it challenging to control where sensitive data might end up. 

Kelcey Morgan, senior manager of product management at Rapid7, emphasizes that businesses and individuals should take a broad approach to AI security. Instead of focusing solely on DeepSeek, companies should develop comprehensive practices to protect their data, regardless of the latest AI trend. The potential for China to use DeepSeek’s data for intelligence is not far-fetched, according to cybersecurity experts. 

With significant computing power and data processing capabilities, the CCP could combine information from multiple sources to create detailed profiles of American users. Though this might not seem urgent now, experts warn that today’s young, casual users could grow into influential figures worth targeting in the future. 

To stay safe, experts advise treating AI interactions with the same caution as any online activity. Users should avoid sharing sensitive information, be skeptical of unusual questions, and thoroughly review an app’s terms and conditions. Ultimately, staying informed and vigilant about where and how data is shared will be critical as AI technologies continue to evolve and become more integrated into everyday life.

AI Models at Risk from TPUXtract Exploit

 


A team of researchers has demonstrated that it is possible to steal an artificial intelligence (AI) model without actually gaining access to the device that is running the model. The uniqueness of the technique lies in the fact that it works efficiently even if the thief may not have any prior knowledge as to how the AI works in the first place, or how the computer is structured. 

According to North Carolina State University's Department of Electrical and Computer Engineering, the method is known as TPUXtract, and it is a product of their department. With the help of a team of four scientists, who used high-end equipment and a technique known as "online template-building", they were able to deduce the hyperparameters of a convolutional neural network (CNN) running on Google Edge Tensor Processing Unit (TPU), which is the settings that define its structure and behaviour, with a 99.91% accuracy rate. 

The TPUXtract is an advanced side-channel attack technique devised by researchers at the North Carolina State University, designed to protect servers from attacks. A convolutional neural network (CNN) running on a Google Edge Tensor Processing Unit (TPU) is targeted in the attack, and electromagnetic signals are exploited to extract hyperparameters and configurations of the model without the need for previous knowledge of its architecture and software. 

A significant risk to the security of AI models and the integrity of intellectual property is posed by these types of attacks, which manifest themselves across three distinct phases, each of which is based on advanced methods to compromise the AI models' integrity. Attackers in the Profiling Phase observe and capture side-channel emissions produced by the target TPU as it processes known input data as part of the Profiling Phase. As a result, they have been able to decode unique patterns which correspond to specific operations such as convolutional layers and activation functions by using advanced methods like Differential Power Analysis (DPA) and Cache Timing Analysis. 

The Reconstruction Phase begins with the extraction and analysis of these patterns, and they are meticulously matched to known processing behaviours This enables adversaries to make an inference about the architecture of the AI model, including the layers that have been configured, the connections made, and the parameters that are relevant such as weight and bias. Through a series of repeated simulations and output comparisons, they can refine their understanding of the model in a way that enables precise reconstruction of the original model. 

Finally, the Validation Phase ensures that the replicated model is accurate. During the testing process, it is subject to rigorous testing with fresh inputs to ensure that it performs similarly to that of the original, thus providing reliable proof of its success. The threat that TPUXtract poses to intellectual property (IP) is composed of the fact that it enables attackers to steal and duplicate artificial intelligence models, bypassing the significant resources that are needed to develop them.

The competition could recreate and mimic models such as ChatGPT without having to invest in costly infrastructure or train their employees. In addition to IP theft, TPUXtract exposed cybersecurity risks by revealing an AI model's structure that provided visibility into its development and capabilities. This information could be used to identify vulnerabilities and enable cyberattacks, as well as expose sensitive data from a variety of industries, including healthcare and automotive.

Further, the attack requires specific equipment, such as a Riscure Electromagnetic Probe Station, high-sensitivity probes, and Picoscope oscilloscope, so only well-funded groups, for example, corporate competitors or state-sponsored actors, can execute it. As a result of the technical and financial requirements for the attack, it can only be executed by well-funded groups. With the understanding that any electronic device will emit electromagnetic radiation as a byproduct of its operations, the nature and composition of that radiation will be affected by what the device does. 

To conduct their experiments, the researchers placed an EM probe on top of the TPU after removing any obstructions such as cooling fans and centring it over the part of the chip emitting the strongest electromagnetic signals. The machine then emitted signals as a result of input data, and the signals were recorded. The researchers used the Google Edge TPU for this demonstration because it is a commercially available chip that is widely used to run AI models on edge devices meaning devices utilized by end users in the field, as opposed to AI systems that are used for database applications. During the demonstration, electromagnetic signals were monitored as a part of the technique used to conduct the demonstration.

A TPU chip was placed on top of a probe that was used by researchers to determine the structure and layer details of an AI model by recording changes in the electromagnetic field of the TPU during AI processing. The probe provided real-time data about changes in the electromagnetic field of the TPU during AI processing. To verify the model's electromagnetic signature, the researchers compared it to other signatures made by AI models made on a similar device - in this case, another Google Edge TPU. Using this technique, Kurian says, AI models can be stolen from a variety of different devices, including smartphones, tablets and computers. 

The attacker should be able to use this technique as long as they know the device from which they want to steal, have access to it while it is running an AI model, and have access to another device with similar specifications According to Kurian, the electromagnetic data from the sensor is essentially a ‘signature’ of the way AI processes information. There is a lot of work that goes into pulling off TPUXtract. The process not only requires a great deal of technical expertise, but it also requires a great deal of expensive and niche equipment as well. To scan the chip's surface, NCSU researchers used a Riscure EM probe station equipped with a motorized XYZ table, and a high-sensitivity electromagnetic probe to capture the weak signals emanating from it. 

It is said that the traces were recorded using a Picoscope 6000E oscilloscope, and Riscure's icWaves FPGA device aligned them in real-time, and the icWaves transceiver translated and filtered out the irrelevant signals using bandpass filters and AM/FM demodulation, respectively. While this may seem difficult and costly for a hacker to do on their own, Kurian explains, "It is possible for a rival company to do this within a couple of days, regardless of how difficult and expensive it will be. 

Taking the threat of TPUXtract into account, this model poses a formidable challenge to AI model security, highlighting the importance of proactive measures. As an organization, it is crucial to understand how such attacks work, implement robust defences, and ensure that they can safeguard their intellectual property while maintaining trust in their artificial intelligence systems. The AI and cybersecurity communities must learn continuously and collaborate to stay ahead of the changing threats as they arise.

The Role of Confidential Computing in AI and Web3

 

 
The rise of artificial intelligence (AI) has amplified the demand for privacy-focused computing technologies, ushering in a transformative era for confidential computing. At the forefront of this movement is the integration of these technologies within the AI and Web3 ecosystems, where maintaining privacy while enabling innovation has become a pressing challenge. A major event in this sphere, the DeCC x Shielding Summit in Bangkok, brought together more than 60 experts to discuss the future of confidential computing.

Pioneering Confidential Computing in Web3

Lisa Loud, Executive Director of the Secret Network Foundation, emphasized in her keynote that Secret Network has been pioneering confidential computing in Web3 since its launch in 2020. According to Loud, the focus now is to mainstream this technology alongside blockchain and decentralized AI, addressing concerns with centralized AI systems and ensuring data privacy.

Yannik Schrade, CEO of Arcium, highlighted the growing necessity for decentralized confidential computing, calling it the “missing link” for distributed systems. He stressed that as AI models play an increasingly central role in decision-making, conducting computations in encrypted environments is no longer optional but essential.

Schrade also noted the potential of confidential computing in improving applications like decentralized finance (DeFi) by integrating robust privacy measures while maintaining accessibility for end users. However, achieving a balance between privacy and scalability remains a significant hurdle. Schrade pointed out that privacy safeguards often compromise user experience, which can hinder broader adoption. He emphasized that for confidential computing to succeed, it must be seamlessly integrated so users remain unaware they are engaging with such technologies.

Shahaf Bar-Geffen, CEO of COTI, underscored the role of federated learning in training AI models on decentralized datasets without exposing raw data. This approach is particularly valuable in sensitive sectors like healthcare and finance, where confidentiality and compliance are critical.

Innovations in Privacy and Scalability

Henry de Valence, founder of Penumbra Labs, discussed the importance of aligning cryptographic systems with user expectations. Drawing parallels with secure messaging apps like Signal, he emphasized that cryptography should function invisibly, enabling users to interact with systems without technical expertise. De Valence stressed that privacy-first infrastructure is vital as AI’s capabilities to analyze and exploit data grow more advanced.

Other leaders in the field, such as Martin Leclerc of iEXEC, highlighted the complexity of achieving privacy, usability, and regulatory compliance. Innovative approaches like zero-knowledge proof technology, as demonstrated by Lasha Antadze of Rarimo, offer promising solutions. Antadze explained how this technology enables users to prove eligibility for actions like voting or purchasing age-restricted goods without exposing personal data, making blockchain interactions more accessible.

Dominik Schmidt, co-founder of Polygon Miden, reflected on lessons from legacy systems like Ethereum to address challenges in privacy and scalability. By leveraging zero-knowledge proofs and collaborating with decentralized storage providers, his team aims to enhance both developer and user experiences.

As confidential computing evolves, it is clear that privacy and usability must go hand in hand to address the needs of an increasingly data-driven world. Through innovation and collaboration, these technologies are set to redefine how privacy is maintained in AI and Web3 applications.