Search This Blog
Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label GenAI. Show all posts
Technology
Agentforce AI Cloud GenAI LLMs Salesforce Technology

Salesforce Pulls Back from AI LLMs Citing Reliability Issues


Salesforce, a famous enterprise software company, is withdrawing from its heavy dependence on large language models (LLMs) after facing reliability issues that the executive didn't like. The company believes that trust in AI LLMs has declined in the past year, according to The Information. 

Parulekar, senior VP of product marketing said, “All of us were more confident about large language models a year ago.” This means the company has shifted away from GenAI towards more “deterministic” automation in its flagship product Agentforce.

In its official statement, the company said, “While LLMs are amazing, they can’t run your business by themselves. Companies need to connect AI to accurate data, business logic, and governance to turn the raw intelligence that LLMs provide into trusted, predictable outcomes.”

Salesforce cut down its staff from 9,000 to 5,000 employees due to AI agent deployment. The company emphasizes that Agentforce can help "eliminate the inherent randomness of large models.” 

Failing models, missing surveys

Salesforce experienced various technical issues with LLMs during real-world applications. According to CTO Muralidhar Krishnaprasad, when given more than eight prompts, the LLMs started missing commands. This was a serious flaw for precision-dependent tasks. 

Home security company Vivint used Agentforce for handling its customer support for 2.5 million customers and faced reliability issues. Even after giving clear instructions to send satisfaction surveys after each customer conversation, Agentforce sometimes failed to send surveys for unknown reasons. 

Another challenge was the AI drift, according to executive Phil Mui. This happens when users ask irrelevant questions causing AI agents to lose focus on their main goals. 

AI expectations vs reality hit Salesforce 

The withdrawal from LLMs shows an ironic twist for CEO Marc Benioff, who often advocates for AI transformation. In his conversation with Business Insider, Benioff talked about drafting the company's annually strategic document, prioritizing data foundations, not AI models due to “hallucinations” issues. He also suggests rebranding the company as Agentforce. 

Although Agentforce is expected to earn over $500 million in sales annually, the company's stock has dropped about 34% from its peak in December 2024. Thousands of businesses that presently rely on this technology may be impacted by Salesforce's partial pullback from large models as the company attempts to bridge the gap between AI innovation and useful business application.

Read more »
Technology
ChatGPT Cloud Content GenAI Internet Misinformation Technology

The New Content Provenance Report Will Address GenAI Misinformation


The GenAI problem 

Today's information environment includes a wide range of communication. Social media platforms have enabled reposting, and comments. The platform is useful for both content consumers and creators, but it has its own challenges.

The rapid adoption of Generative AI has led to a significant increase in misleading content online. These chatbots have a tendency of generating false information which has no factual backing. 

What is AI slop?

The internet is filled with AI slop- content that is made with minimal human input and is like junk. There is currently no mechanism to limit such massive production of harmful or misleading content that can impact human cognition and critical thinking. This calls for a robust mechanism that can address the new challenges that the current system is failing to tackle. 

The content provenance report 

For restoring the integrity of digital information, Canada's Centre for Cyber Security (CCCS) and the UK's National Cyber Security Centre (NCSC) have launched a new report on public content provenance. Provenance means "place of origin." For building stronger trust with external audiences, businesses and organisations must improve the way they manage the source of their information.

NSSC chief technology officer said that the "new publication examines the emerging field of content provenance technologies and offers clear insights using a range of cyber security perspectives on how these risks may be managed.” 

What is next for Content Integrity?

The industry is implementing few measures to address content provenance challenges like Coalition for Content Provenance and Authenticity (C2PA). It will benefit from the help of Generative AI and tech giants like Meta, Google, OpenAI, and Microsoft. 

Currently, there is a pressing need for interoperable standards across various media types such as image, video, and text documents. Although there are content provenance technologies, this area is still in nascent stage. 

What is needed?

The main tech includes genuine timestamps and cryptographically-proof meta to prove that the content isn't tampered. But there are still obstacles in development of these secure technologies, like how and when they are executed.

The present technology places the pressure on the end user to understand the provenance data. 

A provenance system must allow a user to see who or what made the content, the time and the edits/changes that were made. Threat actors have started using GenAI media to make scams believable, it has become difficult to differentiate between what is fake and real. Which is why a mechanism that can track the origin and edit history of digital media is needed. The NCSC and CCCS report will help others to navigate this gray area with more clarity.


Read more »
Phishing Campaigns
AI Cloud Cyber Security GenAI Internet Microsoft Phishing Campaigns

Microsoft Stops Phishing Scam Which Used Gen-AI Codes to Fool Victims


AI: Boon or Curse?

AI code is in use across sectors for variety of tasks, particularly cybersecurity, and both threat actors and security teams have turned to LLMs for supporting their work. 

Security experts use AI to track and address to threats at scale as hackers are experimenting with AI to make phishing traps, create obfuscated codes, and make spoofed malicious payloads. 

Microsoft Threat Intelligence recently found and stopped a phishing campaign that allegedly used AI-generated code to cover payload within an SVG file. 

About the campaign 

The campaign used a small business email account to send self addressed mails with actual victims coveted in BCC fields, and the attachment looked like a PDF but consisted SVG script content. 

The SVG file consisted hidden elements that made it look like an original business dashboard, while a secretly embedded script changed business words into code that exposed a secret payload. Once opened, the file redirects users to a CAPTCHA gate, a standard social engineering tactical that leads to a scanned sign in page used to steal credentials. 

The hidden process combined business words and formulaic code patterns instead of cryptographic techniques. 

Security Copilot studied the file and listed markers in lines with LLM output. These things made the code look fancy on the surface, however, it made the experts think it was AI generated. 

Combating the threat

The experts used AI powered tools in Microsoft Defender for Office 375 to club together hints that were difficult for hackers to push under the rug. 

The AI tool flagged the rare self-addressed email trend , the unusual SVG file hidden as a PDF, the redirecting to a famous phishing site, the covert code within the file, and the detection tactics deployed on the phishing page. 

The incident was contained, and blocked without much effort, mainly targeting US based organizations, Microsoft, however, said that the attack show how threat actors are aggressively toying with AI to make believable tracks and sophisticated payloads.

Read more »
Technology
AI Business ChatGPT Cloud Data Data Leak GenAI Technology

CISOs fear material losses amid rising cyberattacks


Chief information security officers (CISOs) are worried about the dangers of a cyberattack, and there is an anxiety due to the material losses of data that organizations have suffered in the past year.

According to a report by Proofpoint, the majority of CISOs fear a material cyberattack in the next 12 months. These concerns highlight the increasing risks and cultural shifts among CISOs.

Changing roles of CISOs

“76% of CISOs anticipate a material cyberattack in the next year, with human risk and GenAI-driven data loss topping their concerns,” Proofpoint said. In this situation, corporate stakeholders are trying to get a better understanding of the risks involved when it comes to tech and whether they are safe or not. 

Experts believe that CISOs are being more open about these attacks, thanks to SEC disclosure rules, strict regulations, board expectations, and enquiries. The report surveyed 1,600 CISOs worldwide; all the organizations had more than 1000 employees. 

Doing business is a concern

The study highlights a rising concern about doing business amid incidents of cyberattacks. Although the majority of CISOs are confident about their cybersecurity culture, six out of 10 CISOs said their organizations are not prepared for a cyberattack. The majority of the CISOs were found in favour of paying ransoms to avoid the leak of sensitive data.

AI: Saviour or danger?

AI has risen both as a top concern as well as a top priority for CISOs. Two-thirds of CISOs believe that enabling GenAI tools is a top priority over the next two years, despite the ongoing risks. In the US, however, 80% CISOs worry about possible data breaches through GenAI platforms. 

With adoption rates rising, organizations have started to move from restriction to governance. “Most are responding with guardrails: 67% have implemented usage guidelines, and 68% are exploring AI-powered defenses, though enthusiasm has cooled from 87% last year. More than half (59%) restrict employee use of GenAI tools altogether,” Proofpoint said.

Read more »
LLM security
AI Models AI technology AI tools CISO Cyber Security Cybersecurity Researchers GenAI Generative AI generative AI tools LLM LLM Model LLM security

Security Teams Struggle to Keep Up With Generative AI Threats, Cobalt Warns

 

A growing number of cybersecurity professionals are expressing concern that generative AI is evolving too rapidly for their teams to manage. 

According to new research by penetration testing company Cobalt, over one-third of security leaders and practitioners admit that the pace of genAI development has outstripped their ability to respond. Nearly half of those surveyed (48%) said they wish they could pause and reassess their defense strategies in light of these emerging threats—though they acknowledge that such a break isn’t realistic. 

In fact, 72% of respondents listed generative AI-related attacks as their top IT security risk. Despite this, one in three organizations still isn’t conducting regular security evaluations of their large language model (LLM) deployments, including basic penetration testing. 

Cobalt CTO Gunter Ollmann warned that the security landscape is shifting, and the foundational controls many organizations rely on are quickly becoming outdated. “Our research shows that while generative AI is transforming how businesses operate, it’s also exposing them to risks they’re not prepared for,” said Ollmann. 
“Security frameworks must evolve or risk falling behind.” The study revealed a divide between leadership and practitioners. Executives such as CISOs and VPs are more concerned about long-term threats like adversarial AI attacks, with 76% listing them as a top issue. Meanwhile, 45% of practitioners are more focused on immediate operational challenges such as model inaccuracies, compared to 36% of executives. 

A majority of leaders—52%—are open to rethinking their cybersecurity strategies to address genAI threats. Among practitioners, only 43% shared this view. The top genAI-related concerns identified by the survey included the risk of sensitive information disclosure (46%), model poisoning or theft (42%), data inaccuracies (40%), and leakage of training data (37%). Around half of respondents also expressed a desire for more transparency from software vendors about how vulnerabilities are identified and patched, highlighting a widening trust gap in the AI supply chain. 

Cobalt’s internal pentest data shows a worrying trend: while 69% of high-risk vulnerabilities are typically fixed across all test types, only 21% of critical flaws found in LLM tests are resolved. This is especially alarming considering that nearly one-third of LLM vulnerabilities are classified as serious. Interestingly, the average time to resolve these LLM-specific vulnerabilities is just 19 days—the fastest across all categories. 

However, researchers noted this may be because organizations prioritize easier, low-effort fixes rather than tackling more complex threats embedded in foundational AI models. Ollmann compared the current scenario to the early days of cloud adoption, where innovation outpaced security readiness. He emphasized that traditional controls aren’t enough in the age of LLMs. “Security teams can’t afford to be reactive anymore,” he concluded. “They must move toward continuous, programmatic AI testing if they want to keep up.”
Read more »
Technology
AI ChatGPT CISA FBI GenAI NSA Technology

Governments Release New Regulatory AI Policy


Regulatory AI Policy 

The CISA, NSA, and FBI teamed with cybersecurity agencies from the UK, Australia, and New Zealand to make a best-practices policy for safe AI development. The principles laid down in this document offer a strong foundation for protecting AI data and securing the reliability and accuracy of AI-driven outcomes.

The advisory comes at a crucial point, as many businesses rush to integrate AI into their workplace, but this can be a risky situation also. Governments in the West have become cautious as they believe that China, Russia, and other actors will find means to abuse AI vulnerabilities in unexpected ways. 

Addressing New Risks 

The risks are increasing swiftly as critical infrastructure operators develop AI into operational tech that controls important parts of daily life, from scheduling meetings to paying bills to doing your taxes.

From foundational elements of AI to data consulting, the document outlines ways to protect your data at different stages of the AI life cycle such as planning, data collection, model development, installment and operations. 

It requests people to use digital signature that verify modifications, secure infrastructure that prevents suspicious access and ongoing risk assessments that can track emerging threats. 

Key Issues

The document addresses ways to prevent data quality issues, whether intentional or accidental, from compromising the reliability and safety of AI models. 

Cryptographic hashes make sure that taw data is not changed once it is incorporated into a model, according to the document, and frequent curation can cancel out problems with data sets available on the web. The document also advises the use of anomaly detection algorithms that can eliminate “malicious or suspicious data points before training."

The joint guidance also highlights issues such as incorrect information, duplicate records and “data drift”, statistics bias, a natural limitation in the characteristics of the input data.

Read more »
HR Department
Ai Data AI technology AI tools Cyber Security Data Services Data Technology GenAI HR Department

How GenAI Is Revolutionizing HR Analytics for CHROs and Business Leaders

 

Generative AI (GenAI) is redefining how HR leaders interact with data, removing the steep learning curve traditionally associated with people analytics tools. When faced with a spike in hourly employee turnover, Sameer Raut, Vice President of HRIS at Sunstate Equipment, didn’t need to build a custom report or consult data scientists. Instead, he typed a plain-language query into a GenAI-powered chatbot: 

“What are the top reasons for hourly employee terminations in the past 12 months?” Within seconds, he had his answer. This shift in how HR professionals access data marks a significant evolution in workforce analytics. Tools powered by large language models (LLMs) are now integrated into leading analytics platforms such as Visier, Microsoft Power BI, Tableau, Qlik, and Sisense. These platforms are leveraging GenAI to interpret natural language questions and deliver real-time, actionable insights without requiring technical expertise. 

One of the major advantages of GenAI is its ability to unify fragmented HR data sources. It streamlines data cleansing, ensures consistency, and improves the accuracy of workforce metrics like headcount growth, recruitment gaps, and attrition trends. As Raut notes, tools like Visier’s GenAI assistant “Vee” allow him to make quick decisions during meetings, helping HR become more responsive and strategic. This evolution is particularly valuable in a landscape where 39% of HR leaders cite limited analytics expertise as their biggest challenge, according to a 2023 Aptitude Research study. 

GenAI removes this barrier by enabling intuitive data exploration across familiar platforms like Slack and Microsoft Teams. Frontline managers who may never open a BI dashboard can now access performance metrics and workforce trends instantly. Experts believe this transformation is just beginning. While some analytics platforms are still improving their natural language processing capabilities, others are leading with more advanced and user-friendly GenAI chatbots. 

These tools can even create automated visualizations and summaries tailored to executive audiences, enabling CHROs to tell compelling data stories during high-level meetings. However, this transformation doesn’t come without risk. Data privacy remains a top concern, especially as GenAI tools engage with sensitive workforce data. HR leaders must ensure that platforms offer strict entitlement management and avoid training AI models on private customer data. Providers like Visier mitigate these risks by training their models solely on anonymized queries rather than real-world employee information. 

As GenAI continues to evolve, it’s clear that its role in HR will only expand. From democratizing access to HR data to enhancing real-time decision-making and storytelling, this technology is becoming indispensable for organizations looking to stay agile and informed.
Read more »
Technology
AI Alibaba China Cloud DeepSeek GenAI Technology

Alibaba Launches Latest Open-source AI Model from Qwen Series for ‘Cost-effective AI agents’

Alibaba Launches Lates Open-source AI Model from Qwen Series for ‘Cost-effective AI agents’

Last week, Alibaba Cloud launched its latest AI model in its “Qwen series,” as large language model (LLM) competition in China continues to intensify after the launch of famous “DeepSeek” AI.

The latest "Qwen2.5-Omni-7B" is a multimodal model- it can process inputs like audio/video, text, and images- while also creating real-time text and natural speech responses, Alibaba’s cloud website reports. It also said that the model can be used on edge devices such as smartphones, providing higher efficiency without giving up on performance. 

According to Alibaba, the “unique combination makes it the perfect foundation for developing agile, cost-effective AI agents that deliver tangible value, especially intelligent voice applications.” For instance, the AI can be used to assist visually impaired individuals to navigate their environment via real-time audio description. 

The latest model is open-sourced on forums GitHub and Hugging Face, after a rising trend in China post DeepSeek breakthrough R1 model open-source. Open-source means a software in which the source code is created freely on web for potential modification and redistribution. 

In recent years, Alibaba claims it has open-sourced more that 200 generative AI models. In the noise of China’s AI dominance intensified by DeepSeek due to its shoe string budget and capabilities, Alibaba and genAI competitors are also releasing new, cost-cutting models and services an exceptional case.

Last week, Chinese tech mammoth Baidu launched a new multimodal foundational model and its first reasoning-based model. Likewise, Alibaba introduced its updated Qwen 2.5 AI model in January and also launched a new variant of its AI assistant tool Quark this month. 

Alibaba has also made strong commitments to its AI plan, recently, it announced a plan to put $53 billion in its cloud computing and AI infrastructure over the next three years, even surpassing its spending in the space over the past decade. 

CNBC talked with Kai Wang, Asia Senior equity analyst at Morningstar, Mr Kai told CNBC that “large Chinese tech players such as Alibaba, which build data centers to meet the computing needs of AI in addition to building their own LLMs, are well positioned to benefit from China's post-DeepSeek AI boom.” According to CNBC, “Alibaba secured a major win for its AI business last month when it confirmed that the company was partnering with Apple to roll out AI integration for iPhones sold in China.”

Read more »
Subscribe to: Comments (Atom)