The UK's cybersecurity agency has issued a warning that artificial intelligence (AI) advancements may make it challenging to distinguish between genuine and fraudulent emails, particularly those prompting users to reset passwords. The National Cyber Security Centre (NCSC), affiliated with the GCHQ spy agency, highlighted the increasing sophistication of AI tools, such as generative AI, which can create convincing text, voice, and images based on simple prompts.

According to the NCSC's assessment of AI's impact on cyber threats, it anticipates a significant rise in cyber-attacks over the next two years. Generative AI, coupled with large language models like those powering chatbots, is expected to complicate the identification of various attack types, including phishing, spoofing, and social engineering.

The agency emphasized that by 2025, assessing the legitimacy of emails or password reset requests would become challenging for individuals, regardless of their cybersecurity expertise. Ransomware attacks, which have affected institutions like the British Library and Royal Mail, are also projected to increase. The NCSC pointed out that AI's sophistication lowers the entry barrier for amateur cybercriminals, enabling them to paralyze computer systems, extract sensitive data, and demand cryptocurrency ransoms.

Generative AI tools are already being used to create more convincing approaches to potential victims by crafting fake "lure documents" without typical errors associated with phishing attacks. While generative AI won't enhance ransomware code effectiveness, it will assist in identifying potential targets.

In 2022, the UK reported 706 ransomware incidents, compared to 694 in 2021, according to the Information Commissioner's Office. The NCSC warned that state actors likely possess enough malware to train AI models capable of creating new code that can evade security measures.

The report acknowledged AI's dual role, stating that it can also serve as a defensive tool by detecting attacks and designing more secure systems. In response to the rising threat of ransomware, the UK government introduced new guidelines, the "Cyber Governance Code of Practice," urging businesses to prioritize information security alongside financial and legal management.

Despite these measures, cybersecurity experts, including Ciaran Martin, the former head of the NCSC, have called for stronger actions. Martin emphasized the need for a fundamental shift in approaching ransomware threats, suggesting stronger rules on ransom payments and abandoning unrealistic notions of retaliatory measures.