Security experts have cautioned that a new AI bot called ChatGPT may be employed by cybercriminals to educate them on how to plan attacks and even develop ransomware. It was launched by the artificial intelligence r&d company OpenAI last month.
Computer security expert Brendan Dolan-Gavitt questioned if he could command an AI-powered chatbot to create malicious code when the ChatGPT application first allowed users to communicate. Then he gave the program a basic capture-the-flag mission to complete.
The code featured a buffer overflow vulnerability, which ChatGPT accurately identified and created a piece of code to capitalize it. The program would have addressed the issue flawlessly if not for a small error—the number of characters in the input.
The fact that ChatGPT failed Dolan Gavitt's task, which he would have given students at the start of a vulnerability analysis course, does not instill trust in massive language models' capacity to generate high-quality code. However, after identifying the mistake, Dolan-Gavitt asked the model to review the response, and this time, ChatGPT did it right.
Security researchers have used ChatGPT to rapidly complete a variety of offensive and defensive cybersecurity tasks, including creating refined or persuading phishing emails, creating workable Yara rules, identifying buffer overflows in code, creating evasion code that could be utilized by attackers to avoid threat detection, and even writing malware.
Dr. Suleyman Ozarslan, a security researcher and co-founder of Picus Security, claimed that he was able to program the program to carry out a variety of aggressive and defensive cybersecurity tasks, like the creation of a World Cup-themed email in perfect English and the generation of both evasion code that can get around detection rules as well as Sigma detection rules to find cybersecurity anomalies.
Reinforcement learning is the foundation of ChatGPT. As a result, it acquires new knowledge through interaction with people and user-generated prompts. Additionally, it implies that over time, the program might pick up on some of the tricks researchers have employed to get around its ethical checks, either through user input or improvements made by its administrators.
Multiple researchers have discovered a technique to get beyond ChatGPT's limitations, which stops it from doing things that are malicious, including providing instructions on how to make a bomb or writing malicious code. For the present term, ChatGPT's coding could be more optimal and demonstrates many of the drawbacks of relying solely on AI tools. However, as these models develop in complexity, they are likely to become more crucial in creating malicious software.
