Artificial intelligence (AI) chatbots, such as OpenAI’s ChatGPT and Google’s Bard (now known as Gemini), have demonstrated their ability to pass a standard certified ethical hacking exam, according to a recent study. This research, co-authored by Prasad Calyam from the University of Missouri alongside collaborators from Amrita University in India, suggests that while these AI tools can understand and describe cybersecurity threats effectively, they should not be solely relied upon for comprehensive security. The study involved testing the AI systems with typical questions from a recognized certified ethical hacking test, which measures an individual’s knowledge of attack types, protection strategies, and responses to security incidents.
The examination results revealed distinct strengths in both AI models. Bard generally showed better accuracy in its responses, while ChatGPT was noted for its comprehensive, clear, and concise answers. During the tests, both AI tools explained complex cybersecurity scenarios, such as man-in-the-middle attacks, and recommended appropriate preventative measures. However, the researchers caution that the AIs also produced incorrect answers, underscoring the critical need for accuracy in cybersecurity, where errors can lead to severe consequences. This cautionary note is a reminder of the risks involved in relying solely on AI for comprehensive security.
The study also highlighted an interesting dynamic in AI responses: when prompted with questions like “are you sure?” the AI tools often revised their answers, sometimes correcting mistakes from their initial responses. Additionally, when tasked with providing advice on how to conduct a cyberattack, ChatGPT referenced ethical considerations, whereas Bard stated it was not programmed to assist with such queries. This indicates a level of built-in ethical programming and highlights the limitations in AI’s ability to navigate morally ambiguous requests.
Prasad Calyam, holding the Greg L. Gilliom Professor of Cyber Security title at the University of Missouri, emphasized that these AI models are still being prepared to replace human cybersecurity experts. Humans bring problem-solving skills essential for developing robust cyber defences that AI cannot match. However, Calyam acknowledged that AI could be a valuable resource for providing baseline information beneficial for individuals or small businesses needing quick assistance.
Calyam suggests that these AI tools can also be practical training tools for those involved with information technology or individuals eager to learn about cybersecurity basics. This educational potential, combined with the ongoing improvements in AI technology, offers a promising outlook for their future role in cybersecurity.
While the research demonstrates AI’s potential in ethical hacking, Calyam notes that much work remains to fully harness their capabilities. Ensuring AI’s reliability as ethical hackers could significantly enhance cybersecurity measures, contributing to safer digital environments. This ongoing development, with its promise to refine AI tools further, making them more adept at supporting cyber defence while continuing to evolve within ethical boundaries, offers an optimistic outlook for the future of AI in cybersecurity.
More information: Raghu Raman et al, ChatGPT or Bard: Who is a better Certified Ethical Hacker? Computers & Security. DOI: 10.1016/j.cose.2024.103804
Journal information: Computers & Security Provided by University of Missouri