In-house AI solutions are being manipulated by attackers

87% of security professionals report that their organisation has encountered an AI-driven cyberattack in the last year, according to a new study by SoSafe, Europe’s largest security awareness and human risk management solution.  

The finding comes from SoSafe’s 2025 Cybercrime Trends, a comprehensive survey of 500 global security professionals as well as 100 SoSafe customers across 10 countries. It examines social engineering tactics and the escalating risks facing organisations.

The report also highlights a growing global tension between the widespread adoption of AI and the inherent security risks that it can pose. Notably, 91% of all security experts anticipate a significant surge in AI-driven threats over the next three years. However, only 26% express high confidence in their ability to detect these attacks – showing how dangerously exposed organisations are today.

Andrew Rose, chief security officer, SoSafe, said: “AI is dramatically scaling the sophistication and personalisation of cyberattacks. While organisations seem to be aware of the threat, our data shows businesses are not confident in their ability to detect and react to these attacks”.

Rise of multichannel attacks

Advancements in AI are enabling multichannel cyberattacks, blending tactics across email, SMS, social media and collaboration platforms. 95% of cybersecurity professionals agree they’ve noticed an increase in this style of attack in the past 2 years. A clear example is the attack on WWP’s CEO, where the attackers combined WhatsApp to build trust, Microsoft Teams for further interaction, and an AI-generated deepfake voice call to extract sensitive information and money.

“Targeting victims across a combination of communications platforms allows them to mimic normal communication patterns, appearing more legitimate,” said Rose. “Simplistic email attacks are evolving into 3D phishing, seamlessly integrating voice, videos or text-based elements to create AI-powered, advanced scams.”

AI’s dual threat: attack vector and expanded attack surface

Beyond AI-based attacks, in-house adoption of AI is inadvertently expanding organisations’ attack surfaces, subjecting themselves to new innovative attacks such as data poisoning and AI hallucinations.

“Even the benevolent AI that organisations adopt for their own benefit can be abused by attackers to locate valuable information, key assets or bypass other controls. Many firms create AI chatbots to provide their staff with assistance, but few have thought through the scenario of their chatbot becoming an accomplice in an attack by aiding the attacker to collect sensitive data, identify key individuals and identify useful corporate insight.”

SoSafe’s survey found that 55% of businesses have not fully implemented controls to manage the risks associated with their in-house AI solutions.

“It is imperative that businesses couple their own AI adoption with a rigorous approach to security that protects against both technological and human vulnerabilities.”

Leading concerns for cybersecurity professionals

SoSafe’s report reveals a range of concerns among security professionals regarding AI-powered attacks.

Obfuscation techniques, such as AI-generated methods to mask the origins and intent of attacks, were cited as the top concern by over 51% of security leaders. Additionally, 45% reported that the creation of entirely new attack methods was their biggest worry, while two fifths (38%) cited the scale and speed of automated attacks.

AI: a balancing act

 “While AI undoubtedly presents new challenges, it also remains one of our greatest allies in protecting organisations against ever-evolving threats. However, AI-driven security is only as strong as the people who use it. Cybersecurity awareness is critical. Without informed employees who can recognise and respond to AI-driven threats, even the best technology falls short. By combining human expertise, security awareness and the careful application of AI, we can stay ahead of the curve and build stronger, more resilient organisations,” said Niklas Hellemann, chief executive officer of SoSafe.

Image: FlyD on Unsplash

Robert Welbourn
Robert Welbourn is an experienced financial writer. He has worked for a number of high street banks and trading platforms. He's also a published author and freelance writer and editor.