Security tools spent years learning to spot bad links and bad files. Attackers noticed. So they stopped sending bad files and started sending believable people.
AI-powered social engineering does not break your firewall. It borrows a voice, a face, or a writing style your staff already trust, then asks for something routine. The payment goes out. Nothing looks like an attack.
What Is AI-Powered Social Engineering?
Social engineering means manipulating a person into an action that helps the attacker. AI makes that manipulation cheap, fast, and personal.
It arrives in three forms:
- Text. Clean emails and chat messages written with real context about you.
- Voice. Cloned speech built from a short public clip.
- Video. Deepfaked faces on a live call.
The goal is old. A payment, a password, or access. Only the delivery changed.
How Does It Work?
The chain is simple and repeatable:
- Collect public data on the target from social profiles, press pages, and leaks.
- Build a persona of someone the target already trusts.
- Pick a channel that carries authority, such as email, a phone call, or video.
- Add urgency and secrecy so the target skips normal checks.
- Collect the payment, the credentials, or the access.
Step 2 used to take skill. Now it takes a sample and a few minutes.
Why Does It Matter?
Most security controls hunt for a malicious artifact. A bad attachment. A known domain. A strange login.
This attack has none of that. The email is clean. The transfer is a real transfer through a real banking portal, approved by a real employee. Your tooling sees normal business.
The volume is climbing too. Darktrace reported a 135% rise in novel social engineering attacks between January and February 2023. That rise tracked the spread of generative AI tools. It shows a link, not proof of cause, but the trend is clear.
Real-World Example: The Arup Deepfake Fraud
In January 2024, a staff member at British engineering firm Arup joined a video call about a private deal. The chief financial officer was on screen. So were other colleagues.
Every one of them was AI-generated.
The employee made a series of transfers to five Hong Kong bank accounts totalling roughly HK$200 million, about US$25 million. The fraud was only caught after a follow-up with head office. Arup confirmed in May 2024 that false voices and images were used.
Note what did not happen. No malware. No stolen password. No breached system. The attackers targeted a process, not a network.
Common AI Attack Techniques
- Business email compromise. Generated messages that match a real thread and writing style.
- Voice cloning. A short recording is enough to fake a familiar caller.
- Live deepfake calls. Fake participants in a meeting, as in the Arup case.
- Executive impersonation. Authority plus urgency, aimed at finance and payroll.
- Help desk fraud. Callers who pass identity checks using scraped personal data.
How to Prevent It
Verify people through a separate channel, and never inside the channel making the request.
Process controls:
- Call back on a number already in your directory. The FBI advises finding that number yourself, not using one supplied in the message.
- Require two approvers for payments above a set limit.
- Never approve funds on the strength of a single call or video, however convincing.
- Agree a spoken code word for high-value requests.
- Give staff written permission to pause a payment. Urgency and secrecy together are the warning sign.
Technical controls:
- Deploy phishing-resistant MFA so stolen credentials alone go nowhere.
- Harden help desk identity checks beyond data an attacker can scrape.
- Log and alert on new payee creation and bank detail changes.
Do not rely on spotting the fake. Detection is falling behind the fakes.
What Should Security Testing Cover?
Ask your testers to target the process, not just the perimeter:
- Simulated executive impersonation against finance and payroll
- Help desk social engineering, including password and MFA reset attempts
- Whether staff really call back when under pressure
- How much executive voice and video is public and easy to clone
- Whether staff who pause a suspicious request face any friction
Also Read: Red Teaming and Adversary Simulation
How OraSec Can Help
OraSec runs social engineering assessments that test people and process together. We map your public exposure, run realistic impersonation, and report where the checks broke down. You get the failing step, not a generic awareness score.
Conclusion
AI did not invent deception. It removed the cost, the accent, and the typos that used to give deception away.
Assume voice and video can be faked. Move trust out of the conversation and into a check the attacker cannot reach. That single change stops most of these attacks.
FAQs
Is a deepfake attack a hack? Not in the technical sense. Nothing is breached. The attacker exploits a process gap and human trust.
Can we detect deepfakes with software? They help, but they lag behind the fakes. Treat them as a signal, never as the check that releases money.
How much audio does voice cloning need? Modern tools work from short public clips. Assume any executive who speaks publicly can be cloned.
Does MFA stop this? Phishing-resistant MFA stops credential theft. It does not stop a staff member willingly approving a fraudulent transfer.
Who should be trained first? Finance, payroll, executive assistants, and the help desk. They hold approval power and face the most impersonation attempts.
