Home / Technology / AI Cyber Risks: 10 Terms Explained
AI Cyber Risks: 10 Terms Explained
18 Nov
Summary
- AI acts as both a tool for attackers and a target for defense.
- Terms like prompt injection exploit how AI models process input.
- NIST published a voluntary AI Risk Management Framework in January 2023.
Artificial intelligence plays a dual role in cybersecurity, enabling sophisticated attacks while also offering defensive capabilities. Attackers leverage AI for crafting convincing phishing messages and deepfakes, while defenders use it to detect threats. This duality necessitates a shared vocabulary beyond security teams, impacting finance, management, and boards reviewing incidents.
Key terms are emerging to describe these risks, including agentic AI, prompt injection, and data poisoning, which relate to how AI models are built and trained. Older security concepts like breakout time and red teaming are being reapplied to AI contexts. The U.S. National Institute of Standards and Technology (NIST) released its voluntary AI Risk Management Framework 1.0 in January 2023, offering a structured approach to AI risk management.
Understanding specific threats like prompt injection, where malicious instructions are hidden within text fed to AI models, is vital. This can lead to models disregarding intended instructions and acting on attacker commands, either directly typed or indirectly embedded in content. Data poisoning corrupts the training data, causing models to behave incorrectly or develop hidden backdoors.
Other critical terms include deepfakes, AI-generated or altered media used for impersonation, and agentic AI, which refers to autonomous AI systems capable of planning and executing tasks with limited human oversight. Red teaming involves proactively attacking AI systems to discover vulnerabilities before malicious actors do, ensuring systems are robust against misuse and bias.