AI, Security, and Adversarial Use

Powerful technology can be dual-use- protect or harm; build or destroy; inform or deceive.

AI is unprecedented in speed, scale and opacity.

1. The Anthropic AI-Enabled Cyber Threats Case

  • Anthropic analyzed 832 accounts banned for malicious cyber activity and mapped those activities onto the MITRE ATT&CK framework. 67% used AI.
  • AI can act as a capability multiplier.

    • What AI Adds to Cyberattacks

  • Malware preparation
  • Attack planning
  • Post-compromise activity
  • Lowering the skill barrier

    • The Trust Problem

    A malicious message can be polished, context-aware, and personalized. A fake technical explanation can sound authoritative. A malicious script can be wrapped in plausible documentation. A fraudulent request can imitate the tone, writing style, and vocabulary of a real colleague.

    The Attribution Problem

    AI makes it more difficult to figure out who is behind an attack.

    The Responsibility Gap Revisited

    Responsibility is distributed across many actors:
  • the malicious user
  • the AI company that provides the model
  • the platform that detects and bans misuse
  • the organization that fails to secure its systems
  • the government that sets or fails to set rules
  • the broader market that rewards speed and capability before safety

    • The Philosophical Lesson

    What happens when AI makes more people capable of carrying out attacks that previously required advanced expertise?

    2. Offensive AI: What Becomes Possible: Social Engineering at Scale

    AI dramatically enhances social engineering:
  • Spear phishing at scale.
  • Voice deepfakes
  • Video deepfakes

    • 3. Governance and Regulation

    Existing Frameworks and Their Limitations

  • Export controls
  • Responsible disclosure
  • Terms of service restrictions
  • International agreements

    • The Philosophical Question: Arms Races and the Prisoner's Dilemma

  • Each state (and non-state actor) faces a choice between restraint (not developing
  • offensive AI capabilities) and competition (developing them).
  • The dominant strategy is to compete.
  • Nuclear weapons regulation may be offer a solution.
  • Are we obligated to attempt international AI security governance if current prospects are poor?
  • What is the moral calculus of restraint in a context where restraint may leave you vulnerable while your adversaries continue developing offensive capabilities?