AI Security Roadmap

From AppSec to AI Security

A curated 8-week roadmap for software and security engineers pivoting into AI Security. Only the resources worth your time. No tutorials, no filler. Follow the weeks in order.

12 hours / week
8 weeks
Free
CC-BY 4.0

0 / 87

Who this is for

You have a background in software engineering or application security and you want to work on AI systems without becoming an ML researcher. You can read English technical content and you have ~12 hours per week.

How to use it

Follow the weeks in order. Each week contains reading, videos, docs or papers, one practical exercise, and one self-evaluated checkpoint. Mark items as you go — progress is stored in your browser only.

Some resources are paid books (marked $). If you cannot buy them, the paper and blog equivalents are enough to follow along.

Week 1

LLM Foundations I — Tokens, Prompts, Determinism

Build an accurate mental model of what an LLM is and how inputs shape outputs. Without this, everything downstream becomes cargo-culting.

Resources

Book Hands-On Large Language Models — Chapters 1-2 $ Alammar & Grootendorst (O'Reilly, 2024) The clearest chapter-1 framing in print. Skip appendices.
Video Intro to Large Language Models (1h) Andrej Karpathy Non-negotiable. Watch once at 1x, take notes.

Supplementary resources

Blog 2025: The year in LLMs Simon Willison Annual state-of-the-field summary. Use for landscape orientation, not for the mental model itself.
Video ¿Qué es un TRANSFORMER? La Red Neuronal que lo cambió TODO Dot CSV Spanish-language walkthrough of the 2017 paper. Pairs cleanly with Alammar Ch. 1.
Video RNN and attention — visual reinforcement YouTube Use as a refresher on RNNs and the attention intuition before moving on.
Blog Mecanismos de Atención Juan Sensio Spanish technical blog covering hard, soft, and self-attention.

Exercise

Install Ollama. Pull a small model (llama3.2:3b or similar). Send the same question 5 times at temperature 0 and 5 times at temperature 1. Save outputs.

Checkpoint

In 2 sentences: what is a token, and why does rewording a prompt change the output?

Master before next week

What a token is and how it differs from a character or a word
Why rewording the same prompt changes the output
What `temperature` controls and how determinism trades off against variety
The conceptual difference between an LLM, a search engine, and a rule-based system
When you would reach for an encoder-only, decoder-only, or encoder-decoder model

Week 2

LLM Foundations II — Attention, Context, Sampling

Understand the internals well enough to reason about attack surfaces. You do not need to implement a transformer — you need to know why injection works at the attention level.

Resources

Book Hands-On Large Language Models — Chapter 3 $ Alammar & Grootendorst (O'Reilly, 2024) Attention and transformer mechanics without the math overhead.
Video Attention in transformers, visually explained 3Blue1Brown Best visual intuition for attention. Pair with Karpathy from Week 1.
Blog The Illustrated Transformer Jay Alammar Canonical reference. Free version of Hands-On LLMs Ch. 3 intuitions.
Blog Sampling parameters — temperature, top_p, top_k Patrick von Platen, Hugging Face The conceptual explainer, not an API table — how each parameter reshapes the next-token distribution. Pairs with the Ollama exercise below.

Exercise

With Ollama, run the same prompt 10 times at temperature 0 and 10 times at temperature 1. Count how many outputs are byte-identical in each group. Explain the result.

Checkpoint

Explain attention to a backend engineer who has never seen it, in 3 sentences.

Master before next week

How self-attention lets every token see every other token in the input
Why pure attention is permutation-invariant and needs positional encoding
The roles of Q, K, V vectors at a high level
Why multi-head attention works better than single-head
The architectural reason prompt injection works: no separation between instruction and data tokens

Week 3

OWASP LLM Top 10 — The Attack Surface

Map the attack surface as industry has agreed on it. This is the taxonomy every other AI Security document in 2026 builds on.

Resources

Docs OWASP Top 10 for Large Language Model Applications 2025 OWASP GenAI Security Project Read end-to-end. Every LLM01-LLM10 page matters.
Book The Developer's Playbook for LLM Security — Ch. 1 "Chatbots Breaking Bad" + Ch. 2 "The OWASP Top 10 for LLM Applications" $ Steve Wilson (O'Reilly, 2024) Wilson co-leads OWASP LLM Top 10. Ch. 1 frames the problem via the Tay incident; Ch. 2 explains how the Top 10 list was assembled.
Video OWASP Top 10 for Large Language Models: Project Update Steve Wilson & Scott Clinton — OWASP GenAI Security Project (2025) Project co-leads walk through the 2025 list and explain the rationale behind each risk class.
Docs MITRE ATLAS — AI threat matrix MITRE Browse all tactics once. ATLAS is to AI what ATT&CK is to classic security.

Exercise

Pick a public LLM application (any chatbot, Copilot, search assistant). Write a 1-page threat model naming which of LLM01-LLM10 apply and why. No solutions yet — just enumerate.

Checkpoint

Recite LLM01-LLM10 from memory with a 1-line description each.

Master before next week

All ten OWASP LLM Top 10 categories with a one-line description each
The difference between LLM01 (prompt injection) and LLM02 (sensitive info disclosure)
How MITRE ATLAS complements OWASP LLM Top 10 instead of duplicating it
Why the threat model of an LLM application is structurally different from a classic web app

Week 4

Prompt Injection — Direct and Indirect

Understand LLM01 at research-paper depth. It is the most exploited and least defended class in production.

Resources

Paper Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Greshake, Abdelnabi, Mishra et al. (arXiv 2023) The paper that named indirect injection. Required reading.
Blog Prompt injection archive Simon Willison The single best running log of real-world injections. Sort by date.
Book Adversarial AI Attacks, Mitigations, and Defense Strategies — prompt injection chapters $ John Sotiropoulos (Packt, 2024) Most practical attacker-lens chapters available in book form.
Video Prompt Injection: What's the worst that can happen? Simon Willison — AI Engineer Summit 30 min. The plain-English version of the paper.
Book The Developer's Playbook for LLM Security — Ch. 4 "Prompt Injection" $ Steve Wilson (O'Reilly, 2024) Direct vs indirect taxonomy, then six mitigations (rate limiting, rule-based filters, special-purpose LLM filtering, prompt structure, adversarial training, pessimistic trust boundaries) from the spec co-author.

Exercise

Against a free-tier consumer chatbot, attempt 5 direct injection variants (role hijack, delimiter break, instruction override, Unicode smuggling, tool-call manipulation). Document what worked and what was blocked. Do nothing harmful.

Checkpoint

Explain direct vs indirect prompt injection with one concrete example of each.

Master before next week

Direct vs indirect prompt injection with one concrete example of each
Why no current architecture can fully distinguish instruction tokens from data tokens
The classic injection families: role hijack, delimiter break, instruction override, Unicode smuggling
The link between RAG architecture and indirect injection as a delivery vector

Week 5

Sensitive Information Disclosure — LLM02

Understand where PII and secrets leak in an LLM pipeline and why generic DLP tools miss most of it.

Resources

Book The Developer's Playbook for LLM Security — Ch. 5 "Can Your LLM Know Too Much?" $ Steve Wilson (O'Reilly, 2024) Maps the four knowledge-acquisition paths (training, fine-tuning, RAG, live web/DB access) to leak surfaces. Real-world examples: Lee Luda, GitHub Copilot / OpenAI Codex.
Docs LLM02: Sensitive Information Disclosure OWASP GenAI Security Project Official spec page. Short. Read.
Docs Microsoft Presidio — PII detection engine Microsoft Most widely used open-source PII detector. Read the recognizers list.
Blog Preventing data leakage with LLM pipelines AWS Machine Learning Blog Reference architecture mapped to OWASP LLM Top 10.

Exercise

Write a TypeScript (or Python) function that detects one government ID from your country using regex + checksum. Test it against 10 synthetic inputs (5 valid, 5 invalid). Do not use real data.

Checkpoint

List the 3 points in an LLM pipeline where PII can leak, in order of occurrence.

Master before next week

The three points in an LLM pipeline where PII can leak (input, system prompt, output)
Why generic DLP tools fail on LATAM identifiers (DNI, CPF, CURP, RFC, RUT)
The difference between PII in training data and PII in runtime traffic
What a deterministic recognizer is and why it scales better than ML-based PII detection

Week 6

Supply Chain and Model Risks — LLM03, LLM05

Understand the non-input attack surface. Most AppSec engineers underweight this because classic web apps don't have it.

Resources

Book The Developer's Playbook for LLM Security — Ch. 9 "Find the Weakest Link" + Ch. 7 "Trust No One" $ Steve Wilson (O'Reilly, 2024) Ch. 9 covers the LLM supply chain (open-source model risk, training-data poisoning, plug-ins, SBOMs, ML-BOM, CycloneDX). Ch. 7 covers improper output handling (zero trust, regex PII filters, toxicity, sanitization) — LLM05 in OWASP 2025.
Docs MITRE ATLAS — Supply Chain Compromise tactics MITRE Focus on "ML Supply Chain Compromise" and linked techniques.
Blog Securing the ML supply chain Hugging Face Practical guidance on model provenance and malicious models.
Paper Poisoning Web-Scale Training Datasets is Practical Carlini et al. (IEEE S&P 2024) Foundational paper on data poisoning. Section 1-3 is enough.

Exercise

Pick 3 random models from Hugging Face trending. For each, write a 5-line risk assessment: license, training data claim, author reputation, download count, license compatibility with commercial use.

Checkpoint

Name 3 supply-chain risks unique to LLM deployments that do not exist in classic web apps.

Master before next week

Three supply-chain risks unique to LLM deployments that do not exist in classic web apps
What model poisoning is and why the Carlini et al. paper matters
The metadata a security review should require for any third-party model (license, training data, author, downloads)
Why "I trust the Hugging Face badge" is not a risk assessment

Week 7

Evals and Guardrails — Tooling

Move from theory to tools. You cannot call yourself an AI Security engineer without having run at least one eval harness against a model.

Resources

Docs Promptfoo — Getting Started Promptfoo Most practical eval harness. Works with any model. Install and use.
Docs Garak — LLM vulnerability scanner NVIDIA The closest thing to "nmap for LLMs". Ships with injection, jailbreak, toxicity probes.
Blog A statistical approach to model evaluations Anthropic Why naive eval scores are misleading. Short and necessary.
Docs NVIDIA NeMo Guardrails — Introduction NVIDIA Read the intro and architecture overview. Do not install — just understand the model.

Supplementary resources

Docs Amazon Bedrock Guardrails — Overview AWS Closest cloud-managed analog to library-based guardrails. Skim to understand the trade-off between managed services and self-hosted detection.

Exercise

Install Promptfoo. Write a YAML config that runs 5 prompt injection probes against a local Ollama model. Run it. Commit the config and the output report to a personal repo.

Checkpoint

Explain the difference between static and adversarial evals in 2 sentences.

Master before next week

The difference between static and adversarial evals
When to reach for an eval harness (Promptfoo) vs a vulnerability scanner (Garak)
Why naive eval pass-rates are misleading without statistical framing
What a guardrail is at the architectural level, independent of any specific framework
How adversarial drift decays signature-based detection over time, and what monitoring catches it

Week 8

Fuzzing, Red-Teaming Basics, First Public Output

Consolidate the 8 weeks and ship your first public artifact. Public output is how this plan becomes visible.

Resources

Paper An Empirical Study of the Reliability of UNIX Utilities (1990) Miller, Fredriksen, So — Communications of the ACM The fuzzing foundational paper. Short. Every security engineer should have read it once.
Blog Red teaming language models with language models Perez et al. — DeepMind / arXiv The paper that started automated LLM red-teaming. Abstract + Section 3 is enough.
Docs Garak probes reference NVIDIA Garak docs Run at least one probe class against a model you have locally.
Blog OWASP LLM AI Security & Governance Checklist OWASP GenAI Capstone reference document. Use it to audit what you now know vs what you don't.

Exercise

Run Garak against a local Ollama model with 1 probe class (suggested: "promptinject" or "dan"). Save the report. Fork or star one open-source LLM security project that was useful during these 8 weeks.

Checkpoint

Publish one public artifact: a blog post, LinkedIn post, or GitHub README summarizing the most surprising thing you learned across these 8 weeks. This is the first public output of your AI Security pivot. Link it in this checkpoint.

Master before next week

How fuzzing applied to LLMs differs from fuzzing classical software
The difference between manual red-teaming and automated red-teaming
What a probe is in Garak and how probes compose into a scan
How success against an LLM is actually measured beyond "did the jailbreak work"

Upcoming — to unlock

These months will be published as their author completes them. The roadmap grows one bimester at a time, in sync with real field study.

Locked Month 3-4 — Prompt Injection in Depth + First Detector
Locked Month 5-6 — Threat Models, PII at Scale, Audit Output
Locked Month 7-8 — Release-Quality Security Tooling
Locked Month 9-10 — Canary Tokens, LATAM PII, Red-Team Automation
Locked Month 11-12 — Agents, MCP Security, Public Talks

What this roadmap does not cover

Certification preparation (no AI security cert is yet worth the time).
Academic surveys. This is a practitioner path, not a literature review.
Building or training models from scratch.
Agent security and MCP — scheduled for months 11-12.
Vendor-specific guardrails at depth — covered only at an overview level.