What AI Actually Finds: LLM-Assisted Work Across Pentest, SOC and Malware
I've folded LLMs into day-to-day security work for a while now. This is an honest account of what they find, where they save hours, and where they hand you a confident wrong answer that costs you more than they saved.
Offense: pattern-spotting at scale
The model is strong at the boring, high-volume review that humans skim.
- Reads a whole handler set and flags the one route missing an authorization check
- Spots IDOR / BOLA shapes across dozens of endpoints faster than manual diffing
- Generates payload and mutation lists, regexes, and one-off parser scripts
- Weak at business logic and multi-step chains — it needs you to frame the abuse case
SOC: detection-as-code and triage
This is where it pays off most. It drafts Splunk SPL and Sigma rules from a plain-English description, summarizes noisy alert clusters into a timeline, and explains an unfamiliar Windows event ID in context. You still review every rule — it will invent a field name that doesn't exist in your data model.
EDR and malware: deobfuscation and first-pass triage
Feed it an obfuscated PowerShell or JScript loader and it will unwind the string-building and base64 layers in minutes, name the likely technique, and draft a YARA rule from the decoded artifacts. Treat the YARA rule as a starting point and test it for false positives before it ships.
Where it fails — plan for this
Hallucinated CVE numbers, exploit code that looks right and isn't, missed context because it never saw the running system, and total confidence while being wrong. Every AI-surfaced finding gets manually reproduced before it enters a report.
What this kind of project turns up
Representative findings from real engagements. Details are altered and unlinked from any client — the pattern and severity are what matter.
The model flagged one REST handler in a set of ~40 that trusted a client-supplied user ID with no ownership check. Manual review had skimmed past it. Reproduced and confirmed as BOLA.
A plain-English prompt produced an SPL query for `schtasks`/`at` creation with unusual parent processes; it fired on activity analysts had previously filtered as noise.
Multi-layer base64 + char-code obfuscation unwound by the model; decoded C2 domain and mutex became IOCs and a YARA rule.
Model flagged a string-concatenated log line as SQLi; the query was actually parameterized. Included here as the failure mode: always verify.
Tools mentioned
AI is a force multiplier for review-at-scale, SPL and Sigma drafting, and malware deobfuscation — not a scanner. It finds real issues a tired human misses, and invents issues a tired human might believe. Reproduce everything.