IdeaWell

Project ideas from Hacker News discussions.

The coming industrialisation of exploit generation with LLMs

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

1. LLMs excel at exploit generation due to clear success metrics but struggle with subjective tasks like bug reporting.

The discussion notes that exploit development has a verifiable goal (e.g., executing a shell), allowing LLM agents to iterate without human intervention, while bug reporting requires triage and is prone to low-quality, unverified submissions.

  • "An exploit tends to involve building a capability to allow you to do something you shouldn’t be able to do. If, after running the exploit, you can do that thing, then you’ve won." - moyix
  • "LLMs produce good output and bad output. The trick is figuring out which is which. They excel at tasks where good output is easily distinguished." - wat10000
  • "Both are true. Exploits are a very narrow problem with unambiguous success metrics. While also naturally complementing the ingrained persistence of LLMs. Bug reports are much more fuzzy by comparison..." - GoatInGrey

2. Human expertise remains crucial for setup, verification, and interpreting results, challenging claims of full automation.

Despite claims of "industrialisation" where agents work without humans, many argue that human skill is essential for designing the experiment, setting up the environment, and ensuring the verifier is accurate, making the process not truly autonomous.

  • "My expectation is that any organization that attempts this will need subject matter experts to both setup and run the swarm of exploit finding agents for them." - simonw
  • "You need an objective criterion you can use as a success metric. 'Design of verifiers' is a specific form of domain expertise." - adw
  • "The people sending useless bug reports aren't checking for good output." - wat10000

3. The asymmetry favors attackers, as they only need to find one exploit to succeed, while defenders must secure the entire system.

This dynamic is exacerbated by LLMs, which allow attackers to find vulnerabilities at scale with minimal cost. Defenders face the impossible task of fixing all potential issues, making the security landscape increasingly challenging for them.

  • "If the offender runs these tools, then any bug they find becomes a cyberweapon. If the defender runs these tools, they will not thwart the offender unless they find and fix all of the bugs." - pizlonator
  • "The defensive side needs everything to go right, all the time. The offensive side only needs something to go wrong once." - digdugdirk
  • "The only guaranteed winner is the LLM companies, who get to sell tokens to both sides." - pixl97

🚀 Project Ideas

Automatic Verifier Generator

Summary

  • Solves the problem of needing human-expert-created verification harnesses to enable autonomous exploit-finding agents.
  • Provides a tool that automatically generates working "verifiers" for specific security goals (e.g., "did we spawn a shell?", "was the file written?") given a vulnerability type and target environment.
  • Core value proposition: Democratizes autonomous security research by removing the need for expensive human setup time for verification infrastructure.

Details

Key Value
Target Audience Security researchers, red teams, and companies running automated vulnerability discovery (e.g., "Big Sleep" style operations).
Core Feature Parses vulnerability descriptions and target code, then outputs a minimal, executable test harness that strictly validates the exploit's success criteria without human intervention.
Tech Stack Python (for orchestration), Docker (sandboxing), LLVM/Clang (for binary instrumentation).
Difficulty Medium
Monetization Revenue-ready: SaaS API for generating verifiers; Enterprise license for on-prem execution.

Notes

  • HN commenters emphasized that "The verifier needs to be accurate, fast and again not involve a human." This tool directly addresses that bottleneck.
  • Practical utility is high as it turns the theoretical "industrialisation" mentioned in the article into a practical, repeatable pipeline for defenders and researchers.

Agent-Harness-as-a-Service

Summary

  • Addresses the confusion and spam in bug bounty programs caused by non-experts pasting raw LLM output into issue trackers.
  • Provides a managed environment where users input a target codebase and specific testing constraints; the system spins up LLM agents with tool access to find, verify, and report only working exploits.
  • Core value proposition: Filters "AI slop" by enforcing a strict "verify or discard" loop, ensuring that reported issues include demonstrable, working proof-of-concepts.

Details

Key Value
Target Audience Bug bounty hunters (to improve signal-to-noise ratio), open source maintainers (to triage reports), and enterprise security teams.
Core Feature A sandboxed environment where LLM agents can write, compile, and execute code against the target, submitting reports only when a verifiable success condition is met.
Tech Stack Kubernetes (for scaling agents), FastAPI (backend), React (frontend), OpenAI/Anthropic SDKs.
Difficulty Medium
Monetization Revenue-ready: Usage-based pricing (compute time + token usage) for running the harness.

Notes

  • Aligns with Simon Willison’s observation: "An 'agent harness' here is software that directly writes and executes code to test that it works."
  • Solves the frustration voiced by users like SchemaLoad regarding AI-generated issues wasting maintainer time by ensuring PoCs are tested before submission.

Formal Spec to Exploit DSL Converter

Summary

  • Bridges the gap between formal verification languages (TLA+, Lean) and the "money cannon" approach of LLM exploit generation.
  • Converts high-level formal specifications of system properties into a Domain Specific Language (DSL) that LLM agents can use to systematically search for logic bugs and vulnerabilities.
  • Core value proposition: Allows developers to define security invariants in high-level formal languages, which are then automatically translated into adversarial testing prompts for LLMs.

Details

Key Value
Target Audience High-assurance software developers, formal verification engineers, and security architects.
Core Feature Translates formal specifications (e.g., TLA+ specs) into executable LLM prompts and test harnesses that hunt for violations of those specs.
Tech Stack Rust (for robust parsing/transpilation), TLA+ parser libraries, LLM API integration.
Difficulty High
Monetization Revenue-ready: Enterprise license for integrating into CI/CD pipelines of safety-critical software.

Notes

  • Responds to adw's insight: "The design of verifiers is a specific form of domain expertise... given a clear enough, verifiable enough objective, you can point the money cannon at it."
  • Provides a concrete tool to operationalize the "objective criterion" required for industrialized exploit generation mentioned in the article.

Asymmetric Security Simulator

Summary

  • Addresses the "Asymmetric Warfare" dilemma where defenders must secure 100% of the surface while attackers need only one flaw.
  • A simulation environment that models the "Token Budget" race: pitting defensive LLM agents (patching/securing) against offensive LLM agents (exploiting) within a controlled codebase.
  • Core value proposition: Allows organizations to visualize the "token cost" required to secure their specific software stack against AI-driven attacks, justifying defensive budget and tooling.
Key Value
Target Audience CTOs, Security VPs, and Risk Analysts needing to quantify AI-driven security threats.
Core Feature Users upload their codebase; the system spawns parallel offensive and defensive agents, generating a risk heatmap and "Time-to-Exploit" estimates based on token spend.
Tech Stack Python (Simulation logic), Streamlit (Dashboard), Docker (Agent isolation).
Difficulty Medium
Monetization Revenue-ready: Subscription model for enterprise risk assessment reports.

Notes

  • Directly addresses pizlonator's point: "Any vs all is not symmetric."
  • Provides a practical tool for the "LLM Red Team" concepts mentioned by er4hn and azakai, allowing teams to benchmark their defenses against simulated industrialized attacks.

Read Later