IdeaWell

Project ideas from Hacker News discussions.

Test, don't just verify

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

1. AI Accelerates Coding but Verification Remains the Bottleneck

AI enables rapid code generation, outpacing testing and deployment. "We can write code a lot faster than we can safely deploy it at the moment" (getregistered). Historically true, but AI exacerbates it: "a single developer with Claude code can very easily overwhelm even a couple of testers" (lelanthran).

2. Debate on AI Making Formal Verification Mainstream

Optimism that AI lowers barriers to formal methods like VGD or Lean, but skepticism persists. "AI is making formal verification go mainstream... The landscape has changed qualitatively" (baq). Counter: "This nonsense again. No. No it isn’t" (badgersnake). "Formal verification is going mainstream as watercooler weekend project fodder" (Ericson2314).

3. Static Typing and TDD as Practical Alternatives to Full Formal Verification

Types and tests offer accessible verification over full proofs; dynamic languages debated. "The most useful and used type of test is type checking" (anon-3988). "AI assisted coding is going to cause a resurgence of interest in XP... AI makes it easy to write well-tested code" (andrewmutz). TDD risks with AI: LLMs may "invent something that ensures that it is balanced" (9rx).

🚀 Project Ideas

VGD Assistant

Summary

  • An AI-powered VS Code extension that implements Verification-Guided Development (VGD) by generating formal specs from natural language prompts, suggesting proofs in Lean/Alloy, and iteratively refining code based on verification feedback.
  • Core value: Bridges the verification bottleneck for AI-assisted coding, making formal methods accessible as a "watercooler weekend project."

Details

Key Value
Target Audience Developers using AI tools like Claude/Cursor, interested in formal verification
Core Feature Prompt → spec → code → verify loop with integrated proof assistant
Tech Stack VS Code API, Lean 4, Alloy, OpenAI/Claude API, LSP for feedback
Difficulty Medium
Monetization Revenue-ready: Freemium (basic free, pro proofs $10/mo)

Notes

  • "Perhaps you could write a follow-up article or tutorial on your favored approach, Verification-Guided Development? This is new to most people" (esafak); "Formal verification is going mainstream as watercooler weekend project fodder" (Ericson2314).
  • HN would love the practical tutorial-in-tool form; high utility for regulated industries or AI code safety.

Adversarial Test Agent

Summary

  • A multi-agent service using frontier LLMs (e.g., Claude with computer use) that generates adversarial test cases, runs E2E QA on AI-generated code, and flags issues before deployment.
  • Core value: Scales testing to match AI code velocity, reducing manual QA overload without full formal verification.

Details

Key Value
Target Audience Solo devs/startups overwhelmed by AI code volume, no dedicated QA
Core Feature Upload code → adversarial agents probe (fuzzing, edge cases, smoke tests) → report with fixes
Tech Stack Anthropic API, Docker for sandboxed execution, Playwright for E2E, LangChain for agents
Difficulty Medium
Monetization Revenue-ready: Pay-per-run ($0.01/LOC)

Notes

  • "a single developer with Claude code can very easily overwhelm even a couple of testers" (lelanthran); "setting up adversarial agents for testing/verification - that has been a big win... avoiding the manual babysitting" (tgtweak).
  • Sparks HN debates on AI QA limits; immediate utility for fast deploys.

AI TDD Pair Programmer

Summary

  • Web-based tool that acts as an XP-style pair programmer: generates TDD cycles (tests first, then code), refactors for invariants, and enforces types in dynamic langs via gradual typing.
  • Core value: Revives TDD for AI era, providing "high confidence" verification cheaper than formal proofs.

Details

Key Value
Target Audience Python/JS devs using AI, seeking better-than-dynamic safety without Rust migration
Core Feature Chat interface: "Write sorted records API" → auto-generates failing tests → code → green → refactor with types/invariants
Tech Stack Next.js, Pyright/mypy for typing, Vitest/Jest, GPT-4o/Claude for generation
Difficulty Low
Monetization Hobby

Notes

  • "AI assisted coding is going to cause a resurgence of interest in XP... AI makes it easy to write well-tested code. The 'pairing' method... is also a great model" (andrewmutz); "TDD gets you a lot of them as well... much easier than formally verifiable codebase."
  • HN loves TDD nostalgia + AI twist; practical for daily coding, fosters type discipline discussions.

Read Later