IdeaWell

Project ideas from Hacker News discussions.

Human mathematicians are being outcounterexampled

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Four dominant themes in the discussion

Theme Supporting quotation
1. AI is now finding counterexamples faster than humans “Human mathematicians have been being out‑counterexampled for at least two decades … the sizes of counterexample we’re seeing are around about what I’d expect pre‑generative‑AI counterexample search systems to be able to find.” — wizzwizz4
2. LLMs are not trained per‑task; they share a single next‑token objective “They are not trained for each task individually. Pretraining just dumps every piece of content in the mix, and only has one objective - next token prediction.” — NitpickLawyer
3. A counterexample by itself does not provide deep understanding “Proofs by counterexample are effective but ultimately unsatisfying. they get you to an answer but they don’t help help you understand and bend you r mind into seeing how the math works and lead you on to the new set of questions.” — zeroonetwothree
4. Ethical and practical concerns about relying on LLMs in mathematics “LLMs are indefensibly evil because of stolen training data and enormous environmental impact.” — Diogenesian

🚀 Project Ideas

Counterexample Compiler

Summary

  • Systematically generate and verify counterexamples to open mathematical conjectures.
  • Provides automated search, verification, and reporting pipeline for researchers.

Details

Key Value
Target Audience Professional mathematicians, research students
Core Feature End‑to‑end counterexample discovery with Lean formal verification
Tech Stack Python, SageMath, PyTorch, Lean 4, Docker
Difficulty High
Monetization Revenue-ready: Tiered subscription

Notes

  • Directly addresses the frustration expressed about “easy” counterexamples that humans missed.
  • Enables systematic exploration of large parameter spaces, reducing reliance on ad‑hoc human intuition.

Conjecture Refiner AI

Summary

  • AI‑driven tool that iteratively proposes, tests, and refines conjectures based on discovered counterexamples.
  • Turns failed attempts into stronger, more precise statements.

Details

Key Value
Target Audience Pure mathematicians, graduate students
Core Feature Generates refined conjecture versions and suggests new conditions
Tech Stack LLM API, SymPy, Z3 solver, Jupyter notebooks
Difficulty Medium
Monetization Hobby

Notes

  • Mirrors the discussion about “adding conditions to avoid counterexamples” and wanting deeper insight beyond a single counterexample.
  • Helps users move from “just a counterexample” to a family of insights, matching the desire for explanatory power.

MathConfidence Dashboard

Summary

  • Interactive UI that extracts confidence scores and uncertainty markers from LLM‑generated mathematical outputs.
  • Flags low‑confidence claims for manual review or additional verification.

Details

Key Value
Target Audience Researchers using LLMs for theorem proving or conjecture testing
Core Feature Confidence labeling, verification status tracking, export to Lean/Isabelle
Tech Stack React, TypeScript, OpenAI/Anthropic API, Lean 4
Difficulty Low
Monetization Revenue-ready: SaaS per‑user pricing

Notes

  • Tackles the pain point of over‑confident AI outputs mentioned in the thread (e.g., “confidently incorrect” answers).
  • Provides the community‑wide need for reliable uncertainty reporting when exploring counterexamples.

Community Counterexample Repository

Summary

  • Crowdsourced, searchable database of AI‑discovered counterexamples with metadata (degree, variables, search context).
  • Includes benchmarking tools and citation tracking for scholarly use.

Details

Key Value
Target Audience Math communities, educators, AI researchers
Core Feature Search, filter, rate, and download verified counterexamples; API for integration
Tech Stack PostgreSQL, Django, ElasticSearch, Docker
Difficulty Medium
Monetization Hobby

Notes

  • Addresses the call for “systematic” collection of counterexamples and the desire for a “vibe‑counterexample” culture.
  • Enables reproducible research and collective progress, aligning with the discussion’s emphasis on shared insight.

Read Later