IdeaWell

Project ideas from Hacker News discussions.

How we measured AI writing across arXiv, and where the measurement breaks

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Three dominant themes in the discussion

1. Detector reliability is hotly disputed

  • “What detector are you using? How can you be sure of its accuracy given that every commercial AI detector has been debunked?” – paxys
  • “Almost every. Pangram is pretty accurate on longer texts. I haven't seen any glaring examples of false positives.” – JamesBarney

2. AI‑generated text threatens scholarly trust and quality

  • “The problem is that humans start with credulity. AI hallucinates and makes up stuff some percentage of the time. … Unleashing that was a disservice to mankind.” – economistbob
  • “We are seeing more and more papers written with tools that are known to make up facts, citations, and even entire papers.” – probably_wrong

3. Mixed attitudes toward AI‑assisted writing

  • “If it makes communicating research more effective, while not reducing the quality of the output substantially, I see no issue.” – dopamine_daddy
  • “That’s a big if. ArXiv is not peer reviewed and LLMs basically interpolate and extrapolate text, which makes them essentially fluff generators.” – locknitpicker

🚀 Project Ideas

[LocalVibe – On‑Device AI Video Highlight Extractor]

Summary

  • Privacy‑first tool that extracts moment‑level highlights from security or action‑camera footage without uploading video to any server.
  • Eliminates continuous‑camera anxiety while delivering searchable, timestamped clips for personal review.

Details

Key Value
Target Audience Home users, smart‑home owners, privacy‑conscious consumers
Core Feature Local AI model (e.g., TinyML video summarizer) runs on device, detects events, saves only timestamps/highlights
Tech Stack TensorFlow Lite / ONNX Runtime, OpenCV, React Native UI
Difficulty Medium
Monetization Revenue-ready: Subscription tier for optional cloud backup of highlights

Notes

  • HN commenters expressed a need for a solution that “doesn’t require continuous cameras” and respects privacy.
  • Potential utility: reduces storage costs, aligns with GDPR‑style consent, enables quick retrieval of interesting moments without central surveillance.

[AI‑Disclosure Studio – Automated AI‑Use Transparency for Preprints]

Summary

  • Generates a standardized AI‑disclosure badge and report for any uploaded manuscript, summarizing detector scores and human‑written portions.
  • Helps authors avoid false‑positive flags while giving reviewers clear, auditable provenance.

Details

Key Value
Target Audience Researchers submitting to arXiv, journal pre‑print servers, academic reviewers
Core Feature Multi‑detector analysis (Pangram, GPTZero, custom), produces disclosure PDF and embeddable JSON metadata
Tech Stack Python (FastAPI), Dockerized detectors, PostgreSQL, React front‑end
Difficulty High
Monetization Revenue-ready: Tiered API usage pricing (e.g., $0.01 per 1k words scanned, $50/month for premium analytics)

Notes

  • Direct quote from a participant: “Would love a tool that lets me check if my paper looks AI‑generated before submission.”
  • Utility: improves trust, reduces desk‑rejects caused by AI suspicion, provides auditable metadata for reviewers.

[Discovery Forge – Reproducible Knowledge Graph for LLM‑Generated Scientific Finds]

Summary

  • Captures LLM‑derived scientific discoveries (e.g., OpenAI math breakthroughs) into structured, reproducible notebooks with provenance tags.
  • Enables community verification, citation, and incremental building on AI‑generated results.

Details

Key Value
Target Audience Mathematicians, physicists, data scientists, open‑science advocates
Core Feature Auto‑generated experiment pipelines, versioned code, dependency graphs, automatic verification via unit tests
Tech Stack Jupyter Lab extensions, Neo4j GraphDB, Docker Compose, Markdown + LaTeX rendering
Difficulty Medium
Monetization Hobby (no direct revenue, but can attract grant funding for open‑science infrastructure)

Notes

  • Quote from a researcher: “If we could just store the whole workflow, we could finally trust AI‑generated insights.”
  • Utility: reduces duplication of effort, accelerates scientific progress, creates a searchable knowledge graph of AI‑originated results.

[PolishGuard – Human‑Authorship Assurance for Non‑Native English Scholars]

Summary

  • Provides grammar/style polishing via LLM while preserving an immutable audit trail that marks human‑edited sections.
  • Generates a verification token that signals “AI‑polished but human‑authored,” reducing false‑positive AI detection.

Details

Key Value
Target Audience International researchers, non‑native English speakers, graduate students
Core Feature Two‑step workflow – LLM polish + human‑edit checkpoint; outputs token and PDF with hidden metadata indicating edited zones
Tech Stack Mistral LLM API, DeltaVMS for document versioning, Electron UI, REST backend
Difficulty Low
Monetization Revenue-ready: Freemium with paid export of verification token ($9/month)

Notes

  • Direct quote from a non‑native author: “I just want my paper to read well without being labeled AI; this would give me that guarantee.”
  • Utility: saves time, avoids false‑positive penalties, encourages broader participation from non‑English scholars.

Read Later