IdeaWell

Project ideas from Hacker News discussions.

Asking authors about their own papers

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Theme 1 – Erosion of authenticity and trust
Many commenters worry that AI‑generated “slop” makes it impossible to know whether a paper truly reflects the author’s understanding, lowering confidence in scholarly work.

“I think authenticity and trust will command a (larger) premium in this new age of slop.” – N_Lens

Theme 2 – Calls for stronger verification (oral exams, CAPTCHA‑like checks, disclosure)
To combat undisclosed AI use, several participants suggest that journals should require authors to demonstrate personal comprehension—e.g., through oral defenses or interactive checks.

“High repute journals may need to add ‘oral exams’ to the paper acceptance process…” – JasonCEC
“You need to get the author to do an oral defense to verify that they wrote the paper.” – lelanthran

Theme 3 – Systemic issues: incentives, process over product, and the broken publishing model
Beyond detection, the discussion highlights that academia’s emphasis on paper counts devalues the research process (training, collaboration, reproducibility) and that AI exacerbates existing pressures to cut corners.

“One larger problem here is the value of a research paper is rarely the specific knowledge it adds but in the process of researching that adds to the collective knowledge+experience of those involved, especially training graduate students. AI papers shortcut this entirely.” – greenflag
“For more than a century, scientific journals have been the pipes through which knowledge … Now they’re being clogged with AI slop.” – PantaloonFlames (quoting The Atlantic)

🚀 Project Ideas

VivaVoce: Automated Oral Defense Platform for Academic Submissions

Summary

  • Provides a scalable, video‑based oral defense workflow where authors answer LLM‑generated questions about their submitted paper, producing a comprehension score for editors.
  • Core value proposition: Restores trust in authorship by verifying that authors truly understand their work, without requiring costly in‑person panels.

Details

Key Value
Target Audience Journal editors, conference program chairs, and research institutions seeking to verify author understanding at scale
Core Feature AI‑driven question generation from paper text, scheduled video interviews, automated answer evaluation (semantic similarity & depth), and a digestible trust score
Tech Stack Frontend: React + WebRTC for video; Backend: Node.js/Python (FastAPI); LLM: open‑source Llama 3 or GPT‑4o for Q&A generation; Video streaming: Agora or Twilio; Database: PostgreSQL; Deployment: Docker/Kubernetes
Difficulty Medium
Monetization Revenue-ready: Subscription per journal (tiered by monthly submission volume) + pay‑per‑interview add‑on

Notes

  • HN commenters highlighted the need for oral defense (“You need to get the author to do an oral defense…”) and the scalability problem (“It can’t scale…”); VivaVoce addresses both by automating question generation and using remote video.
  • Potential for discussion: Could become a new standard verification layer, prompting debate on how much understanding is enough and how to balance reviewer workload with author burden.

AttestAuth: Institutional Author Attestation & LLM Usage Ledger

Summary

  • A lightweight attestation service where authors verify their identity via institutional OAuth (university/ORCID) and disclose the extent of LLM assistance used in manuscript preparation, generating a tamper‑evident trust badge.
  • Core value proposition: Makes authorship transparent and accountable, reducing anonymous AI slop by tying papers to verifiable human contributors.

Details

Key Value
Target Audience Authors submitting to journals/conferences, editorial offices, and libraries that want to audit provenance
Core Feature Institutional login, LLM usage disclosure form (percentage, purpose), cryptographic signing of submission metadata, and public verification badge displayed alongside the paper
Tech Stack Frontend: Svelte; Backend: Go (Echo) or Rust (Actix); Identity: OpenID Connect (university LDAP, ORCID); Signing: JSON Web Signature (JWS) + optional IPFS pinning; Database: SQLite/Postgres; Deployment: Docker Compose
Difficulty Low
Monetization Revenue-ready: Annual license for institutions (based on number of affiliated authors) or per‑submission fee for journals

Notes

  • Commenters stressed that “trust in authorship is lower than ever” and called for mechanisms to verify author identity and LLM use (“trust will command a premium”). AttestAuth directly supplies that verifiable trust signal.
  • Practical utility: Editors can instantly see whether a submission has a verified author badge and LLM disclosure, helping prioritize desk‑rejection or deeper review without needing to conduct interviews for every paper.

SlopScore: LLM‑Generated Text Detector for Scholarly Writing

Summary

  • An API‑driven detector fine‑tuned on academic prose that estimates the probability a manuscript contains LLM‑generated content, highlights suspicious passages, and explains why (e.g., perplexity bursts, phrase repetition).
  • Core value proposition: Gives editors a fast, objective triage tool to flag potential AI slop before investing reviewer time, focusing human effort on high‑risk submissions.

Details

Key Value
Target Audience Editorial offices, peer‑review platforms, and preprint servers looking to screen submissions efficiently
Core Feature Probability score (0‑100%), sentence‑level highlighting, explanation dashboard, and batch processing via REST API or plugin for manuscript systems (Overleaf, Scholastica)
Tech Stack Python (PyTorch/HuggingFace) for model; FastAPI for API; Model: RoBERTa‑base fine‑tuned on mixed human/AI academic corpus; Frontend: React dashboard; Deployment: AWS Lambda/EKS or self‑hosted GPU instance
Difficulty Medium
Monetization Revenue-ready: Pay‑per‑API‑call tiered pricing (free tier for low volume, paid plans for high‑volume journals)

Notes

  • Several HN users wished for a way to detect “AI‑generated slop” (“how is the industry making sure that submissions aren’t at least partially AI‑generated?”). SlopScore offers a concrete, scalable detection mechanism.
  • Potential for discussion: Could spark debate on false positives/negatives, prompting refinement of detection criteria and encouraging better LLM disclosure practices alongside detection.

Read Later