Project ideas from Hacker News discussions.

More questions about whether researchers can trust OpenAI with unpublished math

📝 Discussion Summary (Click to expand)

Six Prevalent Themes in the HN Discussion

  1. Allegations of Unauthorized Data Usage
    Users accuse OpenAI of training on private conversations despite opt-out settings, claiming intellectual property theft.

    "OpenAI takes credit for what it's model spits out in response to other people's prompts. That's even worse." – znnajdla
    "exploitation, or appropriation, maybe it's time scientists and engineers revisited basic ethical notions." – calf
    "mathematicians I know... realized that they'd all gotten some emails from people they now know to be affiliated with OpenAI/Anthropic asking questions about their research in a way that seemed like scooping attempts." – viccis

  2. AI as Synthesizer vs. True Innovator
    Debate centers on whether AI creates novel work or merely connects existing ideas like human cognition.

    "Nobody ever invented or discovered anything in a vacuum - all discovery is synthesis of existing ideas... Einstein described the universe with it [Riemann's work]." – madaxe_again
    "Einstein didn't literally have someone prompting him towards his result." – znnajdla (countering madaxe_again)

  3. Corporate PR/IPO Motivations Overriding Scientific Integrity
    Suspicion that labs prioritize hype for investors over genuine discovery or ethical collaboration.

    "The big LLM providers, desperate for good PR before their IPOs, are all actively looking for 'almost finished' hard problems... claim 'they have solved it'." – mentalgear
    "All of this so that OpenAI and Anthropic can get into math result dick measuring to gas up their IPOs." – viccis

  4. Distrust in Opt-Out Mechanisms and Transparency
    Widespread skepticism that data privacy controls are effective or honestly implemented.

    "'Improve the model for everyone' [...] It's on by default. We can debate whether or not it should be opt in or opt out..." – jrflo
    "merely using the thumbs up/down button in a chat could share your entire conversation with them for model training." – ayewo
    "The very fact that you go to 'chatgpt.com' and write to them... there is no reason they would not receive and process your data, unless explicitly promising not to." – kzrdude

  5. Threats to Academic Careers and Knowledge Pipeline
    Concerns that AI undermines incentives for human researchers, discouraging new entrants to fields like mathematics.

    "my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math... they don't want to work for another half decade or more just to have to start their careers all over." – viccis
    "exploitative process... exploiting the knowledge/academia domain for scraps with a thin veneer of 'for science' PR." – mentalgear

  6. AI as a Useful Tool for Accelerating Discovery
    Counterarguments defending AI's role in advancing research through collaboration or synthesis, regardless of attribution disputes.

    "human/AI collaboration is indeed super charging discovery." – drivebyhooting
    "If the proof is valid, OpenAI has produced something new." – glitchc
    "synthesis is synthesis whether a human, a machine, or a duck does it." – madaxe_again


🚀 Project Ideas

PromptVault: Encrypted Prompt Logger & Audit Trail

Summary

  • Logs every prompt sent to AI APIs in an encrypted local store, with tamper‑evident hashes and optional signatures.
  • Enables users to prove whether their private prompts were used for training by submitting verifiable logs to a third‑party auditor.
  • Core value: transparent, user‑controlled audit trail that turns opaque data‑usage claims into provable facts.

Details

Key Value
Target Audience Researchers, engineers, and anyone sending private prompts to cloud AI services
Core Feature End‑to‑end encrypted prompt logging with cryptographic proof of ingestion and optional attestation to auditors
Tech Stack Electron/Tauri desktop app, AES‑256 local storage, libsodium for signatures, optional integration with OpenAI/Anthropic APIs via middleware
Difficulty Medium
Monetization Hobby

Notes

  • HN users complained about opaque opt‑outs and the inability to verify if their data was used (e.g., profsummergig: “For my (private) prompts, I need a warning telling me they may be used for training.”). PromptVault gives them the evidence they need.
  • Provides a concrete way to settle disputes like the Navier‑Stokes controversy by showing whether specific prompts entered the model’s training data.

PrivateLLM Desktop: Self‑Hosted LLM Workspace

Summary

  • A portable, open‑source desktop application that runs a fully private LLM (e.g., Llama‑3, Mistral) on the user’s own hardware, ensuring no data leaves the machine.
  • Includes chat UI, prompt history, and optional local fine‑tuning, all air‑gapped from the cloud.
  • Core value: guarantees data privacy by eliminating reliance on external AI providers for sensitive work.

Details

Key Value
Target Audience Mathematicians, scientists, and professionals handling confidential or unpublished research
Core Feature Local LLM inference with optional fine‑tuning, zero data egress, and encrypted prompt/vault storage
Tech Stack Rust backend, llama.cpp inference, Svelte frontend, optional Docker for cross‑platform distribution
Difficulty High
Monetization Hobby

Notes

  • Many commenters urged running models locally to avoid IP theft (e.g., vrganj: “Get an open Chinese model and host it yourself somewhere… nobody will be able to steal your innovations.”). PrivateLLM makes this turnkey.
  • Eliminates the dark‑pattern risk of opt‑out toggles being reset, because data never leaves the user’s control.

PromptWatermark: Invisible Prompt Watermarking Service

Summary

  • Adds a cryptographically undetectable watermark to user prompts before they are sent to any AI API.
  • If the model later outputs text containing the watermark, the user can prove their prompt contributed to training or generation.
  • Core value: enables provable attribution without revealing the prompt content, countering stealthy data harvesting.

Details

Key Value
Target Audience Power users, researchers, and companies sending sensitive prompts to third‑party AI services
Core Feature Prompt‑level watermark embedding and detection API (statistically robust, low‑false‑positive)
Tech Stack Python service using token‑level perturbation models, REST/gRPC API, optional WASM widget for browser integration
Difficulty Medium
Monetization Revenue-ready: subscription tiered by monthly watermarked prompts (e.g., $0.001 per prompt)

Notes

  • Users like postalcoder advocated watermarking prompts: “If only prompts could also be watermarked.” This gives them exactly that.
  • Provides a technical countermeasure to the fear that labs train on private chats without consent, turning suspicion into testable evidence.

AttributionChain: Blockchain‑Based AI Credit Ledger

Summary

  • A decentralized ledger where researchers can register timestamps, hashes, and zero‑knowledge proofs of their ideas, prompts, or intermediate work.
  • When an AI lab publishes a result, they must check the ledger for matching entries; any matches are automatically attributed.
  • Core value: creates a tamper‑proof, public record of pre‑AI work to settle credit disputes and encourage honest attribution.

Details

Key Value
Target Audience Academics, independent researchers, and innovation teams seeking proof of prior work
Core Feature Immutable registration of research artifacts with optional ZK‑proofs to preserve secrecy until disclosure
Tech Stack Ethereum Layer‑2 (Polygon) smart contracts, IPFS for off‑chain storage, zk‑SNARKs for private verification
Difficulty High
Monetization Hobby

Notes

  • The discussion repeatedly cited the need for attribution (e.g., nmfisher: “it is academically unethical to fail to credit the people whose specific conversations were fed into a model”). AttributionChain gives a concrete mechanism.
  • Enables researchers to prove they were “scooped” without having to reveal their unpublished work publicly until they choose.

OptOutGuard: Local Privacy Proxy for AI APIs

Summary

  • Runs as a local HTTP/HTTPS proxy that intercepts all traffic to known AI endpoints, enforcing user‑defined privacy policies (e.g., stripping identifying metadata, adding opt‑out headers, blocking feedback‑button data).
  • Alerts the user if the provider changes policies or if opt‑out toggles are silently re‑enabled.
  • Core value: enforces the user’s intent to keep data private, counteracting dark‑pattern resets and hidden data collection.

Details

Key Value
Target Audience Privacy‑conscious developers and power users who rely on cloud AI APIs
Core Feature Policy‑engine proxy that modifies/blocks requests and logs all outgoing AI traffic for audit
Tech Stack Go or Rust proxy with MITM TLS interception (using user‑installed CA), rule engine (YAML), Prometheus metrics for alerts
Difficulty Medium
Monetization Hobby

Notes

  • Users complained that the “Improve the model for everyone” toggle gets reset by app updates (asimpleusecase: “Old Facebook trick - likely resetting that box each time the app is updated.”). OptOutGuard prevents such silent re‑enrollment.
  • Provides logs that can be fed into PromptVault for end‑to‑end auditing.

DataLeakCheck: Membership‑Inference-as-a-Service

Summary

  • Offers a simple API (or CLI) that queries a target AI model (black‑box) to estimate whether a given piece of text (e.g., a private prompt) was likely present in its training data, using state‑of‑the‑art membership‑inference techniques.
  • Returns a confidence score and supporting statistics, letting users test if their data was used without needing model internals.
  • Core value: turns the opaque question “Did they train on my chats?” into a measurable, actionable test.

Details

Key Value
Target Audience Researchers, journalists, and anyone wanting to verify if their private data contributed to a model’s output
Core Feature Black‑box membership inference service with calibrated confidence scores and optional batch testing
Tech Stack Python service leveraging recent MI attacks (e.g., loss‑threshold, shadow‑model APIs), FastAPI, optional Docker
Difficulty Medium
Monetization Revenue-ready: pay‑per‑query credits (e.g., $0.001 per query) or monthly subscription for high‑volume testing

Notes

  • Commenters like hoj and postalcoder asked for ways to detect leakage: “how precisely do we … find this, given said companies are 100% non‑auditable…”. DataLeakCheck offers a practical workaround.
  • Empowers users to challenge unverified claims of “no training on user data” with empirical evidence, addressing the core frustration over lack of transparency.

Read Later