Project ideas from Hacker News discussions.

We Replaced MMAP with Io_uring in Our Rust Query Engine. It Got Slower

📝 Discussion Summary (Click to expand)
  • The article is widely perceived as LLM‑generated / low quality
  • “Yay, first article (for me!) that i actually click on because the title is interesting, and then it puts me off because it's LLM generated.” — nottorp
  • “If you read through the LLM generated garbage it becomes clear that io_uring isn’t the problem, their code was just slop / bad.” — cbarnes99
  • “The article's subheadings suggest this is AI slop. Seeing a section titles ‘The Production Symptom’ is a Claudism in itself.” — muragekibicho

  • Performance and architectural trade‑offs between io_uring and mmap are debated

  • “Rust isn't inherently bad at io_uring but at least Tokio currently is… Tokio shares one buffer pool for all threads so as you scale to 100+ threads the whole thing grinds to a halt.” — orlp
  • “APIs like io_uring, combined with O_DIRECT, allow you to design your own workload‑specific userspace scheduler… The benefits of io_uring are limited without a commitment to designing your own schedulers.” — jandrewrogers
  • “Blindly throwing io_uring for mmap and hoping for better perfomance in a highly concurrent environment is a recipe for thread contention and latency spike.” — elendilm

  • Many see the effort as a classic “second‑system” overengineering trap

  • “Project exists → New employees come up with idea for efficiency improvement → Months spent implementing… Efficiency of new thing turns out worse than original…” — londons_explore
  • “Second System Effect” — jamesfmilne (link to Wikipedia)
  • “Then they fix the new thing implementation and realise the original gains that were promised.” — sceptic123

🚀 Project Ideas

LLM Article Detector for Hacker News

Summary

  • Browser extension that scores submitted HN articles for likelihood of being LLM-generated using linguistic heuristics and classifier.
  • Helps users skip low-quality, AI‑slop content and focus on genuine technical writing.

Details

Key Value
Target Audience Hacker News readers who want to filter out AI‑generated articles
Core Feature Real‑time LLM‑likelihood score overlay on HN article titles
Tech Stack TypeScript, React, TensorFlow.js (or ONNX runtime) for client‑side model; optional backend for model updates
Difficulty Medium
Monetization Hobby

Notes

  • Addresses complaints from nottorp, muragekibicho, and others who found the article “LLM generated garbage”.
  • Could spark discussion on detection techniques and improve signal‑to‑noise on HN.
  • Easy to ship as a userscript or extension; open‑source encourages community improvements.

io_uring Custom Scheduler Toolkit (Rust)

Summary

  • Provides a composable, async‑runtime‑agnostic scheduler built on io_uring with O_DIRECT support, letting developers avoid Tokio’s shared buffer pool contention.
  • Includes benchmarking harness and tutorial for building workload‑specific schedulers.

Details

Key Value
Target Audience Rust systems programmers using io_uring for high‑concurrency I/O (e.g., databases, network proxies)
Core Feature Plug‑and‑play io_uring scheduler API + O_DIRECT helpers + built‑in latency/throughput benchmarks
Tech Stack Rust, io_uring crate, async‑std or tokio‑agnostic core, criterion.rs for benchmarks
Difficulty High
Monetization Revenue-ready: Sponsored support / consulting tiers

Notes

  • Directly responds to orlp’s Tokio buffer‑pool issue and jandrewrogers’ point about needing custom schedulers to reap io_uring benefits.
  • Offers a practical solution that HN commenters would love to see benchmarked and discussed.
  • Could become a go‑to library for “design your own scheduler” guides, reducing second‑system effect.

Mmap vs io_uring Benchmark-as-a-Service

Summary

  • Cloud‑hosted benchmarking platform where users upload a workload description (e.g., random read/write, sequential) and receive side‑by‑side performance reports for mmap, io_uring, and alternative approaches.
  • Helps teams avoid costly “second system” rewrites by providing data‑driven decisions.

Details

Key Value
Target Audience Performance engineers, database/kernel developers, Rust/C teams evaluating I/O APIs
Core Feature Automated benchmark harness (Linux, configurable kernel versions) generating reproducible reports with latency, throughput, CPU usage
Tech Stack Go or Python backend, Kubernetes for job isolation, Prometheus/Grafana for metrics, static site for results
Difficulty Medium
Monetization Revenue-ready: Pay‑per‑benchmark subscription or tiered usage

Notes

  • Addresses londons_explore’s observation about the second‑system effect and elendilm’s advice to test before adopting io_uring.
  • Provides concrete data that HN users love to debate and reference.
  • Could be offered as a free tier for open‑source projects, encouraging community contributions and discussion.

Read Later