IdeaWell

Project ideas from Hacker News discussions.

Show HN: Screen memory without screenshots, just text to Markdown

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Theme 1 – Preferring Accessibility API over screenshot + OCR
Many participants favor reading text directly via the OS Accessibility API to avoid the macOS screen‑sharing warning that comes with capturing the viewport.

“Because you then have the macOS orange screen sharing warning/icon. I don't really want to record my screen, just the text is enough.” – Dramatize

Theme 2 – Limitations of raw accessibility extraction
The discussion repeatedly notes that most applications don’t expose clean, structured accessibility data, requiring per‑app tweaks and resulting in loss of formatting.

“most apps just aren’t wired up right… you end up having to hand code a lot of specific profiles for specific apps… you don’t quite get the right level of detail.” – etchalon

Theme 3 – Hoping for smarter, model‑based interpretation
Several commenters envision lightweight vision or language models that could infer UI semantics from limited text, possibly enriched with file/URL references or ambient context, to make the extracted data more useful.

“I hope in near future … that layer of abstraction … could probably become a light‑weight fine‑tuned vision model … that can run fully locally.” – albert_e
“The lossiness is also why the file/URL references exist… the LLM reading the file can open the original if the fragments aren’t enough.” – Dramatize

🚀 Project Ideas

Generating project ideas…

FocusText

Summary

  • A background daemon that periodically reads the text of the focused window using the OS Accessibility API (with optional Tesseract OCR fallback) and writes plain‑markdown snippets enriched with timestamps, app name, window title, and source file/URL references.
  • Provides a plug‑in system for app‑specific extraction profiles so users can tune noisy or non‑standard UIs without screen‑recording warnings.

Details

Key Value
Target Audience Developers, writers, researchers who want a lightweight, privacy‑first work log
Core Feature Capture focused‑window text every few seconds, output markdown with metadata, support custom app profiles
Tech Stack Swift (macOS), C# (Windows), Rust core logic, Tauri for cross‑platform UI, SQLite storage, Tesseract WASM for OCR fallback
Difficulty Medium
Monetization Revenue-ready: Subscription $5/mo (cloud sync, profile sharing, premium support)

Notes

  • HN users complained about macOS screen‑sharing warnings when screenshotting and wanted “just the text is enough” (Dramatize) – FocusText avoids any screen capture.
  • The discussion highlighted the need for per‑app profiles to handle apps “not wired up right” (etchalon) – the plug‑in system directly addresses that pain point.
  • Could spark conversation about standardizing accessibility‑based logging and enable new ambient‑context tools.

UIContextLM

Summary

  • An on‑device tiny vision‑language model that interprets UI elements (buttons, labels, fields) from the accessibility tree or a lightweight screenshot, producing semantic tags that improve text extraction and contextual enrichment.
  • Runs locally, ensuring privacy, and can be used as a plug‑in for FocusText or similar loggers to replace hand‑coded profiles with learned understanding.

Details

Key Value
Target Audience Privacy‑conscious power users, developers building accessibility tools, indie hackers
Core Feature Real‑time UI understanding model that labels elements and maps them to meaning, boosting extraction accuracy
Tech Stack PyTorch Mobile / Core ML / TensorFlow Lite, Rust bindings, TinyLLaVA‑style model (~200MB), optional ONNX runtime, integrated via FocusText plug‑in API
Difficulty High
Monetization Hobby (open‑source core, optional paid model‑training service)

Notes

  • Commenter albert_e hoped for a “light‑weight fine‑tuned vision model” that could reason about UI meaning locally – UIContextLM delivers exactly that.
  • By providing semantic labels, it reduces the “lossiness” noted when trying to reconstruct formatting from accessibility trees (Dramatize).
  • Could enable richer discussion on balancing model size, accuracy, and on‑device latency for productivity tools.

AmbientWorkLog

Summary

  • A personal knowledge‑base app that indexes the markdown snippets produced by FocusText (or any similar logger), offers full‑text search, tagging, automatic summarization, and bidirectional links back to the original document/window via the file:/url: reference.
  • Includes plugins for popular note‑taking tools (Obsidian, Notion, Logseq) to seamlessly integrate ambient work logs into existing workflows.

Details

Key Value
Target Audience Knowledge workers, researchers, journalists who need to recall and reuse past computer interactions
Core Feature Indexed, searchable archive of captured text snippets with summarization, tagging, and source linking
Tech Stack Electron/Tauri frontend, Rust backend, Tantivy (or MeiliSearch) for full‑text search, Llama.cpp for local summarization, SQLite/FS storage, plugin system using WebAssembly or native hooks
Difficulty Medium
Monetization Revenue-ready: Subscription $4/mo (sync across devices, advanced AI summarization, premium plugins)

Notes

  • Users wanted “ambient context about your daily usage/work” (Dramatize) and a way to “clean that extracted text” (hetsaraiya) – AmbientWorkLog provides the cleaning, indexing, and retrieval layer.
  • The file:/url: reference already lets users jump back to the source; AmbientWorkLog makes that actionable with search and summarization.
  • Could generate practical utility by turning fleeting UI text into a reusable personal wiki, prompting HN discussion on productivity and data ownership.

Read Later