IdeaWell

Project ideas from Hacker News discussions.

RTK reports token savings, but our cost benchmarks disagree

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Theme 1 – Skepticism that token‑saving hacks are mostly snakeoil
Many commenters dismiss the claimed savings as overhyped or unsubstantiated.

aeneas_ory: “All of these 'hacks' are snakeoil and I think deep down we all know.”

Theme 2 – Such hacks can confuse the model and actually raise costs
Several users argue that altering tool output breaks the model’s expectations, leading to extra tool calls or wrong results.

kgeist: “If the output is not what it expects, an LLM may issue more tool calls than before, because it will assume the tool is broken, the arguments passed to it were wrong, or it's a newer/older version of the tool etc => more tokens.”

Theme 3 – Practical alternatives and nuanced takes
A few contributors note that certain approaches (semantic search, quiet‑mode tooling, IDE‑native actions) can help when used carefully, but they stress the need for rigorous benchmarking.

aeneas_ory (on Lumen): “Why does it work you may ask? Well, LLMs basically brute force words/phrases … Semantic search looks for similarities so you have to do less brute forcing.”
dist‑epoch: “One quick win is to just avoid wasteful tokens, for example run all the QA tools like the unit tests in --quiet mode, which only prints warnings/failures.”

🚀 Project Ideas

AgentBench: Transparent Token‑Usage Benchmark Suite for AI Coding Agents

Summary

  • Provides a reproducible, open‑source benchmark harness to measure token consumption, latency, and correctness of AI coding agents when using various token‑saving hacks (RTK, Lumen, semantic search, etc.).
  • Core value proposition: eliminates vague marketing claims by delivering objective, comparable metrics across models, prompts, and tool configurations.

Details

Key Value
Target Audience AI agent developers, tool makers, and engineering teams evaluating productivity hacks
Core Feature Harness that runs a suite of coding tasks (code completion, refactoring, bug fixing) with and without a token‑saving wrapper, logs token usage via LiteLLM, and verifies output correctness
Tech Stack Python, LiteLLM for model interfacing, Pydantic for schema, Docker for isolation, GitHub Actions for CI, optional Streamlit dashboard
Difficulty Medium
Monetization Hobby

Notes

  • HN commenters stressed the need for "independent benchmark" and distrust of vendor claims (e.g., "We need some kind of independent benchmark." – ProjectBarks).
  • Enables community‑driven validation, fostering discussion on which techniques actually reduce token waste without harming agent reasoning.

CtxMCP: Local Semantic Search MCP Server for AI Coding Agents

Summary

  • Runs a self‑hosted Model Context Protocol server that indexes a codebase with lightweight embeddings and returns relevant snippets via natural‑language queries, reducing the need for agents to brute‑force grep.
  • Core value proposition: gives coding agents precise, context‑aware code retrieval locally, cutting token usage while preserving accuracy and privacy.

Details

Key Value
Target Audience Developers using Claude Code, Codex, OpenCode, or similar AI coding assistants who work with large or monorepo codebases
Core Feature Embedding‑based code search exposed as MCP tools (search, get‑snippet, find‑symbol) with incremental updates and support for multiple languages via tree‑sitter parsers
Tech Stack Python, FastAPI for MCP endpoint, sentence‑transformers or MiniLM embeddings, ChromaDB or FAISS vector store, tree‑sitter for language‑aware chunking, optional Ollama for local LLM reranking
Difficulty Medium
Monetization Revenue-ready: Subscription: $10/user/mo (hosted managed service with SSO, team workspaces, and usage analytics)

Notes

  • Commenters praised local embedding approaches ("I indexed the codebase with a dedicated local code embedding model… reduced token use significantly" – aeneas_ory) and desired editor‑agnostic tooling ("PRs welcomed!" – aeneas_ory).
  • Provides a transparent, self‑hostable alternative to opaque cloud‑based retrieval, encouraging community contributions and trust.

TrimTok: Deterministic Output Truncation & Summarization Proxy for Agent Tool Calls

Summary

  • Acts as a middleware proxy that intercepts common CLI tool outputs (e.g., ls, grep, test runners) and applies rule‑based truncation or summarization that preserves essential structural information (headers, summary stats, JSON keys) while cutting unnecessary tokens.
  • Core value proposition: reduces token load from noisy tool output without breaking agent expectations, addressing the pitfalls of blunt tools like RTK.

Details

Key Value
Target Audience AI agent orchestrators and developers who rely on tool‑call heavy agents (Claude Code, Codex, etc.) and want safe output compression
Core Feature Configurable rules per command (e.g., keep first/last N lines, keep column headers for tabular output, preserve JSON schema) with deterministic behavior and optional confidence scoring
Tech Stack Rust for low‑overhead subprocess handling and parsing, or Go; uses regex and language‑specific parsers (e.g., jq for JSON, csvkit for CSV); configurable via TOML/YAML
Difficulty Medium-High
Monetization Hobby

Notes

  • HN users warned that RTK "subverts the model's expectations" and produces "plain wrong results" (GodelNumbering), advocating for smarter truncation ("Cap large/unknown command output" – kriskrunch).
  • TrimTok offers a principled, transparent way to limit output, aligning with the community's call for tools that don't confuse the model while saving tokens.

Read Later