IdeaWell

Project ideas from Hacker News discussions.

'AI' Is Not a Tool by Rich Hickey

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

1. Semantic debate: “tool” vs. “automation/agent”
Many commenters argue over whether LLMs should be called tools, pointing to dictionary definitions or to the idea that meaning is based on use.
- “I agree 100% to what Rich has to say here, especially because, as Rich always writes, he goes to the dictionary to find correct definitions to support his argument.” – adelmotsjr
- “Whether something is a tool or not is a matter of family resemblances and is not susceptible to proof or disproof. I recommend that Hickey put down the dictionary and pick up Wittgenstein.” – xhevahir

2. Spectrum of toolness → agency
Several users note that LLMs (and humans) lie on a gradient from predictable tools to autonomous agents, with capability affecting predictability.
- “LLM's exist on a vast gradient of 'toolness -> agent'. Humans exist on that spectrum too.” – atleastoptimal
- “The smarter an LLM gets, the more its wants are disconnected from its tasks… the more you have to grapple with the depth of their thinking between the input and output.” – atleastoptimal

3. Practical use and caution
Commenters stress that, even if useful, AI must be handled carefully—building harnesses, accepting indeterminism, and watching for misalignment.
- “If you're using an agentic AI as a tool, be careful as it may stop being a tool.” – pixl97
- “I think there's an immense value there… the harness I built to work with them definitely is one.” – iLemming
- “a semantics discussion but I think it would be fair to call them 'indeterministic tools' … People try to wrangle the indeterministic outputs through deterministic layers.” – erelong

🚀 Project Ideas

Generating project ideas…

ToolLock: Deterministic LLM Harness

Summary

  • Provides a deterministic wrapper around LLM calls that enforces strict input/output schemas, logging, and sandboxed execution.
  • Core value proposition: Turns unpredictable LLMs into reliable, repeatable tools for engineering workflows.

Details

Key Value
Target Audience Software engineers, DevOps, ML engineers integrating LLMs into pipelines
Core Feature Schema‑driven validation and enforcement layer with deterministic fallback and full audit trail
Tech Stack Python/FastAPI, Pydantic for schema validation, Docker sandbox (or WASM), Redis for state
Difficulty Medium
Monetization Revenue-ready: SaaS subscription tiered by API call volume

Notes

  • HN commenters asked for a harness: “I think there's an immense value there, way before you even start talking about code‑generation…” (iLemming) and noted the need for deterministic layers to wrangle indeterministic outputs (erelong).
  • Could spark discussion on the tool vs. agent distinction and become a de‑facto standard for safe LLM usage.

LLMTestBench: Reliability Testing & Evaluation Platform

Summary

  • Enables teams to define task‑specific test suites, run LLMs against them, measure hallucination, latency, and compliance, and receive actionable reports.
  • Core value proposition: Continuous reliability monitoring that catches drift before it impacts production.

Details

Key Value
Target Audience Product teams using LLMs for customer‑facing features, QA engineers, AI safety officers
Core Feature Test harness with predefined metrics (factuality, policy adherence, format) and CI/CD integration
Tech Stack React frontend, Node.js backend, HuggingFace inference API or self‑hosted models, PostgreSQL, GitHub Actions
Difficulty Medium
Monetization Revenue-ready: per‑seat pricing plus usage‑based test run fees

Notes

  • Commenters highlighted LLM unpredictability: “its wants are unpredictable and hard to specify…” (atleastoptimal) and the need to convert indeterminism to something determined through deterministic layers (erelong).
  • Practical utility: gives teams confidence to ship LLM features while reducing risk of rogue or off‑spec behavior.

GuardLLM: Real‑Time LLM Guardrails & Intervention Service

Summary

  • Acts as a middleware that monitors LLM streams for policy violations, hallucinations, or undesired behavior and can trigger fallback, rewrites, or alerts.
  • Core value proposition: Protects applications from harmful or off‑spec LLM outputs in real time.

Details

Key Value
Target Audience SaaS providers, chatbot platforms, any service exposing LLM APIs to end‑users
Core Feature Real‑time scoring engine with customizable rules, fallback to deterministic rules or human‑in‑the‑loop, detailed audit logging
Tech Stack Go low‑latency proxy, Envoy/NGINX plugin, TensorFlow Lite toxicity models, Redis rule cache, Prometheus metrics
Difficulty High
Monetization Revenue-ready: usage‑based pricing per million tokens processed

Notes

  • Users warned: “if you're using an agentic AI as a tool, be careful as it may stop being a tool.” (pixl97) and expressed the need for deterministic layers to convert indeterminism to something determined (erelong).
  • Potential for discussion: raises questions about oversight vs. autonomy and could inspire new safety standards for LLM‑powered services.

Read Later