IdeaWell

Project ideas from Hacker News discussions.

Show HN: GoModel – an open-source AI gateway in Go

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

1. Open‑source sustainability &licensing
“I would love to keep it open source forever, but I can't promise that for now.” — santiago‑pl
The discussion repeatedly raises concerns about whether GoModel can stay truly open‑source, with users asking for a clear license commitment and worrying about future commercial restrictions.

2. Technical design: semantic caching and unified API
“GoModel embeds requests and does vector similarity lookup before proxying… Regarding the cache invalidation, there is no 'purging' involved – the model is part of the namespace (params_hash includes the LLM model, path, guardrails hash, etc). TTL takes care of the cleanup later.” — giorgi_pro

“Yes, we have an OpenAI‑compatible API and we develop GoModel with Postel’s law in mind … If they make a minor API‑level change, GoModel will handle it without any code changes.” — santiago‑pl
The project’s architecture emphasizes a flexible, standards‑based surface that can adapt to upstream API tweaks with minimal friction.

3. Practical value for multi‑provider routing & usage tracking
“I've been building an AI platform (HOCKS AI) where I route different tasks to different providers … The biggest pain point has been exactly what you describe: switching models without changing app code.” — goodkiwi

“Currently we have a unified concept of User‑Paths. Once you add a specific header OR assign User‑Path to an API key, you can track the usage based on this.” — santiago‑pl
Users appreciate the ability to switch LLM providers seamlessly and request richer telemetry (e.g., per‑user usage, cost breakdown).

🚀 Project Ideas

Generating project ideas…

[OpenAPI‑Key Vault Integration for AI Proxy]

Summary- Centralized vault‑backed storage for API keys across multiple LLMs, eliminating per‑provider secret sprawl.

  • Automatic per‑request cost attribution via User‑Path headers, solving hgo’s request for end‑user tracking.
  • Secure audit logs that let teams comply with internal security policies while using services like Litellm or Bifrost.

Details

Key Value
Target Audience Platform engineers, DevOps teams, AI service developers
Core Feature Vault‑integrated key provisioning with per‑user cost tracking via HTTP headers
Tech Stack Go, PostgreSQL, HashiCorp Vault, Prometheus‑Grafana
Difficulty Medium
Monetization Revenue-ready: Tiered SaaS pricing (Free up to 1 M requests, Pro $19/mo)

Notes- Why HN commenters would love it: “I miss integration with a service where I can inspect and debug completion traffic” (hgo) and “the biggest pain point has been exactly what you describe” (santiago‑pl).

  • Potential for discussion or practical utility: Provides a reusable secret‑management pattern for any Go‑based LLM gateway and enables granular usage billing.

[Live API Compatibility Checker & Auto‑Updater]

Summary

  • Periodic fetches of upstream provider OpenAPI specs and auto‑generates diff reports. - Generates pull‑requests or patches that keep the GoModel codebase in sync with breaking changes.
  • Includes a lightweight changelog viewer for contributors to see impact instantly.

Details

Key Value
Target Audience Maintainers of GoModel, Bifrost, and similar LLM routing projects
Core Feature Automated detection of API changes + auto‑generated PRs or patches
Tech Stack Go, Go‑npm, Docker, GitHub Actions, Markdown diff viewer
Difficulty High
Monetization Hobby

Notes

  • Why HN commenters would love it: “I’ll dig into this topic more broadly over the next few days” (santiago‑pl) shows appetite for ongoing compatibility work; talderigi’s query about semantic caching invalidation highlights the need for stable APIs.
  • Potential for discussion or practical utility: Reduces maintenance overhead for projects that must stay compatible with rapidly evolving model APIs.

[Multi‑Provider LLM Traffic Mirroring with End‑User Header Propagation]

Summary

  • Proxy layer that injects a configurable “X‑User‑Path” header into every outgoing request, enabling per‑end‑user usage tracking.
  • Adds a UI dashboard that visualizes request flow, latency, and cost per header value.
  • Supports OpenTelemetry tracing and can be extended with DLP or threat‑mitigation hooks.

Details

Key Value
Target Audience AI platform engineers building multi‑provider routing stacks
Core Feature Header‑based user identification + unified traffic dashboard
Tech Stack Go (proxy), React (dashboard), OpenTelemetry, Loki/Prometheus
Difficulty Medium
Monetization Revenue-ready: SaaS add‑on $0.01 per 1k traced requests

Notes

  • Why HN commenters would love it: “I don’t see if I’d be able to track usage from individual end‑users through a header” (hgo) and santiago‑pl’s reply that “User‑Path might be your end‑user” provides a concrete solution. - Potential for discussion or practical utility: Directly addresses the debugging and usage‑tracking gaps highlighted throughout the thread, offering a drop‑in component for any Go‑based gateway.

Read Later