IdeaWell

Project ideas from Hacker News discussions.

Ollaya – Ollama for open-source, Jev-style decision models

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

Theme 1 – Jev is essentially a classifier, not a fundamentally new technology
Many commenters argue that the “decision model” label is mostly marketing hype and that Jev works like a standard classifier.
- “Decision model = classifier … system one model = small non‑reasoning LLM” — hbrn
- “If you have an eval set for a fixed task, a trained classifier is the better choice.” — cobanov
- “Their marketing language is misleading. They must still use some transformer language model backbone …” — rockinghigh
- “Laya author is spitting more BS than Typesafe … the (incomplete) papers are nothing like Jev.” — verdverm

Theme 2 – Open‑source alternatives are rapidly closing the gap, threatening Jev’s moat
Discussion frequently notes that open models are catching up quickly and that any competitive advantage may rely on data pipelines rather than the model itself.
- “The best open ones are close to Jev now, but they're big models … The open models that get close to Jev are much bigger, and I'm adding support for those next.” — cobanov
- “I think the point being made is that Jev is great but it has no competitive moat, and open source versions will very soon catch up if their secret sauce is just synthetic data.” — not_a_bot_4sho
- “I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?).” — pradn
- “The moat is the RL synthetic data pipeline they set up to train jev.” — _menelaus

Theme 3 – Jev delivers practical speed, low cost, and ease‑of‑use benefits for real‑world tasks
Several users highlight concrete advantages: low latency, cheap inference, and simple zero‑shot deployment that save money and engineering effort.
- “It really does just work … Saves me about 75% of costs for the section its working in.” — shepardrtc
- “It's a fast classifier you can use out‑the‑box, ~1.5bn tokens is about $40 … It just works … a whole bunch of low‑level/low‑importance workflow stuff …” — mtkd
- “I put 250 MB / 1M lines of logs through Grev and it cost ~$10 USD, DSv4.1 would be at least 10x that and much, much, much slower.” — devttyeu
- “If you don't want to spend a lot and want low latency, e.g. home automation … it will then turn on the lights and heater nearly instantly.” — colordrops
- “Main difference is that laya/jev/et‑al give you a zero‑shot classifier that requires no training.” — Swizec

🚀 Project Ideas

DecisionBench

Summary

  • An open-source benchmarking platform that lets developers evaluate and compare Jev-like decision models on standardized, real-world decision tasks (navigation, traffic, moderation, etc.).
  • Core value: Provides trustworthy, apples-to-apples scores so teams can pick the best model for latency, cost, and accuracy without guesswork.

Details

Key Value
Target Audience ML engineers, product teams building decision-driven features, and open-source model maintainers
Core Feature Automated evaluation suite with leaderboards, Dockerized task runners, and API for submitting custom models
Tech Stack Python, FastAPI, PostgreSQL, Docker, HuggingFace Datasets, GitHub Actions for CI
Difficulty Medium
Monetization Hobby

Notes

  • HN commenters repeatedly asked for better benchmarks (e.g., "Has anyone actually seen better or the same results with Laya compared to Jev?" and linked to benchmarkheaven.com); DecisionBench gives them a community-run, transparent alternative.
  • Enables discussion around what constitutes a "general purpose" classifier and encourages open models to close the gap with Jev by highlighting weak spots.

JevForge

Summary

  • A CLI toolkit that generates synthetic training data using LLMs and RL-inspired reward signals to fine-tune compact decision models (Jev/Laya style) for any domain.
  • Core value: Lowers the barrier to create custom, high-performance decision models without needing large labeled datasets or expert ML knowledge.

Details

Key Value
Target Audience Developers who want domain-specific decision models (e.g., support bots, IoT agents) but lack labeled data
Core Feature Synthetic data generator + LoRA fine-tuning pipeline with calibration utilities, compatible with HuggingFace Transformers
Tech Stack Python, HuggingFace PEFT, vLLM for data generation, PyTorch, Typer CLI
Difficulty Medium
Monetization Hobby

Notes

  • Users expressed desire for an open "RL synthetic data pipeline" (_menelaus: "The moat is the RL synthetic data pipeline they set up to train jev. Open sourcing that would be the coup..."); JevForge directly addresses that.
  • Provides practical utility for the many who want to "distill Jev for your task" (_menelaus) and avoid paying for black-box APIs.

DeciGate

Summary

  • A lightweight inference gateway that hosts Jev-like decision models with a Jev/Ollaya-compatible API, enabling hot-swapping, batching, and GPU/CPU fallback.
  • Core value: Gives teams a simple, Ollama‑style way to run decision models locally or in the cloud, reducing vendor lock‑in and latency for real-time use cases.

Details

Key Value
Target Audience Backend engineers integrating decision models into products, hobbyists running local AI pipelines
Core Feature Unified REST/gRPC endpoint supporting multiple decision model backends (vLLM, llama.cpp, HuggingFace TGI) with model versioning and metrics
Tech Stack Go (or Rust), gRPC, Prometheus, OpenAPI, Docker, CUDA support
Difficulty High
Monetization Revenue-ready: hosted SaaS tier ($9/mo per concurrent model) + open-source core

Notes

  • Commenters wished for Ollama to support decision models ("adityamwagh: Hey Claude, make ollama for Jev like models") and noted the value of a gateway ("gav: GoModel (gateway) already supports Jev like endpoints"); DeciGate offers a dedicated, extensible solution.
  • Enables discussion on performance trade‑offs and practical adoption, addressing the frustration that "it’s just a classifier" but still costly to serve at scale.

Read Later