IdeaWell

Project ideas from Hacker News discussions.

History LLMs: Models trained exclusively on pre-1913 texts

Original Article

Hacker News Discussion

📝 Discussion Summary (Click to expand)

1. Intrigue in Time-Locked Historical Simulation

Users express fascination with LLMs embodying pre-1913 perspectives, free of modern hindsight like WWI knowledge, enabling "genuine" period conversations.
"saaaaaam": “Time-locked models don't roleplay; they embody their training data. Ranke-4B-1913 doesn't know about WWI because WWI hasn't happened in its textual universe.”
"observationist": "This is definitely fascinating - being able to do AI brain surgery, and selectively tuning its knowledge and priors..."

2. LLMs as Autocomplete vs. Emergent Capabilities

Intense debate on whether LLMs are mere statistical predictors or capable of novel problem-solving via scale, RLHF, and emergence.
"eek2121": "LLMs are just seemingly intelligent autocomplete engines... Ask it to solve something no human has, you'll get a fabrication."
"libraryofbabel": "Don’t let some factoid about how they are pretrained on autocomplete-like next token prediction fool you... they can absolutely solve new problems that aren’t in their training set."

3. Critiques of Limitations, Biases, and Value

Skepticism over hallucinations, data leakage, historical biases, modern prose contamination, and practical utility beyond parlor tricks.
"root_axis": "ultimately, Opus 4.5 is the same thing as GPT2, it's only that coherence lasts a few pages rather than a few sentences."
"TSiege": "This is just make believe... a parlor trick, a seance masquerading as science."

🚀 Project Ideas

Historical LLM Playground

Summary

  • Open web-based platform for running, fine-tuning, and sharing small historical LLMs (e.g., pre-1913 cutoffs) with built-in disclaimers, output logging, and community voting on hallucinations.
  • Core value: Democratizes access to "time-locked" models, enabling safe experimentation without researcher fears of misuse or backlash.

Details

Key Value
Target Audience Historians, writers, educators, HN enthusiasts
Core Feature Upload datasets, select cutoffs, generate/share chats with auto-disclaimers and hallucination flags
Tech Stack Hugging Face Transformers, Gradio/Streamlit for UI, Pinecone for vector search validation
Difficulty Medium
Monetization Revenue-ready: Freemium (free small models, paid GPU for large/custom)

Notes

  • Addresses "responsible access framework" fears: "We're developing a responsible access framework... preventing misuse" – provides disclaimers and moderation.
  • HN would love community tinkering: "just release the model", "tinkering with the models" for insights like relativity discovery tests.

Archive Transcription Accelerator

Summary

  • AI service using vision-language models paired with historical LLMs to OCR/transcribe/translate scanned archives (e.g., newspapers, letters), outputting cleaned text for LLM training.
  • Core value: Solves data scarcity ("not enough historical tokens"), unlocking trillions of tokens from undigitized sources like national archives.

Details

Key Value
Target Audience Researchers, libraries, digitization projects
Core Feature Batch upload scans, period-specific OCR + translation, export tokenized datasets
Tech Stack Tesseract + LlamaVision/Mistral-VLM, LayoutLM for docs, DeOldify for image enhancement
Difficulty High
Monetization Revenue-ready: Pay-per-page (e.g., $0.01/page)

Notes

  • Directly tackles "very little text before the internet": "trillions of tokens... tucked away in national archives", "less than 30% scanned".
  • Practical utility: "accelerate the transcription process", praised by historians like "scans of 18th century english documents".

Overton Window Tracker

Summary

  • SaaS tool training/running decade-by-decade LLMs (e.g., 1900, 1910, 1920) on timestamped corpora, with a query interface to compare worldview shifts on topics like gender, empire.
  • Core value: Visualizes evolving norms without hindsight bias, for research/writing ("period accurate books and screenplays").

Details

Key Value
Target Audience Academics, journalists, authors
Core Feature Auto-train on public datasets, side-by-side chat comparisons, sentiment timelines
Tech Stack LoRA for efficient fine-tuning, LangChain for multi-model querying, Plotly for viz
Difficulty Medium
Monetization Revenue-ready: Subscription ($10/mo researchers, $50/mo teams)

Notes

  • Fills gap: "models by decades... ask the same questions... when homosexuality... accepted" – tracks "Overton window on many issues".
  • Sparks discussion: "compare architectures... pre-1905 about relativity", utility for "historical fiction... genuinely from the period".

Read Later