1. StarCraft as a benchmark for LLM strategy and tactics
- “This is a great idea for a benchmark. Something all the benchmarks seem to be missing is strategy, tactical solutions …” – malfist
- “I predict LLMs will reach superhuman level and beat even that model in the next 12 months.” – benswerd
- References to GoBench and similar efforts show interest in measuring LLMs’ strategic ability via games. – GodelNumbering
2. Nostalgia and personal ties to the StarCraft community
- “I love StarCraft. I started playing it right from the beginning… I literally met people that have spread to almost every continent …” – pelagicAustral
- “Later we even listened to Eminem and played violent video games. Most of have never even been charged with a crime…” – sidewndr46
- “Oh boy, I've spent more hours playing it than I dare to admit. I won over 10 000 battle.net games …” – snicky
3. Technical challenges and limits for LLM agents in real‑time StarCraft
- “Starcraft is APM‑dependent. Unless the latency will improve greatly in frontier reasoning LLMs (which is unlikely), it will remain a bit like knitting with an excavator.” – orbital‑decay
- “Yeah but Starcraft needs, like, 10‑20x the APM these agents are doing.” – loeg
- “Playing these games autoregressively isn't even the right way to use the LLM for this task … it ends up on how much go transcripts ended up in the training…” – nullc