9 prevalent themes in the discussion
-
Apple’s hardware gives it a clear edge for running large AI models locally
“M5 Ultra features a massive amount of high‑bandwidth unified memory, up to 512GB, and delivers a staggering 1.2TB/s of unified memory bandwidth that is 50 percent higher than M3 Ultra.” – rvz
“Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.” – rvz -
Apple’s AI strategy leans heavily on hardware while its software efforts are still maturing
“As someone who works in AI now, I have found it pretty amazing that Apple basically didn't do much with AI software, and focused more on the hardware side. I think this is what the future of AI is going to look like, local models run on your mac for your workflow.” – nasaeclipse
“Apple has been dumping enormous sums into their own AI development. Their failure to have a lot to show for it doesn't void the fact that they tried really, really hard.” – llm_nerd -
Performance is often compared to NVIDIA GPUs, with trade‑offs in speed vs. memory
“My old 3090 is typically significantly faster (almost 2× token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.” – mhast
“1.2TB/s is 2/3 the speed of an nVidia 5090. But you get a generic computer and much more RAM.” – LeBit -
Linux support on Apple Silicon is limited; workarounds involve VMs or Asahi
“You can't run Linux directly on these. Asahi Linux supports up to M2 only. Linux runs very well in a VM on macOS… However, macOS on M-series macs is excellent for local models.” – jlokier
-
Economic analysis pits local hardware against cloud/subscription AI services
“The monthly workstation cost is about $550 capex and $150 electricity → $700/month … You would need about 6 Pro accounts to reach that capacity, which would cost you $1,200 a month.” – ltbarcly3
“Unless you need privacy for your inference this instant, paying for credits can get 80 to 90 percent of people everything they need.” – bilbo0s -
Motivations for buying include privacy, agent‑style workflows, and creative workloads
“I think it'll become considered totally reasonable to consider spending the cost of a small car on a computer, for many families.” – ericd
“Having it at home means that you can use it as a personal agent that always puts your interests first… you can stash in it your medical data, what you buy, what you make, your worries, hopes, and dreams.” – ericd -
RAM and memory prices are driven by AI‑fueled demand and show cyclical behavior
“The DRAM market is cyclical. I don’t think anyone truly knows when, but it will happen.” – dannyw
“The memory companies report that they're sold out through 2027… so it might be a while.” – unsupp0rted -
Future outlook favors efficiency gains and smaller, good‑enough models
“Massive models have proven to be very strong, but small models have proven to be good enough (especially with the recent Qwen 2.8 27B drop) and that's where I imagine the future will lie for consumers.” – jjice
“Tools like turbo fieldfare are really reducing the memory requirements to run large models and I don’t see it stopping soon.” – twobitshifter -
Skepticism remains about cost‑effectiveness and real‑world utility
“Not if it's 5-10x slower than a remote inference server. Mac prefill latency is exhausting.” – bigyabai
“Do you think monthly AI ‘subscriptions’ are going to be $100 a month in 5 years? These people using these would probably be on $200/month subscriptions… makes no sense.” – AbsurdCensor