Here are the 3 most prevalent themes from the discussion:
1. The Essential Distinction Between Statistical Prediction and Causal Understanding A central debate revolves around whether sophisticated statistical models (like LLMs) provide genuine insight or merely excel at curve-fitting without capturing underlying reality. Critics argue that Norvig's essay and modern ML ignore the fundamental need for causal models, while defenders contend that predictive accuracy itself is a valuable scientific goal, even without explicit causal explanations.
"There is a difference between discovering causes and fitting curves. The search for causes guides the design of experiments... Norvig seems to be confusing the map (data, models) for the territory (causal reality)." — intalentive
"I don't want to engage much with the arguments because it starts on the wrong foot and begins by making, in my opinion, an incoherent / unsound distinction, while also ignoring... the actual scientific and philosophical progress." — D-Machine
2. The Divergence Between Engineering Success and Theoretical Linguistics Commentators vigorously debated the respective merits of Chomsky's theoretical framework versus the practical achievements of statistical language models. Many argued that Chomsky's work, while influential in computer science, has failed to produce useful models of natural language, whereas LLMs demonstrate undeniable success in generation and prediction, even if they don't address the same scientific questions.
"He was impressively early to the concept, but I think even those skeptical of the ultimate value of LLMs must agree that his position [that 'probability of a sentence' is useless] has aged terribly." — tripletao
"Chomsky is interested in modeling the range of grammatical structures and associated interpretations possible in natural languages... LLMs are not really a competitor in that sense anyway." — foldr
3. The Falsifiability and Practical Value of Scientific Theories A recurring theme is whether Chomsky's research program (and similar theoretical work) meets scientific standards of falsifiability and practical utility. Critics argue that his theories are unfalsifiable or self-referential, lacking external validation or real-world application, while supporters defend pure theoretical inquiry and point to Chomsky's broader influence on cognitive science.
"Everything that I see turns inward, valuable only within the framework that he himself constructed. Anyone can build such a framework, so that's not an accomplishment." — tripletao
"Most scientific work doesn't [have practical applications]. It's just that, for obvious reasons, you tend to hear more about the work that does... Has geology accomplished something considered difficult outside of geology?" — foldr