LLMs as a Cognitive Virus

We need to produce exactly 3 concrete project ideas addressing pain points expressed in the discussion. Identify pain points: cognitive offloading leading to reduced independent thinking (cognitive debt), difficulty verifying AI outputs, reliance on LLMs causing atrophy, non-technical folks speaking about tech, need for outside observers, quantifying cognitive debt, concerns about AI training on low-quality data, need for tools to detect AI-generated content, need for ways to maintain thinking skills, tooling for outside observers to evaluate AI impact, tracking AI adoption and its effects, maybe a "cognitive debt dashboard", "AI literacy checker", "Human-AI collaboration guardrails", "LLM output quality validator", "Cognitive offload tracker".

Let's parse discussion:

• tptacek: distinguishing social mechanism LLMs seem to have from something as simple as a "meme".
• Avicebron: contagion more accurate.
• grebc: STD outbreak maps better.
• pixl97: created new SCPs (maybe referencing SCP foundation) - implies need for containment? Maybe need for AI safety/ containment tools.
• j45: Have to take commentary by non tech folks speaking about tech with a grain of salt.
• tptacek: they're not using term virus pejoratively but distinguishing social mechanism LLMs seem to have from something as simple as a "meme".
• Avicebron: Contagion probably more accurate.
• tptacek: Well, a meme is contagious, but it doesn't do the same work a virus can at altering the structure of its host.
• Avicebron: I'm actually not too sure about that, viruses have their own mechanisms of infiltrating the host and replicating inside of it. But so do ideas.
• grebc: STD outbreak maps better.
• pixl97: Ah yes, the cognito-hazards.
• We've created new SCPs.
• j45: I would agree contangion is a much better word.
• Social Contagions can be memes, plus doom scrolling or anything.
• djeastm: It helps to have outside observers looking at the effects of a new technology and industry and not just depending on those within it to report for us.
• podgietaru: I can't disagree enough with this. Not everything needs to be understood in a technical context. Technology intersects with society, and understanding its impacts on that society is vital.
• Or in other words: the humanities are good.
• j45: I'll clarify in case it helps - need everyone at the table, not just only tech (or non-tech only) as you may be assuming.
• Technology takes without a tech perspective present can greatly skew it.
• I'm not referring to a technology context at all. Just like you shouldn't trust the IT department to run accounting... the opposite will also vary.
• Not being able to track the possibilities and capabilities of a type of software that is unlike the software before it can really outdate things quickly.
• einpoklum: > The viral analogy does not imply that LLM-human interactions are intrinsically parasitic.
• The analogy doesn't, but LLMs are trained on text, or content, generated by humans, with the aim of their output being acceptable to human interlocutors. That's one parasitic aspect. A second one is the nature and behavior of the organizational entities providing and promoting the popular LLM services: Microsoft, Google, Meta, Anthropic etc - their influence on the economy, e.g. in energy and water use and the drawing of capital/human efforts - also has a somewhat parasitic aspect.
• varispeed: > with the aim of their output being acceptable to human interlocutors.
• The more I use LLMs I find that their output drifts toward being acceptable to other LLMs. I find myself quite often telling the model: "this is not human readable.". Especially Opus and Fable write in such a way that it is difficult to follow. The often put sentences out of context, the context itself is fragmented. Often start with what you would think is end of a story so when you start reading you are like wtf it is about.
• If think the issue might be that LLMs read text all at once whereas humans are somewhat linear. So when LLM puts out text it understands, to human it is close to random gibberish that looks like English.
• ionetan: Leads me to think of a related topic, the Cost of Cognitive Debt. I think it is about time that we start to quantify what it means that we no longer grasp major parts of our systems and infrastructure. Once we quantify it, we can start to reason about how to address it.
• I explored this concept in this post: https://ljtn.github.io/epiq/blog/cost-of-cognitive-debt.html
• tylerchilds: If this is interesting, message me about my “elves”
• They are the cognitive construct I fortified in my mind and in my code before the models began producing their own training materials. My approach is based on two decades of coding without ai.
• The immunization is becoming increasingly more valuable as people begin understanding the market in which I do business.
• Suhinnall27: LLMs are both a blessing and a curse, they magnificent tools for those who know how to employ them , but a crutch for those who rely on them.
• dspillett: And like a cult to those of us preferring not to engage, for whatever reason, with everyone and their dog telling us that we should (and sometimes getting quite offended, like our refusal to play that game is a personal attack on them for choosing to).
• tosapple: they're the undead, fed by the slough of our skin.
• greed malice pestilence lust envy...
• ragequittah: ... etc.
• grebc: You can find something interesting without brow-beating people about it.
• ragequittah: You can also find something uninteresting without brow-beating people about it. Goes both ways. Just giving my experience with it, take it or leave it. The comment I was replying to seemed to wonder why people didn't understand non-engagement with the tech, I gave my reasoning. The cost of engaging is also so slim (and it's freaking everywhere) that it has to be an almost religious aversion to not, which I find quite perplexing in itself.
• bigstrat2003: > It's finally fun to be a nerd again!
• I genuinely cannot understand how you can possibly find fun in LLMs. They are poor tools that can't do the things they are claimed to be able to do, they are being shoved down the throat of the entire industry by massive FOMO, and if they ever can actually live up to the hype it's clear that we will all (every human, not just programmers) be out of jobs as the oligarchs replace us with machines who don't get tired or require a cut of the money. So they are either useless toys, or the most dystopian thing the tech industry has ever created. Where, exactly, is the fun in either of those outcomes?
• mitxela: Did you ever get to play AI Dungeon?
• mitxela: Surely this was the case in 2020-2021 when GPT-3 had just come out, not any more. Now it's big corporate business money-taking tech.
• chrisjj: And a scourge for those against whom these bots are deployed.
• CityOfThrowaway: Zero surprise this is coming from a European institution. The continent is destined to be left behind.
• canjobear: It is a collaboration across many institutions including those from Europe, the US, and Japan
• kmeisthax: I am very confused as to why LLM proponents insist that I can be left behind by a technology that takes almost 0 seconds to adopt. In the event that I realize "Oh no, I got left behind!" I can just ask an LLM to catch me up.
• CityOfThrowaway: Europe loves to regulate, which absolutely reduces your ability to adopt
• goatlover: Left behind in corruption?
• chrisjj: > The continent is destined to be left behind
• The best place to be in a rush over a cliff edge.
• qarl2: Ok... but are automobiles also a virus? Because we've become dependent on them? Should we also facilitate reversibility in our automobile usage?
• What about cooking food? We've become entirely dependent on cooking food.
• Retr0id: > Should we also facilitate reversibility in our automobile usage
• Yes?
• qarl2: That's why I added the part about food. Or heck, computers. Or telephones. Or any incredibly useful technology.
• Toothbrushes! What will we do if we come to rely on toothbrushes??? The use of toothbrushes travels from person to person exactly like a virus!!!
• pixl97: >Ok... but are automobiles also a virus?
• Yes.

• Because we've become dependent on them?

• Because we're burning the planet down with them.

• Should we also facilitate reversibility in our automobile usage?

• Yes, because in places we do people enjoy higher qualities of life and happiness. Along with things like health improvement from doing more excersize.

• What about cooking food? We've become entirely dependent on cooking food.

• Moreso we've become dependent on eating processed junk over the past 50 years and gotten fat as shit.
• So yea, maybe think a little more deeply about your analogies?

• What about cooking food? We've become entirely dependent on cooking food.

• qarl2: Don't straw man me. My question was not about processed food. My question was about cooking food.
• Is cooking food a virus? Should we facilitate reversibility in our food cooking?
• pixl97: Well, do you consider humans a virus like entity that's taken over a planet and is trying to spread to another?
• qarl: You continue to refuse to answer the question in my argument. I am not inclined to answer yours, nor do I see the point of it.
• goatlover: Not anymore than ants or any other species that's spread across the planet. A virus is something co-opts the replication machinery of a host to make more copies. That's not what humans do.
• Since this idea may have come from the first Matrix movie, it's ironic that the character who espoused the idea, Agent Smith, became an actual virus in the 2nd and third movies.
• qarl: > Agent Smith became an actual virus in the 2nd and third movies.
• OOO. I never put that together before. Very nice. Very hypocritical of him.
• FailMore: Hmm maybe it always feels "different" to every generation going through automation of something previously un-automated, but don't you feel that to some extent this is different.
• Humans have been the dominant species because of our ability to think, not our ability to get from A to B in any particular way/cook food. If we abstract ~the bulk of our thinking to LLMs, don't we lose something as a species which makes us more vulnerable... Or is it always the case during an automation?
• qarl: In cooking food we lose a great deal of our immunity to parasitic infections. Cooking food absolutely does make us more vulnerable to uncooked food.

• Or is it always the case during an automation?

• Obviously. Any time we make a useful discovery and begin using it, we become dependent on using it.
• chrisjj: > In cooking food we loose a great deal of our immunity to parasitic infections
• ... and eliminate many parasites.
• qarl: Yes. That's my point.
• pmg101: Jared Diamond explores this in his book Collapse.
• The core idea being that each time humanity was going to exceed the carrying capacity of its environment, we invented a new technology which increased the headroom. Obviously this ratchet only goes in one direction. The only way to go backwards is collapse.
• But... It's worked so far?
• camgunz: No, because walking around or eating raw food wasn't essential to being human. Thinking is.
• tripleee: I'm not sure what the point you're making is.
• We're in the midst of figuring out what the tradeoffs are of LLM usage - all the things you list are already mostly known and found worth it. You trade a tiny amount of time and fuel to cook food in exchange for not getting sick. How is that relevant to anything?
• qarl: > I'm not sure what the point you're making is.
• I'm making the point that the viral sensationalism in this article can be equally applied to automobiles, computer, telephones, calculators, and toothbrushes.
• If the article had merely been what you said "is AI worth the cost" then I would not have taken issue with it.
• canjobear: > What about cooking food? We've become entirely dependent on cooking food.
• In evolutionary time, we lost the ability to digest raw meat because we offloaded so much of process of the digestion to the cultural technology of cooking.
• Do you like the looks of a future where that happens but for thinking?
• qarl: Do you truly believe that AI will prevent people from thinking?
• It seems clear to me that people will continue to think, but in different ways.
• I hate to bring this analogy up again, but it seems impossible to escape: did losing the ability to do long division really impact humanity?
• chrisjj: > Do you truly believe that AI will prevent people from thinking?
• It is already relieving people of need for thinking. Today the need; tomorrow the ability.
• canjobear: I’ve encountered several people now who consult LLMs constantly for nearly every decision. Some proportion of people will absolutely cede their thinking to these things, and some already have.
• qarl: To be fair - I know several people for whom this would be an improvement.
• tripleee: Even if AI were superhuman at all thinking tasks, that doesn't mean we'd lose our abilities.
• Chess and games can be a lot more demanding than a lot of people's work lives - and AI is already superhuman there.
• The heaviest thing I need to lift in my daily life is a laptop and yet I've gotten significantly stronger over the last few years from strength training.
• If AI took over all the boring but hard things I'm required to do it'd free up lots of time for me to spend thinking about things, and solve problems I find more interesting.
• Murfalo: I love this line of thinking but the framing comes across to me as a bit inflammatory and uncharitable. Just about anything involved in the exchanged of ideas can be viewed as a virus! That is because, when viewed from an evolutionary biology perspective, ideas are the cognitive equivalent of genes (in fact, this is where the term meme comes from).
• There's a whole field, evolutionary memetics, dedicated to this. Cultural values, marketing, religion, social media, education, propaganda, etc. can all be viewed quite naturally through this lense. Even the terms we use in every day language (eg, going "viral") seem to intuitively grasp this.
• Is my friend a virus for recommending me their favorite book? Well, yes I guess, but that's kind of just how ideas work. Just my two cents
• cyanydeez: Memes are what we are talking about.
• IshKebab: Dawkins memes.
• Muromec: Our enlightened ideas and their dangerously agitating propaganda (literally a virus).
• The framing of something as a disease is itself slightly into the fascist territory.
• drdaeman: > The framing of something as a disease
• Is a well-known cognitive quirk, Sapolsky covered that in “Behave” and “Determined”. As I get it, our morality piggybacks on the sense of disgust from insula activation, which lights up the same way as with pathogen avoidance. So comparing social groups to rodents, insects, or disease is a popular propaganda trick. Same here with viruses.
• goatlover: Wasn't aware that memes were an accepted part of evolutionary biology. They're not inherited like genes through reproduction that are subject to natural selection via biological fitness. Or any other evolutionary mechanism.
• robotresearcher: The word was invented by Richard Dawkins, an evolutionary biologist. He described ideas as reproducing with variation under selection pressure, which is the definition of the evolutionary mechanism.
• There is a more recent use of ‘meme’ to refer to what we used to call an ‘internet image macro’. So the meme of the idea ‘meme’ has itself evolved to have a more specific variant of the general idea.
• mitxela: Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
• loose-cannon: >Memes are inherited. You brush your teeth because your parents do. Memes are subject to selection via biological fitness. The people who didn't brush their teeth died sooner of tooth disease, so those who brushed their teeth had more children. Memes mutate. You don't brush your teeth identically to your parents.
• That's parenting and the passing of tradition. I wouldn't call it a meme. Also I can't tell if you're joking with people dying due to teeth disease.
• beloch: >"We model transitions among uncoupled, coupled, and persistently dependent users, and show that the interplay between social transmission, recovery, and collective reinforcement can generate tipping points and technological lock-in."
• You can finish reading your friend's favorite book or put it down if you don't like it. Neither are you economically compelled to read it by your workplace or forced to build work on top of it that will become difficult or impossible to modify/maintain if you bin the book.
• I get that the title sounds a bit sensational, but there really is something to AI adoption beyond book-sharing.
• Upvoter33: Not sure why this is interesting; it seems like anything that becomes "popular" could be cast as a "virus", including things like "having a fridge". Why does this give particular insight?
• E-Reverance: Smart fridges are a pathological virus, yes /jk
• WJW: Even normal fridges are a memetic virus, fridges clearly live in a symbiotic relationship with humans. They make our lives better and in return we create more fridges. If they weren't useful to us, we
• Monetization: Hobby