to lie about the results and to generate more anxiety in others to keep up with your made-up achievements
Emperor’s New Clothes style “we all need to pretend to like it” is an unfortunately common effect of decision-makers deciding they know some brilliant thing and any naysayers just aren’t suited to appreciate the brilliant thing.
I think at least some of the wasteful or even harmful ways you describe of using LLMs come from this push to use it and “be more productive” with it.
Some, sure.
Others from a fundamental misunderstanding of the nature of language models. They’re text processors and generators designed to sound human. They can’t tell facts from filler.
Just earlier, I saw a post elsewhere about someone having generated an article or something which cited three experts – wrongly, because it doesn’t actually know what the relation between the text in quotes and some supposed source is or why it needs to be verbatim to be a correct quote. That’s not a bug, nor a hallucination or whatever anthropomorphic euphemism people come up with for “random output happened to be wrong” (though, to be fair, “random” glosses over a highly complex prediction system that can predict plausible text quite impressively, even if it can’t predict truth).
Students relying LLMs to generate their coursework are falling into that trap without any pressure of productivity. They don’t get that the purpose of coursework is to learn about the source material and the structure of academic writing rather than just produce text. They also don’t get that the LLM won’t look up, interpret and cite sources accurately in accordance with the subject of the question. It will generate a plausible-sounding answer to the question, and therein lies the danger: If you don’t already know the answer, how could you tell if it’s true?
The same goes with people “looking up” information. Gemini will produce some text statistically correlated to the text it has read, but you never know whether that correlation reflects facts or whether it falsely attributes some shady business to companies who had nothing to do with it (about which there was a court case in Germany recently).
Vibe coders without programming experience cannot qualify the output of their generator. It’s always harder to understand code you didn’t write (or maybe wrote long ago), but if you don’t even know how to write code, you’ll have no experience to compare it to.
The common thread behind these is that these AIs lack the understanding of the concepts they’re producing text about and semantic connections between them, and accordingly cannot treat these things with the same nuance and precision that humans can.
But the ways they’re harmful doesn’t immediately become apparent. “Report where it’s harmful” doesn’t really work if it takes two years for a critical security flaw to surface that some code generator produced and nobody with experience caught. You may never notice your ability to deal with stress being eroded until some day you can’t ask your robot buddy for help and just crack instead.
They plant traps in your education, your knowledge, your work, your psyche. To encourage people to use them without thoroughly preparing them for those traps is reckless.
Emperor’s New Clothes style “we all need to pretend to like it” is an unfortunately common effect of decision-makers deciding they know some brilliant thing and any naysayers just aren’t suited to appreciate the brilliant thing.
Some, sure.
Others from a fundamental misunderstanding of the nature of language models. They’re text processors and generators designed to sound human. They can’t tell facts from filler.
Just earlier, I saw a post elsewhere about someone having generated an article or something which cited three experts – wrongly, because it doesn’t actually know what the relation between the text in quotes and some supposed source is or why it needs to be verbatim to be a correct quote. That’s not a bug, nor a hallucination or whatever anthropomorphic euphemism people come up with for “random output happened to be wrong” (though, to be fair, “random” glosses over a highly complex prediction system that can predict plausible text quite impressively, even if it can’t predict truth).
Students relying LLMs to generate their coursework are falling into that trap without any pressure of productivity. They don’t get that the purpose of coursework is to learn about the source material and the structure of academic writing rather than just produce text. They also don’t get that the LLM won’t look up, interpret and cite sources accurately in accordance with the subject of the question. It will generate a plausible-sounding answer to the question, and therein lies the danger: If you don’t already know the answer, how could you tell if it’s true?
The same goes with people “looking up” information. Gemini will produce some text statistically correlated to the text it has read, but you never know whether that correlation reflects facts or whether it falsely attributes some shady business to companies who had nothing to do with it (about which there was a court case in Germany recently).
Vibe coders without programming experience cannot qualify the output of their generator. It’s always harder to understand code you didn’t write (or maybe wrote long ago), but if you don’t even know how to write code, you’ll have no experience to compare it to.
People using AI for coping with stress may run into a trap where they end up unlearning to cope on their own and potentially take on even more stress.
The common thread behind these is that these AIs lack the understanding of the concepts they’re producing text about and semantic connections between them, and accordingly cannot treat these things with the same nuance and precision that humans can.
But the ways they’re harmful doesn’t immediately become apparent. “Report where it’s harmful” doesn’t really work if it takes two years for a critical security flaw to surface that some code generator produced and nobody with experience caught. You may never notice your ability to deal with stress being eroded until some day you can’t ask your robot buddy for help and just crack instead.
They plant traps in your education, your knowledge, your work, your psyche. To encourage people to use them without thoroughly preparing them for those traps is reckless.