Not below average dev necessarily, but when posting code examples on the internet people often try to get a point across. Like how do I solve X? Here is code that solves X perfectly, the rest of the code is total crap, ignore that and focus on the X part. Because it’s just an example, it doesn’t really matter. But when it’s used to train an LLM it’s all just code. It doesn’t know which parts are important and which aren’t.
And this becomes worse when small little bits of code are included in things like tutorials. That means it’s copy pasted all over the place, on forums, social media, stackoverflow etc. So it’s weighted way more heavily. And the part where the tutorial said: “Warning, this code is really bad and insecure, it’s just an example to show this one thing” gets lost in the shuffle.
Same thing when an often used pattern when using a framework gets replaced by new code where the framework does a little bit more so the same pattern isn’t needed anymore. The LLM will just continue with the old pattern, even though there’s often a good reason it got replaced (for example security issues). And if the new and old version aren’t compatible with each other, you are in for a world of hurt trying to use an LLM.
And now with AI slop flooding all of these places where they used to get their data, it just becomes worse and worse.
These are just some of the issues why using an LLM for coding is probably a really bad idea.
Yeah, once you get the LLM’s response you still have to go to the documentation to check whether it’s telling the truth and the APIs it recommends are current. You’re no better off than if you did an internet search and tried to figure out who’s giving good advice, or just fumbled your own way through the docs in the first place.
You’re no better off than if you did an internet search and tried to figure out who’s giving good advice, or just fumbled your own way through the docs in the first place.
These have their own problems ime. Often the documentation (if it exists) won’t tell you how to do something, or it’s really buried, or inaccurate. Sometimes the person posting StackOverflow answers didn’t actually try running their code, and it doesn’t run without errors. There are a lot of situations where a LLM will somehow give you better answers than these options. It’s inconsistent, and the reverse is true also, but the most efficient way to do it is to use all of these options situationally and as backups to each other.
Yes, it can be useful in leading you to look in the right place for more information, or orienting you with the basics when you’re working with a technology that’s new to you. But I think it wastes my time as often as not.
That’s implying that the quality of information from other sources is always better, but I’m saying that’s sometimes not true; when you’re trying to figure out the syntax for something, documentation and search engines have failed you, and the traditional next step would be to start contacting people or trying to find the answer in unfamiliar source code, sometimes a LLM can somehow just tell you the answer at that point and save the trouble. Of course you have to test that answer because more often than not it will just make up a fake one but that just takes a few seconds.
There are some situations I’m going back to search engines as a first option though, like error messages, LLMs seem to like to get tunnel vision on the literal topic of the error, while search results will show you an unintuitive solution to the same problem if it’s a very common one.
“Truth” implies understanding that these don’t have, and because of the underlying method the models use to generate plausible-looking responses based on training data, there is no “truth” or “lying” because they don’t actually “know” any of it.
I know this comes off probably as super pedantic, and it definitely is at least a little pedantic, but the anthropomorphism shown towards these things is half the reason they’re trusted.
Yeah, it has no notion of being truthful. But we do, so I was bringing in a human perspective there. We know what it says may be true or false, and it’s natural for us to call the former “telling the truth”, but as you say we need to be careful not to impute to the LLM any intention to tell the truth, any awareness of telling the truth, or any intention or awareness at all. All it’s doing is math that spits out words according to patterns in the training material.
Not below average dev necessarily, but when posting code examples on the internet people often try to get a point across. Like how do I solve X? Here is code that solves X perfectly, the rest of the code is total crap, ignore that and focus on the X part. Because it’s just an example, it doesn’t really matter. But when it’s used to train an LLM it’s all just code. It doesn’t know which parts are important and which aren’t.
And this becomes worse when small little bits of code are included in things like tutorials. That means it’s copy pasted all over the place, on forums, social media, stackoverflow etc. So it’s weighted way more heavily. And the part where the tutorial said: “Warning, this code is really bad and insecure, it’s just an example to show this one thing” gets lost in the shuffle.
Same thing when an often used pattern when using a framework gets replaced by new code where the framework does a little bit more so the same pattern isn’t needed anymore. The LLM will just continue with the old pattern, even though there’s often a good reason it got replaced (for example security issues). And if the new and old version aren’t compatible with each other, you are in for a world of hurt trying to use an LLM.
And now with AI slop flooding all of these places where they used to get their data, it just becomes worse and worse.
These are just some of the issues why using an LLM for coding is probably a really bad idea.
Yeah, once you get the LLM’s response you still have to go to the documentation to check whether it’s telling the truth and the APIs it recommends are current. You’re no better off than if you did an internet search and tried to figure out who’s giving good advice, or just fumbled your own way through the docs in the first place.
These have their own problems ime. Often the documentation (if it exists) won’t tell you how to do something, or it’s really buried, or inaccurate. Sometimes the person posting StackOverflow answers didn’t actually try running their code, and it doesn’t run without errors. There are a lot of situations where a LLM will somehow give you better answers than these options. It’s inconsistent, and the reverse is true also, but the most efficient way to do it is to use all of these options situationally and as backups to each other.
Yes, it can be useful in leading you to look in the right place for more information, or orienting you with the basics when you’re working with a technology that’s new to you. But I think it wastes my time as often as not.
That’s implying that the quality of information from other sources is always better, but I’m saying that’s sometimes not true; when you’re trying to figure out the syntax for something, documentation and search engines have failed you, and the traditional next step would be to start contacting people or trying to find the answer in unfamiliar source code, sometimes a LLM can somehow just tell you the answer at that point and save the trouble. Of course you have to test that answer because more often than not it will just make up a fake one but that just takes a few seconds.
There are some situations I’m going back to search engines as a first option though, like error messages, LLMs seem to like to get tunnel vision on the literal topic of the error, while search results will show you an unintuitive solution to the same problem if it’s a very common one.
“whether the output is correct or a mishmash”
“Truth” implies understanding that these don’t have, and because of the underlying method the models use to generate plausible-looking responses based on training data, there is no “truth” or “lying” because they don’t actually “know” any of it.
I know this comes off probably as super pedantic, and it definitely is at least a little pedantic, but the anthropomorphism shown towards these things is half the reason they’re trusted.
That and how much ChatGPT flatters people.
Yeah, it has no notion of being truthful. But we do, so I was bringing in a human perspective there. We know what it says may be true or false, and it’s natural for us to call the former “telling the truth”, but as you say we need to be careful not to impute to the LLM any intention to tell the truth, any awareness of telling the truth, or any intention or awareness at all. All it’s doing is math that spits out words according to patterns in the training material.
Didn’t expect this much. I don’t think about tuto example being weighted heavier. This make sense.