- cross-posted to:
- [email protected]
- cross-posted to:
- [email protected]
Four months ago, we asked Are LLMs making Stack Overflow irrelevant? Data at the time suggested that the answer is likely “yes:”
Four months ago, we asked Are LLMs making Stack Overflow irrelevant? Data at the time suggested that the answer is likely “yes:”
Problem being that someone else asked the question 10 years ago and the answer is now irrelevant due to version changes. People with high scores are just early adopters who answered all of the easy questions. Hostile users generally can’t understand the question. The issue with llms answering your question is that they are going to be stuck in the current time period. In the future their answers will also be irrelevant due to version changes.
Earlier today I googled how to toggle full screen in dosbox-x and the AI-generated answer said to use alt+enter. Tried it and it didn’t work, so I look in the documentation and it turns out that they changed it to F12+f a while ago (probably to avoid interfering with actual dos input).
This is definitely already a problem.
Every LLM is shit at dealing with version changes. They don’t understand it as a concept, despite all their training data.
I mean that is already a problem, if you ask a question you have to be ready for the answer to be a mismatch of version conflicts.
But that is ok. ChatGPT is a tool that can either help you or hurt you. I like to think of it like a power hammer. If you are doing a roofing job, it can help you get things done faster compared to a manual hammer, but you still need to know how to build a roof to get started.
ChatGPT is great at helping you organize your thoughts or finding an answer to some error message buried in some log file, but you still need to know what questions to ask and you need to be ready for it to give you a stupid answer and how to get around that.