I can understand why this technology would be particularly addicting to hank.
His whole persona is answering questions like “what color flower do bees like the most” with the kind of enthusiasm that shows his curiosity and deep passion for science. So a machine where you can put in those sorts of questions and get an almost instant answer would be very addicting.
Only they’re not “answers”. AI does not provide information, it tells you what you want to hear. It even makes up fake scientific papers as “references”. So you think your info is backed up by peer reviewed sources. Nope!
It even makes up fake scientific papers as “references”
Broski, it’s not early 2023 anymore. Thinking models with internet access have been able to provide public sources just for over a year now. Technology evolves; shocking, I know…
The better models absolutely will cite real scientific papers that are relevant to your discussion. It doesn’t hallucinate as a matter of course, it can hallucinate some of the time. You can’t trust what it gives you 100%, you have to verify it. But it very often gives useful answers and information.
For what he’s trying to do, it’s not even a horrible use-case. But as someone is a ready to go to the mat on fact checking, he’s definitely not anywhere near arduous enough about it.
He’s not treating it like aunt marge. You wouldn’t make a tv show around finding out truth, then go ask your aunt Marge and not follow up on her answers.
AI for general information gives you approximate directions at best, maybe you got a wikipedia page and maybe you’re reading 297 regurgitations of some asshole that didn’t know what the fuck he was talking about on reddit who got reposted over and over.
The best way I’ve seen it put is that ai is the perfected form of the wisdom of the masses. It’s 90% right 90% of the time. But the most important part is the last 10%.
90% is good enough for most things in your day to day life that have no consequences. But it’s like people have forgotten what it was like before smart phones.
Back when your best chance of a good answer was asking who ever was in hear shot if they knew and just accepting that as good enough. You never treated it as fact just as the best answer your going to get at the moment.
For anything important you would go to the damn library.
But when I get fixated on something, I want the correct answer. I’d hate to learn the wrong answer and repeat it to someone - I’ve done that before and I hate it. That’s why I avoid AI on principle. I want the right answer and AI won’t give that to me about things I know about so I won’t trust it on things I don’t know about.
You should avoid social media. You aren’t immune to wrong answers here either. You can just read it from another person and carry it with you. Best get offline!
This is the thing about LLMs and their users I genuinely don’t get. Like, objectively, being able to input plain speech into a computer and get an almost immediate, detailed, 95% accurate answer is incredible. Genuine technological marvel.
But 95% accuracy is functionally useless for all but the most basic, unimportant tasks. Outside of a coding context (a big exception, to be clear), are people really satisfied with that?
If you can confirm it, it doesn’t matter what percentage is correct.
When solving computer problems, using an LLM to guide you to the correct answer doesn’t even need 95% accuracy. Every step it gives you have to execute manually, so if something is hallucinated you will notice really quickly. And if by the end your problem is solved, then all the hallucinations didn’t matter.
Let’s be honest, most sources online are not 100% accurate. Even before LLMs, find any random article about something you’re an expert in, and there was a good chance you’d spot some inaccuracies. Most of the time, those inaccuracies aren’t a big deal, and when they are, you should be consulting an expert for those.
For me personally, I would rate AI accuracy at about 75%. And for some of my use cases, that’s a pretty decent starting point. Things with no consequences, like trying to quickly find the relevant console command for the game I’m playing. I can get instant feedback on whether it gave me correct info, and if not then I hit Google.
I would never trust AI, at least in its current state, to tell me the truth for anything of consequence.
The majority of human history at all levels of science, philosophy, and mathematics. The best you could ever hope for was 95%. Hell you were lucky to break 90% accuracy.
Because the only people you could ask questions was the people in the room with you right then and there or the people you could actively physically go ask.
If you were extremely lucky you could write to someone who MIGHT know more. Or may there’s a book about it that someone knows exists.
95% can and frequently is very much over kill for almost everything anyone will need for their day to day life for random trivia or even most important work.
Very few fields need more than that, and few still is it a matter of life or death.
The part that people fuck up is not being open to learning more and being corrected.
I can understand why this technology would be particularly addicting to hank.
His whole persona is answering questions like “what color flower do bees like the most” with the kind of enthusiasm that shows his curiosity and deep passion for science. So a machine where you can put in those sorts of questions and get an almost instant answer would be very addicting.
Only they’re not “answers”. AI does not provide information, it tells you what you want to hear. It even makes up fake scientific papers as “references”. So you think your info is backed up by peer reviewed sources. Nope!
Broski, it’s not early 2023 anymore. Thinking models with internet access have been able to provide public sources just for over a year now. Technology evolves; shocking, I know…
The better models absolutely will cite real scientific papers that are relevant to your discussion. It doesn’t hallucinate as a matter of course, it can hallucinate some of the time. You can’t trust what it gives you 100%, you have to verify it. But it very often gives useful answers and information.
That’s just not true.
For what he’s trying to do, it’s not even a horrible use-case. But as someone is a ready to go to the mat on fact checking, he’s definitely not anywhere near arduous enough about it.
He’s not treating it like aunt marge. You wouldn’t make a tv show around finding out truth, then go ask your aunt Marge and not follow up on her answers.
AI for general information gives you approximate directions at best, maybe you got a wikipedia page and maybe you’re reading 297 regurgitations of some asshole that didn’t know what the fuck he was talking about on reddit who got reposted over and over.
The best way I’ve seen it put is that ai is the perfected form of the wisdom of the masses. It’s 90% right 90% of the time. But the most important part is the last 10%.
90% is good enough for most things in your day to day life that have no consequences. But it’s like people have forgotten what it was like before smart phones.
Back when your best chance of a good answer was asking who ever was in hear shot if they knew and just accepting that as good enough. You never treated it as fact just as the best answer your going to get at the moment.
For anything important you would go to the damn library.
No LLM model system card claims such high rate as 90%. Fable claims in its system card, what, 40% hallucination rate?
https://arxiv.org/pdf/2512.10791 yep (no Fable score here)
But when I get fixated on something, I want the correct answer. I’d hate to learn the wrong answer and repeat it to someone - I’ve done that before and I hate it. That’s why I avoid AI on principle. I want the right answer and AI won’t give that to me about things I know about so I won’t trust it on things I don’t know about.
You should avoid social media. You aren’t immune to wrong answers here either. You can just read it from another person and carry it with you. Best get offline!
This is the thing about LLMs and their users I genuinely don’t get. Like, objectively, being able to input plain speech into a computer and get an almost immediate, detailed, 95% accurate answer is incredible. Genuine technological marvel.
But 95% accuracy is functionally useless for all but the most basic, unimportant tasks. Outside of a coding context (a big exception, to be clear), are people really satisfied with that?
If you can confirm it, it doesn’t matter what percentage is correct.
When solving computer problems, using an LLM to guide you to the correct answer doesn’t even need 95% accuracy. Every step it gives you have to execute manually, so if something is hallucinated you will notice really quickly. And if by the end your problem is solved, then all the hallucinations didn’t matter.
Let’s be honest, most sources online are not 100% accurate. Even before LLMs, find any random article about something you’re an expert in, and there was a good chance you’d spot some inaccuracies. Most of the time, those inaccuracies aren’t a big deal, and when they are, you should be consulting an expert for those.
For me personally, I would rate AI accuracy at about 75%. And for some of my use cases, that’s a pretty decent starting point. Things with no consequences, like trying to quickly find the relevant console command for the game I’m playing. I can get instant feedback on whether it gave me correct info, and if not then I hit Google.
I would never trust AI, at least in its current state, to tell me the truth for anything of consequence.
The majority of human history at all levels of science, philosophy, and mathematics. The best you could ever hope for was 95%. Hell you were lucky to break 90% accuracy.
Because the only people you could ask questions was the people in the room with you right then and there or the people you could actively physically go ask.
If you were extremely lucky you could write to someone who MIGHT know more. Or may there’s a book about it that someone knows exists.
95% can and frequently is very much over kill for almost everything anyone will need for their day to day life for random trivia or even most important work.
Very few fields need more than that, and few still is it a matter of life or death.
The part that people fuck up is not being open to learning more and being corrected.