I’m all sceptical of AI and the hype but maybe the curl codebase is just quite secure and there are not many vulnerabilities? Not finding a bunch of things doesn’t mean the model sucks. That’s a stupid conclusion.
Daniel has been quite vocal about his views on AI slop reports, but he’s also been honest about how some AI systems have been able to identify issues in the curl code, ranging from documentation drift to actual vulnerabilities. It’s not that Mythos isn’t finding vulns. It’s that Mythos is not noticeably better at finding them than other tools (LLM or non-LLM), unlike what Anthropic are claiming.
I’m all sceptical of AI and the hype but maybe the curl codebase is just quite secure and there are not many vulnerabilities? Not finding a bunch of things doesn’t mean the model sucks. That’s a stupid conclusion.
Daniel has been quite vocal about his views on AI slop reports, but he’s also been honest about how some AI systems have been able to identify issues in the curl code, ranging from documentation drift to actual vulnerabilities. It’s not that Mythos isn’t finding vulns. It’s that Mythos is not noticeably better at finding them than other tools (LLM or non-LLM), unlike what Anthropic are claiming.