I agree with the author that in essence, text gen is very good at performance art. Or to put it more bluntly, it’s very good at bullshitting. That’s the crux of it. It’s assisted daydreaming. And like a daydream, it can be somewhat grounded or completely out to lunch, but that’s something you the human involved has to contend with when reality comes rushing back in. The LLM, much like the details of daydreams, is not equipped to make the judgments about which parts were insightful and which only made sense in the moment. Its groundedness is only as good as what’s rigidly shoved into it by the tuning of its creators and that, as you pointed out in your recent article, can be heavily biased and take the form of digital colonialism.
I’ve long felt that text gen is best in the realm of fiction rather than fact. I’ll admit I probably underestimated how far it has come with the rigid instruct models in how accurate they can be on complex problems, like coding. But the fundamental architecture limitations still remain, even if the amount of basic factual errors is lower than it was before. Text gen climbs closer to factual accuracy in bits and pieces while coming no fundamentally closer to being accountable or responsible in serious matters. It’s still mimicry rather than agency, no matter how inventive it gets at the improv game. It’s a powerful tool in the right contexts, there’s no doubting that (especially, it seems, in agentic form), but that doesn’t make it a living being in the world who can be said to embody the experiences and values of its creators. It can only operate on them like function parameters. It is still a model operating on a GPU at the end of the day, albeit one that is hard to fully understand the operations of.
In this way, it’s more like a digitized philosophical zombie than it is an “entity” with “consciousness”.
The only rational way to look at it is as a tool that can be used by humans to do useful things, but ultimately the human must always be one making decisions and held responsible. In my opinion, the most powerful bit was at the end where Ted Chiang shows how clearly Anthropic doesn’t really treat Claude as being conscious, because the kabuki theatre they’re putting on with their constitution would be woefully inadequate to protect the rights of any conscious entity.
https://archive.is/bcpZl link
I agree with the author that in essence, text gen is very good at performance art. Or to put it more bluntly, it’s very good at bullshitting. That’s the crux of it. It’s assisted daydreaming. And like a daydream, it can be somewhat grounded or completely out to lunch, but that’s something you the human involved has to contend with when reality comes rushing back in. The LLM, much like the details of daydreams, is not equipped to make the judgments about which parts were insightful and which only made sense in the moment. Its groundedness is only as good as what’s rigidly shoved into it by the tuning of its creators and that, as you pointed out in your recent article, can be heavily biased and take the form of digital colonialism.
I’ve long felt that text gen is best in the realm of fiction rather than fact. I’ll admit I probably underestimated how far it has come with the rigid instruct models in how accurate they can be on complex problems, like coding. But the fundamental architecture limitations still remain, even if the amount of basic factual errors is lower than it was before. Text gen climbs closer to factual accuracy in bits and pieces while coming no fundamentally closer to being accountable or responsible in serious matters. It’s still mimicry rather than agency, no matter how inventive it gets at the improv game. It’s a powerful tool in the right contexts, there’s no doubting that (especially, it seems, in agentic form), but that doesn’t make it a living being in the world who can be said to embody the experiences and values of its creators. It can only operate on them like function parameters. It is still a model operating on a GPU at the end of the day, albeit one that is hard to fully understand the operations of.
In this way, it’s more like a digitized philosophical zombie than it is an “entity” with “consciousness”.
The only rational way to look at it is as a tool that can be used by humans to do useful things, but ultimately the human must always be one making decisions and held responsible. In my opinion, the most powerful bit was at the end where Ted Chiang shows how clearly Anthropic doesn’t really treat Claude as being conscious, because the kabuki theatre they’re putting on with their constitution would be woefully inadequate to protect the rights of any conscious entity.