Today the International Committee of the Fourth International inaugurates Socialism AI, a transformative application of advanced technological development to the political education and mobilization of the international working class.
I have not read the article yet but i think this is a good topic to discuss here.
They can be great at it tho, it all depends on the data fed to the LLM. If someone built one with all socialist literature it could be a great tool. This specific one seems more like a front end to another model.
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the “voice” of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power…
LLMs work by hallucinating, the wild shit that gets shared isn’t an accident, it’s how they generate all their output.
people have trained models on internal document sets and it gets things wrong, they are simply not useful for facts. they don’t think, they don’t have knowledge, they just pull scrabble tiles in a clever statistical way that fools you into trusting it.
LLMs are useless for factual information
They can be great at it tho, it all depends on the data fed to the LLM. If someone built one with all socialist literature it could be a great tool. This specific one seems more like a front end to another model.
that is precisely what it is: a thin wrapper around the chatgpt API with a RAG database of all their articles
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
It might be cheap enough to make a LoRA actually. It would still probably take a week or so, but doable. :)
we have to do this actually
https://github.com/percy-raskova/marxists.org-rag-db
It is in progress, if you’d like to help develop this let me know!
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the “voice” of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power…
LLMs work by hallucinating, the wild shit that gets shared isn’t an accident, it’s how they generate all their output.
people have trained models on internal document sets and it gets things wrong, they are simply not useful for facts. they don’t think, they don’t have knowledge, they just pull scrabble tiles in a clever statistical way that fools you into trusting it.
thats a tooling/prompting/context window management problem. it can be solved with proper programming procedures and smart memory management