Today the International Committee of the Fourth International inaugurates Socialism AI, a transformative application of advanced technological development to the political education and mobilization of the international working class.
I have not read the article yet but i think this is a good topic to discuss here.
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the “voice” of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power…
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
It might be cheap enough to make a LoRA actually. It would still probably take a week or so, but doable. :)
we have to do this actually
https://github.com/percy-raskova/marxists.org-rag-db
It is in progress, if you’d like to help develop this let me know!
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the “voice” of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power…