I have not read the article yet but i think this is a good topic to discuss here.
When people stop treating LLMs as magic and start thinking of them as lossy information retrieval tools, then it’ll be worth it.
Nothing wrong with making the massive amounts of theory we have collectively produced more easily accessible and explorable, but when you offload your material analysis to a text generator you’re bound to make grotesque mistakes.
Though I would’ve be much happier with that effort being put into making marxists.org more accessible instead, reducing the dependence on external search engines.
When people stop treating LLMs as magic and start thinking of them as lossy information retrieval tools, then it’ll be worth it.
^this. Very much this (well for westerners anyway)
Good point! Thanks for sharing it.
https://github.com/percy-raskova/marxists.org-rag-db
one step ahead of you
This looks awesome (starred)!
Could I ask what did you have to self-teach to be able to build/code that? I am a layman when it comes to this; looking to expand my skillset over the next few years (though I may not be able to do what you just did I would like to work towards it).
Nice!
Yeah I’m actually in the process of fine-turning and developing an actual AI
the trotskyists just use a wrapper around vanilla Chat GPT and give it a database of Trotskyist articles lmfao
vanilla chat gpt is free btw
I mean sure its free but it is also a privacy nightmare so you couldnt pay me to use it or to tell comrades to use it. :)
Trot garbage.
Trotskyist AI
Damn. I had to write down my thoughts to see how this is a problem.
The trot’ doctrine is doing nothing and claiming moral victory over the issue.
doing nothing and claiming moral victory over the issue
That’s the Trotskyism is all about comrade
Communist-leaning LLMs could have some value but the way they are hyping this is really weird. At best it will be a little bit helpful but they talk like it will be a major breakthrough for socialism at large. This article itself did not need to be this long and is seasoned liberally with communist jargon. They also don’t disclose some important technical details like who is the vendor, is it based on an open weight model etc.
It is also a trotskyist AI so…
everything trots print out is a great victoy for the workers and socialism lol
its literally just them doing the whole 'selling newspapers" thing but with generative AI
I think this is a great idea personally. If we don’t like the biases found in current models, then the way to address that is to build our own models that embed Marxist biases. And while LLMs are not reliable for factual information, they can be a great way to explore ideas and to stimulate thinking. If you can have a discussion with an LLM trained explicitly on Marxist material, then it will help you develop dialectical way to approach things. It doesn’t need to be completely accurate because the goal is to develop a way of thinking as opposed to learn specific facts.
good idea in theory, sucks that trots were the first to get to it lol. I also wonder what they use under the hood, if it’s just a system prompt being injected into a commercial model or they fine-tuned a model or what. Dozens of different ways one could make this LLM with varying amount of effort and resources needed for varying results. I doubt they trained an entire model from scratch because you still need more than their 250,000 WSWS articles they put in it (which is also just wild to think about that they have 250k articles over 25 years. Do Trots ever only write stuff? Apparently so.)
We have given out ProleWiki content (as txt files) to a dev to make it available for RAG on github, which is basically a compendium of data the LLM looks at before answering. We have no reason to paywall or restrict the use of our content, so it was only natural to start making these - with the understanding that they make the data available to everyone of course, so that it’s ready to use for anyone who wants to try their hand with it (over 5000 PW pages + library books we rehost). They’re currently working through it to format it for LLM use since you don’t just give it the raw text files, and previously they’d also worked on porting the MIA in the same way to a github repo (https://github.com/percy-raskova/marxists.org-rag-db). Oh they also said it was easier to work with the heavily-formatted mediawiki data over the plain html non marked-up data from MIA hah. Even our library books are formatted with clear reference tags and heading/section tags. They also contain metadata in a clearly formatted infobox (on purpose, not every wiki does this) which also allows the llm to natively understand the metadata it’s looking at.
What could come out of it though I’m not sure. That is to say I’m not sure if the best thing you can do with a socialist llm is the novelty of “wow marx chatbot” like wsws seems to be doing. I could definitely see it being queried on much higher-level questions like presenting it with your current material conditions and asking what is the best party form for it based on theory, submitting your organizational ideas to it and having it challenge them with theory. But for that it needs to have a vast corpus of texts available, and definitely MIA + ProleWiki’s current library is far from enough. MIA has a lot of works but they don’t republish if it’s not fair use or from living authors. ProleWiki republishes everything, but we don’t have a lot of works compared to all that exists.
edit: damn they’re just yapping in that press release. get to the point lol.
Sidenote: PW is a goldmine when it comes to ML theory. I’m proud to have helped a little and will most likely be helping more in the future.
The LLM idea is much more versatile than people think. Your examples would already mean quantum leaps in ML agitation and helping people understand without accepting revisionism or something. its really great. my additional idea is something like grok but for MLs so as to get it to cite theory or analysis on social media (mastodon for example).
My personal wet dream is having the individual authors read their works in all languages in color, in their voice. People tend to underestimate the power of personality that comes with movement and language. Stalin at the drums and lenin at the microphone was really a sight to behold (available on red creators network on youtube) although it was of course not authentic as lenin was not a singer afaik. The emotions it triggers are next level. Especially since there are video and audio files of stalin, i would love to have some more of the OGs that gives the incredible theoretical prowess the well deserved personality that people nowadays definitely need.
I’m kind of wary about cult of personality, even among our heroes. I’m constantly reminding myself (after forgetting for a little bit, usually lol) that heros are human, with human falliabilities, partly as a reality check, partly as innoculation against disillusionment with a whole, rather than individual quirks, I think.
I understand that. But after reading a lot of theory, marx, engels, lenin and stalin are amongst the greatest heroes that ever lived, objectively. Not because i count everything that happened while they were in power but by the theory they wrote that will forever help people survive, liberate, change, cooperate, etc. and even proved them to be true. Its the thanos “okay i’m gonna do it myself” meme in a way. Of course they werent alone, they never acted alone, materialism even suggests they did not really have a chance but to write it but i choose to hold them in high regard, place pictures of them at my home, etc. to remind me every day what I aspire to achieve. You can of course do differently or plain disagree on that idea but i dont think this constitutes a “cult”. Its plain admiration of people who managed to do the thing. :)
And there are of course others who deserve admiration. Its also part of my personality to have mentors which helps me learn faster through mimmicking. The important distinction is to basically get off at the correct stop (talking in trains since i’m riding a train atm) and not loosing the bigger picture for plain parasocial relationship.
I’m saying there’s a fine line between admiration and hero worship; and that I cross it too often. Ymmv
We’ve looked at integrating TTS natively onto our pages previously but it’s not great. Firefox had an API to use the rudimentary pre-installed voice packages but apparently they are deprecating it and it’s unsure if it’ll work for everyone.
And of course since we change the content sometimes, having a static TTS file means we need to recreate and reupload it every time we make an edit. Very costly if we start looking at AI TTS voices, unless we are able to start running some on our server locally.
You could totally have a choice of voices such as Lenin, Stalin, anyone for whom we have voice recordings basically lol. And yes neural networks are especially good at TTS and even different languages, it’s one of the first things they perfected - it’s pretty surreal listening to yourself speak perfect Chinese or Danish in your own voice and shows that we are not so different after all from each other.
The first time i saw this was when watching cpt traore speak perfect english and only later discovering that he does not speak english, at the very least at that level.
That was when I first thought “this might actually help us A LOT.”
But the most cruel thing will be when you’re discussing something on grad and you can just do cite and !stalin to have an llm use original works of stalin to analyze the quote for its truthfulness and theoretical quality.
Edit: or maybe just @stalin to get the bot to respond. That would be awesome.
It’s 100% a system prompt on a commercially available model/API - it even uses the “sign in with Google” option (lol)
lmao
edit: okay I’m trying to dig more into how they provide this service and the terms of service are pretty funny.
Left Insight LLC does not return any results online (I might try to scan deeper later). They probably set it up to own the chatbot.
There’s a subscription because of course there is. Greatest victory for the workers and socialism but also pay us to unlock this victory.
Lengthy intellectual property bit. Can’t use the “chatbot” to compete against wsws was a good one.
This doesn’t hold up in US court where they operate from. They can make licences but nobody owns the output of an LLM. Also more funny copyright bit. The trotskyist revolution will not be televized because the workers will not have paid the TV fees.
I also wanted to look at their html to see if I could find more information there such as interface used or maybe even a comment but it’s a huge one-line block that I can’t lint. I threw deepseek at it, will report when it’s done (oh no that counts as reverse-engineering they will terminate my non-existent account with them!)
New comment for clarity, crush is still crushing through the problem (hah) but it has found some cool stuff already. Their homepage calls json files which contain the models they offer, and we see there is at least GPT4.1 and Deepseek. No word yet on how they use these models exactly but since GPT is closed-source they have to be contacting the service and forwarding the requests, can’t run it yourself.
Will update as it works through the tasks.
edit: so we can only look at the JSON and JS files their homepage calls (don’t really feel like making an account with them) and there is some interesting info. Seems to simply be RAG from the way the service parses the data (there’s a file with the steps it will show to the user on the interface).
No fine-tuning or running their own model and BECAUSE OPENAI IS PROPRIETARY, everything you send through the service ends up with OAI.
hey! i’m the developer of the Marxist RAG and the ProleWIki RAG so if you have any questions/ideas/suggestions/want to participate reply and let me know!!!
❤️ thank you
Socialism with AI characteristics…
Anyway, I would like to see a counter hegemony is getting stronger but LLM isn’t what I was hoping for. For example, Instead of CIApedia, Prolewiki should be mainstream to explain ML perspective. I used it a lot when I explain myself why Russia is getting critical support in Russo-Ukraine war.
LLM might lose information on the way. It doesn’t understand text. It doesn’t know the truth. It doesn’t reason (in the human sense). This might be creating a dogmatic tendency amongst the new comrades.
heh, i’ve told deepseek to respond to me with a historical materialist, dialectical materialist, marxist leninist pov and they’ve done very well, i dont know if this is necessary and if they have the resources
sounds good. I think it would be more efficient to have a specifically trained llm though. i’m looking into training a pre trained llm on ML works. no idea how long this will take but it sounds like a good idea.
i’m actually working on this! https://github.com/percy-raskova/pw-mcp
Nice! Though I wonder if you’d get better performance using LoRA fine tuning?
Awesome. I’ll have a look. Thanks for mentioning it.
Ask it about Kevin Spacey or Russel Brand
If done well, I think LLMs could be a pretty decent entry point for newbies. They don’t need 100% perfect dialectical analysis, they just need to be pointed vaguely in the right direction. As long as you can train it to emphasize the importance of reading actual theory rather than just relying it on information, it wouldn’t necessarily be a bad thing.
they just need to be pointed vaguely in the right direction.
And that’s really important.
I tried to see the feasability of using local LLMs to sift through news/propaganda to make me a personalized report, and perhaps even send alerts of important things, so I tried it with something related to what’s going on in Ukraine and wow was it bad.
Anything Russia said was immediatelly false and whatever Ukraine said was fact, no questions asked. It couldn’t even be asked to summarize things without just repeating propaganda that wasn’t even in context.
And that was a Chinese open-weight LLM as well.
So yeah, just having LLMs that don’t repeat propaganda blindly would be really helpfull, and if the LLM had a Marxist-Leninist background and gave a lot of weight to such views, perhaps through training using real and synthetic discussions from such a PoV, would make a lot of things possible.
Oh yeah, even the Chinese LLMs are pretty lib because the most of the internet (especially the English part) is dominated by liberal content, and the model makers (the Chinese ones, I mean) optimise them for coding or math performance without caring much about ideology.
There is definitely an opportunity for someone to fine-tune it with Marxist content, but the prerequisite for this is to build large enough datasets with the right information and analysis. This is why stuff like Prolewiki is so important.
Exactly, and them being interactive is a really valuable aspect. It lets people explore things in a way that makes most sense to them, and to pull on the threads they want to pull on easily.
True AI? Maybe, depends on how it’s designed.
LLM? Even Trots probably wouldn’t train it in a particularly wasteful or environmentally damaging way. If it works well could help people get a beginner socialist education (even Trots can manage that much). If it doesn’t, then it wastes the time of Trots. Might be a win/win.
LLMs are useless for factual information
They can be great at it tho, it all depends on the data fed to the LLM. If someone built one with all socialist literature it could be a great tool. This specific one seems more like a front end to another model.
that is precisely what it is: a thin wrapper around the chatgpt API with a RAG database of all their articles
It would probably be wisest to use a pretrained model and feed it the prolewiki with particular focus on marxist leninist writers as to give it the correct bias.
mixture-of-experts but the experts are the five heads.
this is technically possible already, just that it takes so much resources that not many people could make this model and the cost would be prohibitive for what is a hobby project. This is why China is developing forces of production first.
It might be cheap enough to make a LoRA actually. It would still probably take a week or so, but doable. :)
we have to do this actually
https://github.com/percy-raskova/marxists.org-rag-db
It is in progress, if you’d like to help develop this let me know!
i love this idea but i must be honest: fine-tuning a mistral model to specifically embed the weights with the “voice” of Marx Engels Lenin Stalin and Mao would be an immense machine learning and engineering task. Now Abliterated DeepSeek 7B (abliterated means uncensored) = true power…
LLMs work by hallucinating, the wild shit that gets shared isn’t an accident, it’s how they generate all their output.
people have trained models on internal document sets and it gets things wrong, they are simply not useful for facts. they don’t think, they don’t have knowledge, they just pull scrabble tiles in a clever statistical way that fools you into trusting it.
thats a tooling/prompting/context window management problem. it can be solved with proper programming procedures and smart memory management
btw
https://github.com/Kimonarrow/ChatGPT-4o-Jailbreak
knock yourself out! jailbreak their AI and go wild
