And it’s only going to get faster…

(Note that this is the Q4 model, but otherwise still the full 671b params and thinking mode enabled)

https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/ it’s even better, this rig doesn’t even have a GPU.

it’s not any better than perplexity if you want to use it as a search engine in my experience. but to do tech stuff like debugging it works

I understand where you’re coming from, but I think there might be some misconceptions about the resource requirements. You can actually host LLMs on a local computer without needing a $10,000 GPU. For example, it’s possible to self-host the full Deepseek model on a $2000 setup and open it to your organization for browser-based use, or smaller models on a 400$ GPU.

I also find it compelling that LLMs like Deepseek are designed to be very efficient in their cloud versions, especially when compared to Western tech that isn’t incentivized to prioritize environmental concerns because there are no mechanisms in place to force them to care about the environment. This (the fact that capitalism won’t save the environment) is a much stronger argument than a blanket “no datacenters,” since a datacenter is powering Lemmygrad as we speak. To put it in perspective, China has about 450 datacenters while the US has over 4000, yet their tech sector is just as advanced. It shows there are different, more efficient ways of doing things that we (the state) can tap into if we only wanted to.

This also seems like it could erode trust in Communist organizations

To be perfectly honest, I think you overestimate the existing level of trust the general masses have in communist organizations.

I’m coming from a place of wanting our movements to succeed globally, it’s just that it worries me when I see us hesitating to adopt tools that could give us a real edge. We already use technology, including the internet and automated stuff in our organizational work. I believe we need to move past a certain hesitation toward new tech (a sort of “return to Pravda” mindset) and embrace whatever makes our praxis more effective. We don’t have the luxury of refusing efficient tools. Looking at how China integrates technology provides a practical, existing blueprint for this.

I’ve often seen proposals to automate tasks or improve efficiency in orgs get shut down with responses like, “Oh, that sounds complicated,” or “I like the way we do things already.” But we have to try new things if we want to close the gap. I’d be happy to help build out a tech stack if given the chance! And yet many still prefer to rely on manual email lists when a simple Telegram channel could coordinate communication.

It’s a bit like how the MIA gained its foothold in the 90s while other communists were still debating whether the internet was a fad. We got shown up by trots!

Just recently, we launched a Telegram broadcast channel with ProleWiki to share news. It’s only the first week, and we already have 80 subscribers. That’s 80 people we can reach directly, without being subject to algorithmic filters. The bot for the channel was coded by our dev with some LLM assistance, it uses RSS feeds and custom filters to select the headlines we want and posts them automatically on a schedule. Eventually, we might use something like Deepseek to scrape sites that no longer offer RSS, and maybe even analyze the articles for relevance before posting. At this moment the channel runs automatically, it requires literally 0 labor to sustain. I’m not aware of any org that have a low-stakes, public-facing point of entry like this. They seem to assume that the more labor they put into something the greater its impact and this results in a lot of wasted effort. This automated approach lets us maintain a presence with 0 effort while freeing up energy for other things we want to work on. It’s essentially self-sustaining, I mean, how cool is that!

perpetuate a surveillance state

I mean by many metrics China would be considered a surveillance state (and not just liberal metrics). They have a different cultural and legal approach to online privacy and device security, in fact researchers that work in China like that accessing data, even medical data, is more straightforward there. Our distrust should be directed at capitalism, not the ‘surveillance’ itself.

deepseek cloud tbh. 5$ on the API gets you around 9 million words (input tokens are half the price). I have no idea what I’m gonna do with that amount of tokens but I’m probably going to be riding that 5$ for years to come lol. I also like that they don’t have auto billing and you have to top up your credit balance manually.

deepseek also has a 128,000 tokens context window which is just huge, that’s like 100k words. You could basically send deepseek a whole novel (60k words) and it will still have 30k words with which to write an output. But due to how context currently works in llms it will probably get confused or completely forget some parts of it, so I wouldn’t recommend doing that. But compare to chatGPT which gets lost and automatically cuts off your prompt after 3 paragraphs.

However don’t think deepseek is secure. Your data is still stored on your account and has to transit over the open web even if it ends up in servers in China. With local LLMs you can set it to delete chat history as soon as you close the program, so once it’s generated, it’s gone.

As for some uses,

• translation of theory that was previously not available in X language. With a script and API access you could just automate this and pump out translations to 10 languages as quickly as you find the books. Now that I have some deepseek API access I might actually work on a pipeline for this.
• producing agitprop, incl for example images you can quickly print on flyers prior to a protest or event, or stickers. These are flyers that will often be discarded anyway so the image doesn’t need to be great. But I’m thinking for example of the Samuel L. Jackson edit (the L stands for Lenin), people loved it when it came out. It’s one of the top-rated post on r/AIart which is just wild to think about, it’s also the only political post there. We just need to find the right idea and then AI can execute.
• coding, of course. any stuff that you need to automate for your org or project, deepseek can probably code
• generating questions to chapter you are reading in book club (then go through a process of perfecting them every time you run the book club and eventually you’ll have a standard set of questions that you serve every time you run the book. In fact I kinda want to do that for ProleWiki and make them accessible so orgs don’t have to make their own individually)
• submit drafts to steelman them prior to release, though we might want a fine-tuned model for that. Can also make sure language works and will speak to people but again I’m not sure llms are quite there yet. Maybe if you give it a persona whom you want the draft to speak to. But marxism being so specific and developed I don’t think llms can quite grasp it at the level veteran comrades do.
• Create content cascade for your online agitation work. Submit your argument about a topical issue to the LLM and ask it to derive it into a tweet, an instagram story, a bullet-point list for a flyer etc. imo this is something we don’t do enough of, there’s no need to reinvent the wheel each time and we need automated tech stacks to handle this work.
• deepseek gave me a surprising observation considering it comes from an LLM, “Problem: The right-wing dominates online meme culture, which shapes common sense.” even it is aware of this lol. It also offered this example solution “Generate an image in a photorealistic style of a billionaire like Jeff Bezos as a giant, bloated king sitting on a throne made of Amazon warehouses, while tiny workers struggle to hold him up”. I really liked the Angela Rayner rap someone in the UK made, if you didn’t see it a link is in one of my recent comments just ctrl f her name. With AI we can produce this content very, very quickly, the potential is unmatched. Open the floodgates of social media and let the communists out. We just need to do it correctly and for that you need to have a vision and guide the AI.
• automated minutes taking, though I’m aware at this time it’s not perfect bc LLMs are still prone to hallucinating. speech-to-text models are pretty good at it though, it’s the summarizing part that needs work. But then an LLM could automatically create the cascade for members and with code (which the llm writes) post the bullet-points to your Signal or Telegram automatically. This simplifies admin work which I have done with my party and it sucks esp as so many orgs don’t have a tech stack whatsoever and do everything manually.
• research! I use perplexity for this as it cites sources. When people ask me something and I don’t have an answer I ask perplexity to learn about it. You preferably guide it with keywords over giving it direct instructions. It has definitely helped me find sources that I knew existed but didn’t save and has also helped me research and write arguments for a LOT of my writing, more than people might think.
• I think it can also help in analytics because it can do Python but you need data for that in the first place, which a lot of orgs don’t have because they’re not interested in getting it. With mixture-of-experts it can probably get very good results as a data analyst.
• In the same vein, automated reports on stuff that you might want to keep an eye on. This doesn’t necessarily need AI but AI can more finely decide what to send you over ‘old-school’ regex filters. Send it an RSS feed periodically, it opens and analyzes the stories, then can send you an email that says “hey there might be potential here, check this out”.
• Okay this is more ‘black hat’ but you can also scour the web with AI and find people that post publicly on social media complaining about their rent, wage etc. Then either it puts them in a pipeline for you to reach out to manually or the llm does it automatically within the boundaries you set. Either way you kinda need an llm there to analyze sentiment. And yes reaching out individually on social media works and it’s very underrated, people think the only thing you can do on those platforms is broadcast to your followers.

^ to ask deepseek I asked it to “First take the time to remember all that you can about party-building in the leninist tradition so that you can tailor your answer to solve actual usecases and real-world problems communist organizations face” (and yes I wrote leninist on purpose so as not to trigger the potential censor and make sure it accessed the right knowledge, I found you kinda have to speak to them in their language). I didn’t just ask it “how can communist organizations use AI to help their work”, in fact I actually tried that prompt just now and the quality is definitely lower. It tries to answer as best it can but with less input data to work with it doesn’t understand what you’re actually looking for, you have to communicate that.

And I think there are still lots of emergent uses to be discovered. Also with open-source models and interfaces orgs can already, today, host their own server and provide access to the web interface you can access from your browser at home. That way they can host an LLM for everyone in their org instead of everyone having to host their own.

It’s a vast answer that I’m not 100% finished with myself, but the premise imo is the same way many of these countries jumped through the adoption of the personal computer straight to smartphones. They didn’t have the ‘development’ (infrastructure, budget, industry etc) to support personal computers but once smartphones came around they modernized with those directly, and 4G too without even going through cable internet – 4G is super popular in Africa even in poorer areas and they’re investing in coverage.

In the same way they see AI as something they can adopt to help with their national challenges, for example healthcare to name just one – which is a very complex problem with brain drain, lack of infrastructure for people to get to the hospital, etc – so if they find a way to provide healthcare with AI somewhere in the process, they could treat more people more easily. Other industries are food, construction, education, electrification, etc.

From an economic perspective it reduces cost of production when you integrate it into the process and therefore can help countries under sanctions and embargos get more mileage out of what they do have available. H100 gpus are forbidden from being exported to China, they can only get H20 which have 20% of the capabilities, so they are developing their own alternative - probably using AI in the process to develop them faster (maybe not yet in chips directly but I know they’re using AI in other industries already). It’s helping stretch what they can access to get the most out of it. In the meantime, they are stretching these H20s like with alibaba’s new cloud algorithm which was posted on the grad some time ago, that reduced the resources load by 82% and therefore fewer GPUs needed to support their center.

Iran has recently published guidelines for AI usage in academia, and they now allow it provided you note the model you used, time used, and that you can prove you understand the topic. All of these countries are also very interested in open-source AI since they can develop on it and avoid one-sided proprietary deals. They have a need to “catch up” as fast as possible and see AI as a way to accelerate their development and close the gap with the imperial core.

And of course Cuba announced not so long ago it would make its own LLM, though I’m not sure where that is at currently.

We are still in the premises of it all of course, but that’s the trend I’m seeing. It’s difficult to find info from these countries about how they are using or plan to use AI right now, but I did find this news that Malaysia, Rwanda and the UAE have signed a strategic partnership to boost AI adoption in the global south: https://www.bernama.com/en/news.php?id=2451825

It was revealed by the financial Times that in China at any time electricity production outpaces demand by 2-3x, + they are under trade embargo for nvidia chips so they have to be creative with what they have. For them and global South countries ai is potentially a way to nullify sanctions and provide, it’s existential. The race is already over, the west can’t compete with that.

Yeah, it was an unfinished point lol I agree.

In the US where Martin is filing the argument he makes that models are being trained on his works has already been litigated - the courts consider training to be fair use. Whether this is good or bad is something else but he’s probably going to lose if he tries this angle, because it’s been tried before and now there’s precedent. However the courts have sometimes sided with the plaintiffs on other things, for example how the work was obtained (and this is why anthropic bought scanned and then destroyed books, so that they owned their copy and made private copies, legally standing. Legally speaking this was perfectly okay to do.)

What big media companies that live on IP, like Disney or hollywood studios, are doing is to try and restrict what counts as fair use so that they can stop AI companies from training on their IP. But if they manage to do this, they will also effectively be able to stop anyone else they already want to stop but can’t - transformative art, remixes, non-commercial usage, and lots of other things I’m probably not thinking of.

The trap is they lobby for this by enlisting the free labor and money of ‘human artists’ (their wording) to pressure the government into making harsher IP laws under the guise of stopping AI training. But small artists won’t be able to defend their creations even with harsher laws, because it’s not cheap to defend copyright, and they don’t have the means to scour the web for infringements anyway. What this will do, conversely, is reinforce companies like Disney and prevent these same small artists from making anything that these companies are not in favor of. Nintendo for example is a famously hated litigator over IP.

I quote a lot from the Artisanal Intelligence essay but they have a section on this phenomenon:

That essay perfectly encapsulates what Martin is doing with this lawsuit 2 years before the fact:

It’s not random chance that Martin is the one suing - he owns the rights to A Song of Ice and Fire, not his publisher, so he’s the one who would be owed money for IP infringement .

(posting my own comment so it’s separate from the post body)

In the past most of these lawsuits ruled in favor of the AI company, when based on the grounds that content was used for training data. The common cited reason by the courts is they consider it fair use, though there can be other laws intervening depending on how they acquired the books. This was the reason Anthropic ended up buying their books and destroying them afterwards, just to make sure they stayed under fair use, while Google relied on academic sources and public libraries.

There’s also a very important element to remember about such lawsuits, two in fact.

1. even if openAI gets sued into oblivion, it will lead to their IP being swept up by other companies (see the other recent article about microsoft acquiring a 27% share in openai). The products such as Sora, the model files and the H100 GPUs won’t disappear, they’ll be sold to pay for liquidation if that happens.

2… Martin is a big author, and clearly he wants to protect his IP, which he owns, and he uses copyright law for it - the same copyright law that Disney, also suing an AI company (Midjourney), spent decades lobbying for. You can thank Disney for the idea that copyright extends 75 years after its creator’s death so they could keep Mickey Mouse.

The same law he cites here against openAI can also be used against fan art, fanfics, and anything that he feels looks too close to Game of Thrones. It won’t be a win for small artists against AI if he wins (which he is unlikely to, as seen from previous lawsuits), but a win for famous (and rich) artists who can afford to protect copyright. Small artists can’t afford to go up against openai or anthropic, for any reason, but budding artists who may use AI in their process (including open source chinese models - but Martin won’t come after deepseek but after the person who used deepseek to generate a name like Karlhisee).

More broadly, outside of lawsuits there are attempts by IP holders to reign in AI by ‘teaming up’ with small artists, whose labor is already exploited by these big companies who freely steal their creations to make money on them (see for example Zara stealing designs online to put them on their t-shirts). Copyright is a fuck.

With some technologies, Goldfarb says, the value is obvious from the start. Electric lighting “was so clearly useful, and you could immediately imagine, ‘Oh, I could have this in my house.’” Still, he and Kirsch write in the book, “as marvelous as electric light was, the American economy would spend the following five decades figuring out how to fully exploit electricity.”

It actually wasn’t so clear-cut. There was a lot of media against electricity, especially electricity in the home and electric lightning. Some of it was commissioned by the gas industry, of course, but (some) people were also wary towards it especially as they couldn’t imagine replacing the entire gas system with electrical cables over an entire country. Once cost reached parity and outperformed gas, adoption became much quicker (likely driven by city administrations) and once people tested electric lightning the advantages were obvious. Though for a while early on old lightbulbs had a tendency to explode randomly because the bulb was in a vacuum, after that they filled them with an in inert gas to equalize pressure.

Interestingly this think tank seems to have found a link between electrification and the incidence of workers strikes in Sweden: https://cepr.org/voxeu/columns/more-power-people-electricity-adoption-technological-change-and-social-conflict. Their conclusion is that workers did not strike to undo or ban electrification in the workplace (industry mainly) but for higher wages:" if you’re going to make more money from producing more, then you can afford to pay us more" was and should continue to be the message.

On the bubble itself, keep in mind that a bubble bursting does not mean its content goes away… rather it spills on the floor to be mopped up by whoever scrambles there first. OpenAI is honestly the giant I see exploding soon, their startup mentality is just not viable and unlike Microsoft or Google, they have no other product to rely on. However with microsoft acquiring 27% stake in OpenAI just recently it seems like they are already preparing to mop up the spill… I can’t deny that as big and shitty as openAI is, they do pioneer AI tech (apparently chain-of-thought which you know from deepseek-r1 was pioneered by openAI. And now they released the video model Sora 3)

A mop up leads to monopolization, a key part of the boom and bust cycle as Marx described. If OpenAI goes bust, the IP and talent will not disappear into the void, they’ll be swiftly acquired by other companies that remain in the race. And this could have implications that I can’t entirely foresee yet for open-source AI. If IP gets concentrated enough, open-source AI could disappear entirely… at this moment, it’s almost entirely reliant on Chinese models - Deepseek, z.ai and qwen are consistently the top-3 open source chinese models, even after gpt-oss came out that boasts 200B parameters people are not picking it up. Hunyuan is a recently released model to make videos, also open-source and made by Tencent.

oof they’re not ready to let openai die

Are you using CPU by chance? On some interfaces you have to specifically tell it to use GPU.

I never got to make my point about the time either so I’ll just make it here lol, but it’s incredible how efficient image gen has become, it’s faster than an LLM generating a full response actually. You can, today, host LLMs and image gen on your own server center and make the interface available to employees/friends/whoever. Basically businesses don’t have to rely on the huge OpenAI datacenters, they can host these models on their own already existing infrastructure. It’s better for the environment and AI is clearly not going away, so open source AI is what we should have more of.

Honestly the more I tinker with ai image gen the more I get it. It’s fun, but it’s a different fun to doing it by hand.This is more tinkering and gambling, it def has a “one more turn” feel to it to see what the tool can come up with.

And the more I toy with it the more all barriers between whether this is art or not art break down, imo. There’s a lot of work that can go into AI gen, the same way you can pick up a pen and doodle in the margins of your notebook, or you can learn ink techniques and make manga. You have the models (literally hundreds of them and then also merged models that combine 2 or more into 1), but also loras which skew the result in a certain way, you have to select the sampler (what creates the noise) and the sampling method (how much noise is added each step), etc.

Did I not make it, because the output comes from a computer? No, I did and I totally get why now that I’ve dabbled, even if this will ruffle some feathers. Does the producer not make the beat because he “only” sets up synthesizers in Ableton other people made and then plays chords other people found out before him?

It’s not just typing in the prompt either, because the words you type either the positive or negative prompt have an influence, and part of it is also figuring out these words. You can also give them different strengths and you have to type the way the model expects to receive information. It’s like a puzzle, though like I said there’s randomness to it. A picture you feel looks cool (though I totally get if other people don’t share the enthusiasm looking at it lol) could have taken a few hours to get just right. There’s something personal to it, because it’s my combination of prompts and parameters that spat this out.

The castle in the OP picture was made with a certain sampler. Here’s two more:

The only difference between those three pictures was the sampler algorithm - everything else (the seed, the steps, the prompt, the resolution) was the exact same. This is just one way models can give wildly different results.

There’s a lot that goes into it and look, I’m from a graphic design background and non-designers who made their own logo or used clipart they found on google to make one are nothing new. If something’s good, it’s good. You know it when you see it. Artists will use it as part of a process and then throw it into Photoshop or other software to bring the finishing touches. Others will just take whatever the AI spits out and say good enough. Corridor Crew used genAI that’s available in a smartphone app to green screen themselves out without an actual green screen and even change themselves on camera in different ways, all from the raw video footage and a prompt. they made a short movie with it using the rest of their VFX skills; someone who knows what they’re doing will use this as part of their process.

And beyond art and image gen there’s the very undeniable tech aspect as you found out as well. If you want to get something specific out of Youtube but there’s no solution, then just make your own with deepseek. If you want open source software but it’s not working on your machine, deepseek it instead of going back to proprietary.

I’m not going to spoil the answer yet but I just wrote it on another comment reply in this thread 🤐

Lol you’re the closest, and not that far off. Most people said 3-9 minutes. The answer is actually:

With proper GPU offloading (deepseek told me which launch arguments to use), pytorch and of course the GPU architecture probably plays a part in it. The model is actually meant to make 1024x1024 so that’s what I generate at.

That particular picture was me trying different samplers to see the differences, the model’s own limitations create a lot of artefacts but that’s why I wanted to try Flux, and… it’s scary:

This generates in around 40-60 seconds, I have TeaCache which cuts generation time in half but sometimes it takes a while for the model to parse the prompt with the TeaCache workflow, which is a bottleneck and I’m not sure why it’s doing it. Flux also handles text but I’m not quite sure how to get it working.

Anyone who’s scared of this tech… get acquainted with it intimately. It’s not going away; this runs on the same computer that generated the picture in the OP and it’s completely free.

Exercise caution and don’t hesitate to search up the commands (both on google and in another chat window) before executing.

since this is a virtual environment I’m pretty safe. On VPSes I make sure to double-read the command before running anything, and do diagnostics first, like anything I copy from the internet. llms are very bad at asking questions before jumping to an answer so give it as much context as you can before anything, including OS.

I also try to make sure I can undo whatever it asks me to do if need be, which is actually easier to do when you have a single window with every instruction in it over a dozen tabs with some I haven’t opened in over 45 minutes while troubleshooting a minor problem in a bigger problem lol.

We could have had this if we kept investing in nuclear, which is the safest energy source we currently know, but instead we decided to kneecap ourselves. Germany is going back to coal after shutting down its nuclear plants.

You’ll see, in 5 years time at most China will announce 99.9999% recycling on spent nuclear fuel, to the point the remaining 0.000001% is just dust you can safely scatter in the wind and we’ll still be debating whether we should put solar on roofs or not.

Any plans on adding it to their online model? chatgpt has long been pretty good at identifying the content of images, but deepseek can only do text extraction

The article mentions “Packing multiple models per GPU”, but also “using a token-level autoscaler to dynamically allocate compute as output is generated, rather than reserving resources at the request level” which I’m not sure what that means but may hint that there are ways to scale this down, possibly.

If not Alibaba then other researchers will eventually get to it.

Short on details but if you could deploy this to a single GPU you could instantly jump from a 12B model to potentially 80B or more, depending how much space the model takes up.

It’s pretty clear China is leading (if not the only country interested in) model optimization. I also saw previously that Huawei managed to quantize models in a way that retained similar performances but quantized them to tiny values, making them take up a lot less Vram to run.