Mastodon, an alternative social network to Twitter, has a serious problem with child sexual abuse material according to researchers from Stanford University. In just two days, researchers found over 100 instances of known CSAM across over 325,000 posts on Mastodon. The researchers found hundreds of posts containing CSAM related hashtags and links pointing to CSAM trading and grooming of minors. One Mastodon server was even taken down for a period of time due to CSAM being posted. The researchers suggest that decentralized networks like Mastodon need to implement more robust moderation tools and reporting mechanisms to address the prevalence of CSAM.
While the study itself is a good read and I agree with the conclusions—Mastodon, and decentralized social media need better moderation tools—it’s hard to not read the Verge headline as misleading. One of the study authors gives more context here https://hachyderm.io/@det/110769470058276368. Basically most of the hits came from a large Japanese instance that no one federates with; the author even calls out that the blunt instrument most Mastodon admins use is to blanket defederate with instances hosted in Japan due to their more lax (than the US) laws around CSAM. But the headline seems to imply that there’s a giant seedy underbelly to places like mastodon.social[1] that are rife with abuse material. I suppose that’s a marketing problem of federated software in general.
- There is a seedy underbelly of mainstream Mastodon instances, but it’s mostly people telling you how you’re supposed to use Mastodon if you previously used Twitter.
In my opinion the biggest issue the author points out is that cached materials are sometimes retained even after moderator action. Which honestly just sounds like a straight up bug more than anything. Though if I were running an instance, the feds showing up at my door with a warrant because I’ve been accidentally distributing CSAM would be my nightmare scenario. And of course jurisdiction plays a part, too: an American user on a Canadian server might see drawn depictions of sexualized minors, think “weird but not illegal,” and now the Canadian admin has content that’s illegal in Canada on their Canadian server and has no idea.
IMO I think the best solution to this is something similar to what Renaud Chaput (Mastodon’s resident infra boffin) described in his recent blog post. Effectively, give admins a way to hand this off to pluggable third-party services. Admins that are worried about this sort of thing can then have some degree of safety via e.g. PhotoDNA, whereas others can take on additional risk and preserve additional privacy.
All that said: yeah the headline makes it sound like .social is some 8chan-esque hellhole, whereas in reality my feed is 99% German programmers sharing milquetoast political takes.
deleted by creator
The person outright rejects defederation as a solution when it IS the solution, if an instance is in favor of this kind of thing you don’t want to federate with them, period.
I also find worrying the amount of calls for a “Fediverse police” in that thread, scanning every image that gets uploaded to your instance with a 3rd party tool is an issue too, on one side you definitely don’t want this kinda shit to even touch your servers and on the other you don’t want anybody dictating that, say, anti-union or similar memes are marked, denounced and the person who made them marked, targeted and receiving a nice Pinkerton visit.
This is a complicated problem.
Edit: I see somebody suggested checking the observations against the common and well used Mastodon blocklists, to see if the shit is contained on defederated instances, and the author said this was something they wanted to check, so i hope there’s a followup
Yeah I recall that the Japanese instances have a big problem with that shit. As for the rest of us, Facebook actually open sourced some efficient hashing algorithms for use for dealing with CSAM; Fediverse platforms could implement these, which would just leave the issue of getting an image hash database to check against. All the big platforms could probably chip in to get access to one of those private databases and then release a public service for use with the ecosystem.
That’d be useless though, because first, it’d probably opt-in via configuration settings and even if it wasn’t, people would just fork and modify the code base or simply switch to another ActivityPub implementation.
We’re not gonna fix society using tech unless we’re all hooked up to some all knowing AI under government control.
That’d be useless though, because first, it’d probably opt-in via configuration settings and even if it wasn’t, people would just fork and modify the code base or simply switch to another ActivityPub implementation.
No it wouldn’t, because it’d still be significantly easier for instances to deal with CSAM content with this functionality built into the platforms. And I highly doubt there’s going to be a mass migration from any Fediverse platform that implements such a feature (though honestly I’d be down to defederate with any instance that takes serious issue with this).
And the instances who want to engage with that material would all opt for the fork and be done with it. That’s all I meant.
Right, and the rest of us would be able to more effectively filter it out from our instances.
That’s not the point. Yes, child porn sites can host child porn. Other sites/instances can’t stop that. But what other instances can stop, is redistributing said child porn. And for that purpose, such technology would be useful.
Facebook actually being good for once? Unheard of
As much as we can (and should) lambast Facebook/Meta’s C-Suite for terrible decisions, their engineers are generally pretty legit.
I’m not fully sure about the logic and perhaps hinted conclusions here. The internet itself is a network with major CSAM problems (so maybe we shouldn’t use it?).
It doesn’t help to bring whataboutism into this discussion. This is a known problem with the open nature of federation. So is bigotry and hate speech. To address these problems, it’s important to first acknowledge that they exist.
Also, since fed is still in the early stages, now is the time to experiment with mechanisms to control them. Saying that the problem is innate to networks is only sweeping it under the rug. At some point there will be a watershed event that’ll force these conversations anyway.
The challenge is in moderating such content without being ham-fisted. I must admit I have absolutely no idea how, this is just my read of the situation.
@mudeth @pglpm you really don’t beyond our current tools and reporting to authorities.
This is not a single monolithic platform, it’s like attributing the bad behavior of some websites to HTTP.
Our existing moderation tools are already remarkably robust and defederating is absolutely how this is approached. If a server shares content that’s illegal in your country (or otherwise just objectionable) and they have no interest in self-moderating, you stop federating with them.
Moderation is not about stamping out the existence of these things, it’s about protecting your users from them.
If they’re not willing to take action against this material on their servers, then the only thing further that can be done is reporting it to the authorities or the court of public opinion.
Maybe my comment wasn’t clear or you misread it. It wasn’t meant to be sarcastic. Obviously there’s a problem and we want (not just need) to do something about it. But it’s also important to be careful about how the problem is presented - and manipulated - and about how fingers are pointed. One can’t point a finger at “Mastodon” the same way one could point it at “Twitter”. Doing so has some similarities to pointing a finger at the http protocol.
Edit: see for instance the comment by @[email protected] to this post.
Understood, thanks. Yes I did misread it as sarcasm. Thanks for clearing that up :)
However I disagree with @[email protected] in that Lemmy, and the Fediverse, are interfaced with as monolithic entities. Not just by people from the outside, but even by its own users. There are people here saying how they love the community on Lemmy for example. It’s just the way people group things, and no amount of technical explanation will prevent this semantic grouping.
For example, the person who was arrested for CSAM recently was running a Tor exit node, but that didn’t help his case. As shiri pointed out, defederation works for black-and-white cases. But what about in cases like disagreement, where things are a bit more gray? Like hard political viewpoints? We’ve already seen the open internet devolve into bubbles with no productive discourse. Federation has a unique opportunity to solve that problem starting from scratch, and learning from previous mistakes. Defed is not the solution, it isn’t granular enough for one.
Another problem defederation is that it is after-the-fact and depends on moderators and admins. There will inevitably be a backlog (pointed out in the article). With enough community reports, could there be a holding-cell style mechanism in federated networks? I think there is space to explore this deeper, and the study does the useful job of pointing out liabilities in the current state-of-the-art.
@mudeth @pglpm The grey area is all down to personal choices and how “fascist” your admin is (which goes on to which instance is best for you?)
Defederation is a double-edged sword, because if you defederate constantly for frivolous reasons all you do is isolate your node. This is also why it’s the *final* step in moderation.
The reality is that it’s a whole bunch of entirely separate environments and we’ve walked this path well with email (the granddaddy of federated social networks). The only moderation we can perform outside of our own instance is to defederate, everything else is just typical blocking you can do yourself.
The process here on Mastodon is to decide for yourself what is worth taking action on. If it’s not your instance, you report it to the admin of that instance and they decide if they want to take action and what action to take. And if they decide it’s acceptable, you decide whether or not this is a personal problem (just block the user or domain on in your user account but leave it federating) or if it’s a problem for your whole server (in which case you defederate to protect your users).
Automated action is bad because there’s no automated identity verification here and it’s an open door to denial of service attacks (harasser generates a bunch of different accounts, uses them all the report a user until that user is auto-suspended).
The backlog problem however is an intrinsic problem to moderation that every platform struggles with. You can automate moderation, but then that gets abused and has countless cases of it taking action on harmless content, and you can farm out moderation but then you get sloppiness.
The fediverse actually helps in moderation because each admin is responsible for a group of users and the rest of the fediverse basically decides whether they’re doing their job acceptably via federation and defederation (ie. if you show that you have no issue with open Nazis on your platform, then most other instances aren’t going to want to connect to you)
I share and promote this attitude. If I must be honest it feels a little hopeless: it seems that since the 1970s or 1980s humanity has been going down the drain. I fear “fediverse wars”. It’s 2023 and we basically have a World War III going on, illiteracy and misinformation steadily increase, corporations play the role of governments, science and scientific truth have become anti-Galilean based on “authorities” and majority votes, and natural stupidity is used to train artificial intelligence. I just feel sad.
But I don’t mean to be defeatist. No matter the chances we can fight for what’s right.
This is exactly what I thought. The story here is that the human race has a massive child abuse material problem.
The internet itself is a network with major CSAM problems
Is it, though?
Over the last year, I’ve seen several reports on TV of IRL group abuse of children, by other children… which left everyone scratching their heads as to what to do since none of the perpetrators are legally imputable.
During that same time, I’ve seen exactly 0 (zero) instances of CSAM on the Internet.
Sounds to me like IRL has a major CSAM, and general sex abuse, problem.
One of the problems with the fediverse is that each server keeps its own copy of the content. It is definitely a worry that bad actors push content to federated servers to get them taken down due to the content they now are storing.
What’s the reason for that? Caching purposes?
General idea is that if there is only one copy, taking something down is knocking that server out of service.
I think so. My Lemmy instance for example is currently storing several gigabytes of images in my cloud buckets, but with my 4 users I’m reasonably confident it didn’t all come from us.
Yes it’s a caching thing.
The researchers suggest that decentralized networks like Mastodon need to implement more robust moderation tools and reporting mechanisms to address the prevalence of CSAM.
I agree, but who’s going to pay for it? Those aren’t just freely available additions to any application that you only need to toggle on.
One way to do this is to block hashes. This is a slippery slope though because it could be used maliciously. Only way to do this and protect freedom of information is to make this fully open source.
Block hash lists then? Something like a community driven hashlist for CSAM would work, of the majority of federated instances report it as that type then it would get added to the list. Instances could then choose what lists they wanted to block.
…instances could also show what lists they subscribe to so they users could see what sort of moderation they choose
This is kind of problematic… By creating a community driven hashlist that is freely shared, you’ve also kind of created an index of CSAM content that could easily be extrapolated for people actively looking to find/share that content.
Doesn’t anyone looking for that material already know what to look for?
Surely a list of hashes wouldn’t be that useful?
only if they are crypto hashes (hash functions that back btc, ltc, other cryptos) as they are irreversible*
*i wont explain, use your internet in the pocket
Super useful, it’s very similar to how magnet links for torrenting works. I know of a few less popular file sharing services that can act and search for files based on hash alone.
A lot of other areas online make use of hashes as identifiers already too. If you search for a hash of a file you’ve downloaded, just the hash and nothing else, there’s a very good chance you’ll get multiple results.
So the standard approach to this is so-called “perceptual hashing.” Effectively, using cryptographic hashes (sha256, etc.) doesn’t really work well in this case. Given a piece of illegal content, that content is likely to still be just as illegal with a single pixel changed – however, it’ll have a completely different cryptographic hash. So instead, a hash function that determines how “similar-looking” two images are, ignoring things like dimensions, color palette, JPEG compression artifacts, etc. This is obviously way fuzzier, and is prone to both false positives and negatives.
Because all this is inherently kinda fuzzy, the exact database of hashes is usually “secret sauce” if you will. If it were public, it would be super easy to circumvent. As an example, given an illegal image:
- Is the image’s hash in the DB?
- No? All done, you can post it with impunity.
- Yes? Change one random pixel, GOTO 1.
As a result even “public” databases are distributed with NDAs etc. This obviously does not jive well with an open source, federated network like Mastodon, and I have my doubts as to how willing the relevant agencies would be to give their databases to every rando with $5 to spin up a Pleroma instance on a VPS. A public DB might help in some cases, but unfortunately more illegal content is produced every day, and so it would be extremely hard to keep up with the bad actors.
Image hashes? That could work. It could be a simple system like uBlock where you import filter lists to your instance and they’re easy to disable if their caretakers fill them with garbage data.
I agree, but who’s going to pay for it?
How about police/the tax payer?
If university researchers can find the stuff, then police can find it too. There should be an established way to flag the user (or even the entire instance) so that content can be removed from the fediverse while simultaneously asking for all data that is available to try to catch the criminals.
And of course, if regular users come across anything illegal they will report it too, and it should be removed quickly (I’d hope immediately in many cases, especially if the post was by a brand new/untrusted account).
A decentralised platform like the Fediverses won’t easily work with nation states and their taxes. Even with Wikipedia today, it’s not funded directly via any government - but rather by certain universities giving some money to it + all the private doners.
And even if we get that working, power politics will mess this up like so often when things actually get troublesome.
It might be interesting to explore cryptocurrencies as for donations here though. They do have international liquidity and they can’t be misused foe power politics.
I’m not suggesting Beehaw/etc should be government funded. Rather I’m suggesting it’s already possible for basically anyone in the fediverse to report a post as needing urgent moderator attention.
I think there will be tax payer funded efforts, donation funded efforts, volunteers, etc that are unaffiliated with any specific instance but go through major instances and hit the report button where they consider it to be appropriate — not just manually with people but also with automated tools such as searching for images by a hash of their contents or maybe even running messages through a Large Language Model to check if it is, for example, a form of targeted harassment.
And yes, the report feature will be abused. That’s unavoidable and needs to be taken into account when deciding how to respond to a report. An algorithm could easily prioritise reports based on the history of past reports made by the same person / organisation.
Stack Exchange has a pretty good system - decisions by individuals are not trusted. Rather those trigger a review by a randomly selected (and trusted) individual to get a second opinion. And even after a decision has been made and an action has been taken (ban a user, etc) there’s often a third or even fourth review. And there are processes to appeal and question decisions.
It’s not an easy problem to solve, but as the creator of mastodon said - many hands make light work. The fediverse can some day have a billion people doing moderation tasks - where even simple acts like hitting the upvote button become part of the moderation system (upvote would imply that this account holder tends to make valuable contributions to the community, and should make the moderation system less likely to come down with a ban hammer).
And I also think there is scope for some communities to be entirely government funded. For example I’d love for every city in the world to run an offical community, with official local government anouncements as well as moderated discussions relevant to people who live in or are visiting the city.
I am exactly doubting your suggestion of tax paid donations. I don’t think this will happen, unless we actually come together and try to actually enforce this on the political level in various countries.
After all, open source software is an essential and critical foundation since many decades - but I’m not sure, whether there is any government that has made a pledge to donate a certain amount of money per year into the development and funding of such general purpose software. (Maybe I’m wrong though.)
Before the fediverse can get any public funding, we need to make some political efforts. the UN is the largest such institution - and it took all the fiasco with the 2 world war to get many countries pledge to donate to it every year…
I am exactly doubting your suggestion of tax paid donations. I don’t think this will happen, unless we actually come together and try to actually enforce this on the political level in various countries.
I meant private donations, which are already happening.
I think tax revenue would be spent on government employees looking over content in search of evidence of crimes/etc, which I’m sure is also already happening. I hope they don’t just look - they should be reporting whatever they find.
I’m not sure, whether there is any government that has made a pledge to donate a certain amount of money per year into the development and funding of such general purpose software
Tor (The Onion Router) was invented by a United States Naval Intelligence Unit. They released the source code as open source and handed control over to the EFF but several US Government agencies continued to provide substantial funding (especially the Bureau of Democracy, Human Rights and Labor Affairs). As far as I know they continue to fund it.
There are definitely examples of Governments funding open source software, especially things that are as valuable as a social network.
The researchers can’t be taken seriously if they don’t acknowledge that you can’t force free software to do something you don’t want it to.
Even if we started way down at the stack and we added a CSAM hash scanner to the Linux kernel, people would just fork the kernel and use their own build without it.
Same goes for nginx or any other web server or web proxy. Same goes for Tor. Same goes for Mastodon or any other Fedi/ActivityPub implementation.
It. Does. Not*. Work.
* Please, prove me wrong, I’m not all knowing, but short of total surveillance, I see no technical solution to this.
This is one of the things I don’t like about the whole Twitter format. There’s no moderator layer. Every lemmy community must be created by a moderator and that mod can be held accountable.
There isn’t even a concept of communities on Twitter / Mastodon. Hashtags? Nobody owns monitoring them, and they can be freely improvised at will. It really is just the instance and its zillion users with nothing in between. Imagine a lemmy instance admin being responsible for all the moderation… would never work.
Is there any way mastodon stands out from other self hosted websites? Would the CSAM material be harder to distribute or easier to prosecute if they ran, say, a self-hosted bulletin board for it instead?
Pedos that got banned from platforms turn to other platform who hasnt done it yet
In other news: the sky is blue
While white knights propose ways to control everyone everywhere everytime, in the name of catching the pedos who will just hop to the next platform (or have already).
I don’t trust stanford to not work on behalf of the CIA or other 3 alphabet orgs. They kind of turn a blind eye to CSA in churches but a federated media? This sounds like a smear job.
I really don’t think the CIA cares to be honest…
Mastodon.art doesn’t.
And the beauty of Mastodon is you can block an entire instance, as can your admin, when something awful is posted. Mastodon even has a hashtag they use as an alert for this kind of thing. (#Fediblock)
Removed by mod
This is a whataboutist counterpoint at best. Universities and their researchers are not a monolith.
This is just bad press. The actual study is quite good and offers good recommendations on how to improve moderation on the fediverse
I browsed through an anime instance while trying to convince myself to like Mastodon and unfortunately I believe I’ve found some of this myself. I wasn’t going to confirm it was real, I just reported and closed out but considering I’ve never seen such content on other websites and this instance was rife with it, I don’t find this article hard to believe at all.
I think some of the problematic instances have been defederated, IIRC there’s a large japanese instance that was defederated long time ago due to child abuse content. But still since I’ve been seeing increases of hate speech and dog whistling misogyny and homophobia in some instances, I won’t be surprised if CSAM stuff has been trading under our noses.
The main issue is that, with so many users nowadays and small moderation teams, especially in the larger instances, it’s hard to moderate and tackle CSAM problems effectively. I really wish larger instances would limit user registrations or start splitting off into smaller manageable ones.
Also, since they are trading using certain hashtags, blocking those hashtags might not be a bad idea.
I for one am all for instances being forcibly taken down by police if they can’t moderate CSAM appropriately.
Moderation is a very real challenge. The internet at large aimed to solved it by centralizing everything to a few mega corps with AI moderation. The fediverse aims to solve it by keeping instances small and holding both mods and users accountable.
not surprised at all. this is a growing pain here too because this was previously a thing handled invisibly by platforms and federation makes it fall to individual sysadmins and whoever they have on staff. the tools for this stuff are, in general, not here yet–and as people have noted there are potential conflicts with some of the principles of federation introduced by those tools that can’t be totally handwaved.
