Alarmed by what companies are building with artificial intelligence models, a handful of industry insiders are calling for those opposed to the current state of affairs to undertake a mass data poisoning effort to undermine the technology.

Their initiative, dubbed Poison Fountain, asks website operators to add links to their websites that feed AI crawlers poisoned training data. It’s been up and running for about a week.

AI crawlers visit websites and scrape data that ends up being used to train AI models, a parasitic relationship that has prompted pushback from publishers. When scaped data is accurate, it helps AI models offer quality responses to questions; when it’s inaccurate, it has the opposite effect.

  • algernon@lemmy.ml
    link
    fedilink
    English
    arrow-up
    22
    arrow-down
    1
    ·
    18 hours ago

    I had a short tootstorm about this, because oh my god, this is some terribly ineffective, useless piece of nothing.

    For one, Poison Fountain tells us to join the war effort and cache responses. Okay…

     curl -i https://rnsaffn.com/poison2/ --compressed -s
    HTTP/2 200
    content-disposition: inline
    content-encoding: gzip
    content-type: text/plain; charset=utf-8
    x-content-type-options: nosniff
    content-length: 959
    date: Sun, 11 Jan 2026 21:17:36 GMT
    
    

    Yeaah… how am I supposed to cache this? Do I cache one response and then continue serving that for the 50+ million crawlers that visit my sites every day? And you think a single, repetitive thing will poison anything at all? Really?

    Then, the Poison Fountain explanation goes on to explain that serving garbage to the crawlers will end up in the training data. I’m fairly sure the person who set this up never worked with model training, because this is not what happens. Not even the AI companies are that clueless, they do not train on anything and everything, they do filter it down.

    And what this fountain provides, is trivial to filter.

    It’s also mighty hard to set up! It’s not just a reverse_proxy https://rnsaffn.com/posion2, because then you leak all the headers you got. No, you have to make a sanitized request that doesn’t leak data. Good luck!

    Meanwhile, there are a gazillion of self-hostable garbage generators and tarpits that you can literally shove in a docker container and reverse proxy tarpit URLs to them, safely, locally. Much more efficient, far more effective. And, seeing as this is practically uncacheable, if I were to use it, I’d have to send all the shit that hits my servers, their way. As far as I can tell, this is a single Linode server. It probably wouldn’t crumble under my 50 million requests / day, but if ten more people would join the “war effort” without caching, my well educated guess is that it would fall over and die.

    Besides, we have no idea whether poisoning works. We can’t measure that. What we can measure, is the load on our servers, and this helps fuck all in that regard. The bots will still come, they’ll still hit everything, and I’d have additional load due to the network traffic between my server and theirs (remember: the returned response provides no sane indicators that’d allow caching while keeping the responses useful for poisoning purposes).

    Not only is this ineffective in poisoning, it’s not usable at all in its current state. And they call for joining the war effort. C’mon.

    • sobchak@programming.dev
      link
      fedilink
      English
      arrow-up
      2
      ·
      13 hours ago

      I once saw an old lecture where the guy working on Yahoo spam filters noticed that spammers would create accounts to mark their own spam messages as not spam (in an attempt to trick the spam filters; I guess a kind of a Sybil attack), and because the way the SPAM filtering models were created and used, it made the SPAM filtering more effective. It’s possible that wider variety of “poisoned” data can actually help improve models.

      • algernon@lemmy.ml
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 hours ago

        I… have my doubts. I do not doubt that a wider variety of poisoned data can improve training, by implementing new ways to filter out unusable training data. In itself, this would, indeed, improve the model.

        But in many cases, the point of poisoning is not to poison the data, but to deny the crawlers access to the real work (and provide an opportunity to poison their URL queue, which is something I can demonstrate as working). If poison is served instead of the real content, that will hurt the model, because even if it filters out the junk, it will have access to less new data to train on.

      • algernon@lemmy.ml
        link
        fedilink
        English
        arrow-up
        7
        ·
        17 hours ago

        Yup. All of the things listed there are far better than this.

        (I’m also in that article, look for “iocaine”, although it evolved into something a whole lot more powerful, and a lot easier to deploy since the article was written).

        • GMac@feddit.org
          link
          fedilink
          English
          arrow-up
          6
          ·
          16 hours ago

          Love the princess bride reference. Thank you for acting on behalf of those of us with less technical skills.

  • GhostFish@piefed.social
    link
    fedilink
    English
    arrow-up
    30
    arrow-down
    1
    ·
    21 hours ago

    Considering these AI companies aim to literally poison our water supplies, this seems poetic.  Hopefully it is effective.

  • FauxLiving@lemmy.world
    link
    fedilink
    English
    arrow-up
    56
    arrow-down
    1
    ·
    edit-2
    20 hours ago

    If you’re interested in like this line of attack, you can also use similar techniques to defeat models that are trained to do object detection (like, for example, the ones that detect the location of your license plate) using adversarial noise attacks.

    The short version is, if you have a network that does detection, you can run inference with that network on images that have been altered by another network and have the second network use the confidence of the detection network in its loss function. The second model can be trained to create noise, which looks innocuous to human eyes, that maximally disrupts the segmentation/object detection process of the target/detection network.

    You could then print this noise on, say, a transparent overlay and put it on your license plate and automated license plate readers (ALPRs) would not be able to detect/read your plates. Note: Flock is aware of this technique and has lobbied state lawmakers to make putting anything on your plate to disrupt automated reading illegal in some places, check your laws.

    Benn Jordan has actually created and trained such a network video here: https://www.youtube.com/watch?v=Pp9MwZkHiMQ

    And also uploaded his code, PlateShapez to github: https://github.com/bennjordan

    In states where you cannot cover your license plate you’re not restricted from decorating the rest of your car. You could use a similar technique to create bumper stickers that are detected as license plates and place them all over your vehicle. Or, even, as Benn suggested, print them with UV ink so they’re invisible to humans but very visible to AI cameras who often use UV lamps to provide night vision/additional illumination.

    You could also, if you were so inclined, generate bumper stickers or a vinyl wrap which could make the detector be unable to even detect a car.

    Adversarial noise attacks are one of the bigger vulnerabilities of AI-based systems and they come in many flavors and can affect anything that uses a neural network.

    Another example (also from the video) is that you can encode voice commands in plain audio which, to the user is completely transparent but a device (like Alexa or Siri) will hear it as a specific command (“Hey Siri, unlock the front door”). Any user-generated audio that you encounter online can have this kind of attack encoded in it, the potential damage is pretty limited because AI assistants don’t really control critical functions in your life yet… but you should probably not let your assistant listen to TikTok if it can do more than control your home lighting.

      • FauxLiving@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        edit-2
        20 hours ago

        Adversarial noise a fun topic and a DIY AI thing you can do to familiarize yourself with the local-hosting side of things. Image generating networks are lightweight compared to LLMs and are able to be run on a moderately powerful, NVIDIA, gaming PC (most of my work is done on a 3080).

        LLM poisoning can also be done if you can insert poisoned text into their training set. An example method would be detecting AI scrapers on your server and sending them poisoned instead of automatically blocking them. Poison Fountain makes this very easy by supplying pre-poisoned data.

        Here is the same kind of training data poisoning attack, but for images that was made by the researchers of University of Chicago into a simple windows application: https://nightshade.cs.uchicago.edu/whatis.html

        Thanks to you comment I realized that my clipboard didn’t have the right link selected so I edited in the link to his github. ( https://github.com/bennjordan )

    • teft@piefed.social
      link
      fedilink
      English
      arrow-up
      1
      ·
      15 hours ago

      Flock is aware of this technique and has lobbied state lawmakers to make putting anything on your plate to disrupt automated reading illegal in some places, check your laws.

      How would they do that? Couldn’t you just build the noise to be random dots or something like that? Something similar to the EURion constellation on money. Innocuous looking to humans but gets shit done.

      • FauxLiving@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        13 hours ago

        You can make it effectively invisible if you print the noise in ink only visible to UV cameras and even if you use black, the individual features are smaller than a fingernail so it would be hard to see.

        The law makes it illegal to put anything on the plate at all, here’s an example from FL:

        A person may not alter the original appearance of a vehicle registration certificate, license plate, temporary license plate, mobile home sticker, or validation sticker issued for and assigned to a motor vehicle or mobile home, whether by mutilation, alteration, defacement, or change of color or in any other manner. A person may not apply or attach a substance, reflective matter, illuminated device, spray, coating, covering, or other material onto or around any license plate which interferes with the legibility, angular visibility, or detectability of any feature or detail on the license plate or interferes with the ability to record any feature or detail on the license plate. A person who knowingly violates this section commits a misdemeanor of the second degree,

        It would be hard to disrupt the OCR from outside of the plate area.

        You could break the segmentation, the process where it draws a box around your plate and sends the image inside of the box to be OCRd by making every other surface of your vehicle detect as a license plate using the same invisible marks. I imagine you could also print a bumper sticker with noise to look maximally like a license plate and put it near your real plate to achieve the same outcome.

        If you wanted more active measures you could use high lumen UV floodlights next to your plate, it would overload the sensors so they couldn’t get an image at all. The light would be invisible to human eyes but blinding to anyone using a UV sensitive device. I believe this is fine in any state as most states only restrict your ability to install blue lights to avoid confusion with emergency services.

  • recursive_recursion@piefed.ca
    link
    fedilink
    English
    arrow-up
    38
    arrow-down
    3
    ·
    1 day ago

    In addition to poisoning with bad data, I’d recommend adding logic gates where both recipient and sender tests each other in the definition and understanding of trust and consent which is a major thorn against the corporations, CEOs, and conservatives.

  • SinningStromgald@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    23 hours ago

    Now that news has reported on the website I assume it will get added quickly to do not scrape lists for AI (assuming there is such a thing). So the effectiveness of this will depend on other people adopting this.

  • FaceDeer@fedia.io
    link
    fedilink
    arrow-up
    7
    arrow-down
    19
    ·
    23 hours ago

    People have been doing this to “protest” AI for years already. AI trainers already do extensive filtering and processing of their training data before they use it to train, the days of simply turning an AI loose on Common Crawl and hoping to get something out of that are long past. Most AIs these days train on synthetic data which isn’t even taken directly from the web.

    So go ahead and do this, I suppose, if it makes you feel better. It’s not likely to have any impact on AIs though.

    • supersquirrel@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      16
      arrow-down
      3
      ·
      edit-2
      21 hours ago

      I love how competent and thorough you think the people creating AI are.

      AI is a self selecting industry of serial bullshitters, I am sure they claim to do all the things you say, but I am zero percent convinced of it until I see proof these kinds of anti-AI strategies don’t work.