• 8uurg@lemmy.world
      link
      fedilink
      English
      arrow-up
      13
      ·
      19 hours ago

      Classic bots (e.g. standard web crawlers, as used by search engines, that report their both nature inside their user agent) or the modern LLM-controlled bots that control a full modern browser that also runs javascript?

      Because I can imagine they account for the first category, while missing the latter, which have become increasingly more prominent.

      • azuth@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        14 hours ago

        The kind that wants to not appear as a bot is unlikely to send a Linux user agent even if running on Linux.

        • 8uurg@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          9 hours ago

          It doesn’t have to be on purpose. Web crawlers like Google’s specifically identify themselves as a bot. A headless browser just acts like any normal browser, and uses the same user agent as one, just controlled by code rather than a keyboard and mouse.