New York is set to become the first state to impose guardrails on “stealth crawlers,” or unauthorized software that trawls news sources to scrape content.
Inaccurate headline. The bill doesn’t ban web scraping, it just requires that bots accurately identify themselves through the user agent string, and maybe some additional requirements to disclose the purpose of scraping the data.
Inaccurate headline. The bill doesn’t ban web scraping, it just requires that bots accurately identify themselves through the user agent string, and maybe some additional requirements to disclose the purpose of scraping the data.
And there’s a fine if the company doesn’t comply which is basically now gonna be considered the cost of doing business