TTB White LOGO TB
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Trending
How to Fix Problem Ejecting USB Mass Storage Device in Windows 10 & 11
iPad Not Charging When Plugged In: How to Fix in 2025
How to View and Turn on Radar Alerts in Google Maps
How to Test and Use Claude 3.5 Sonnet: A Complete Guide
How to Set Up a TikTok Ads Manager & Business Center in 2025
Tuesday, Jun 3, 2025
The Tech BasicThe Tech Basic
Font ResizerAa
Search
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Follow US
Anthropic's Aggressive Web Crawling Disrupts Websites
The Tech Basic > News > Anthropic’s Aggressive Web Crawling Disrupts Websites
News

Anthropic’s Aggressive Web Crawling Disrupts Websites

Evelyn Blake
Last updated: 28 July 2024 18:03
Evelyn Blake
Share
SHARE

Several website operators have accused the company Anthropic and its chatbot Claude of excessively crawling their websites, disrupting normal operations. Matt Barrie, CEO of Freelancer.com, claims that Anthropic is “by far the most aggressive crawler” of his freelance platform.

Please follow us on Twitter and Facebook

According to data provided to the Financial Times, Freelancer.com received 3.5 million visits from a web crawler connected to Anthropic in just four hours. This amount of data reportedly far exceeds that of other AI crawlers. Barrie estimates that Anthropic’s crawling activity is “probably about five times that of the second largest AI crawler.”

The problem is not limited to Freelancer.com. Kyle Wiens, CEO of iFixit.com, reported that his repair website received one million hits from Anthropic bots within 24 hours.

Impacts and Responses to AI Website Scraping

Anthropic sees itself as a developer of responsible AI systems. The company said it is investigating the reported incidents and respecting the wishes of publishers, reports the Financial Times. The AI startup claims to adhere to standard web protocols such as robots.txt, which govern crawler behavior, but some website operators report that their attempts to deny access to Anthropic’s bots have been unsuccessful.

Eric Holscher, co-founder of document hosting website Read the Docs, pointed out the financial impact of AI crawlers, citing the significant cost of traffic in a recent blog post. As AI companies compete to develop more powerful models, they are looking for new sources of training data. Some, like OpenAI, have partnered with content providers. Anthropic has not yet announced any similar agreements.

Read Also: According to the AI Act, Meta’s New Llama 3.1s Are a “Systemic Risk”: The Issue with TFLOPS

TAGGED:AI
Share This Article
Facebook Reddit Copy Link Print
Share
Evelyn Blake
By Evelyn Blake
Follow:
Evelyn Blake is an investor in technology and journalist who has been in the nascent space since 2014. Her love and passion for technological innovations made her delve deeper into the world of technology evolution. As a journalist, Evelyn has been covering latest trends and emerging gadgetries. She is a philanthropist and human rights activist.

Let's Connect

FacebookLike
XFollow
PinterestPin
InstagramFollow
Google NewsFollow
FlipboardFollow

Popular Posts

Problem Ejecting USB Mass Storage

How to Fix Problem Ejecting USB Mass Storage Device in Windows 10 & 11

S.Dyema Zandria
iPad Not Charging

iPad Not Charging When Plugged In: How to Fix in 2025

S.Dyema Zandria
Radar Alerts in Google Maps

How to View and Turn on Radar Alerts in Google Maps

S.Dyema Zandria
Claude 3.5 Sonnet

How to Test and Use Claude 3.5 Sonnet: A Complete Guide

S.Dyema Zandria

You Might Also Like

Character AI
Blog

Character AI: Review – Is it Safe for Teens and Kids?

New Gemini Feature
News

No More Reading Long Emails? Google’s New Gemini Feature

Grammarly
News

Grammarly’s $1 Billion Boost to Build the Future of AI Productivity

Google Veo 3
BlogNews

Google Veo 3: The Next Frontier in AI-Generated Video Content

Social Networks

Facebook-f Twitter Instagram Pinterest Rss

Company

  • About Us
  • Our Team
  • Contact Us

Policies

  • Disclaimer
  • Privacy Policy
  • Cookies Policy
Latest
How Instagram Edits Empowers Creators with Advanced AI Video Editing Tools
ChatGPT 4.5 Whats New, Features, Access and Comparison with ChatGPT 4.0
Apple Prepares a New Games App for WWDC
Opera Neon: The AI Browser That Works While You Sleep
US Now Orders Chrome Users to Update by June 5 Amid Hack Threat

© 2024 The Tech Basic INC. 700 – 2 Park Avenue New York, NY.

TTB White LOGO TB
Follow US
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?