TTB White LOGO TB
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Trending
iPhone 17 Pro Vapor Chamber Cooling System Leaked: Thermal Upgrade
Google Launches Hands-Free AI Chat in Search for Android, iOS
Snap Acquires Saturn: Social Calendar App for Students to Enhance Snapchat
AI at Risk? Anthropic Flags Industry-Wide Threat of Model Manipulation
Meta Courts Scale AI as Backup After Failed Perplexity Bid
Monday, Jun 23, 2025
The Tech BasicThe Tech Basic
Font ResizerAa
Search
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Follow US
Anthropic's Aggressive Web Crawling Disrupts Websites
The Tech Basic > News > Anthropic’s Aggressive Web Crawling Disrupts Websites
News

Anthropic’s Aggressive Web Crawling Disrupts Websites

Evelyn Blake
Last updated: 28 July 2024 18:03
Evelyn Blake
Share
SHARE

Several website operators have accused the company Anthropic and its chatbot Claude of excessively crawling their websites, disrupting normal operations. Matt Barrie, CEO of Freelancer.com, claims that Anthropic is “by far the most aggressive crawler” of his freelance platform.

Please follow us on Twitter and Facebook

According to data provided to the Financial Times, Freelancer.com received 3.5 million visits from a web crawler connected to Anthropic in just four hours. This amount of data reportedly far exceeds that of other AI crawlers. Barrie estimates that Anthropic’s crawling activity is “probably about five times that of the second largest AI crawler.”

The problem is not limited to Freelancer.com. Kyle Wiens, CEO of iFixit.com, reported that his repair website received one million hits from Anthropic bots within 24 hours.

Impacts and Responses to AI Website Scraping

Anthropic sees itself as a developer of responsible AI systems. The company said it is investigating the reported incidents and respecting the wishes of publishers, reports the Financial Times. The AI startup claims to adhere to standard web protocols such as robots.txt, which govern crawler behavior, but some website operators report that their attempts to deny access to Anthropic’s bots have been unsuccessful.

Eric Holscher, co-founder of document hosting website Read the Docs, pointed out the financial impact of AI crawlers, citing the significant cost of traffic in a recent blog post. As AI companies compete to develop more powerful models, they are looking for new sources of training data. Some, like OpenAI, have partnered with content providers. Anthropic has not yet announced any similar agreements.

Read Also: According to the AI Act, Meta’s New Llama 3.1s Are a “Systemic Risk”: The Issue with TFLOPS

TAGGED:AI
Share This Article
Facebook Reddit Copy Link Print
Share
Evelyn Blake
By Evelyn Blake
Follow:
Evelyn Blake is an investor in technology and journalist who has been in the nascent space since 2014. Her love and passion for technological innovations made her delve deeper into the world of technology evolution. As a journalist, Evelyn has been covering latest trends and emerging gadgetries. She is a philanthropist and human rights activist.

Let's Connect

FacebookLike
XFollow
PinterestPin
InstagramFollow
Google NewsFollow
FlipboardFollow

Popular Posts

Vapor Chamber Cooling System

iPhone 17 Pro Vapor Chamber Cooling System Leaked: Thermal Upgrade

Salman Akhtar
Engadget

Google Launches Hands-Free AI Chat in Search for Android, iOS

Salman Akhtar
SnapChat

Snap Acquires Saturn: Social Calendar App for Students to Enhance Snapchat

Salman Akhtar
Anthropic

AI at Risk? Anthropic Flags Industry-Wide Threat of Model Manipulation

Salman Akhtar

You Might Also Like

Meta
News

Meta Courts Scale AI as Backup After Failed Perplexity Bid

Meta
News

Meta’s Next Smart Glasses Feature Oakley Design and 3K Recording

Siri
How To

How to Use Siri with ChatGPT for Instant AI Help via Apple Intelligence

Midjourney
News

Midjourney V1 AI Video Model Launches: Image to Video on Discord

Social Networks

Facebook-f Twitter Instagram Pinterest Rss

Company

  • About Us
  • Our Team
  • Contact Us

Policies

  • Disclaimer
  • Privacy Policy
  • Cookies Policy
Latest
TikTok’s Legal Limbo: Trump’s Third Extension Defies Supreme Court Ruling
Adobe’s Latest Camera App Brings Smart Photography to Your iPhone
Punch-Hole Camera, Under-Screen Face ID Still Expected With iPhone 18 Pro
Microsoft Password Deletion: Act Now Before August Deadline
Facebook Messenger Passkey Login: iOS Android Security

© 2024 The Tech Basic INC. 700 – 2 Park Avenue New York, NY.

TTB White LOGO TB
Follow US
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?