TTB White LOGO TB
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Trending
Why Modular Phones Failed to Change the Smartphone Game
What is Calm Technology and Why it Matters in a Busy World
Inside the Terahertz Communication – How 6G Will Reshape Wireless Connectivity
iOS 26 Redefines CarPlay: Sleek New Features Enhance Driving
Apple AI Roadmap: Contextual Siri 2026, Knowledge Chatbot & Copilot
Saturday, Jun 14, 2025
The Tech BasicThe Tech Basic
Font ResizerAa
Search
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Follow US
Anthropic's Aggressive Web Crawling Disrupts Websites
The Tech Basic > News > Anthropic’s Aggressive Web Crawling Disrupts Websites
News

Anthropic’s Aggressive Web Crawling Disrupts Websites

Evelyn Blake
Last updated: 28 July 2024 18:03
Evelyn Blake
Share
SHARE

Several website operators have accused the company Anthropic and its chatbot Claude of excessively crawling their websites, disrupting normal operations. Matt Barrie, CEO of Freelancer.com, claims that Anthropic is “by far the most aggressive crawler” of his freelance platform.

Please follow us on Twitter and Facebook

According to data provided to the Financial Times, Freelancer.com received 3.5 million visits from a web crawler connected to Anthropic in just four hours. This amount of data reportedly far exceeds that of other AI crawlers. Barrie estimates that Anthropic’s crawling activity is “probably about five times that of the second largest AI crawler.”

The problem is not limited to Freelancer.com. Kyle Wiens, CEO of iFixit.com, reported that his repair website received one million hits from Anthropic bots within 24 hours.

Impacts and Responses to AI Website Scraping

Anthropic sees itself as a developer of responsible AI systems. The company said it is investigating the reported incidents and respecting the wishes of publishers, reports the Financial Times. The AI startup claims to adhere to standard web protocols such as robots.txt, which govern crawler behavior, but some website operators report that their attempts to deny access to Anthropic’s bots have been unsuccessful.

Eric Holscher, co-founder of document hosting website Read the Docs, pointed out the financial impact of AI crawlers, citing the significant cost of traffic in a recent blog post. As AI companies compete to develop more powerful models, they are looking for new sources of training data. Some, like OpenAI, have partnered with content providers. Anthropic has not yet announced any similar agreements.

Read Also: According to the AI Act, Meta’s New Llama 3.1s Are a “Systemic Risk”: The Issue with TFLOPS

TAGGED:AI
Share This Article
Facebook Reddit Copy Link Print
Share
Evelyn Blake
By Evelyn Blake
Follow:
Evelyn Blake is an investor in technology and journalist who has been in the nascent space since 2014. Her love and passion for technological innovations made her delve deeper into the world of technology evolution. As a journalist, Evelyn has been covering latest trends and emerging gadgetries. She is a philanthropist and human rights activist.

Let's Connect

FacebookLike
XFollow
PinterestPin
InstagramFollow
Google NewsFollow
FlipboardFollow

Popular Posts

Modular Phones

Why Modular Phones Failed to Change the Smartphone Game

Salman Akhtar
Calm Technology

What is Calm Technology and Why it Matters in a Busy World

Salman Akhtar
Terahertz Communication

Inside the Terahertz Communication – How 6G Will Reshape Wireless Connectivity

Salman Akhtar
iOS 26

iOS 26 Redefines CarPlay: Sleek New Features Enhance Driving

Salman Akhtar

You Might Also Like

Apple AI
News

Apple AI Roadmap: Contextual Siri 2026, Knowledge Chatbot & Copilot

Nudify Apps
News

Meta Slams Down on Nudify Apps After Critical Exposure

AI Gaming Studio
Blog

Elon Musk’s AI Gaming Studio: What It Is and What It Means for the Future

macOS Tahoe
Blog

macOS Tahoe 26 Features: Liquid Glass Design, Apple Intelligence, Spotlight

Social Networks

Facebook-f Twitter Instagram Pinterest Rss

Company

  • About Us
  • Our Team
  • Contact Us

Policies

  • Disclaimer
  • Privacy Policy
  • Cookies Policy
Latest
Your Photos Reborn: iOS 26’s Tabbed Interface and AI-Powered 3D Revolution
New from Logitech: Sleek iPad Case with a Compact, Removable Wireless Keyboard
iOS 26 Introduces Smarter Management Tools to Safeguard Child Accounts
Apple Music in iOS 26 Redefines the Lock Screen with Stunning Motion, Color, and Depth
iOS 26 vs iOS 18: Is Apple’s Liquid Glass the Future of UI?

© 2024 The Tech Basic INC. 700 – 2 Park Avenue New York, NY.

TTB White LOGO TB
Follow US
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?