TTB White LOGO TB
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Trending
Nvidia to Resume H20 AI Chip Sales in China After Whiplash
Meta Follows YouTube with Crackdown on Unoriginal Facebook Posts
iPhone 17 Pro Copper-Orange Color Leaked With Full Series Palette
China Demand Dips as Apple Sees Double‑Digit Gains Elsewhere
Google Gemini Flaw Exposes Email Summaries to Hidden Phishing
Wednesday, Jul 16, 2025
The Tech BasicThe Tech Basic
Font ResizerAa
Search
  • News
  • PC & Hardware
  • Mobiles
  • Gaming
  • Electronics
  • Gadget
  • Reviews
  • How To
Follow US
Anthropic's Aggressive Web Crawling Disrupts Websites
The Tech Basic > News > Anthropic’s Aggressive Web Crawling Disrupts Websites
News

Anthropic’s Aggressive Web Crawling Disrupts Websites

Evelyn Blake
Last updated: 28 July 2024 18:03
Evelyn Blake
Share
SHARE

Several website operators have accused the company Anthropic and its chatbot Claude of excessively crawling their websites, disrupting normal operations. Matt Barrie, CEO of Freelancer.com, claims that Anthropic is “by far the most aggressive crawler” of his freelance platform.

Please follow us on Twitter and Facebook

According to data provided to the Financial Times, Freelancer.com received 3.5 million visits from a web crawler connected to Anthropic in just four hours. This amount of data reportedly far exceeds that of other AI crawlers. Barrie estimates that Anthropic’s crawling activity is “probably about five times that of the second largest AI crawler.”

The problem is not limited to Freelancer.com. Kyle Wiens, CEO of iFixit.com, reported that his repair website received one million hits from Anthropic bots within 24 hours.

Impacts and Responses to AI Website Scraping

Anthropic sees itself as a developer of responsible AI systems. The company said it is investigating the reported incidents and respecting the wishes of publishers, reports the Financial Times. The AI startup claims to adhere to standard web protocols such as robots.txt, which govern crawler behavior, but some website operators report that their attempts to deny access to Anthropic’s bots have been unsuccessful.

Eric Holscher, co-founder of document hosting website Read the Docs, pointed out the financial impact of AI crawlers, citing the significant cost of traffic in a recent blog post. As AI companies compete to develop more powerful models, they are looking for new sources of training data. Some, like OpenAI, have partnered with content providers. Anthropic has not yet announced any similar agreements.

Read Also: According to the AI Act, Meta’s New Llama 3.1s Are a “Systemic Risk”: The Issue with TFLOPS

TAGGED:AI
Share This Article
Facebook Reddit Copy Link Print
Share
Evelyn Blake
By Evelyn Blake
Follow:
Evelyn Blake is an investor in technology and journalist who has been in the nascent space since 2014. Her love and passion for technological innovations made her delve deeper into the world of technology evolution. As a journalist, Evelyn has been covering latest trends and emerging gadgetries. She is a philanthropist and human rights activist.

Let's Connect

FacebookLike
XFollow
PinterestPin
InstagramFollow
Google NewsFollow
FlipboardFollow

Popular Posts

Nvidia

Nvidia to Resume H20 AI Chip Sales in China After Whiplash

Salman Akhtar
Meta

Meta Follows YouTube with Crackdown on Unoriginal Facebook Posts

Salman Akhtar
iPhone 17 Pro

iPhone 17 Pro Copper-Orange Color Leaked With Full Series Palette

Salman Akhtar
Apple

China Demand Dips as Apple Sees Double‑Digit Gains Elsewhere

Salman Akhtar

You Might Also Like

Google Gemini
News

Google Gemini Flaw Exposes Email Summaries to Hidden Phishing

xAI and Grok
News

xAI and Grok Apologize After Chatbot’s Antisemitic Outburst

Meta Acquires Play AI
News

Meta Acquires Play AI to Advance Its Generative Voice Technology

Apple Smart Home Hub
News

Delayed by Siri Enhancements Apple Smart Home Hub Will Arrive in 2026

Social Networks

Facebook-f Twitter Instagram Pinterest Rss

Company

  • About Us
  • Our Team
  • Contact Us

Policies

  • Disclaimer
  • Privacy Policy
  • Cookies Policy
Latest
Google Merges Chrome OS and Android into One Unified Platform
iPhone 17 Debut Scheduled for Second Week of September
AWS to Debut AI Agent Marketplace at New York Summit with Anthropic
RealSense Breaks Free from Intel, Raises $50 Million to Grow
Google DeepMind Snaps Up Windsurf CEO After OpenAI Deal Unravels

© 2024 The Tech Basic INC. 700 – 2 Park Avenue New York, NY.

TTB White LOGO TB
Follow US
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?