On Friday, I did a Reddit AMA alongside the r/WebScrapingInsider subreddit concerning the law of web scraping. You can see the complete conversation here.
I posted an introduction to the AMA, which I am sharing here:
__
Hi everyone, and gratitude for joining this Reddit AMA on web scraping! I idea it would be helpful to commencement alongside several introductory remarks.
First, and most importantly, a reminder concerning the dirt rule. Nothing in this AMA constitutes lawful advice. I volition conversation several broad lawful principles, but I can’t opine how those principles use to your circumstances. Further, many of the topics we’re going to conversation are contested, and everyone has distinct tolerances for lawful risk. Before you create any decisions, you need to conversation alongside a attorney concerning how the law applies to your circumstance and what flat of lawful hazard you’re consenting to accept.
Also, as Ian mentioned, I am focused on US law. The rules exterior the US could appearance fairly different!
Some backdrop concerning me: I am an OG Internet Law expert. I got my archetypal email document in 1991, published my archetypal Internet Law document in 1993, started practicing Internet Law in 1994, and started instruction an Internet Law way in 1996—a way I motionless instruct today, 30 years later. I have been a full-time law prof since 2002.
As an Internet Law expert who cast his intelligent character in the 1990s, I motionless method Internet Law as a generalist, which was the norm rear then. I don’t concentration on any one topic of Internet Law. Instead, I motionless track and pay notice to dozens of Internet Law topics. The scraping/trespass to chattels doctrines are fair a subset of my general portfolio of interests. You can see my scholarly writings at https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=332758 Some of my another scholarly priorities contain liability for user-generated satisfied (UGC)/Section 230, online trademark law, and emoji law. My forthcoming publish project volition be concerning the Internet’s pro-social benefits and how lawful regulation jeopardizes those benefits.
Despite my generalist orientation, I discover the law of abrasion particularly interesting. I got my archetypal client inquiry concerning abrasion likely about 1997. When I was in-house, abrasion was an evergreen topic. I routinely test on abrasion in my Internet Law course. For example, this year’s midterm was according to Vox Marketing Group v. Prodigy Promos, 556 F. Supp. 3d 1280 (2021).
* * *
Part of why I discover the topic so fascinating is that I have complex feelings concerning abrasion from a guideline standpoint.
On the one hand, I accept asset owners should mostly have the capability to decide how their asset is used. Scraping can affect unwanted and unconsented uses of a asset owner’s property, in ways that can be pernicious.
On the another hand, I accept in admission to information. (My mom was a librarian). To the degree that abrasion enables improved insights, improved knowledge, and stronger markets, it’s a social benefit.
Furthermore, the Internet succeeded in part since of its seamless, low-friction, low-transactions-cost navigation. To keep that architecture, browsing the Internet have to be a permission-less experience. Anti-scraping doctrines endanger to change web browsing into a permissioned experience, anywhere there are checkpoints at all site’s perimeter and approval is needed for access. Those barriers endanger to undermine the Internet’s navigability and utility, in ways that volition disadvantage all of us.
[Note: Trespass to chattels doctrines are fair one way that services can enforce permissioned admission into web navigation. Copyright law is another. Yet another is the online age authentication mandates sweeping the globe. For additional on my opposition to age authentication mandates, see https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5208739 An apparent point: if age authentication checkpoints rotate into ubiquitous in the Internet, that volition have important impacts on abrasion since most satisfied volition live rearward age authentication walls].
I have never completely written up my complex thoughts concerning these topics, but this part volition provision you a awareness of my thinking. https://blog.ericgoldman.org/archives/2013/04/rethinking_onli.htm
Here’s how I example several guideline considerations of abrasion for students:
On the affirmative side: Scraping can allow services that create important social benefits, specified as hunt engines, cost difference bots, and AI modelbuilding. Also, anti-scraping efforts are frequently anti-competitive, i.e., an intent to industry or keep monopolies. In particular, anti-scraping strategies can portray an attempt to get IP-like rights, equal if the assistance doesn’t meet requirements for IP protection. You can see several of these dynamics in the Register.com v. Verio case, anywhere Register.com was using contracts and trespass to chattels to defend non-copyrightable matter (WHOIS registrant communication info) to maintain its first-mover-advantage authority complete that communication information.
On the minus side, abrasion can appearance akin a DOS assault (intentionally or not). It can facilitate enormous copying, including perchance copyright infringements. And abrasion can be a enormous intrusion of person privacy.
So, from a guideline standpoint, is abrasion a fine or bad thing? ¿Por qué no los dos?: Or, to put it into legalese, “it depends.”
* * *
As for the law of scraping, it’s a mess, particularly the trespass to chattels doctrines (I summarize them as “TTC”). For concerning two decades, I taught the Intel v. Hamidi and Register.com v. Verio cases. Around 2024, I felt akin those cases no longer reflected the current lawful state, so I regretfully took them out of my Internet Law casebook. [For additional on the casebook, see https://blog.ericgoldman.org/archives/2026/07/announcing-the-2026-edition-of-my-internet-law-casebook.htm] I have not yet established a fine replacement case to add to the casebook, so my casebook safety of the topic is now adrift. I now instruct the X v. Bright Data ruling (the archetypal one granting the motion to dismiss) and it’s not extremely satisfying. Because there is no fine sole case or set of cases to instruct the topic, and since the law is so unsettled, I inform my students that the topic has rotate into “unteachable.” You can ideate how much they akin hearing that LOL.
I instruct students to run through a four-step procedure to decide if/how trespass to chattels applies:
Step 1: does the plaintiff own the affected chattel? Usually, the chattel is the servers or another bodily hardware that is used to run the server and nexus it to the Internet. The trespass to chattels doctrines are a fine example of the interplay between the law of bodily area and activities taking location in a virtual environment. Students sometimes battle to cover their heads about the distinctions between tangible items (i.e., “chattel”) and intangible things, specified as data or online accounts, which may be protected by another lawful doctrines (copyright or another IP) but don’t legally meet requirements as the “chattel.”
Step 2: did the accused “use” the chattel? Usually, sending or receiving packets from a server qualifies.
Created by ChatGPT Dec. 2025
Step 3: was the defendant’s use “unauthorized”? The law power assume that a server owner impliedly permits average Internet traffic merely by making the choice to nexus to the open Internet. How can the server owners eliminate this implied permission? This is among the areas that is unteachable. There is no apparent answer to how a server owner can construction this conversation alongside Internet users. Some of the imaginable tools that a server owner MIGHT be capable to use to eliminate any implied authority to use their chattel:
- TOSes
- C&D letters
- Technical admission controls akin registration walls
- Technical admission controls akin IP location blocking
- Signals specified as robot elimination headers
But, none of these mechanisms are guaranteed to work. This is an area of doctrinal chaos. I’m certain we’ll expend several period discussing if and whenever those efforts are effective. In general, for all bullet, I have additional questions than answers.
Step 4: did the defendant’s unauthorized use create legally recognizable harm? This is additionally an unteachable aspect because, again, I typically have additional questions than answers concerning what damage volition suffice. The court’s flip-flop in the X v. Bright Data case is a fine example of how unpredictable this topic is.
* * *
For my students, I summarize the law of abrasion alongside the following tagline: “Scraping is ubiquitous but not plainly legal.” This breaks my students’ brains. If it’s so ubiquitous, it must be plainly legal…right?
In command for a abrasion run to be legal, we have to navigate ALL of the following doctrines:
- Trespass to chattels. I inform my students that they should continually conversation all three of these doctrines as a troika:
- Federal statutes, particularly the Computer Fraud and Abuse Act (CFAA).
- State statutes, specified as the province device infringement laws. Here in California, we concentration on Cal. Penal Code 502.
- Common law trespass to chattels, specified as the Hamidi and Bright Data cases.
- Copyright law (106). I expend several period explaining why average web browsing power be copyright infringement since copyrighted matter necessarily is downloaded into the browser’s RAM. The students really don’t akin that conversation LOL. If web browsing is infringement, afterward abrasion would be as well. In supplement to the first downloading, scrapers power encounter additional 106 difficulty alongside republishing several or all of the scraped material.
- Copyright law DMCA (1201/1202). I don’t shield this viewpoint in category yet since it’s too underdeveloped and chaotic, but I’m certain we’ll expend several period on 1201 today. To provision you a preview, the 1201 rulings to date interrupt my brain, and I have additional questions than answers.
- Many TOSes merge anti-scraping provisions, but it’s not apparent if the TOSes are correctly formed. In my Internet Law course, we expend a complete 2 weeks discussing TOS formation. The law evolved a lot in the 2025 decisions in Chabolla and Godun in the 9th Circuit.
There may be another lawful doctrines raised by scraping, specified as trademarks and privacy rights.
So here’s why I say that abrasion is not plainly legal. In command for a abrasion run to be permissible, it must last all and all among the doctrines enumerated above. If the run fails any among the doctrines, it’s not lawful and the scraper can visage important consequences (including, in several cases, offender liability). It is daunting to build a abrasion run that can successfully navigate so many lawful doctrines perfectly.
* * *
My apologies for specified a long-winded introduction (although, I prompt you that I’m a law professor, so loquaciousness isn’t really that surprising, is it?) The law of abrasion is so complex and multi-faceted that it’s uncomplicated for the conversation to go downward rabbitholes or for conversants to conversation former all other. I anticipation this intro provides several helpful construction and environment for my additional detailed comments.
Thanks again for engaging alongside the AMA. A covering apology ongoing for all of the times I answer “it depends,” “I don’t know,” or “that’s a fine question.” And in all cases, don’t obtain my term as the final one—talk alongside your attorney if you are making any decisions!
