Is web scraping legal? What business owners should know
Published
Scraping publicly available information is common and, in many countries, generally allowed. The risk depends on what you collect, how you collect it and what you do with it afterwards. This guide is general information, not legal advice. For a specific project, ask a lawyer in your country.
Usually lower risk
- Facts that anyone can see without logging in, such as product names, prices and stock
- Public business listings like company names, addresses and business phone numbers
- Your own data on a platform you have an account with
Where you need to be careful
Personal data. Laws such as GDPR in Europe treat information about people, even when public, as personal data. Collecting it needs a lawful reason and careful handling.
Logins and paywalls. Getting around a login, paywall or technical block you are not allowed through is a different situation from reading a public page.
Copyrighted content. Prices and facts are one thing. Copying full articles, photos or reviews to republish them as your own can break copyright.
Website terms. Many sites have terms that restrict automated access. Whether those terms are enforceable varies, but they matter.
Server load. Sending thousands of requests a second can harm a site. Responsible scrapers slow down.
The rules I follow on every job
- I only collect publicly available data, or data from accounts the client owns.
- I rate-limit every scraper so target sites are not overloaded.
- I do not get around paywalls or security measures.
- I turn down jobs that look like they are meant to harm people, such as collecting private details for spam or harassment.
- Client files are used for that job only and deleted after delivery.
If you are not sure whether your project is a good idea, send me the details. I will tell you honestly, before any payment.