1

So let's say that after 50 requests in a period of 30 seconds, a user's IP gets blacklisted. They can still bypass this block by changing their IP, which is quite easy and fast nowadays.

So, what other methods could I use to prevent my public API's limit to get bypassed? There is no user auth at all. My API involves no browser usage, just a terminal so I don't know if using cookies or something like that is even an option.

Thanks for your help!

Gustavo
  • 11
  • 1

1 Answers1

2

Data scraping is now a global industry. Some of these businesses (ie, Mozenda) have 10k+ IP addresses they cycle thru for the same api pull. So what may look like different users to you may in fact be 1 user sucking your data dry by making a copy.

Blocking a single IP address only targets small users. And browsers can be emulated very easily using headless browsers.

The question is why wouldn't you add user authentication? Its easy to add and ensures you know who is using your data. And you can also set limits that way. Or you can charge the user a fair price which would mean they dont have to scrape your data. OR better yet sell them full access to your data annualized. Hope this helps.

Jim

user3097022
  • 21
  • 1