4

I have a secret web page, let's call it something123.php, and looking through my Apache webserver access.log I waas surprised to see that two entities have learned of this page.

  1. Google
  2. Grapeshot.co.uk
  3. Comscore

I have the page in my bookmarks, but I noticed something especially odd.

In one case, 20 seconds after I accessed the secret page, Google did too.

In another case, 4 seconds after I accessed it, Comscore did too and then 5 minutes later, Grapeshot did.

I am curious to learn how this could be. I use Firefox, and my extensions consist of only one: UBlock Origin. I have a theme installed from Mozilla for dark mode. That's it.

I may have been running Ungoogled Chromium at the same time as Firefox but I would think it wouldn't leak information.

It feels like I have spyware on my computer.

wudude
  • 153
  • 4

1 Answers1

5

"It feels like I have spyware on my computer."

Oh, it's much worse than that. There is spyware on everything but your computer, probably.

Grapeshot is an Oracle service. Here are a few quotes from the crawler explanation page:

Oracle Data Cloud assists advertisers placing contextual advertising on web pages.

Pages are only visited on demand. If Oracle Data Cloud Crawler has visited your site it means that an ad was recently placed on a page where the information was either not yet available or needed to be refreshed. For this reason, you will often see a request from Oracle Data Cloud Crawler shortly after a user has visited a page.

That means you probably have ads running on the page. When you visit the page, Grapeshot and other crawlers also visit. Basically, your ads ratted you out and now the crawlers want to know what this secret page is all about.

A significant chain of systems may cause Oracle Data Cloud to analyze your site. Oracle Data Cloud Crawler provides real-time contextual information to a number of Real Time Bidding (RTB) systems, such as Rubicon, AppNexus, and more. These RTB systems are often used by third-party ad server systems as part of their ad serving strategy.

I like the way they worded this. "A significant chain of systems" is their way of saying "we don't feel comfortable listing all of the reasons for a crawl."

The other two crawlers are most likely there for the same reason. You can block the crawlers, but it could cause problems with the ads if this reporting is required. I would leave them to do what they want unless it's a problem (stressing your server).

Trich
  • 601
  • 3
  • 4