14

If I bought a domain like a9j47fn83jd8j49.tld, and only a friend and I ever visit it, who would know about it? It sounds like for most TLDs, there is some reporting that has to happen after you buy a domain, but does it always work like that?

Maybe having just ~4 billion IPv4 addresses makes this inevitable. In an IPv6 world, would it be harder to map out the whole internet? Not trying to rely on any secrecy necessarily, but I do notice that my personal sites always get scanned by bots very quickly, and it got me curious about whether it's inevitable that everyone knows every name out there. Certainly someone needs to know if a domain is taken or not, but that doesn't necessarily mean they always have to publish a list of all domains, right?

dcc310
  • 243
  • 2
  • 5
  • 6
    You cpuld create a subdomain - provided you run the nameservers for it you would have good visibility and control of it, and it would not automatically get into any lists. – davidgo Dec 21 '21 at 07:08
  • 1
    A domain is like a entry in the phone book, that you buy. Of course you can create your own phone book to use, but it's not "the" phone book then. – Thomas Dec 21 '21 at 13:40
  • 1
    @davidgo Sure, as long as clients only ever query a9j47fn83jd8j49.tld IN NS publicly, and only send queries like secret-subdomain.a9j47fn83jd8j49.tld IN AAAA to the authoritative nameservers for a9j47fn83jd8j49.tld over an encrypted channel, but ensuring this is impossible without bespoke tooling or doing everything by hand. Else, whatever recursive name server or middleman you're talking to will just learn the subdomains you're trying to resolve. – Jivan Pal Dec 21 '21 at 14:01
  • With an alternative DNS like Namecoin names (.bit), you can retain secrecy. In the case of Namecoin, the hashes of names are stored on the blockchain, not the names themselves, so you need to know a name in the first place in order to resolve it to any records. In this way, names like "a9j47fn83jd8j49.bit" can stay secret because no one will guess them (unlike e.g. "google.bit"), although name-hashes are public. However, knowing the name-hash alone is not sufficient to get the record data, because that data can just be encrypted using the name itself as a key. – Jivan Pal Dec 21 '21 at 14:03
  • @JivanPal, are the players in the DNS system known to do much with the subdomains that they "overhear" day to day? I assume this varies a lot by country. I've seen some tools online that try to catalogue subdomains for a given domain, but it's imprecise, and I wonder how they attempt it. – dcc310 Dec 21 '21 at 18:12
  • "to the authoritative nameservers for a9j47fn83jd8j49.tld over an encrypted channel" Which does not exist today. DoH and DoT are from clients to recursive resolver, there is no defined standard as of now between recursive and authoritative nameservers. – Patrick Mevzek Dec 21 '21 at 18:21
  • 1
    "are the players in the DNS system known to do much with the subdomains that they "overhear" day to day?" Yes they do. For multiple reasons, from research to combat malware, etc. "but it's imprecise, and I wonder how they attempt it." There are a lot of ways (and questions on this site or other SE ones so you can find details), but you can start by reading https://blog.appsecco.com/a-penetration-testers-guide-to-sub-domain-enumeration-7d842d5570f6 – Patrick Mevzek Dec 21 '21 at 18:24
  • You know about NXT records right? – Joshua Dec 21 '21 at 18:35
  • @PatrickMevzek Authoritative nameservers may support DoT, and recursive nameservers may communicate with them using it, no? Of course, whether this is done in practice is another matter, and I guess what you're saying is that there is no standards document/draft to this effect. In any case, it'll probably end up the same way email relaying has, with plaintext needing to be supported/permitted for eternity, lest remaining legitimate servers without support for TLS be ignored. — EDIT: I somehow missed the part where you literally said there's no standard, but that does still surprise me. – Jivan Pal Dec 22 '21 at 00:53
  • 1
    @JivanPal It is right in the introduction of RFC 7858 defining DoT: " This document focuses on securing stub-to-recursive traffic, as per the charter of the DPRIVE Working Group. It does not prevent future applications of the protocol to recursive-to-authoritative traffic." Yet, these applications do have further problems which is why it is still discuss. At least two: the load of TLS on servers and how to verify the certificates... – Patrick Mevzek Dec 22 '21 at 02:52
  • @Joshua There are no NXT records, this is an abandoned case. You are probably thinking about NSEC, and in some part NSEC3. Yes, they do also provide a way to enumerate a zone. However with one big caveat. DNSSEC has to be enabled on the zone, which is a very low percentage today. – Patrick Mevzek Dec 22 '21 at 02:53
  • See also: How can I figure out how a search engine is finding hidden pages? – Stephen Ostermiller Dec 22 '21 at 15:48
  • If you really want to keep your new web server as "secret" as possible, you could think about not registering a domain for it, and instead using a dedicated IP address. It's a little more expensive, but maybe it's worth it to you. – Jeffiekins Dec 23 '21 at 05:07
  • Your domain being hit by bots has nothing to do with your DNS, and everything to do with your IP address. These automated attacks simply hit entire class C (or higher) networks. They don't care about your domain. – Tom Dec 23 '21 at 14:14

8 Answers8

14

Is it possible to keep the existence of a domain secret?

Secret to whom? And incidentally, why does it need to be secret?

Secret to your OS, applications you use (including plugins in your browser, etc.), ISP, DNS resolver you use (any public one?), registrar and registry involved, certainly not. Which makes already a long list.

(Add to that, potentially, any government or governmental agency of any country in which any of the entities in the above list reside).

Maybe having just ~4 billion IPv4 addresses makes this inevitable.

You are slightly confusing 2 things here also. Registering a domain name does not force it to resolve. Yet the mere fact it was registered, and sometimes to whom, can be considered a valuable information (ex: new companies, or merges, or new movies or games, etc.)

Not trying to rely on any secrecy necessarily, but I do notice that my personal sites always get scanned by bots very quickly

So that is another question, which may have multiple answers. What do you mean exactly by "personal sites"? And why it is a problem that the "bots" (which ones?) do come see your site?

And is it related to when a domain is registered or when you switch ON a new server with a new site attached to it?

They are numerous ways, and the below is certainly not exhaustive:

  • as, albeit wrongly, the other 2 answer states, they exist zonefiles, for gTLDs; almost anyone can get access to them, but it is updated only daily, so that can't explain a visit right after registering a domain
  • your ISP can look at your traffic, and resell patterns. Typically at the DNS level if you use them as DNS recursive resolver, but same for any public resolver you may use, how do you know about its policy regarding data?
  • multiple applications (ex: Skype) scan links in message, purposedly to detect harmful content, but that also mean as soon as you exchange a message with a link, you will get an hit on it; even private things; various other vendors or OS can have the same things, including in smartphones
  • with the world going to HTTPS, browsers rely more and more on Certificate Transparency Logs; public CAs are mandated to publish the certificates they issue, and those certificates contain names; this is almost real time plus in fact the certificates are technically stored there even before the certificate is issued to end client
  • etc. (I only gave the examples above as something you may not think about but are clearly source of data)
Patrick Mevzek
  • 8,375
  • 1
  • 20
  • 42
  • Helpful points! Just to clarify "personal sites", I mean some sites I maintain that have 1-2 intended users, across 2-3 devices (e.g. me and a roommate). Bots could often be friendly, though I've been following that recent "log4shell" exploit, where the mere requesting of a URL could lead to compromise. (But I'm not running Java, so admittedly this is theoretical, and I know at least some of the log4shell attempts in my logs are from benign security-focused companies) – dcc310 Dec 21 '21 at 02:14
  • 5
    "though I've been following that recent "log4shell" exploit, where the mere requesting of a URL could lead to compromise." Against that, trying to think to hide resources is not a solution. Because it is basically impossible to hide completely. If a resource is to be restricted to only some users, then restrict it (by IP, by authentication, etc.). Otherwise monitoring and security updates are the only practical way to go. – Patrick Mevzek Dec 21 '21 at 03:14
  • @PatrickMevzek "If a resource is to be restricted to only some users, then restrict it (by IP, by authentication, etc.)" This. You could even do something like MAC address if it's just you and a couple friends. Not as limiting as IP, no extra auth steps, still limited to your short list. This is not secure for "real" applications, it's an idea to compromise convenience with blocking most/all automated* attempts at access* – TCooper Dec 21 '21 at 19:02
  • 2
    @TCooper " You could even do something like MAC address" Doesn't work out of a local network (same WiFi for example). Plus otherwise, in "public", MAC addresses are not authenticated and hence can be spoofed. – Patrick Mevzek Dec 22 '21 at 02:55
12

No not possible. ICANN, the org maintaining DNS provides something that's called Centralized Zone Data Service (CZDS). Which provides lists of all registered domains.

Here is a bunch of info on that: https://czds.icann.org/help

sgtfrankieboy
  • 229
  • 1
  • 2
8

They do have a list of all domains.

Agencies responsible for many TLDs (top level domain, e.g. .com), publish a zone file, a list of all registered domains in that TLD, along with their DNS servers. It is downloadable as a flat file.

As such, people trying to find all domains simply do that. They don't bang on every IP address, and anyway, that wouldn't tell them about the domains on that IP.

However certain TLDs do not publish zone files. They probably publish new domain names in other ways, however. "Security through obscurity" doesn't work that well.

Harper - Reinstate Monica
  • 836
  • 5
  • 6
  • 2
    "Every agency responsible for every TLD (top level domain, e.g. .com), publishes a list of all registered domains in that TLD, along with their DNS servers. It is downloadable as a flat file." That is absolutely NOT TRUE. Zonefiles (or zone transfers) are available for root, and all gTLDs, per ICANN rules. Certainly not for all ccTLDs. Try to find the zonefile for .de or .fr... good luck! – Patrick Mevzek Dec 21 '21 at 00:35
  • @PatrickMevzek corrected, thanks. Yeah, I was never able to get those. – Harper - Reinstate Monica Dec 21 '21 at 01:35
  • 1
    I still disagree with "However certain TLDs do not publish zone files. Choosing one of them may help with your attempt at "security through obscurity". for many reasons outlined in my answer. For example .fr does not publish a zonefile but publishes a list of daily new names. – Patrick Mevzek Dec 21 '21 at 03:13
  • @Patrick OK I'll fix that too. – Harper - Reinstate Monica Dec 21 '21 at 04:57
  • Not publishing the zone file doesn't really help. As long as the TLD is DNSSEC-compatible, it's trivial to iterate the whole zone by following the NSEC chain. NSEC3 does not preclude this, just makes it more clostly, but the cost is well within practicality thanks to "other prominent applications" of brute forcing hashes. – R.. GitHub STOP HELPING ICE Dec 22 '21 at 18:18
  • "As long as the TLD is DNSSEC-compatible," It is not enough. If it uses "Opt-Out" for NSEC3 (doesn't exist for NSEC just because of not having the idea them) only the signed delegations below will be in the NSEC chain. "of course" big ccTLDs not providing zonefiles are also those using NSEC3+optout. – Patrick Mevzek Dec 22 '21 at 18:47
5

Consider this:

How did you ensure a9j47fn83jd8j49.tld wasn't already taken before buying it?

If your goal is to not be found out then you need to have a local network, a web server, a router-level DNS entry for your domain, and you both need to use a VPN to get to this network and view the website.

MonkeyZeus
  • 281
  • 1
  • 6
4

Alternative of what you are describing might be a Tor hidden service (<random>.onion) and requiring a Tor cookie(v2) or RSA key-pair(v3). That way even though it'd be published (and still rather annoying to retrieve the name), Tor itself would prevent the access to the service for you. Then again, you could just require cert auth via a clearnet webserver, however there's a difference in the DNS name owner's anonymity.

So for the use-case of only you and your friend accessing it, it'll work just fine and even if somebody knew the random name, it wouldn't matter because there's no legal document binding to it (or in other words, one can spawn N new domains in a while loop and nobody knows who it is). For the clearnet DNS record, there's at least the DNS registrar + its connected nodes that will be knowing the name and perhaps even doing a visit to it, you can't never know.

KeyWeeUsr
  • 153
  • 7
1

No... but also Yes!

No, in the typical sense of purchasing and registering a domain name, because the point of registering something is to make it publicly discoverable. Other answers have this covered.

But Yes (for what it sounds like you want to achieve), because the same effect can be achieved by customizing the DNS settings of the devices accessing the site, or the router those devices are on.

Consider Pi Hole, the popular software for ad blocking, among other things. Setting up a Pi Hole as the DNS server on your device/network makes it fail to resolve domain names that are on your block list, thus blocking ads. You could configure a similar DNS server (or possibly even Pi Hole itself) to resolve any arbitrary domain as any arbitrary IP, whether that domain is registered or not.

Doing this from anywhere would require hosting that DNS server online. Insert obligatory boilerplate about network security, if you would consider forwarding a port from your home network

NeatNit
  • 103
  • 2
automaton
  • 111
  • 1
1

I'm not sure why you wish to keep the domain name secret, or why you want a domain name at all if just you and a friend or two will access it. If you're thinking it can provide security, keep in mind that a hacker can just use the IP address directly without bothering with a domain name.

If you just want to make it easier than having to enter the IP address, you could consider skipping the domain registration entirely and just entering a fake domain name in the "hosts" file of any PC that will be used to access it. That wouldn't help if you're wanting to access it via a phone, but depending on your situation, it might be the simplest solution. Under Windows, the file is at C:\Windows\System32\Drivers\etc\hosts.

Wally Hartshorn
  • 181
  • 1
  • 1
  • 7
  • Well, my question is a bit theoretical. I do want my particular sites to be discovered, at the end of the day! In terms of motivation, if people could pay for "unlisted" sites, I bet they would sometimes choose that, like they often opt to have secret Google Docs / Pastebin links, which are accessible from anywhere in the world without authentication. (Though I'm also not saying the Internet should have been built this way) – dcc310 Dec 22 '21 at 22:30
  • Your point about the "hosts" file is a good one. Also, in terms of hackers accessing IP addresses directly, I assume it's feasible and commonplace for a single actor to hit all possible IPv4 addresses over time. But the space of possible domain names or IPv6 addresses is enormous (thus my mention pf it in the original question) – dcc310 Dec 22 '21 at 22:38
  • 1
    Web servers can be (and should be, in my opinion) configured to show an error page for IP address requests without a host name. – Stephen Ostermiller Dec 23 '21 at 11:55
0

A secret domain is a oxy-moron. The whole idea of registering a domain is so that the domain can point end-user to your web servers IP address. If you only have a couple of users you can easily setup your own web server with apache or ngx and just send your friend your IP address, but this would not use a domain. You can auctually visit most website by just entering there IP address in your web browser. Although knowing which one serves your area may be tricky. You may find that out of the 20 - 30 IP addresses that serve YouTube only a certain few will auctually serve your web browser any files.

Also you seem to be confusing domains with IP address. Big sites like YouTube have multiple IP addresses that all serve the same domain. If you want to forbid Google from crawling your website you can always use the noIndex directive in the head tag of you html.

Neil Meyer
  • 135
  • 8
  • "just send your friend your IP address, but this would not use a domain." and it won't work at all for things like HTTPS because of the certificate check (that is based on a name, and not an IP address). So no, just having an IP address instead of a name is not good to establish a connection nowadays. Plus, even outside of certificates and HTTPS, even in HTTP you can have multihoming (multiple servers/websites on the same IP address) so you need to have a name to resolve things at HTTP layer. – Patrick Mevzek Dec 22 '21 at 18:06
  • 3
    "This is in fact how much of the illegal content on the dark web is disseminated." Not at all. What is often described as the dark web is basically Tor, which uses .onion names. – Patrick Mevzek Dec 22 '21 at 18:07
  • "You can auctually visit most website by just entering there IP address in your web browser." Care to give examples for HTTPS? – Patrick Mevzek Dec 22 '21 at 18:07
  • Since you're probably looking for examples where you don't have to create a security exception (I think self-signed certs are perfectly reasonable for a just-for-friends service. Anyway): https://8.8.8.8 (Not sure whether you have to own a CA to pull this off. Also, if you get a CA signed cert, does your IP still count as secret?) – Caesar Dec 24 '21 at 07:05