Outsource the whois server list

[malkusch]

Hi

I see that there are a handful of projects which do maintain their own individual whois server list. How about combining those lists into one single place and just maintain that together?

I created https://github.com/whois-server-list/whois-server-list where you can find a XML list containing domains (TLDs and SLD) with its whois servers from IANA, PSL and several whois client projects. The schema is not fix and could be changed within a new major version.

Also I'd like to use this opportunity to ask you if I may incorporate your list. I use the WTFL license.

[akzhan]

Please use this list w/o any permissions, using of yours is subject of PR.

[RaeesBhatti]

We're hosting our fork of php-whois at https://github.com/hostingini/php-whois and have a pretty good list.
@malkusch, I'm really interested in creating a central server for checking if a domain is available or not. If more people are interested maybe we can come up with a protocol that returns the domain availability in boolean. We are already doing this at HostinGini, after we've decided about the protocol, we can make it public.

[malkusch]

We're hosting our fork of php-whois at https://github.com/hostingini/php-whois and have a pretty good list.

Would you mind to create a PR so that regru would merge that informations as well?

I'm really interested in creating a central server for checking if a domain is available or not.

You know that some whois servers (e.g. whois.nic.de) do have very hard rate limits per IP? So having one central server would very quickly result in getting no response.

[RaeesBhatti]

Yes, I know about the rate limit per IP but lets say that we create a program that can be used to check domain availability like the whois command and listens on a certain port like a WhoIs server. Now when you run a availibility search, for example, isavailable example.com it will forward the request to central server, now central server can route the request to some random server which requested a domain availibity search in the past and return the reponse to requester. All of this can happen under 500ms, which is not bad at all. And we'll have the first ever protocol for domain availability search.
Similar to torrent but different in purpose.

[malkusch]

I once had such a availability service as a web interface. Guess what happened? A bot net hit that web interface with about 5GB traffic, 40,000 ip addresses and 160,000 hits per month just to query if well known domain addresses will become free. You know that this service what you have in mind will attract these kind of users.

But the idea as far as I understand could work.

[RaeesBhatti]

If we do this on a website it will actually be securer, because we can use CloudFlare, they've successfully mitigated the biggest DDoS in the history of internet 400gbps.
But we could also create a protected API, so that only people with the API key will be able to make requests. There could be a paid plan or something to gain higher number of outgoing API request.
And since the requester and the actual provider will never know about each other, they are both secure. A Firewall rule can be created to accept requests from certain IPs only to a specific port.

[malkusch]

@raeesiqbal FYI I did release a Whois API which you might find useful.

[RaeesBhatti]

Cool.... will look into it.