8

I maintain a site with a bunch of downloadable files. It is currently hosted on a server in the U.S., but I have recently acquired a new server in Germany. I would like to mirror the downloads to the server in Germany, and have a PHP script on the first server (hosting the website) detect which file mirror to use based on the user's location. For instance, if the user is in Canada, they should download the file from my current server in the U.S. If they're in France, they should get the file from Germany, rather than downloading across the Atlantic. How, then, can I determine which country they are closer to?

I know about MaxMind GeoIP, and have it installed, but that just gives me a country, and AFAIK, there is no way to automatically determine which of my two mirror countries the given country is closest to. I suppose what I could do is go by continent: have users in Asia, Europe, Africa, and Australia get the content from Germany, and have visitors from North and South America get the file from the U.S. If anyone can think of a better solution, I'm open to suggestions.

Well, I guess I am going to go go with my original idea of checking by continents. For others looking to do this sort of thing, that will be a good place to start. The problem will come when I have multiple mirrors in Europe, but the continent idea will have to work for now.

Bill the Lizard
  • 386,424
  • 207
  • 554
  • 861
Ashley Strout
  • 5,949
  • 5
  • 24
  • 44
  • You could call Google maps through the google API and calculate the distance between the country the IP-user live in, and your two server destinations, based on which of them is closest, set the download mirror. Then you only have two static variables. One for each server location. – Jonas m Jan 31 '12 at 20:51
  • I'm waaay outside my area with this, but I *think* there's a way to use dns to handle this...? A little searching yields this: http://stackoverflow.com/questions/1040545/how-to-dispatch-network-requests-to-the-geographically-closest-server – Aerik Jan 31 '12 at 20:57
  • @Jonasm: not a bad idea, but I believe you can't just use the Maps API to calculate distance, you actually have to show a map, which is definitely unsatisfactory in this case. – Ashley Strout Jan 31 '12 at 21:00
  • You can always hide the map inside a non-visible container and just pull the required data. That will be the fastest and easiest way for you i believe – Jonas m Jan 31 '12 at 21:02
  • @Jonasm you have to actually _display_ the map. See http://code.google.com/apis/maps/terms.html, section 10.1.1 part (g). – Ashley Strout Jan 31 '12 at 21:34
  • Instead of creating your own mirrors, have you looked into amazon cloudfront, or akamai, or one of the many other content distribution networks. They would take care of all of this for you. – bumperbox May 01 '12 at 01:49
  • 1
    @bumperbox In my situation this is not feasible. It is a good idea for those who can afford it, but I'm running this site on a very bare bones budget. Really, I could do without a location-aware mirror system entirely, but I feel like this is something that should be doable and I hope this question (and hopefully a good answer) will be able to be referenced by people needing to do this. It seems a fairly common scenario. – Ashley Strout May 01 '12 at 06:35
  • 1
    Using anycast dns is the best way to do this, to cut a long story short, all your servers get the same ip address and users are seamlessly routed to the closest one without any code being required. This has the advantage of also giving your fall over protection if a server goes down and the ability to add new servers without worrying about a load of changes to your application every time. – Anigel May 02 '12 at 16:35
  • PHP.net has mirror selection / offering by country, http://www.php.net/source.php?url=/include/ip-to-country.inc – hakre May 03 '12 at 15:26

5 Answers5

2

Seems there is a lot of developer overhead in the proposed solutions thus far. If this was a problem I had to address in my own applications, I might save me a few hours of work by opting not to reinvent the wheel on this one.

Determining the Closest Mirror (Using Zip Codes)

  1. Maintain a list of postal codes in an array for those mirror servers available.
  2. Determine the postal code of the user agent (e.g. user input or PHP library)
  3. Calculate the distance between the two postal codes (e.g. PHP library)
  4. Proceed with mirror selection based on distance returned

Please keep in mind that a closer distance does not necessarily constitute a faster response time. In the context of your scenario, however, a mirror in one country will obviously be faster than a mirror in another, assuming both mirrors are up. Continue reading for what I consider to be a more "robust" solution.

Resources & Links

The "Maverick" Approach

In my opinion, Mavericks are also known as those innovators, problem solvers, and inventors of these great libraries and frameworks we all use today. Sometimes mistakenly associated with "hackish" ideas, but we embrace the complement :)

  1. Create your own API service on either one of your mirror servers that will accept either a $_GET or $_POST request.

  2. This API service will take an IP address it is given and ping() it, calculating the response times and then taking the average, returning it to the requesting interface (e.g. your frontend portal through which clients are connecting and/or the server trying to determine the closest mirror). The server that responds with the lowest average ought to be your quickest responding server, albeit not necessarily closest. Which is more important to you? See Ping site and return result in PHP for a working ping() function that does not rely on executing shell commands locally (e.g. platform independent).

  3. Last step, derive the IP address of the requesting client and pass it to your API service running on either mirror server in the background. And we all know how to derive the IP, but not as well as you think we might. If you're load balanced or behind a proxy, you may want to first check to see if any of these headers came through (HTTP_FORWARDED, HTTP_FORWARDED_FOR, HTTP_X_FORWARDED, HTTP_X_FORWARDED_FOR, HTTP_CLIENT_IP). If so, that's probably the real IP address of the user agent.

It is at this point (Step 3) where you would compare the averages of response times that each mirror replied with when they went to ping the user agent. Then proceed with selecting which mirror the user agent should download from. The service flow you will have created then resembles something like this:

  1. User agent visits portal
  2. Portal forwards user agent's IP address to API service running separately on both mirrors using a background AJAX/jQuery request (or traditional POST and redirect).
  3. API service running on mirrors pings the IP address it receives and returns an average of the total number of responses it is configured to fetch.
  4. Portal reads the returned averages and compares them.

Hope that helps and happy coding!

Community
  • 1
  • 1
Matt Borja
  • 1,451
  • 17
  • 38
  • 1
    That is the first really good and easy to implement idea I have seen so far. I like it a lot. One question: how much overhead will this cause? Will the bunch of little requests end up being hard for the server to deal with, figuring about 500 visitors/day? – Ashley Strout May 03 '12 at 19:31
  • PHP is certainly capable to do above and beyond this, so it is up to the programmer to benchmark and create test units for their code to evaluate its performance. One possibility that comes to mind surrounding your concern is coming up with a caching solution where server selection has been predetermined. Determine the network portion of your user agent's IP address and assign the server to that network (e.g. 192.168.0.0). Then, if an incoming IP address matches a network portion in your database, your portal can skip performing the API call and jump right to server provisioning. – Matt Borja May 04 '12 at 04:58
  • Also, another thing to point out - some people configure their routers not to reply to ICMP pings, while others still have no available ports to connect to test (per the ping() reference in my post). It is entirely possible that you will get a request timed out. In such cases, I would update your API service to fall back to an IP to Postal Code distance calculation function which instead of returning the average ping response as a numerical figure, returns the distance as a numerical figure. Again, with the understanding that distance does not necessarily mean a faster connection. – Matt Borja May 04 '12 at 05:11
1

If you have just two mirrors, kick off AJAX requests in your browser that download a 50K file from each server. This is small enough not to represent a huge delay for the user, but large enough to make timer measurement differences significant - though of course you should play with that figure a bit.

Then, once you've got a 'best time', set a JS cookie and redirect to the preferred mirror whenever a download is required. The measurement can be kicked off from a download page in the background, so the user probably won't notice the delay (whilst they are selecting the file they want).

You could even reply with a 'server load' in each AJAX op too, and select the best server not just on response time but on current load also. So, a UK user would use the US server, even though the closest server is in Germany, if the load on the latter is significantly higher than the first.

halfer
  • 19,471
  • 17
  • 87
  • 173
0

I dont remember any library that can do this. But instead of build a system, If I have an idea, that might be able to help you out.

Calculate the distance between two IP's using this distance calculator. Or find out the latitude and longitude of the two IP address (one server) and (one guest) and calculate the distance. Here is a pseudocode to do that

distance = ( 3956 *2 * ASIN( SQRT( POWER( SIN( ( 34.1012181 - ABS( latitude ) ) * PI( ) /180 /2 ) , 2 ) + COS( 34.1012181 * PI( ) /180 ) * COS( ABS( latitude ) * PI( ) /180 ) * POWER( SIN( ( ABS( - 118.325739 ) - ABS( longitude ) ) * PI( ) /180 /2 ) , 2 ) ) ))
Starx
  • 75,098
  • 44
  • 181
  • 258
0

Do a traceroute (configure the traceroute client to not resolve hostnames and with a small timeout).

Based on the number of hops and the location of the traceroute client (I supose it's the same as the PHP script) select between U.S. and Germany.

Geographical distance has nothing to do with network distance and network speed, or bandwidth costs.

Alternatively to the traceroute (since its a hackish, small code solution), I recommend you to use the $_SERVER["REMOTE_ADDR"] and look it up in a geo ip database to get the country code. If the country code is not one of the countries on the American continents, to avoid crossing a crowded internet backbone, fallback to Germany (additionaly you could condition the country code to be from Europe).

Once you setup the geo ip database, I recommend you convert the IP addresses in the ranges from dotted format to integer format for speed and ease of querying.

From my experience with the above geo ip database, it misses so rarely it doesn't matter.

Tiberiu-Ionuț Stan
  • 5,311
  • 6
  • 36
  • 65
0

its not more easy use some library like geoip as u said and use the latitude and longitude to compare the distance betwen the mirrors and the user ?

I think its less complicated and its much easy to implemen,works for N mirrors and u dont need to ask for Zip or another kind of data to make the references

Quaid
  • 233
  • 1
  • 7