DNS Round Robin Configuration using Rsync over SSH

My traffic is growing, alot, and I need to plan how I'm going to maintain scalability, high availability, and redundancy. Scalability is an application's ability to support a growing number of users. High availability can be defined as redundancy or speed. I decided to setup Round Robin DNS for s.askapache.net, which is the "static" subdomain of AskApache that serves all the static assets like images, javascript, css, etc.. (BTW, the z stands for ZAP).. All I needed to attempt setting this up was another hosting account on a separate server. I have hosting accounts with around 10 different companies from working with various clients over the years, like Powweb and I don't use them because they suck in terms of the unix environment. Many of these web hosts are actually very fast bandwidth-wise..

Round Robin Concept

A few months ago I was given a free hosting account on HostGator to evaluate and tempt me away from DreamHost to become a Gator. I get a lot of these types of offers from time to time from the smaller Web companies who read AskApache.. but when I noticed the SSH access was jailed and saw how restrictive the shell was I felt like I was on a windows box.. I want a shell, cpanel sucks. I compile and run everything from the shell so thats was a deal-breaker and I sorta forgot all about it.

The goal is to add the HostGator server to be an exact mirror of the s.askapache.net domain, then to add that server as a 2nd A record to my DNS zone. That way half the visitors to the size will be taking up resources and bandwidth on the HostGator server instead of mine.

Round Robin A records in DNS are intended to evenly distribute queries between each host of the same name. Using some tricks straight out of a hackers toolbox we can verify if the distribution is taking place. (It is.)

DNS - Domain Name System

The Domain Name System is often analogized as a "phone book" for the Internet because it translates human-friendly computer hostnames into IP addresses. For example, www.askapache.com translates to 208.113.134.190. Every request for a human-friendly hostname first needs to be translated to the IP before the server can be queried. Normally each hostname corresponds to exactly 1 IP address. But in a Round Robin setup, the idea is to have the hostname correspond to multiple IP addresses, which are different servers that house the exact same content, resulting in some hosts requesting files from one server, and another host requesting files from the other server, resulting in less CPU resources and bandwidth.

Here is an the zone for s.askapache.net Round Robin records:

QUERY	TTL	CLASS	TYPE	ANSWER
s.askapache.net	13084	IN	A	64.111.114.111
s.askapache.net	13084	IN	A	69.56.174.114

Round Robin DNS

Round robin DNS is a technique of load distribution, load balancing, or fault-tolerance provisioning multiple, redundant Internet Protocol service hosts, e.g., Web servers, FTP servers, by managing the Domain Name System's (DNS) responses to address requests from client computers according to an appropriate statistical model.

In its simplest implementation Round-robin DNS works by responding to DNS requests not only with a single IP address, but a list of IP addresses of several servers that host identical services. The order in which IP addresses from the list are returned is the basis for the term round robin. With each DNS response, the IP address sequence in the list is permuted. Usually, basic IP clients attempt connections with the first address returned from a DNS query so that on different connection attempts clients would receive service from different providers, thus distributing the overall load among servers.

Round robin DNS is often used for balancing the load of geographically-distributed Web servers. For example, a company has one domain name and three identical web sites residing on three servers with three different IP addresses. When one user accesses the home page it will be sent to the first IP address. The second user who accesses the home page will be sent to the next IP address, and the third user will be sent to the third IP address. In each case, once the IP address is given out, it goes to the end of the list. The fourth user, therefore, will be sent to the first IP address, and so forth.

Although easy to implement, round robin DNS has problematic drawbacks, such as those arising from record caching in the DNS hierarchy itself, as well as client-side address caching and reuse, the combination of which can be difficult to manage. Round robin DNS should not solely be relied upon for service availability. If a service at one of the addresses in the list fails, the DNS will continue to hand out that address and clients will still attempt to reach the inoperable service.

Does Round Robin Work

Definately. I can look at the access logs for both servers and see that they are splitting the requests nicely. It is definately not an exact split however, look at these statistics and you'll see what I mean.

$ dig @ns1.dreamhost.com +authority +all s.askapache.net

;; ANSWER SECTION:
s.askapache.net.        14400   IN      A       69.56.174.114
s.askapache.net.        14400   IN      A       64.111.114.111


$ dig @ns1.dreamhost.com +authority +all s.askapache.net

;; ANSWER SECTION:
s.askapache.net.        14400   IN      A       64.111.114.111
s.askapache.net.        14400   IN      A       69.56.174.114

The effects of caching will distort the effectiveness of any IP address allocation algorithm unless a 0 TTL is used which has the effect of significantly increasing the load on the DNS (and is not always implemented consistently). In this case the cure may be worse than the disease Good news we have good load balancing on our web servers. Bad news we need 17 more DNS servers!. Intuitively, and without running any experiments to verify, we would suggest that given a normal TTL (12 hours or more) and ANY IP allocation algorithm other than a single static list, loads should be reasonably balanced .

Full root server query

Tracing to s.askapache.net[a] via A.ROOT-SERVERS.NET, maximum of 3 retries
A.ROOT-SERVERS.NET [.] (198.41.0.4)
 |___ L.GTLD-SERVERS.NET [com] (192.41.162.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ D.GTLD-SERVERS.NET [com] (192.31.80.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ J.GTLD-SERVERS.NET [com] (192.48.79.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ H.GTLD-SERVERS.NET [com] (192.54.112.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ E.GTLD-SERVERS.NET [com] (192.12.94.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ F.GTLD-SERVERS.NET [com] (192.35.51.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ I.GTLD-SERVERS.NET [com] (192.43.172.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ G.GTLD-SERVERS.NET [com] (192.42.93.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ B.GTLD-SERVERS.NET [com] (2001:0503:231d:0000:0000:0000:0002:0030) Not queried
 |___ B.GTLD-SERVERS.NET [com] (192.33.14.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ A.GTLD-SERVERS.NET [com] (2001:0503:a83e:0000:0000:0000:0002:0030) Not queried
 |___ A.GTLD-SERVERS.NET [com] (192.5.6.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ C.GTLD-SERVERS.NET [com] (192.26.92.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
 |___ M.GTLD-SERVERS.NET [com] (192.55.83.30)
 |     |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
 |     |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
 |      ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer
  ___ K.GTLD-SERVERS.NET [com] (192.52.178.30)
       |___ ns3.dreamhost.com [askapache.com] (66.33.216.216) Got authoritative answer
       |___ ns2.dreamhost.com [askapache.com] (208.96.10.221)
        ___ ns1.dreamhost.com [askapache.com] (66.33.206.206) Got authoritative answer

  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 64.111.114.111
  ns1.dreamhost.com (66.33.206.206)       s.askapache.net -> 69.56.174.114
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 64.111.114.111
  ns3.dreamhost.com (66.33.216.216)       s.askapache.net -> 69.56.174.114

Live Online DNS Tools

• DNSstuff.com - Your Destination for DNS and Networking Tools
• ZoneCheck
• Quick Check - Pingability.com
• Squishywishywoo: complete dns traversal checking
• ZoneCut DNS
• DNS Sleuth
• DNS test - Domain name DNS test - pweb.cz
• OpenDNS - Support - CacheCheck
• DNSReport
• DNSTrace
• intoDNS
• DNSthru.com
• DNS Doctor
• Power Check

More Reading

• HOWTO - Configure Load Balancing
• Load Sharing with DNS

RFC's related to DNS

• RFC 920: Specified original TLDs: .arpa, .com, .edu, .org, .gov, .mil and two-character country codes
• RFC 1032: Domain administrators guide
• RFC 1033: Domain administrators operations guide
• RFC 1034: Domain Names - Concepts and Facilities.
• RFC 1035: Domain Names - Implementation and Specification
• RFC 1101: DNS Encodings of Network Names and Other Types
• RFC 1123: Requirements for Internet Hosts -- Application and Support
• RFC 1912: Common DNS Operational and Configuration Errors
• RFC 1995: Incremental Zone Transfer in DNS
• RFC 1996: A Mechanism for Prompt Notification of Zone Changes (DNS NOTIFY)
• RFC 2136: Dynamic Updates in the domain name system (DNS UPDATE)
• RFC 2181: Clarifications to the DNS Specification
• RFC 2182: Selection and Operation of Secondary DNS Servers
• RFC 2308: Negative Caching of DNS Queries (DNS NCACHE)
• RFC 2317: Classless IN-ADDR.ARPA delegation
• RFC 2671: Extension Mechanisms for DNS (EDNS0)
• RFC 3597: Handling of Unknown DNS Resource Record (RR) Types
• RFC 3696: Application Techniques for Checking and Transformation of Names
• RFC 4343: Domain Name System (DNS) Case Insensitivity Clarification
• RFC 4592: The Role of Wildcards in the Domain Name System
• RFC 4892: Requirements for a Mechanism Identifying a Name Server Instance
• RFC 5001: DNS Name Server Identifier Option (NSID)
• RFC 5395: Domain Name System (DNS) IANA Considerations

This page contains content from article and is released under the CC-BY-SA.