The Dutch Prutser's Blog

By: Harald van Breederode

  • Disclaimer

    The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.
  • Subscribe

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 238 other followers

How to setup a private DNS for your virtual cluster

Posted by Harald van Breederode on January 26, 2010

One of the challenges I faced recently was building a virtual cluster based on Oracle 11g Release 2 on top of Oracle Enterprise Linux (OEL) running inside VMware server. Although I have an existing virtual Oracle 11g Release 1 cluster, I decided to build a new one in order to be able to teach both versions of the Oracle University RAC courses I teach. Also, my existing virtual cluster runs OEL4 and for 11gR2 I needed OEL5. I’ll save the story about building the actual cluster for another posting, because I need to perform additional research first, but I’d like to share my solution on how to get around the DNS requirements without making changes to the Oracle corporate DNS server.

Background information

Oracle 11gR2 Clusterware has many new features and two of them require adding resource records to DNS. The features in question are:

  • Single Client Access Name (SCAN)
  • Grid Naming Service (GNS)
Adding resource records to DNS is something that is handled by network administrators in most organizations. However, I consider this a DBA2.0 skill and I think you should gain knowledge in this area if you don’t already have it. A good start would be reading DNS and BIND published by O’Reilly.

Before continuing I will introduce some basic DNS vocabulary. The Domain Name System (DNS) is a client-server architecture which purpose is to translate names into ip-addresses and vice-versa. The server is called the DNS server and the client is called the resolver. The most well-known implementation is the Berkeley Internet Name Domain (BIND) software where the nameserver process is called “named”. As the name suggests DNS is based on domains which are divided into administrative regions which are called zones. A nameserver can be either the master or slave for a zone. The data, called resource records, for a zone is stored in so-called zone files which are maintained on the master nameserver for a particular zone. Slave servers retrieve their zone file from the master server in a process known as a zone transfer. The BIND nameserver is configured by the /etc/named.conf file, and the zone files are usually stored in /var/named.

Making changes to the Oracle corporate DNS server for my virtual cluster is unlikely to happen, nor advisable to do so, and therefore I decided to run a DNS server inside my virtual cluster. This sounds easier than it actually is. The problem is that nameservers are linked to form a tree, thus the nameserver above me links to my nameserver and maybe mine links to a nameserver underneath me. This linking between nameservers is called delegation. Since I am trying to avoid adding resource records to the Oracle corporate nameservers there will be no delegation to my nameserver from the Oracle nameservers. This isn’t a problem as long as clients who wish to connect to my cluster talk to my nameserver and not to the Oracle nameservers. Another problem is that, because of firewalls, my nameserver cannot talk to nameservers outside the corporate Oracle network which means that my nameserver can only resolve names which are within my own zones. Therefore my nameserver should forward any request that it cannot resolve by itself to another nameserver that can.

The implementation

The DNS implementation that I created for my virtual cluster has the following features:

  • I use example.com as my domain.
  • The master DNS server for example.comwill be on my first RAC node.
  • A slave DNS server for example.com will be on my second RAC node.
  • All DNS lookups that fall outside example.com will be forwarded.
  • Each node uses its local nameserver as its primary.
  • Each node uses the other node's nameserver as its secondary.
  • DNS security will not be implemented.

The first step in implementing the above setup is to install the bind rpm either using the rpm or yum command.

The next step is to configure the DNS server. Without the right tool this can be quite challenging. I found h2n To be the best tool to handle this job for me. Basically h2n generates all DNS configuration and zone files based on /etc/hosts or any other file with a similar structure. Mine is called hosts.dns and is shown below:

192.168.40.141  el5n1.example.com
192.168.40.142  el5n2.example.com
#
192.168.40.101  el5n1-vip.example.com
192.168.40.102  el5n2-vip.example.com
#
192.168.40.110  el5n-cluster-scan.example.com
192.168.40.111  el5n-cluster-scan.example.com
192.168.40.112  el5n-cluster-scan.example.com
#
192.168.180.135 el5n1-priv.example.com
192.168.180.136 el5n2-priv.example.com

My RAC nodes are called el5n1 and el5n2, and the IP addresses for the public network interface are shown on lines 1 and 2. Their Virtual IP addresses are on lines 4 and 5. Lines 7, 8 and 9 have the SCAN IP addresses. Finally lines 11 and 12 show the private interface IP addresses.

I created a ‘dns’ sub-directory in my home-directory and stored the above file in there to act as my dns setup area.

The next step is to let h2n generate the DNS configuration and zone files for me based on the above hosts file. I stored the required h2n command line arguments in a .conf file called h2n.conf, to prevent myself from entering them over and over again.

-H hosts.dns
-d example.com
-n 192.168.40
-n 192.168.180
-u root@example.com
-W /var/named
-M
-y
+O forwarders { 192.168.40.2; };
+O forward only;
-s el5n1
-s el5n2
-z 192.168.40.141

Line 1 tells h2n which host file to use, line 2 specifies the domain name, lines 3 and 4 specify the subnets in use. Line 5 is to specify who to contact if something is wrong with my DNS setup. Line 6 specifies where the working directory for the name server is. Lines 7 and 8 are to suppress the generation of MX records and to select a particular serial number format. Lines 9 and 10 are options that h2n places in the named.conf file. Lines 11, 12 and 13 tells h2n which machines act as nameservers and that I want h2n to generate config files for a slave nameserver that transfers its zones from the master DNS server at the specified IP address.

Now that I have the files to drive h2n, the actual generation of the DNS configuration file is straightforward.

$ h2n
Initializing new database files...
Reading host file `hosts.dns'...
Writing database files...
Generating boot and conf files...
Checking NS, MX, and other RRs for various improprieties...
Done.

There is one file h2n cannot generate and that is db.cache Which I downloaded and stored in my dns setup directory. The generated named.conf file should be copied to /etc and the db.* should be copied to /var/named. The setup is now complete and the next step is to start the nameserver and to check its status.

$ sudo service named start
Starting named: [  OK  ]

$ sudo service named status
number of zones: 4
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is OFF
recursive clients: 0/1000
tcp clients: 0/100
server is up and running
named (pid  8656) is running...

The above output shows that my nameserver started successfully and that it runs normally. The next step is to automate the nameserver startup each time the operating system starts using the chkconfig command as shown below:

$ sudo chkconfig named on

$ sudo chkconfig --list named
named           0:off   1:off   2:on    3:on    4:on    5:on    6:off

The DNS server configuration is now complete but before I can test my nameserver I first need to configure the resolver. This step involves editing /etc/resolv.conf and add the IP address of my two nameservers to the nameserver directives. To save typing fully qualified host names I use the search directive to specify my local domain name.

search example.com
nameserver 0.0.0.0
nameserver 192.168.40.142

The final step is to test my master nameserver by resolving several names and IP addresses inside my own domain and by resolving something outside my own domain to test the forwarding part of my setup.

$ host el5n1
el5n1.example.com has address 192.168.40.141

$ host 192.168.40.141
141.40.168.192.in-addr.arpa domain name pointer el5n1.example.com.

$ host el5n-cluster-scan
el5n-cluster-scan.example.com has address 192.168.40.110
el5n-cluster-scan.example.com has address 192.168.40.111
el5n-cluster-scan.example.com has address 192.168.40.112

$ host prutser.wordpress.com
prutser.wordpress.com is an alias for lb.wordpress.com.
lb.wordpress.com has address 72.233.2.58
lb.wordpress.com has address 72.233.2.59
lb.wordpress.com has address 74.200.243.251
lb.wordpress.com has address 74.200.243.253
lb.wordpress.com has address 76.74.254.123
lb.wordpress.com has address 76.74.255.123

According to the above output my master DNS server is working fine and I can now proceed to configure the slave DNS server on my second RAC node. All configuration and zone files for the slave nameserver were already generated by h2n and all I needed to do was to copy the conf.sec.save to /etc/named.conf, and the db.cache and db.127.0.0 to /var/named on my second RAC node. Before starting the slave nameserver I needed to add ENABLE_ZONE_WRITE=yes to the /etc/sysconfig/named file to allow it to save the transferred zone file to disk. The steps to start the slave nameserver manually and to automate the startup after a reboot are the same as for the master nameserver. The /etc/resolv.conf is of course different because it needs the IP address of the master nameserver.

Wrapping up

To add a new host to my DNS configuration all I need to do is to add the hostname and IP address to the hosts.dns file and to let h2n generate the zone files for me. Thereafter I need to copy the new zone files to /var/named on my first RAC node and tell named to re-read the zone files from disk using the rndc command. To ease this process I wrote the following Makefile which will do all the work for me after I edit the hosts.dns file.

named.conf:     hosts.dns h2n.conf
        h2n
        sudo cp named.conf /etc
        sudo cp db.* /var/named
        sudo rndc reload

The above described DNS configuration allows running my own nameservers inside my virtual RAC cluster giving me the possibility to explore the Oracle11gR2 Clusterware. Also, I gained a little bit of experience in setting up and maintaining DNS.
-Harald

About these ads

17 Responses to “How to setup a private DNS for your virtual cluster”

  1. jarneil said

    Hi Harold,

    You say “Also, my existing virtual cluster runs OEL4 and for 11gR2 I needed OEL5″

    Can you expand on that, it’s not a requirement for 11gR2, right? 11gR2 RAC will run on OEL4, wont it?

    jason.

    • Harald van Breederode said

      Hi Jason,

      Part of the 11gR2 Grid infrastructure is ASM Cluster Filesystem (ACFS) and you defenitly need OEL5 for that. I am not sure if you can install the 11gR2 Grid Infrastructure without ACFS on OEL4. I do know however that the 11gR2 database does run on OEL4, but if you want to use ASM you will need Grid Infrastructure for a standalone server which requires OEL5. So in most situations OEL5 is what you need too have.
      -Harald

      • jarneil said

        Hi Harald,

        Absolutely you need oel 5 for ACFS.

        However, you can indeed run Grid Infrastructure on OEL4 (and indeed rhel 4), you get a message of the following type popping up when installing on 4:

        CRS-4123: Oracle High Availability Services has been started.
        ohasd is starting
        ADVM/ACFS is not supported on Redhat 4

        but you still get a working grid infrastructure.

        cheers,

        jason.

  2. A way to ease DNS administration is to use the DNS plugin in webmin. Editting the DNS zone files by hand is extremely error prone (_everybody_ makes a trailing dot error in these files now and then)

    For the sake of completeness: h2n is not part of OEL/RHEL, nor to be found in EPEL. A page which lists DNS tools, including h2n (which lets you download h2n) is here: http://www.dns.net/dnsrd/tools.html

  3. Andre said

    Nice example! There only seems to be a typo in the resolv.conf file shown.
    The IP address “nameserver 192.168.40.1422″ has an extra ’2′.

  4. [...] Harald van Breederode shows how to setup a private DNS for your virtual cluster. [...]

  5. Hello. This is kind of an “unconventional” question , but have other visitors asked you how get the menu bar to look like you’ve got it? I also have a blog and am really looking to alter around the theme, however am scared to death to mess with it for fear of the search engines punishing me. I am very new to all of this …so i am just not positive exactly how to try to to it all yet. I’ll just keep working on it one day at a time Thanks for any help you can offer here.

  6. Hans-Peter Sloot said

    Hi Harald,

    Nice post.
    Pitty only that I did not see it earlier.
    I have just set up a DNS server for 11gR2.
    H2n would have saved me a lot of time.

    Hans-Peter

  7. [...] 6-How to configure DNS for SCAN in 11GR2 for personal tests ? (easier zone configuration with H2N ) Harald van Bredeerode-How to setup a private DNS for your virtual cluster [...]

  8. nice info, thanks for sharing.

  9. [...] on 18/03/2010 My good friend and colleague Harald van Breederode recently posted an article on How to Set Up a Private DNS for Your Virtual Cluster where he discussed DNS and the steps he took to create both master and slave DNS servers on his [...]

  10. forurinfoonly said

    Hi,
    How does the resolve.conf on the node1 and node2 looks like ?

    Thanks

    • Harald van Breederode said

      Hi,

      They have 2 nameserver lines, one with IP address 0.0.0.0 and another one with the IP address of the other node. The 0.0.0.0 entry tells the DNS resolver to talk to the named process on the localhost. They also contains a “search example.com” entry to allow the usage of non-fully qualified hostnames.
      -Harald

  11. kurt said

    I’ve used this as you recommend, and it works. I definitely appreciate you intelligence and ingenuity putting this dns project together.
    Your work is very much appreciated.

  12. rana said

    hi, could you please upload the files generated by h2n..it could be very usefull..

    • Harald van Breederode said

      Hi Rana,

      This is something you can do easily yourselves by running h2n against your own host file.
      -Harald

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 238 other followers

%d bloggers like this: