How to setup a private DNS for your virtual cluster
Posted by Harald van Breederode on January 26, 2010
One of the challenges I faced recently was building a virtual cluster based on Oracle 11g Release 2 on top of Oracle Enterprise Linux (OEL) running inside VMware server. Although I have an existing virtual Oracle 11g Release 1 cluster, I decided to build a new one in order to be able to teach both versions of the Oracle University RAC courses I teach. Also, my existing virtual cluster runs OEL4 and for 11gR2 I needed OEL5. I’ll save the story about building the actual cluster for another posting, because I need to perform additional research first, but I’d like to share my solution on how to get around the DNS requirements without making changes to the Oracle corporate DNS server.
Background information
Oracle 11gR2 Clusterware has many new features and two of them require adding resource records to DNS. The features in question are:
- Single Client Access Name (SCAN)
- Grid Naming Service (GNS)
Before continuing I will introduce some basic DNS vocabulary. The Domain Name System (DNS) is a client-server architecture which purpose is to translate names into ip-addresses and vice-versa. The server is called the DNS server and the client is called the resolver. The most well-known implementation is the Berkeley Internet Name Domain (BIND) software where the nameserver process is called “named”. As the name suggests DNS is based on domains which are divided into administrative regions which are called zones. A nameserver can be either the master or slave for a zone. The data, called resource records, for a zone is stored in so-called zone files which are maintained on the master nameserver for a particular zone. Slave servers retrieve their zone file from the master server in a process known as a zone transfer. The BIND nameserver is configured by the /etc/named.conf
file, and the zone files are usually stored in /var/named
.
Making changes to the Oracle corporate DNS server for my virtual cluster is unlikely to happen, nor advisable to do so, and therefore I decided to run a DNS server inside my virtual cluster. This sounds easier than it actually is. The problem is that nameservers are linked to form a tree, thus the nameserver above me links to my nameserver and maybe mine links to a nameserver underneath me. This linking between nameservers is called delegation. Since I am trying to avoid adding resource records to the Oracle corporate nameservers there will be no delegation to my nameserver from the Oracle nameservers. This isn’t a problem as long as clients who wish to connect to my cluster talk to my nameserver and not to the Oracle nameservers. Another problem is that, because of firewalls, my nameserver cannot talk to nameservers outside the corporate Oracle network which means that my nameserver can only resolve names which are within my own zones. Therefore my nameserver should forward any request that it cannot resolve by itself to another nameserver that can.
The implementation
The DNS implementation that I created for my virtual cluster has the following features:
- I use
example.com
as my domain. - The master DNS server for
example.com
will be on my first RAC node. - A slave DNS server for
example.com
will be on my second RAC node. - All DNS lookups that fall outside
example.com
will be forwarded. - Each node uses its local nameserver as its primary.
- Each node uses the other node’s nameserver as its secondary.
- DNS security will not be implemented.
The first step in implementing the above setup is to install the bind
rpm either using the rpm
or yum
command.
The next step is to configure the DNS server. Without the right tool this can be quite challenging. I found h2n To be the best tool to handle this job for me. Basically h2n
generates all DNS configuration and zone files based on /etc/hosts
or any other file with a similar structure. Mine is called hosts.dns
and is shown below:
192.168.40.141 el5n1.example.com 192.168.40.142 el5n2.example.com # 192.168.40.101 el5n1-vip.example.com 192.168.40.102 el5n2-vip.example.com # 192.168.40.110 el5n-cluster-scan.example.com 192.168.40.111 el5n-cluster-scan.example.com 192.168.40.112 el5n-cluster-scan.example.com # 192.168.180.135 el5n1-priv.example.com 192.168.180.136 el5n2-priv.example.com
My RAC nodes are called el5n1 and el5n2, and the IP addresses for the public network interface are shown on lines 1 and 2. Their Virtual IP addresses are on lines 4 and 5. Lines 7, 8 and 9 have the SCAN IP addresses. Finally lines 11 and 12 show the private interface IP addresses.
I created a ‘dns’ sub-directory in my home-directory and stored the above file in there to act as my dns setup area.
The next step is to let h2n
generate the DNS configuration and zone files for me based on the above hosts file. I stored the required h2n
command line arguments in a .conf file called h2n.conf
, to prevent myself from entering them over and over again.
-H hosts.dns -d example.com -n 192.168.40 -n 192.168.180 -u root@example.com -W /var/named -M -y +O forwarders { 192.168.40.2; }; +O forward only; -s el5n1 -s el5n2 -z 192.168.40.141
Line 1 tells h2n
which host file to use, line 2 specifies the domain name, lines 3 and 4 specify the subnets in use. Line 5 is to specify who to contact if something is wrong with my DNS setup. Line 6 specifies where the working directory for the name server is. Lines 7 and 8 are to suppress the generation of MX records and to select a particular serial number format. Lines 9 and 10 are options that h2n
places in the named.conf
file. Lines 11, 12 and 13 tells h2n
which machines act as nameservers and that I want h2n
to generate config files for a slave nameserver that transfers its zones from the master DNS server at the specified IP address.
Now that I have the files to drive h2n
, the actual generation of the DNS configuration file is straightforward.
$ h2n Initializing new database files... Reading host file `hosts.dns'... Writing database files... Generating boot and conf files... Checking NS, MX, and other RRs for various improprieties... Done.
There is one file h2n
cannot generate and that is db.cache Which I downloaded and stored in my dns setup directory. The generated named.conf
file should be copied to /etc
and the db.*
should be copied to /var/named. The setup is now complete and the next step is to start the nameserver and to check its status.
$ sudo service named start Starting named: [ OK ] $ sudo service named status number of zones: 4 debug level: 0 xfers running: 0 xfers deferred: 0 soa queries in progress: 0 query logging is OFF recursive clients: 0/1000 tcp clients: 0/100 server is up and running named (pid 8656) is running...
The above output shows that my nameserver started successfully and that it runs normally. The next step is to automate the nameserver startup each time the operating system starts using the chkconfig
command as shown below:
$ sudo chkconfig named on $ sudo chkconfig --list named named 0:off 1:off 2:on 3:on 4:on 5:on 6:off
The DNS server configuration is now complete but before I can test my nameserver I first need to configure the resolver. This step involves editing /etc/resolv.conf
and add the IP address of my two nameservers to the nameserver
directives. To save typing fully qualified host names I use the search
directive to specify my local domain name.
search example.com nameserver 0.0.0.0 nameserver 192.168.40.142
The final step is to test my master nameserver by resolving several names and IP addresses inside my own domain and by resolving something outside my own domain to test the forwarding part of my setup.
$ host el5n1
el5n1.example.com has address 192.168.40.141
$ host 192.168.40.141
141.40.168.192.in-addr.arpa domain name pointer el5n1.example.com.
$ host el5n-cluster-scan
el5n-cluster-scan.example.com has address 192.168.40.110
el5n-cluster-scan.example.com has address 192.168.40.111
el5n-cluster-scan.example.com has address 192.168.40.112
$ host prutser.wordpress.com
prutser.wordpress.com is an alias for lb.wordpress.com.
lb.wordpress.com has address 72.233.2.58
lb.wordpress.com has address 72.233.2.59
lb.wordpress.com has address 74.200.243.251
lb.wordpress.com has address 74.200.243.253
lb.wordpress.com has address 76.74.254.123
lb.wordpress.com has address 76.74.255.123
According to the above output my master DNS server is working fine and I can now proceed to configure the slave DNS server on my second RAC node. All configuration and zone files for the slave nameserver were already generated by h2n
and all I needed to do was to copy the conf.sec.save
to /etc/named.conf
, and the db.cache
and db.127.0.0
to /var/named
on my second RAC node. Before starting the slave nameserver I needed to add ENABLE_ZONE_WRITE=yes to the /etc/sysconfig/named
file to allow it to save the transferred zone file to disk. The steps to start the slave nameserver manually and to automate the startup after a reboot are the same as for the master nameserver. The /etc/resolv.conf
is of course different because it needs the IP address of the master nameserver.
Wrapping up
To add a new host to my DNS configuration all I need to do is to add the hostname and IP address to the hosts.dns
file and to let h2n
generate the zone files for me. Thereafter I need to copy the new zone files to /var/named
on my first RAC node and tell named
to re-read the zone files from disk using the rndc
command. To ease this process I wrote the following Makefile which will do all the work for me after I edit the hosts.dns
file.
named.conf: hosts.dns h2n.conf h2n sudo cp named.conf /etc sudo cp db.* /var/named sudo rndc reload
The above described DNS configuration allows running my own nameservers inside my virtual RAC cluster giving me the possibility to explore the Oracle11gR2 Clusterware. Also, I gained a little bit of experience in setting up and maintaining DNS.
-Harald
jarneil said
Hi Harold,
You say “Also, my existing virtual cluster runs OEL4 and for 11gR2 I needed OEL5”
Can you expand on that, it’s not a requirement for 11gR2, right? 11gR2 RAC will run on OEL4, wont it?
jason.
Harald van Breederode said
Hi Jason,
Part of the 11gR2 Grid infrastructure is ASM Cluster Filesystem (ACFS) and you defenitly need OEL5 for that. I am not sure if you can install the 11gR2 Grid Infrastructure without ACFS on OEL4. I do know however that the 11gR2 database does run on OEL4, but if you want to use ASM you will need Grid Infrastructure for a standalone server which requires OEL5. So in most situations OEL5 is what you need too have.
-Harald
jarneil said
Hi Harald,
Absolutely you need oel 5 for ACFS.
However, you can indeed run Grid Infrastructure on OEL4 (and indeed rhel 4), you get a message of the following type popping up when installing on 4:
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
ADVM/ACFS is not supported on Redhat 4
but you still get a working grid infrastructure.
cheers,
jason.
Frits Hoogland said
A way to ease DNS administration is to use the DNS plugin in webmin. Editting the DNS zone files by hand is extremely error prone (_everybody_ makes a trailing dot error in these files now and then)
For the sake of completeness: h2n is not part of OEL/RHEL, nor to be found in EPEL. A page which lists DNS tools, including h2n (which lets you download h2n) is here: http://www.dns.net/dnsrd/tools.html
Andre said
Nice example! There only seems to be a typo in the resolv.conf file shown.
The IP address “nameserver 192.168.40.1422” has an extra ‘2’.
Harald van Breederode said
Thanx for your comment and for catching the typo. I’ve corrected the IP address.
-Harald
Log Buffer #176: a Carnival of the Vanities for DBAs | The Pythian Blog said
[…] Harald van Breederode shows how to setup a private DNS for your virtual cluster. […]
PaSS Police Oral Exam said
Hello. This is kind of an “unconventional” question , but have other visitors asked you how get the menu bar to look like you’ve got it? I also have a blog and am really looking to alter around the theme, however am scared to death to mess with it for fear of the search engines punishing me. I am very new to all of this …so i am just not positive exactly how to try to to it all yet. I’ll just keep working on it one day at a time Thanks for any help you can offer here.
Hans-Peter Sloot said
Hi Harald,
Nice post.
Pitty only that I did not see it earlier.
I have just set up a DNS server for 11gR2.
H2n would have saved me a lot of time.
Hans-Peter
Blogroll Report 22/01/2009 – 29/01/2010 « Coskan’s Approach to Oracle said
[…] 6-How to configure DNS for SCAN in 11GR2 for personal tests ? (easier zone configuration with H2N ) Harald van Bredeerode-How to setup a private DNS for your virtual cluster […]
filesearch.web.id said
nice info, thanks for sharing.
SCAN you LISTEN to this? « The Blog from the DBA Classroom said
[…] on 18/03/2010 My good friend and colleague Harald van Breederode recently posted an article on How to Set Up a Private DNS for Your Virtual Cluster where he discussed DNS and the steps he took to create both master and slave DNS servers on his […]
forurinfoonly said
Hi,
How does the resolve.conf on the node1 and node2 looks like ?
Thanks
Harald van Breederode said
Hi,
They have 2 nameserver lines, one with IP address 0.0.0.0 and another one with the IP address of the other node. The 0.0.0.0 entry tells the DNS resolver to talk to the named process on the localhost. They also contains a “search example.com” entry to allow the usage of non-fully qualified hostnames.
-Harald
kurt said
I’ve used this as you recommend, and it works. I definitely appreciate you intelligence and ingenuity putting this dns project together.
Your work is very much appreciated.
rana said
hi, could you please upload the files generated by h2n..it could be very usefull..
Harald van Breederode said
Hi Rana,
This is something you can do easily yourselves by running h2n against your own host file.
-Harald