I posted earlier about how I was having some bizarre problems with DNS server entries in /etc/resolv.conf on Debian Linux. I see from the Typepad referrer logs that a lot of people are finding The Thin Beer by searching online for Debian+resolv.conf (but not as many as search for beer+pictures).
One of my pet peeves about the Internet is that it is often very easy to find others who have had the same technical problems, but very difficult to find anybody with any answers. Judging from what can be found online, most people never find online answers to their technical questions. Even more frustrating is when the original poster will respond with nothing more than "Never mind, I solved the problem".
So, I have been remiss by not describing the resolution to my DNS problems. If you don't care about such things, you can skip the rest of this post.
The symptom of the problem was that /etc/resolv.conf would unpredictably show up with new nameserver addresses, which were unreachable. I am using Cisco's VPN client software for Linux. It turns out that the VPN client software wants to change the DNS server addresses on the client machine (I think I know why, but it's not important). It does this when the client connects to the VPN.
/etc/resolv.conf is restored to its original status when one of three things happens: when the VPN client disconnects from the VPN, when the kernel module is shut down, or when the kernel module is first started.
The problem would occur when none of these things would happen. Then, on the next boot, the DNS entries would be messed up. The solution is to make sure that at least one of these three things happens.
I do not always remember to disconnect from the VPN when I shut down the computer. That leaves starting the kernel module or stopping the kernel module.
The VPN client installation reported that the kernel module would be started automatically when the machine was booted, but this fails to happen on Debian. Apparently, Debian uses runlevel 2 for what other Linux distributions use runlevel 3. The Cisco VPN client installation places a startup script in /etc/rc3.d (as well as /etc/rc4.d and /etc/rc5.d), but not in /etc/rc2.d. Solution: place a symlink to /etc/init.d/vpnclient_init in /etc/rc2.d that looks exactly like the one in /etc/rc3.d.
This solved the problem, but raised another problem on this machine. When the machine is restarted and enters runlevel 2, the VPN kernel module is started, and the DNS server addresses are restored if they need to be. However, this occurs only after the machine tries to start its mail transfer agent (MTA), exim4. Exim4 hangs (for five minutes or so) if it can't reach a DNS server when it starts. Five minutes is an eternity as part of a boot sequence.
One solution to the latter problem is to change the priority of the VPN startup script in /etc/rc2.d, so that it runs before exim4 (or else change the priority of the exim4 startup to achieve the same effect). I don't know if this solution has any unintended consequences.
The solution that I adopted was to put shutdown links to the vpn client script in runlevels 0 and 6, so that the VPN kernel module will be stopped when the machine is shut down or rebooted. Now, everything works fine.