Funny, isn’t it. The way things may have been setup on a network years ago and everyone thought were running fine. Then one day, small little niggly, nuisance things start happening. That is exactly what we were dealing with yesterday.
At a particular site, users had reported periodically being unable to browse to internal/LAN websites. A reboot or an ipconfig /release then ipconfig /renew would resolve it. Periodic, non-reproducible issues I don’t like but they happen all the time.
So, a quick rundown of the environment:
- Microsoft Active Directory environment
- 6 Windows 2003 Servers
- 4 Ubuntu 6.06 Linux servers
- 30 network drops in the local office
- 2 PTP VPN’s for remote offices
We were finally able to see the behaviour first hand. As we worked throughout the day we noticed that the failures were only being seen for sites residing on the Linux servers. So we threw Wireshark onto one of our laptops and began capturing packets. We cycled through lots of release/renew cycles and finally got the behavior to take place. Bingo!
Eagerly we looked through the packet trace filtering for the DNS queries. Lo and behold, the first requested went to the domain controller but then (2) two seconds later the client sends the dns query to one of the external isp dns servers. Aha! That’s it.
A quick ipconfig /all on a Windows XP desktop revealed that the clients DHCP lease included three dns servers.
- Domain controller
- ISP DNS server
- ISP DNS server
Well there’s your problem. I was always taight that in an active directory environment, your clients should only use the active directory domain controllers for dns resolution. Those active directory domain controllers then are configured to forward unknown zone requests to external dns servers.
What was happening was that the domain controller, also serving as the dns service host was taxed from a system perspective, yet another issue. It failed to respond timely to the client’s dns request. The client then went to the next dns server in its list, the external ISP dns server. Well the external dns server has no idea about the internal servers and sites and therefore fails on the lookup and tells the client.
Thanks to our wonderful folks at Microsoft, Windows XP caches the failed lookup for a period of time. Thus the client can no longer browse to the internal site.
The story ends well we think. We simply changed the DHCP server to have the clients look only at the two internal dns servers.
Problem solved. Isn’t it fun to work with technology. ;o)
Popularity: 9% [?]
