5 min read

Proxy Auto-Configuration Gives Relief From Internet Traffic Chaos

Properly implemented, proxy servers can provide administrators with a way to actively manage Internet traffic. Columnist Eric Hall shows you what you need to know to set one up.
In an effort to further simplify automatic proxy configuration, Microsoft published a specification for Web Proxy Auto-Discovery (WPAD), which essentially describes two different methods for a Web agent to automatically locate proxy control files on the local network. In this model, a client can simply ask the network for the URL of the local PAC file (instead of having to store the URL itself), which the client will then download and process, as described in the preceding section.

The WPAD protocol requires that clients first issue a DHCP request for the location of the shared PAC file, with the answer containing a simple URL that points to the location of the PAC file that the client should use. If the DHCP request fails, then the WPAD specification says that the client should follow up with a DNS lookup for the IP address of a host named "WPAD" in the local domain. If an answer for that query is returned, then the client must issue an HTTP request to the specified host, asking for a file with the exact name of "wpad.dat." The WPAD specification also describes some other lookup mechanisms beyond these first two stabs, but those are extremely rare in the wild, and might as well not actually exist.

The DHCP option is the more flexible of these two auto-configuration mechanisms, since it allows the use of general-purpose URLs pointing to any file, using any protocol, while the DNS option essentially requires using the fixed URL of http://wpad.your.domain/wpad.dat. You can even point to a shared file systems via a "file://" URL if you want. In contrast, the DNS method requires the use of a pre-defined transfer protocol, host name, and file name. However, while most of the proxy clients support the use of the DNS discovery mechanism described above, very few of them use the DHCP mechanism.

For example, the "Auto-detect proxy settings" option shown in the screenshot above is for Firefox, which uses only the DNS method, while Internet Explorer is the only browser I know of that actually supports both DHCP and DNS. As such, if you want to use the WPAD protocol, you are pretty much required to use the DNS mechanism, regardless of whether you also use the DHCP method.

If you decide to go with the DNS-based WPAD method (which you should), then you can take advantage of the mandatory naming to normalize other configuration methods too. For example, since you'll need to have a host named WPAD.your.domain for the DNS lookup to succeed, you can use that host name for your static configurations too. Similarly, you'll need to use a PAC file named WPAD.DAT so you might as well reference that file in any URL-based configurations too.

There are other options for pushing proxy configuration out to clients that do not rely on any of these discovery protocols. For example, you can use predefined Active Directory Group Policy settings to push Internet Explorer proxy settings down to your Windows clients, or you can use legacy NT Domain policy settings with Samba and NAS appliances to do the same basic thing. Using this model, you can configure most of your PCs with the same information in a single operation, without having to rely on auto-discovery methods.

Furthermore, Windows-based applications that reuse Internet Explorer's DLLs for transfer purposes automatically inherit its proxy settings. Unix-derived operating systems and applications can also reuse configuration settings, but this varies by application.Command-line tools typically read the fixed values stored in the environment variables but nothing else, while Gnome applications often use the Gnome-specific fixed-configuration settings, and so forth. Even if the applications understand the same locator mechanism, they might not support features such as user authentication, meaning that consistency can still be spotty even where there is reuse. Overall, there is still a lack of consistency among applications on each platform, not to mention a lack of consistency across them.

One last option worth mentioning before I sign off--due to the kinds of problems described above, some organizations go so far as to use "transparent" proxies on their networks. Transparent proxies essentially work by having a router or forwarder intercept traffic to TCP port 80, and then redirect that traffic to a proxy server, where it is processed like normal. This seems like a good idea on the surface, since you don't have to perform any client configuration whatsoever, but there are also costs associated with this approach that can make it unacceptable.

For example, if a URL points to an odd-ball port number like 8080, then the traffic might not get redirected to the transparent proxy for processing. Furthermore, protocols like FTP may or may not be supported by the proxy server, so you might lose access to alternative protocols that otherwise work with visible proxies. Transparent proxies are also known to have problems with sites that depend on fixed IP addresses for access control, and user authentication problems can crop up too. All told, it's best to configure as many agents as possible with WPAD and network policies, and send the rest of the traffic sources through the transparent proxy if needed.

Editor's Choice
Sara Peters, Editor-in-Chief, InformationWeek / Network Computing
Jessica Davis, Senior Editor
Richard Pallardy, Freelance Writer
Carrie Pallardy, Contributing Reporter
John Edwards, Technology Journalist & Author
Carlo Massimo, Contributing Writer
Salvatore Salamone, Managing Editor, Network Computing