Penguin
Note: You are viewing an old revision of this page. View the current version.

Squid Caching Proxy Server Notes

Problem solving

Resolving name problems

Having problems that http://brian/wherever/whatever doesn't resolve in Squid? This is caused by Squid running its own DNS resolver, instead of using gethostbyname(3). It pulls the IPs of the name servers out of resolv.conf(5). Add a line like this to your squid.conf
append_domain .yourdomain.tla

Any domain without a dot in it will get that domain prepended to it; everything works nicely all of a sudden.

Always get Connection Refused for any website

probably means that squid has run out of disk space...

This is also the default configuration - all users are denied access. If you want to simply allow all users to use squid and are just installing to save bandwidth go into /etc/squid/squid.conf and change the line http_access deny all to http_access allow all and restart squid - typically by /etc/init.d/squid restart

"Unable to load page" error

Microsoft InternetExplorer 6 SP 1 has a bug where if you are using "Basic" auth (eg, with squid), the first page afterwards will display an "Unable to load page" error. This is because MSIE tries to reuse an already closed TCP connection. See KB:331906.

Caching / Proxying Microsoft Windows Update

Windows Update caching works just fine, for the most part. If you have an authenticated proxy, you might want to add ".microsoft.com", ".windowsupdate.com" and "*.akamai.net" as an auth-bypass whitelist.

As of mid December 2004, Windows Update (under XP at least) changed the way it works. It ignores proxy settings, and attempts to make direct connections to a pool of servers. This is fairly annoying, as if you have no default route set on your workstations (a sensible security measure), you can no longer run windows update.

The subnets in question are: 207.46.0.0/16 and (I think) 64.2.21.0/24. It seems that the only solution is to allow these direct access via your firewall.

To use windows update via a proxy you must configure it using proxycfg as below

proxycfg -u

will import your proxy server setting from Internet Explorer.

Add-on utilities for Squid

Log Analysis (sarg)

sarg is a log file analyser for squid. It's partially useful.

Sarg is a reasonably nice tool for generating nice reports for your squid logs. I have only two problems with it currently.

  • Dates on reports spanning weeks or months are often wrong - all the data is there but the title of the report says it only covers 2-5 days.
  • Only shows reports of the percentage of traffic that was/was not served from the cache. Does not give an actual byte count. Sure it is easy to calculate it from the total but it would be even easier if it did it for me.

Log Analysis (srg)

SRG is a fast and flexible log analyser written in C/C++, it was written by MattBrown while working for CRCnet because none of the existing log analysation programs such as sarg were adequate. In particular SRG allows you to generate reports right down to the level of each file requested from a site, and reports can be generated in plain html or using PHP to allow you to easily integrate with your squid authentication system to restrict access to all or parts of the report. Another useful feature of SRG is the ability to generate an email every time a report is generated summarising the traffic used during the reporting period.

SRG is released under the GPL and is under active development.

Find out more about srg at http://www.crc.net.nz/software/srg.php

Graphing Squid data

Here are some other notes on Squid, SNMP and MRTG. This shows sample MRTG config options for graphing some of the info. Note that you can get MRTG to talk directly to Squid's nonstard SNMP port.

Content Blocking

Investigate the following blacklists:

(Note from Daniel Barron, DG author: the SG clause is in violation of the GPL and thus is invalid. The DG license is fully 100% within the GPL. What is asked for is that commercial users pay to download DG. I just thought I'd clarify the FUD.)

Useful configurations and tips

Proxy Auto Detection

To set things up so that your web browsers auto detect your proxy server, investigate WPAD, the Web Proxy Auto Detection script.

Filtering - ACLs in squid

When specifying ACLs, dont set more than one type of acl on a single acl line. Squid ignores them. eg:

 acl lab proxy_auth labuser src 192.168.2.0/32
 acl denylab proxy_auth labuser
 ....
 http_access allow lab
 http_access deny denylab

doesn't work. instead:

 acl labuser proxy_auth labuser
 acl labmachines proxy_auth 192.168.2.0/32
 ....
 http_access allow labuser labmachines
 http_access deny labuser

will do the trick.

URL Blocking

 acl restrictedmachine src ip.ad.dr.ess/255.255.255.255
 acl restrictedmachinesites dstdomain "/etc/squid/list-of-sites"

 http_access allow restrictedmachine restrictedmachinesites
 http_access deny restrictedmachine

list-of-sites takes the form

 # banned sites list
 host.domain.com
 # or
 .domain.com
 # for everything in domain.com

Alternatively, an external redirector such as ufdbGuard is used to block URL categories.

 redirect_program  /local/squid/bin/ufdbGuard -c /local/squid/etc/ufdbGuard.conf
 redirect_children 2

Proxy Auth with NTLM?

A full working example on having a Squid proxy pick up user information from NTLM and a MicrosoftWindows ActiveDirectory. This will allow anyone in the AD Group "Internet" to have full access to the internet, and anyone in "Domain Users" (and not in "Internet") to access sites in the "/etc/squid-allowedsites" file only.

If you are using InternetExplorer or newer Mozilla browsers (on MicrosoftWindows), this will work transparently using NTLM Authentication. If you're using another browser (or are running Linux), you'll be prompted for a username and password.

Using Squid 2.5STABLE3 and Samba 2.2.8a. Squid was recompiled with all winbind helpers and samba was recompiled with challenge-auth.

Initially we tried to use transparent proxying AND NTLM auth, as all indications were that this should work. In practice it does not - it seems there are bugs in squid which prevent this.

After installation of all packages and config files, samba must be joined to the domain with the command "smbpasswd -j DOMAIN -r PDC -U Administrator" - this will prompt you for the admin password.

At every boot, samba and winbind must be started, and winbind must auth to the domain with the command: "winbind -A DOMAIN\\Administrator%password".

Config files:

squid.conf

hierarchy_stoplist cgi-bin ?

auth_param ntlm program /usr/lib/squid/wb_ntlmauth -d
auth_param ntlm children 5
auth_param ntlm max_challenge_reuses 0
auth_param ntlm max_challenge_lifetime 2 minutes

#auth_param basic program /usr/lib/squid/wb_auth -d
#auth_param basic children 5
#auth_param basic realm Squid proxy-caching web server
#auth_param basic credentialsttl 2 hours

acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY

acl all src 0.0.0.0/0.0.0.0
acl allsites dst 0/0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl localnet src 192.168.99.0/255.255.255.0

acl allowedsites dstdomain "/etc/squid-allowedsites"
external_acl_type NT_global_group %LOGIN /usr/lib/squid/wb_group -c
acl FullUsers external NT_global_group "/etc/squid-fullusers"

acl SSL_ports port 443 563

#acl purge method PURGE
#acl CONNECT method CONNECT
#http_access allow manager localhost
#http_access deny manager
#http_access allow purge localhost
#http_access deny purge
#http_access deny CONNECT !SSL_ports
#http_access allow localhost

#http_access allow localnet

http_access allow allowedsites
http_access allow FullUsers
http_access deny all
icp_access allow all
visible_hostname firewall.example.co.nz

/etc/squid-allowedsites

.foo.bar
.foo.bar.baz

/etc/squid-fullusers

Internet

(These are checked against groups only)

/etc/smb.conf

#======================= Global Settings =======================

[global]

workgroup = EXAMPLE
password server = ADSNAME
security = domain
encrypt passwords = true
winbind uid = 10000-20000
winbind gid = 10000-20000
winbind enum users = yes
winbind enum groups = yes
winbind use default domain = yes
winbind cache time = 3
log file = /var/log/samba/log.%m
max log size = 1000
syslog = 0
wins server = 10.7.0.114
wins proxy = yes

Samba 3.x

If you're using Samba 3.x, you can no longer use the Squid wb_* helpers. Instead, you have to use the Samba-supplied helpers themselves:

# NTLM proxy auth
auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp

# HTTP basic proxy auth
auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic

Its a good idea to supply both NTLM and HTTP Basic authenticators so that non-IE browsers can use the proxy as well.

You will also need to allow the user ID Squid is running as to write to the /var/lib/samba/winbindd_privileged directory or you will get authentication failures (with errors written to cache.log).