Speed up repeat downloads with an explicit Squid 7.1 proxy on Ubuntu 24.04

By /

(Build from source • MITM with a Squid-only Sub-CA signed by your root • Broad caching • Run as proxy • Managed by systemd • Shell-only clients)

TL;DR: Build Squid 7.1, generate a Squid-only Sub-CA (key stays on the proxy; signed by your root), enable TLS bump, run as proxy under systemd, and set http_proxy/https_proxy in /etc/profile. Clients either trust your root (broad) or this Sub-CA (narrow).

Reference for step-ca:

Deploying Step CA: Your Own Certificate Authority for Internal Services

Legal / ethics

TLS interception is only for networks you own/manage with informed user consent. Do not intercept traffic you’re not authorized to inspect.

Prerequisites

  • Ubuntu 24.04 (proxy host + clients).
  • A working step-ca you control (root + intermediate exist).
  • Example names you can swap:
    • Proxy host: squid.maksonlee.com
    • LAN: 192.168.0.0/24
    • Ports: 3128/tcp
  1. Build Squid 7.1 (from source)
sudo apt update
sudo apt install -y build-essential autoconf automake libtool pkg-config gettext libssl-dev
git clone https://github.com/squid-cache/squid.git
cd squid
git checkout SQUID_7_1
./bootstrap.sh
./configure \
  --prefix=/usr/local/squid \
  --sysconfdir=/etc/squid \
  --localstatedir=/var \
  --libexecdir=/usr/local/squid/libexec \
  --with-default-user=proxy \
  --with-openssl \
  --enable-ssl-crtd \
  --enable-storeid-rewrite \
  --with-large-files
make -j"$(nproc)"
sudo make install

Create runtime dirs and initialize the leaf-cert cache once:

# dirs owned by 'proxy'
sudo install -d -m 750 -o proxy -g proxy /var/log/squid
sudo install -d -m 750 -o proxy -g proxy /var/spool/squid

# one-time init of ssl_db (note the -c is ONLY for this one-time step)
sudo /usr/local/squid/libexec/security_file_certgen -c -s /var/lib/ssl_db -M 32MB
sudo chown -R proxy:proxy /var/lib/ssl_db
  1. Create a Squid-only Sub-CA (key stays on the proxy)

On the proxy host (generate key + CSR)

sudo openssl req -new -newkey rsa:4096 -nodes \
  -subj "/CN=Squid MITM Sub-CA/O=Makson Lee" \
  -keyout /etc/squid/mitm_ca.key \
  -out /tmp/mitm_ca.csr
sudo chown root:proxy /etc/squid/mitm_ca.key
sudo chmod 640 /etc/squid/mitm_ca.key

# Send CSR to the CA host
scp /tmp/mitm_ca.csr administrator@stepca.maksonlee.com:/tmp/

On the step-ca host (sign the CSR with the root)

sudo -u step step certificate sign \
  /tmp/mitm_ca.csr \
  /etc/step-ca/certs/root_ca.crt \
  /etc/step-ca/secrets/root_ca_key \
  --profile intermediate-ca --path-len 0 --not-after 8760h \
  > /tmp/mitm_ca.crt

# Verify: CA:TRUE, pathlen:0; keyCertSign, cRLSign present
openssl x509 -in /tmp/mitm_ca.crt -noout -text | egrep -A3 'Basic Constraints|Key Usage'

Back on the proxy host (install Sub-CA cert)

scp administrator@stepca.maksonlee.com:/tmp/mitm_ca.crt /tmp/
sudo mv /tmp/mitm_ca.crt /etc/squid/mitm_ca.crt
sudo chown root:proxy /etc/squid/mitm_ca.crt
sudo chmod 644 /etc/squid/mitm_ca.crt
  1. Squid config — /etc/squid/squid-mitm.conf (listen on 3128, run as proxy)

Because the service runs as proxy, do not include cache_effective_user/group in this file.

#############################################
# Explicit proxy + TLS MITM (PROD on :3128)
# Service runs as 'proxy' (systemd User=proxy)
#############################################

# PID / core / logs
pid_filename /run/squid/squid.pid          # /var/run -> /run on Ubuntu
coredump_dir /var/spool/squid
cache_log /var/log/squid/cache.log
access_log stdio:/var/log/squid/access.log
cache_store_log none
logfile_rotate 10

# Listener (3128 production)
http_port 3128 ssl-bump cert=/etc/squid/mitm_ca.crt key=/etc/squid/mitm_ca.key \
  generate-host-certificates=on dynamic_cert_mem_cache_size=32MB

# Trust system roots upstream
tls_outgoing_options cafile=/etc/ssl/certs/ca-certificates.crt

# On-the-fly leaf certificate generator (NO -c here)
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s /var/lib/ssl_db -M 32MB
sslcrtd_children 16 startup=4 idle=2

# TLS bump: peek then bump everything
acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump bump all
# Optional hardening:
# sslproxy_cert_error deny all

#############################################
# Broad caching policy (let origin headers decide)
#############################################
collapsed_forwarding on
range_offset_limit 1 GB
maximum_object_size 40960 MB
cache_mem 2048 MB
cache_dir ufs /var/spool/squid 100000 16 256   # ~100 GB on fast disk

# TTL hints (respect no-store/private unless explicitly overridden)
refresh_pattern -i \.(deb|rpm|msi|cab|zip|tar|gz|xz|dmg|pkg|whl|jar|iso|7z)$      43200 100% 43200
refresh_pattern -i \.(sha256|sha512|sig)$                                         43200  20% 43200
refresh_pattern -i \.(png|jpg|jpeg|gif|webp|svg|ico|avif)$                        43200  90% 43200
refresh_pattern -i \.(css|js)$                                                    10080  90% 43200
refresh_pattern -i \.(woff|woff2|ttf|eot|otf)$                                    43200  90% 43200
refresh_pattern -i \.(mp4|webm|mp3|m4a|ogg)$                                      10080  50% 43200
refresh_pattern .                                                                   0     20% 4320

#############################################
# Access control (adjust to your LAN)
#############################################
acl localnet src 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
http_access allow localnet
http_access deny all

# Privacy
forwarded_for off

Want to keep huge artifacts even longer? Consider:

cache_replacement_policy heap LFUDA
memory_replacement_policy lru
  1. systemd unit — run as proxy

/etc/systemd/system/squid-custom.service:

[Unit]
Description=Squid Web Proxy (custom v7 build)
After=network-online.target
Wants=network-online.target

[Service]
# Everything runs as 'proxy'
User=proxy
Group=proxy

Type=forking
PIDFile=/run/squid/squid.pid

# systemd creates /run/squid owned by 'proxy'
RuntimeDirectory=squid
RuntimeDirectoryMode=0750

# Start daemonized (no -N); -sY logs notices to journal
ExecStart=/usr/local/squid/sbin/squid -f /etc/squid/squid-mitm.conf -sY
ExecReload=/usr/local/squid/sbin/squid -k reconfigure -f /etc/squid/squid-mitm.conf
ExecStop=/usr/local/squid/sbin/squid -k shutdown -f /etc/squid/squid-mitm.conf

KillMode=mixed
Restart=on-failure
RestartSec=2s
LimitNOFILE=1048576
TimeoutStopSec=30s

[Install]
WantedBy=multi-user.target

Enable/start:

# initialize disk cache dirs once (only after install or if cache_dir changes)
sudo install -d -m 750 -o proxy -g proxy /run/squid
sudo -u proxy /usr/local/squid/sbin/squid -z -f /etc/squid/squid-mitm.conf

sudo systemctl daemon-reload
sudo systemctl enable --now squid-custom
sudo systemctl status squid-custom --no-pager
  1. Ubuntu clients
  • Install your Root CA

Copy your root (not the intermediate; no private keys) to each client and install it:

# copy from step-ca host
scp /etc/step-ca/certs/root_ca.crt user@client:/tmp/root_ca.crt

# on the client
sudo install -D -m 0644 /tmp/root_ca.crt /usr/local/share/ca-certificates/maksonlee-root-ca.crt
sudo update-ca-certificates
# Expect: "1 added, 0 removed; done."
  • System-wide proxy for shells via /etc/profile.d/

Append these lines so every interactive shell gets the proxy:

sudo tee /etc/profile.d/90-proxy.sh >/dev/null <<'EOF'
# Explicit proxy for shells
export http_proxy="http://squid.maksonlee.com:3128"
export https_proxy="http://squid.maksonlee.com:3128"
export ftp_proxy="http://squid.maksonlee.com:3128"

export HTTP_PROXY="$http_proxy"
export HTTPS_PROXY="$https_proxy"
export FTP_PROXY="$ftp_proxy"

# Bypass proxy for localhost + domain + LAN
export no_proxy="localhost,127.0.0.1,::1,.maksonlee.com,192.168.0.0/24,192.168.0.*"
export NO_PROXY="$no_proxy"
EOF
sudo chmod 644 /etc/profile.d/90-proxy.sh
  • Apt proxy (root / non-interactive)

/etc/profile.d only affects interactive shells. To force apt, unattended-upgrades, timers, etc. to use the proxy, add /etc/apt/apt.conf.d/80proxy:

Acquire::http::Proxy  "http://squid.maksonlee.com:3128/";
Acquire::https::Proxy "http://squid.maksonlee.com:3128/";
Acquire::ftp::Proxy   "ftp://squid.maksonlee.com:3128/";
  • npm / Node.js behind the proxy (trust the system CA bundle)

npm and Node tooling sometimes ignore environment CA settings unless explicitly pointed at your system trust bundle. Set cafile so npm trusts the same CAs your OS does (which now includes your Root CA):

# Make npm trust the system CA bundle (contains your installed Root CA)
npm config set cafile /etc/ssl/certs/ca-certificates.crt
  1. Verify

Through the proxy:

curl -I https://example.com/

Check TLS chain/issuer (no verify errors expected if Root is trusted):

openssl s_client -proxy squid.maksonlee.com:3128 -connect example.com:443 \
  -verify 5 -CAfile /etc/ssl/certs/ca-certificates.crt -brief </dev/null
# Expect: issuer = your Squid Sub-CA ; verify return code: 0 (ok)

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top