Squid3 Proxy on Ubuntu

Using a personal proxy server can be helpful for a variety of reasons, such as:

  • Performance – network speed and bandwidth
  • Security – filtering and monitoring
  • Debugging – to trace activity

Here are some simple steps to get you started,  obviously you will need to further “harden” security to make it production ready!


sudo apt-get install squid3


cd /etc/squid3/
sudo mv squid.conf squid.orig
sudo vi squid.conf

NOTE: the following configuration works, but will likely need to be adapted for your specific usage.


http_port 3128
visible_hostname proxy.EXAMPLE.com
auth_param digest program /usr/lib/squid3/digest_file_auth -c /etc/squid3/passwords
#auth_param digest program /usr/lib/squid3/digest_pw_auth -c /etc/squid3/passwords
auth_param digest realm proxy
auth_param basic credentialsttl 4 hours
acl authenticated proxy_auth REQUIRED
acl localnet src 10.0.0.0/8 # RFC 1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC 1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC 1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
#acl SSL_ports port 443
#http_access deny to_localhost
#http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access allow localhost
http_access allow authenticated
via on
forwarded_for transparent

Create the users and passwords:

sudo apt-get install apache2-utils (required for htdigest)
sudo htdigest -c /etc/squid3/passwords proxy user1
sudo htdigest /etc/squid3/passwords proxy user2

Open up firewall port (if enabled):

sudo ufw allow 3128

Restart the server and tail the logs:

sudo service squid3 restart
sudo tail -f /var/log/squid3/access.log

OTHER FILE LOCATIONS:

/var/spool/squid3
/etc/squid3

MONITORING with Splunk…

sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid3/access.log -index main -sourcetype Squid3
sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid3/cache.log -index main -sourcetype Squid3

REFERENCES:

Brotli Compression

If you look at HTTP Headers as often as I do, you’ve likely noticed something different in Firefox 44 and Chrome 49. In addition to the usual ‘gzip’, ‘deflate’ and ‘sdhc’ , a new value ‘br’ has started to appear for HTTPS connections.

Request:

Accept-Encoding:br

Response:

Content-Encoding:br

Compared to gzip, Brotli claims to have significantly better (26% smaller) compression density woth comparable decompression speed.

The smaller compressed size allows for better space utilization and faster page loads. We hope that this format will be supported by major browsers in the near future, as the smaller compressed size would give additional benefits to mobile users, such as lower data transfer fees and reduced battery use.

Advantages:

  • Brotli outperforms gzip for typical web assets (e.g. css, html, js) by 17–25 %.
  • Brotli -11 density compared to gzip -9:
  • html (multi-language corpus): 25 % savings
  • js (alexa top 10k): 17 % savings
  • minified js (alexa top 10k): 17 % savings
  • css (alexa top 10k): 20 % savings


NOTE: Brotli is not currently supported Apache HTTPd server (as of 2016feb10), but will likely be added in an upcoming release.

http://mail-archives.apache.org/mod_mbox/httpd-users/201601.mbox/%3C549EBED9-2E3F-43CC-BDED-198B8ED4E257@jaguNET.com%3E

Until there is native support, you can pre-compress files by following instructions here…
https://lyncd.com/2015/11/brotli-support-apache/

REFERENCES:

HTTP Strict Transport Security (HSTS)

The HTTP Strict Transport Security feature lets a web site inform the browser that it should never load the site using HTTP, and should automatically convert all attempts to access the site using HTTP to HTTPS requests instead.

Example Use case:
If a web site accepts a connection through HTTP and redirects to HTTPS, the user in this case may initially talk to the non-encrypted version of the site before being redirected, if, for example, the user types http://www.foo.com/ or even just foo.com.

Problem:
This opens up the potential for a man-in-the-middle attack, where the redirect could be exploited to direct a user to a malicious site instead of the secure version of the original page.

Risk:
For HTTP sites on the same domain it is not recommended to add a HSTS header but to do a permanent redirect (301 status code) to the HTTPS site.

Bonus:
Google is always “tweaking” their search algorithms, and, at least at present time, gives greater weight to secure websites.


# Optionally load the headers module:
LoadModule headers_module modules/mod_headers.so

<VirtualHost *:443>
# Guarantee HTTPS for 1 Year including Sub Domains
Header always set Strict-Transport-Security "max-age=31536000; includeSubdomains; preload"
</VirtualHost>

Then you might (optionally, but recommended) force ALL HTTP users to HTTPS:

# Redirect HTTP connections to HTTPS
<VirtualHost *:80>
ServerAlias *
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTPS} off
#RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [redirect=301]
</IfModule>
</VirtualHost>

That’s it…

REFERENCES:

Poodle.. or rather, what’s all the fuss with SSLv3

The “Poodle” attack on websites and browsers was all over the media a few weeks ago, following in the shadow of Heartbleed.

Here’s what most users need to know… This is an vulnerability that exists in secure internet communication because…

  1. While most newer systems rely on TLS security, they still support older protocols (SSLv3 in particular for this issue)
  2. As secure communications generally attempt to find a “common” method, they will often “drop down” to older supported versions (even if they are now often considered insecure!)
  3. Most browser and server software (unless recently patched) will allow for this “drop down” in security.
  4. Most software provides a mechanism to disable this by the user or in configuration.
  5. Upgrading your software will usually remove these “problematic” vulnerabilities.

Simply put… for a consumer, it’s best to upgrade to a newer browser or find the appropriate configuration to disable SSLv3 if you are unable to upgrade. Server administrators generally should update their sofware on a regular basis for security items such as this one!

NOTE: Many CDN’s such as CloudFlare are proactive and block this vulnerability.

Technical details on the Poodle vulnerability (if you’re into that sort of thing!):

Instructions here are for Apache HTTPd 2.2.23 and newer, other servers will require a similar change:


  1. sudo vi /etc/apache2/mods-enabled/ssl.conf
  2. Change the following line from:
    SSLProtocol All -SSLv2
    to:
    SSLProtocol All -SSLv2 -SSLv3
  3. sudo service apache2 reload
  4. sudo service apache2 restart

Can be tested at the following websites:

REFERENCES:

Install free “recognized” SSL certificates for Apache2

Once you have your server running with a self-signed certificate you might find it useful to have a “real” certificate that does not warn users.

Many of the CA’s provide test certificates that are generally valid for 30-60 days, I’ve recently discovered StartSSL, that generates free certificates that are valid for a full year.

  1. Generating keys and certificates….

    NOTE: this process is rather involved and is documented better elsewhere, here’s what I needed to remember to get the keys and certificates.

    • save ssl.key (private)
    • save ssl.crt (pem encoded)
    • get file from control panel: sub.class1.server.ca.pem
  2. Make sure that you move all three files to the /etc/apache2/ssl/ folder on the server.
  3. Edit the config file…
    sudo vi /etc/apache2/sites-available/default-ssl.conf

    Modify the values related to the keys and certs…

    SSLCertificateFile /etc/apache2/ssl/ssl.crt
    SSLCertificateKeyFile /etc/apache2/ssl/ssl.key
    SSLCertificateChainFile /etc/apache2/ssl/sub.class1.server.ca.pem

  4. Reload the config and restart…

    sudo service apache2 reload
    sudo service apache2 restart
  5. Test it out…
    https://www.ssllabs.com/ssltest/analyze.html?d=YOURDOMAIN.COM

REFERENCES:

Create self-signed SSL certificates for Apache on Ubuntu

To increase the security of your web applications, it is a standard process to enable HTTPS/SSL/TLS. Unfortunately, purchasing certificates can often be very expensive. Luckily, you can create a self-signed certificate for free for casual use or testing.

These steps are for Ubuntu, I wrote similar documentation for the Windows platform that you can find way back in my blog archives!

NOTE: As certificates generated in this manner are not verified by any recognized authority, many browsers will warn users (often in frightening language) about their insecurity. As stated above, these are best used only for internal use.

  1. First you will need to have apache2 installed, at a minimum you need to run:
    sudo apt-get install apache2
  2. Enable the SSL module:
    sudo a2enmod ssl

  3. Create the folder to store the keys and certificates:
    sudo mkdir /etc/apache2/ssl

  4. Generate a private key and certificate:

    sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.crt

    Enter reasonable values for the fields in question.
    For FQDN Common Name enter *.domain.com for wildcard support!

  5. Edit the config file:

    sudo vi /etc/apache2/sites-available/default-ssl.conf

  6. Un-comment or update the following lines:

    ServerName YOURDOMAIN.COM
    ServerAlias WWW.YOURDOMAIN.COM
    SSLCertificateFile /etc/apache2/ssl/apache.crt
    SSLCertificateKeyFile /etc/apache2/ssl/apache.key

  7. Enable to SSL website and restart:

    sudo a2ensite default-ssl.conf
    sudo service apache2 reload
    sudo service apache2 restart

  8. Test it out… provided your firewall routes port 443 to your server.

    https://www.ssllabs.com/ssltest/analyze.html?d=YOURDOMAIN.COM

REFERENCES:

Device Fingerprinting

Often it can be beneficial to ‘more’ uniquely identify your users. For applications this is trivial, but in a web browser this can be accomplished via only a few attributes.

  • HTTP – User-Agent, IP Address, Content types and languages accepted.
  • HTTPS/SSL – the keys and encryption methods available to a specific user may vary for each client configuration.
  • HTML5 – local storage and capabilities supported.
  • Geolocation – this is included in HTML5, but also supported in many clients without full HTML5 support, or via browser extensions.
  • JavaScript – Installed version – and many DOM attributes/capabilities such as timezone, installed plugins, screen sizes and fonts from the browser can be detected.
  • Java (Applet) – Installed version – this can often be used to get additional information regarding the client system directly from the VM or Operating System itself. (* Persistent Cookies possible)
  • Flash – Installed version – this can often be used to get additional information regarding the client system directly from the Operating System itself. (* Persistent Cookies possible)
  • Silverlight (for Microsoft Windows) – – Installed version and additional information from Operating System?
  • GoogleGEARS (deprecated) – Installed version and additional information from Operating System such as Geolocation

REFERENCES:

Accessing VirtualBox client from host

Assuming that you are a developer on Windows that run a VirtualBox instance for a server, you might find it difficult to browse/access the “virtual” server that are running as a client. Here’s an example of the config required for standard HTTP, HTTPS and SSH, you can easily expand for other services as needed. Example assumes that your client is named “Ubuntu64”, you need to change appropriately for each client.

NOTE: if you are using a Linux host, the commands are similar.

cd C:\Program Files\Oracle\VirtualBox
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTP/HostPort 80
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTP/GuestPort 80
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTP/Protocol TCP
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTPS/HostPort 443
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTPS/GuestPort 443
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/ApacheHTTPS/Protocol TCP
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/Tomcat/HostPort 8080
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/Tomcat/GuestPort 8080
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/Tomcat/Protocol TCP
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/SSH/HostPort 22
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/SSH/GuestPort 22
VBoxManage.exe setextradata "Ubuntu64" VBoxInternal/Devices/pcnet/0/LUN#0/Config/SSH/Protocol TCP
VBoxManage.exe getextradata "Ubuntu64" enumerate

Sniff for SSL capability of browser

If you run a secure server, you often have some non-secure content prior to authentication of a secure session. To provide a mechanism to show a page prior to authentication, you can “sniff” for the clients capability with a small bit of JavaScript.

First establish a global variable on the page:
<script type="text/javascript">
var sslok = 0;
</script>

Then, include a JavaScript file that is ONLY available via a secure
<script type="text/javascript" src="https://www.giantgeek.com.com/secure/sniff.js"></script>

The ‘sniff.js’ file should contain (at a minimum):
sslok = 1;

Finally, check and act on the value:
<script type="text/javascript">
if (sslok === 1) {
window.location.href = 'https://www.giantgeek.com/secure/';
}
</script>

Done!

HTTP is deprecated?

I found this several years ago, and ‘most’ of my websites implement this standard.

While “http://” implies IP Port 80, and “https://” implies IP Port 443, the payload of such traffic is no longer limited to HyperText (the “HT” in HTTP). To support future protocols, it is advised that websites not force this upon future implementations of the IP stack.

Unfortunately, there is ONE case that implementation of this process can cause your websites. If you run on non-standard ports (or use HTTPS), you’ll still need the protocol (“http:” or “https:”) on the links used to swap protocols.

Obvious advantage here is that all of your links can be shortened.

More information is available at:

http://www.no-http.org/

Cheers!