Google and Facebook bypassing P3P User Privacy Settings

I wrote about P3P a very long time ago, and have implemented it on several websites. Some history, the W3C crafted the P3P policy.
Microsoft introduced P3P support in IE6 (in 2001) and it remains implemented in all current versions of the browser. The primary intended use is to block 3rd party cookies within the browser on behalf of the user.

Interesting enough, Microsoft has had been a bit of a struggle with Google and Facebook, which send the following HTTP response headers.

Google’s Response:

P3P: CP="This is not a P3P policy! See for more info."

Facebook’s response:

P3P: CP="Facebook does not have a P3P policy. Learn why here:"


Install Google mod_pagespeed for Apache HTTP

Website network performance is usually a very complicated process. Over the years, I’ve outlined many development techniques that can be used toward this goal. I’d heard about mod_pagespeed for some time, but never had the opportunity to experiment with it until recently. My first impression is that it is a VERY EASY means to gain performance improvements without reworking your existing website to implement techniques for establishing far-future expires, cache-busting, minification and static file merging.

Out of the box, most common techniques are automatically applied to your assets and a local server cache is created to utilize them.

Default installation is trivial:

  1. Download the latest version for your server architecture:




  2. sudo dpkg -i mod-pagespeed-*.deb

  3. sudo apt-get -f install
    (if required)

  4. sudo service apache2 restart

NOTE: Using tools like Firebug will enable you to see an HTTP Header indicating the version being used for your requests.

If you need to modify configuration from the default:

sudo vi /etc/apache2/mods-available/pagespeed.conf

For VirtualDomains, you can selectively enable and disable PageSpeed and many of it’s settings in your appropriate configuration files with:

<IfModule pagespeed_module>
ModPagespeed on

NOTE: Appending ?ModPagespeed=off to your URL will disable functions for that request.


Java User-Agent detector and caching

It’s often important for a server side application to understand the client platform. There are two common methods used for this.

  1. On the client itself, “capabilities” can be tested.
  2. Unfortunately, the server cannot easily test these, and as such must usually rely upon the HTTP Header information, notably “User-Agent”.

Example User-agent might typically look like this for a common desktop browser, developers can usually determine the platform without a lot of work.

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)"

Determining robots and mobile platforms, unfortunately is a lot more difficult due to the variations. Libraries as those described below simplify this work as standard Java Objects expose the attributes that are commonly expected.

With Maven, the dependencies are all resolved with the following POM addition:


/* Get an UserAgentStringParser and analyze the requesting client */
final UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();
final ReadableUserAgent agent = parser.parse(request.getHeader("User-Agent"));

out.append("You're a '");
out.append("' on '");

As indicated on the website documentation, running this query for each request uses valuable server resources, it’s best to cache the responses to minimize the impact!

NOTE: the website caching example is hard to copy-paste, here’s a cleaner copy.

package com.example.cache;

import java.util.concurrent.TimeUnit;
import net.sf.uadetector.ReadableUserAgent;
import net.sf.uadetector.UserAgentStringParser;
import net.sf.uadetector.service.UADetectorServiceFactory;

* Caching User Agent parser
* @see
* @author Scott Fredrickson [skotfred]
* @since 2015jan28
* @version 2015jan28
public final class CachedUserAgentStringParser implements UserAgentStringParser {

private final UserAgentStringParser parser = UADetectorServiceFactory.getCachingAndUpdatingParser();
private static final int CACHE_MAX_SIZE = 100;
private static final int CACHE_MAX_HOURS = 2;
* Limited to 100 elements for 2 hours!
private final Cache<String , ReadableUserAgent> cache = CacheBuilder.newBuilder().maximumSize(CACHE_MAX_SIZE).expireAfterWrite(CACHE_MAX_HOURS, TimeUnit.HOURS).build();

* @return {@code String}
public String getDataVersion() {
return parser.getDataVersion();
* @param userAgentString {@code String}
* @return {@link ReadableUserAgent}
public ReadableUserAgent parse(final String userAgentString) {
ReadableUserAgent result = cache.getIfPresent(userAgentString);
if (result == null) {
result = parser.parse(userAgentString);
cache.put(userAgentString, result);
return result;
public void shutdown() {


HTTP Strict Transport Security (HSTS)

The HTTP Strict Transport Security feature lets a web site inform the browser that it should never load the site using HTTP, and should automatically convert all attempts to access the site using HTTP to HTTPS requests instead.

Example Use case:
If a web site accepts a connection through HTTP and redirects to HTTPS, the user in this case may initially talk to the non-encrypted version of the site before being redirected, if, for example, the user types or even just

This opens up the potential for a man-in-the-middle attack, where the redirect could be exploited to direct a user to a malicious site instead of the secure version of the original page.

For HTTP sites on the same domain it is not recommended to add a HSTS header but to do a permanent redirect (301 status code) to the HTTPS site.

Google is always “tweaking” their search algorithms, and, at least at present time, gives greater weight to secure websites.

# Optionally load the headers module:
LoadModule headers_module modules/

<VirtualHost *:443>
# Guarantee HTTPS for 1 Year including Sub Domains
Header always set Strict-Transport-Security "max-age=31536000; includeSubdomains; preload"

Then you might (optionally, but recommended) force ALL HTTP users to HTTPS:

# Redirect HTTP connections to HTTPS
<VirtualHost *:80>
ServerAlias *
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTPS} off
#RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [redirect=301]

That’s it…


Selenium Firefox modifyheaders

A few of my tests require access to modify the HTTP Request headers. Unfortunately, Selenium hides access to them to allow for portability, and to better emulate what “users” generally can change. To work around this a Firefox extension can be used and configured at runtime for this purpose.

NOTE: for Maven, you need to place a copy of the .xpi file referenced into the /src/test/resources folder for Selenium to locate it.

In the example below, I’m setting the HTTP Header for “DNT” to “1”.

public FirefoxDriver createFirefoxDriver() throws URISyntaxException, IOException {
// Specify the install location (if not default)
// Prevent Console log "noise" from the Selenium Firefox plugin
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire", "OFF");
System.setProperty("", "OFF");

final FirefoxProfile profile = new FirefoxProfile();
final URL url = this.getClass().getResource("/modify_headers-");
final File modifyHeaders = modifyHeaders = new File(url.toURI());


profile.setPreference("modifyheaders.headers.count", 1);
profile.setPreference("modifyheaders.headers.action0", "Add");
profile.setPreference("modifyheaders.headers.name0", "DNT");
profile.setPreference("modifyheaders.headers.value0", "1");
profile.setPreference("modifyheaders.headers.enabled0", true);
profile.setPreference("", true);
profile.setPreference("modifyheaders.config.alwaysOn", true);

final DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(FirefoxDriver.PROFILE, profile);
return new FirefoxDriver(capabilities);

Load Testing web application with Selenium and TestNG

I’ve used Selenium for while to do verification tests of web applications, recently I discovered a very simple way to use it with TestNG and Maven to do some performance testing. TestNG allows for the use of annotations to allow multi-threading and iterations.



And as for a simple test to get started with… scripting of steps is available online or could be in a future blog post.

package com.example.selenium;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
* Simple test example for Selenium
public class SeleniumTest {

private static final Logger LOGGER = LoggerFactory.getLogger(SeleniumTest.class);
* TODO Un-comment or change if needed to set your local path!
public void oneTimeSetUp() {
System.out.println("-------------------------------------- init ----------------------------------------");
* NOTE: uses TestNG - behaves differently than JUnit
@Test(invocationCount = 1, threadPoolSize = 5)
public void testLoadApp() {

final String fn = "testLoadApp";
final String baseUrl = "";
LOGGER.debug("[START] Thread Id: {} is started!", Thread.currentThread().getId());

WebDriver driver = null;
final long start = System.currentTimeMillis();
driver = (WebDriver)new FirefoxDriver();

final String actual = driver.getTitle();
LOGGER.debug("Page Title is {}", actual);
final String expected = "GIANTGEEK.COM";
//perform whatever actions, like login, submit form or navigation

}catch(final WebDriverException ex){
}catch(final Exception ex){
finally {
final long elapsed = System.currentTimeMillis() - start;
LOGGER.debug("[END] Thread Id: {}, elapsed={}", Thread.currentThread().getId(),elapsed);
if(driver != null){

WARNING: Selenium Tests MAY fail if the browser used for testing is updated in the Operating System. Updating the pom.xml to a newer release usually helps!


“msapplication-config” and browserconfig.xml

Windows-8/MSIE-11 introduced Tiles, as such server administrators may have started seeing HTTP 404 errors in their server logs as it attempts to look for a “browserconfig.xml” file at the root of a website domain. If you are inclined to use this file, you should definitely look into the documentation for how to best make use of it. Others may just wish to prevent the error from making “noise” in their log files.

To remove the error, add the following to your pages; alternately you COULD define the URL of your file as the ‘content’ attribute:

<meta name="msapplication-config" content="none" />

You can alternately place an empty /browserconfig.xml on your web server for each domain.

An common example of how to use this file is below:

<?xml version="1.0" encoding="utf-8"?>
<square70x70logo src="/mstile-70x70.png"/>
<square150x150logo src="/mstile-150x150.png"/>
<wide310x150logo src="/mstile-310x150.png"/>
<square310x310logo src="/mstile-310x310.png"/>
<TileImage src="/mstile-150x150.png" />


Install Subversion Server on Ubuntu

Subversion is a commonly used central version control system for software development. There are currently still a large number of organizations that rely upon it, many have since moved on to Git.

  1. sudo apt-get install apache2 apache2-utils
  2. sudo apt-get install subversion subversion-tools libapache2-svn
  3. sudo mkdir /home/svn
  4. svnadmin create /home/svn/test
  5. Create a group for subversion users:
    sudo groupadd subversion
  6. sudo adduser USERNAME
  7. Add a user to the group:
    sudo useradd -G USERNAME subversion
  8. sudo chown -R www-data:subversion /home/svn/test
  9. sudo chmod -R g+rws /home/svn/test
  10. sudo a2enmod dav_svn
  11. To create/clobber a new file for the first user:
    sudo htpasswd -c /etc/apache2/.htpasswd YOURUSER
  12. To add additional users:
    sudo htpasswd /etc/apache2/.htpasswd YOURUSER
    (repeat for new users without the -c as that creates/clobbers the file)
  13. sudo vi /etc/apache2/sites-available/000-default.conf
    Then add to the bottom:
    (NOTE1: the LimitExcept can be enabled to allow anonymous access):
    (NOTE2: the LimitXMLRequestBody can be uncomment to allow large commits)

    <Location /svn>
    DAV svn
    SVNParentPath /home/svn
    AuthType Basic
    AuthName "Subversion Repository"
    # AuthUserFile /etc/svn-auth
    AuthUserFile /etc/apache2/.htpasswd
    #LimitXMLRequestBody 0
    Require valid-user
  14. sudo service apache2 reload
  15. sudo service apache2 restart

    NOTE: At this point you should be able to browse and do a remote checkout of the code from another machine….

    svn co http://YOUR-IP-OR-HOSTNAME/svn/test --username YOURUSER --password YOURPASS

  16. sudo vi /etc/init/svnserve.conf
    Add the following:

    # svnserve - Subversion server
    description "Subversion server"
    start on (local-filesystems and net-device-up IFACE=lo and started udev-finish)
    stop on runlevel [06]
    chdir /home/svn
    respawn limit 2 3600
    exec /usr/bin/svnserve --foreground --daemon --config-file /home/svn/repos/conf/svnserve.conf --root /home/svn/repos/
  17. Then execute:
    sudo initctl start svnserve
  18. Back on the client side…
    Create a new folder inside your user folder:
    cd ~/test
  19. Check out the project into this folder:
    svn checkout http://YOUR-IP-OR-HOSTNAME/svn/test
  20. Let us just add a new HTML index file to the folder:
    vi index.html
  21. Add it to version control:
    svn add index.html
    Commit the new file:
    svn commit -m "commit message"
    svn up
  22. That should cover most cases for you…


Security through obscurity – hiding your server version information

I’ve recently spent a lot of time reviewing the OWASP documentation, and (like many corporations) realized that I’d neglected to keep up with this configuration item.

By sharing the exact version of each piece of server software you are using, “hackers” are able to quickly identify unpatched systems and their known vulnerabilities.

To make their work harder, there are a few simple steps that the server admin can take to remove this information from the HTTP Headers and error pages.

Apache HTTPd:

  1. sudo vi /etc/apache2/conf-enabled/security.conf
  2. Add:

    ServerTokens ProductOnly
    ServerSignature Off
  3. If using virtual hosts, add the following to each one:
    ServerSignature Off
  4. sudo service apache2 restart

Apache Tomcat:

  1. vi /opt/tomcat7/conf/server.xml
  2. Find the <Connector > entry and add:
  3. cd /opt/tomcat7/lib
  4. mkdir -p org/apache/catalina/util
  5. vi /opt/tomcat7/lib/org/apache/catalina/util/ Tomcat
  6. sudo service tomcat7 restart

PHP “X-Powered-By: PHP/5.x.x-1ubuntuX.X”

  1. sudo vi /etc/php5/apache2/php.ini
    expose_php = Off
  2. sudo service apache2 restart


Preventing Blackberry browser from messing up your UI

I’ve previously given steps to prevent phone numbers (and other elements) from being automatically reformatted by Skype Toolbar and IOS Safari, there is still a small segment of the user population that uses Blackberry devices that can similarly benefit from a little code.

The following stops auto detection and formatting of phone and email addresses on devices with the BlackBerry Browser.


<meta http-equiv="x-rim-auto-match" content="none" />


<meta name="x-rim-auto-match" http-equiv="x-rim-auto-match" forua="true" content="none" />