Selenium Firefox modifyheaders

A few of my tests require access to modify the HTTP Request headers. Unfortunately, Selenium hides access to them to allow for portability, and to better emulate what “users” generally can change. To work around this a Firefox extension can be used and configured at runtime for this purpose.

NOTE: for Maven, you need to place a copy of the .xpi file referenced into the /src/test/resources folder for Selenium to locate it.

In the example below, I’m setting the HTTP Header for “DNT” to “1”.

public FirefoxDriver createFirefoxDriver() throws URISyntaxException, IOException {
// Specify the install location (if not default)
System.setProperty("webdriver.firefox.bin","C:\\path\\to\\Firefox.exe");
// Prevent Console log "noise" from the Selenium Firefox plugin
System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog");
System.setProperty("org.apache.commons.logging.simplelog.log.httpclient.wire", "OFF");
System.setProperty("org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient", "OFF");

final FirefoxProfile profile = new FirefoxProfile();
final URL url = this.getClass().getResource("/modify_headers-0.7.1.1-fx.xpi");
final File modifyHeaders = modifyHeaders = new File(url.toURI());

profile.setEnableNativeEvents(false);
profile.addExtension(modifyHeaders);

profile.setPreference("modifyheaders.headers.count", 1);
profile.setPreference("modifyheaders.headers.action0", "Add");
profile.setPreference("modifyheaders.headers.name0", "DNT");
profile.setPreference("modifyheaders.headers.value0", "1");
profile.setPreference("modifyheaders.headers.enabled0", true);
profile.setPreference("modifyheaders.config.active", true);
profile.setPreference("modifyheaders.config.alwaysOn", true);

final DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setBrowserName("firefox");
capabilities.setPlatform(org.openqa.selenium.Platform.ANY);
capabilities.setCapability(FirefoxDriver.PROFILE, profile);
return new FirefoxDriver(capabilities);
}

Load Testing web application with Selenium and TestNG

I’ve used Selenium for while to do verification tests of web applications, recently I discovered a very simple way to use it with TestNG and Maven to do some performance testing. TestNG allows for the use of annotations to allow multi-threading and iterations.

pom.xml:

<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.44.0</version>
<scope>test</scope>
</dependency>
<dependencies>

And as for a simple test to get started with… scripting of steps is available online or could be in a future blog post.

/*
* COPYRIGHT. none
*/
package com.example.selenium;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
/**
* Simple test example for Selenium
*/
public class SeleniumTest {

private static final Logger LOGGER = LoggerFactory.getLogger(SeleniumTest.class);
/**
* TODO Un-comment or change if needed to set your local path!
*/
@BeforeClass
public void oneTimeSetUp() {
System.out.println(“————————————– init —————————————-“);
//System.setProperty(“webdriver.firefox.bin”,”C:\\path\\to\\firefox.exe”);
}
/**
* NOTE: uses TestNG – behaves differently than JUnit
*/
@Test(invocationCount = 1, threadPoolSize = 5)
public void testLoadApp() {

final String fn = “testLoadApp”;
final String baseUrl = “http://www.giantgeek.com/index.php”;
LOGGER.debug(“[START] Thread Id: {} is started!”, Thread.currentThread().getId());

WebDriver driver = null;
final long start = System.currentTimeMillis();
try{
driver = (WebDriver)new FirefoxDriver();
driver.get(baseUrl);

final String actual = driver.getTitle();
LOGGER.debug(“Page Title is {}”, actual);
final String expected = “GIANTGEEK.COM”;
Assert.assertEquals(actual,expected);
//perform whatever actions, like login, submit form or navigation


}catch(final WebDriverException ex){
LOGGER.warn(fn+":WebDriverException:{}",ex);
}catch(final Exception ex){
LOGGER.warn(fn+":Exception:{}",ex);
}
finally {
final long elapsed = System.currentTimeMillis() - start;
LOGGER.debug("[END] Thread Id: {}, elapsed={}", Thread.currentThread().getId(),elapsed);
if(driver != null){
driver.quit();
}
}
}
}

WARNING: Selenium Tests MAY fail if the browser used for testing is updated in the Operating System. Updating the pom.xml to a newer release usually helps!

REFERENCES:

“msapplication-config” and browserconfig.xml

Windows-8/MSIE-11 introduced Tiles, as such server administrators may have started seeing HTTP 404 errors in their server logs as it attempts to look for a “browserconfig.xml” file at the root of a website domain. If you are inclined to use this file, you should definitely look into the documentation for how to best make use of it. Others may just wish to prevent the error from making “noise” in their log files.

To remove the error, add the following to your pages; alternately you COULD define the URL of your file as the ‘content’ attribute:

<meta name="msapplication-config" content="none" />

You can alternately place an empty /browserconfig.xml on your web server for each domain.

An common example of how to use this file is below:

<?xml version="1.0" encoding="utf-8"?>
<browserconfig>
<msapplication>
<tile>
<square70x70logo src="/mstile-70x70.png"/>
<square150x150logo src="/mstile-150x150.png"/>
<wide310x150logo src="/mstile-310x150.png"/>
<square310x310logo src="/mstile-310x310.png"/>
<TileColor>#8bc53f</TileColor>
<TileImage src="/mstile-150x150.png" />
</tile>
</msapplication>
</browserconfig>

REFERENCES:

Install Subversion Server on Ubuntu

Subversion is a commonly used central version control system for software development. There are currently still a large number of organizations that rely upon it, many have since moved on to Git.

  1. sudo apt-get install apache2 apache2-utils
  2. sudo apt-get install subversion subversion-tools libapache2-svn
  3. sudo mkdir /home/svn
  4. svnadmin create /home/svn/test
  5. Create a group for subversion users:
    sudo groupadd subversion
  6. sudo adduser USERNAME
  7. Add a user to the group:
    sudo useradd -G USERNAME subversion
  8. sudo chown -R www-data:subversion /home/svn/test
  9. sudo chmod -R g+rws /home/svn/test
  10. sudo a2enmod dav_svn
  11. To create/clobber a new file for the first user:
    sudo htpasswd -c /etc/apache2/.htpasswd YOURUSER
  12. To add additional users:
    sudo htpasswd /etc/apache2/.htpasswd YOURUSER
    (repeat for new users without the -c as that creates/clobbers the file)
  13. sudo vi /etc/apache2/sites-available/000-default.conf
    Then add to the bottom:
    (NOTE1: the LimitExcept can be enabled to allow anonymous access):
    (NOTE2: the LimitXMLRequestBody can be uncomment to allow large commits)

    <Location /svn>
    DAV svn
    SVNParentPath /home/svn
    AuthType Basic
    AuthName "Subversion Repository"
    # AuthUserFile /etc/svn-auth
    AuthUserFile /etc/apache2/.htpasswd
    #LimitXMLRequestBody 0
    #<LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
    #</LimitExcept>
    </Location>
  14. sudo service apache2 reload
  15. sudo service apache2 restart

    NOTE: At this point you should be able to browse and do a remote checkout of the code from another machine….

    http://YOUR-IP-OR-HOSTNAME/svn
    and
    svn co http://YOUR-IP-OR-HOSTNAME/svn/test --username YOURUSER --password YOURPASS

  16. sudo vi /etc/init/svnserve.conf
    Add the following:

    # svnserve - Subversion server
    description "Subversion server"
    start on (local-filesystems and net-device-up IFACE=lo and started udev-finish)
    stop on runlevel [06]
    chdir /home/svn
    respawn
    respawn limit 2 3600
    exec /usr/bin/svnserve --foreground --daemon --config-file /home/svn/repos/conf/svnserve.conf --root /home/svn/repos/
  17. Then execute:
    sudo initctl start svnserve
  18. Back on the client side…
    Create a new folder inside your user folder:
    cd ~/test
  19. Check out the project into this folder:
    svn checkout http://YOUR-IP-OR-HOSTNAME/svn/test
  20. Let us just add a new HTML index file to the folder:
    vi index.html
  21. Add it to version control:
    svn add index.html
    Commit the new file:
    svn commit -m "commit message"
    Update:
    svn up
  22. That should cover most cases for you…

REFERENCES:

Security through obscurity – hiding your server version information

I’ve recently spent a lot of time reviewing the OWASP documentation, and (like many corporations) realized that I’d neglected to keep up with this configuration item.

By sharing the exact version of each piece of server software you are using, “hackers” are able to quickly identify unpatched systems and their known vulnerabilities.

To make their work harder, there are a few simple steps that the server admin can take to remove this information from the HTTP Headers and error pages.

Apache HTTPd:

  1. sudo vi /etc/apache2/conf-enabled/security.conf
  2. Add:

    ServerTokens ProductOnly
    ServerSignature Off
  3. If using virtual hosts, add the following to each one:
    ServerSignature Off
  4. sudo service apache2 restart

Apache Tomcat:

  1. vi /opt/tomcat7/conf/server.xml
  2. Find the <Connector > entry and add:
    server="Apache"
  3. cd /opt/tomcat7/lib
  4. mkdir -p org/apache/catalina/util
  5. vi /opt/tomcat7/lib/org/apache/catalina/util/ServerInfo.properties
    server.info=Apache Tomcat
  6. sudo service tomcat7 restart

PHP “X-Powered-By: PHP/5.x.x-1ubuntuX.X”

  1. sudo vi /etc/php5/apache2/php.ini
    expose_php = Off
  2. sudo service apache2 restart

REFERENCES:

Preventing Blackberry browser from messing up your UI

I’ve previously given steps to prevent phone numbers (and other elements) from being automatically reformatted by Skype Toolbar and IOS Safari, there is still a small segment of the user population that uses Blackberry devices that can similarly benefit from a little code.

The following stops auto detection and formatting of phone and email addresses on devices with the BlackBerry Browser.

HTML:

<meta http-equiv="x-rim-auto-match" content="none" />

WML:

<meta name="x-rim-auto-match" http-equiv="x-rim-auto-match" forua="true" content="none" />

REFERENCES:

Comcast Business Class gateway forwarding port 22 for SSH

For as long as I’ve had Comcast, and other providers for that matter, I’ve been able to configure my internet gateway/router to allow port 22 (SSH) access to an internal machine. It came as a surprise to me earlier this week that I was blocked when I tried to use their web admin console to change the internal forwarding to a newer machine. As usual, Technical Support was less that helpful and said that it was not possible to do so, and never should have been as Comcast uses that port to administer the gateway. To make matters more disturbing, I was told that I could not have similar SSH access to the gateway, and that replacing their hardware, while permitted, would prevent my use of a static IP.

Back to the solution, as I know that I had only setup this forwarding about a year ago, and it was working only minutes before I tried to change it, I knew that the configuration was possible if I could figure out how it was being blocked. The message in the web console was a javascript alert(); and gave me a starting point. I opened up Firefox and used Firebug to look for the message. Here are a few interesting findings from:

http://HOSTNAME/user/feat-firewall-port-forward-edit.asp

var RemoteManagementPortsCgiBase = “8080,8080,1\|8181,8181,1\|2323,2323,1\|22,22,1\|”;

msg += “Public Port Range conflict with Remote Management Ports.\n”;

if (msg.length > 1)
{
alert(msg);
return false;
}
return true;
}

If you even a little bit of javascript (or simple computer programming for that matter), the solution is clear…. if the ‘msg’ value is empty you will not see the alert or be prevented from making the change you desire.

Lesson to be learned by the Comcast developers (or most likely = subcontractors), always validate submitted form data in your application code, NEVER rely upon javascript alone to verify user entered data!

I also find it interesting that they are also preventing 8080, 8081 and 2323… perhaps that’s their other back doors in these gateways for their access. The same approach should work for those ports if you need it!

Name based virtual hosting with Apache2 on Ubuntu

I often find it necessary to setup a temporary server for testing web applications. The easiest way to support this is often to create an integration (beta) test server that can be configured to support multiple hosts.

Initial Installation:

sudo apt-get install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod rewrite
sudo chmod 755 /var/www

To add a new site:

cd /etc/apache2/sites-available
sudo cp 000-default.conf yourdomain.com.conf
sudo vi yourdomain.com.conf
(change values as appropriate)
sudo chown -R YOURUSER:YOURGROUP yourdomain.com.conf
sudo mkdir -p /var/www/yourdomain.com
sudo chmod 755 /var/www
sudo vi /var/www/yourdomain.com/index.html
(add some fluff)
sudo chown -R YOURUSER:YOURGROUP /var/www/yourdomain.com
sudo a2ensite yourdomain.com
sudo service apache2 reload
sudo service apache2 restart

NOTE: replace YOURUSER and YOURGROUP with appropriate values for your server.

Content-Security-Policy HTTP Header

There’s yet another new means to ‘help’ client User-Agents with preventing XSS on your websites.

In it’s simplest form you can simply use the following HTTP Header(s), the second one is for earlier versions of Webkit (Chrome/Safari):

Content-Security-Policy: default-src 'self'
Webkit-CSP: default-src 'self'

You can also add to the above to permit assets to load from other sources.
For example, if you were to permit javascript files from example.com you could include:

Content-Security-Policy: default-src 'self'; script-src http://example.com

Additionally, while failures are noted in the client’s browser console (that most users are not aware of), you can have them sent back to your server by adding a ‘report-uri’ attribute with an appropriate handler:

Content-Security-Policy: default-src 'self'; report-uri http://example.com/csp-report.php

REFERENCES:

Device Fingerprinting

Often it can be beneficial to ‘more’ uniquely identify your users. For applications this is trivial, but in a web browser this can be accomplished via only a few attributes.

  • HTTP – User-Agent, IP Address, Content types and languages accepted.
  • HTTPS/SSL – the keys and encryption methods available to a specific user may vary for each client configuration.
  • HTML5 – local storage and capabilities supported.
  • Geolocation – this is included in HTML5, but also supported in many clients without full HTML5 support, or via browser extensions.
  • JavaScript – Installed version – and many DOM attributes/capabilities such as timezone, installed plugins, screen sizes and fonts from the browser can be detected.
  • Java (Applet) – Installed version – this can often be used to get additional information regarding the client system directly from the VM or Operating System itself. (* Persistent Cookies possible)
  • Flash – Installed version – this can often be used to get additional information regarding the client system directly from the Operating System itself. (* Persistent Cookies possible)
  • Silverlight (for Microsoft Windows) – – Installed version and additional information from Operating System?
  • GoogleGEARS (deprecated) – Installed version and additional information from Operating System such as Geolocation

REFERENCES: