Java User-Agent detector and caching

It’s often important for a server side application to understand the client platform. There are two common methods used for this.

  1. On the client itself, “capabilities” can be tested.
  2. Unfortunately, the server cannot easily test these, and as such must usually rely upon the HTTP Header information, notably “User-Agent”.

Example User-agent might typically look like this for a common desktop browser, developers can usually determine the platform without a lot of work.

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)"

Determining robots and mobile platforms, unfortunately is a lot more difficult due to the variations. Libraries as those described below simplify this work as standard Java Objects expose the attributes that are commonly expected.

With Maven, the dependencies are all resolved with the following POM addition:

<dependency>
<groupid>net.sf.uadetector</groupid>
<artifactid>uadetector-resources</artifactid>
<version>2014.10</version>
</dependency>


/* Get an UserAgentStringParser and analyze the requesting client */
final UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();
final ReadableUserAgent agent = parser.parse(request.getHeader("User-Agent"));

out.append("You're a '");
out.append(agent.getName());
out.append("' on '");
out.append(agent.getOperatingSystem().getName());
out.append("'.");

As indicated on the website documentation, running this query for each request uses valuable server resources, it’s best to cache the responses to minimize the impact!

http://uadetector.sourceforge.net/usage.html#improve_performance

NOTE: the website caching example is hard to copy-paste, here’s a cleaner copy.


/*
* COPYRIGHT.
*/
package com.example.cache;

import java.util.concurrent.TimeUnit;
import net.sf.uadetector.ReadableUserAgent;
import net.sf.uadetector.UserAgentStringParser;
import net.sf.uadetector.service.UADetectorServiceFactory;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;

/**
* Caching User Agent parser
* @see http://uadetector.sourceforge.net/usage.html#improve_performance
* @author Scott Fredrickson [skotfred]
* @since 2015jan28
* @version 2015jan28
*/
public final class CachedUserAgentStringParser implements UserAgentStringParser {

private final UserAgentStringParser parser = UADetectorServiceFactory.getCachingAndUpdatingParser();
private static final int CACHE_MAX_SIZE = 100;
private static final int CACHE_MAX_HOURS = 2;
/**
* Limited to 100 elements for 2 hours!
*/
private final Cache<String , ReadableUserAgent> cache = CacheBuilder.newBuilder().maximumSize(CACHE_MAX_SIZE).expireAfterWrite(CACHE_MAX_HOURS, TimeUnit.HOURS).build();

/**
* @return {@code String}
*/
@Override
public String getDataVersion() {
return parser.getDataVersion();
}
/**
* @param userAgentString {@code String}
* @return {@link ReadableUserAgent}
*/
@Override
public ReadableUserAgent parse(final String userAgentString) {
ReadableUserAgent result = cache.getIfPresent(userAgentString);
if (result == null) {
result = parser.parse(userAgentString);
cache.put(userAgentString, result);
}
return result;
}
/**
*
*/
@Override
public void shutdown() {
parser.shutdown();
}
}

REFERENCES:

Take and save a screenshot capture with Selenium

As I recently discussed Selenium, it might be useful to know how to take screen captures during tests. I’ve found that putting the function into a java method makes usage a LOT easier… here are the relevant code bits (obviously this will not run on it’s own). Feel free to expand on it as needed as this is just a stub.


import java.io.File;
import java.io.IOException;
import org.apache.commons.io.FileUtils;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
/**
* @param driver {@code WebDriver}
* @param filename {@code String}
*/
protected static void takeScreenshot(final WebDriver driver, final String suffix){
final String fn = "takeScreenshot("+ driver.getCurrentUrl() +","+suffix+")";
final String filename = "/tmp/screenshot_" + suffix + ".png";

LOGGER.debug("takeScreenshot("+ driver.getCurrentUrl() +","+filename+")");
final File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
// Now you can do whatever you need to do with it, for example copy somewhere
try{
FileUtils.copyFile(scrFile, new File(filename));
LOGGER.debug("[EXEC] {} {}",filename, fn);
}
catch(final IOException ex){
LOGGER.error("IOException:fn={},file={}:ex={}",fn,filename,ex);
}

}

Some other helpful Selenium methods

Here are a few other helpful functions for use of Selenium testing scripts as you often need to click links, fill in fields, and submit forms.


import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
/**
*
* @param driver
* @param name
* @return
*/
public static WebElement findElementByName(final WebDriver driver, final String name){
final By el = By.name(name);
final WebElement wel = driver.findElement(el);
return wel;
}
/**
*
* @param driver
* @param name
* @param value
*/
public static void sendKeysByFieldName(final WebDriver driver, final String name, final String value){
final WebElement wel = findElementByName(driver, name);
wel.sendKeys(value);
}
/**
*
* @param driver
* @param xpath
*/
public static void clickByXpath(final WebDriver driver, final String xpath){
final By el = By.xpath(xpath);
//LOGGER.info("el is {}", el);
final WebElement wel = driver.findElement(el);
wel.click();
}
/**
*
* @param driver
* @param linktext
*/
public static void waitToClickLinkText(final WebDriver driver, final String linktext){
final WebDriverWait wait = new WebDriverWait(driver, 10);
final By el = By.linkText(linktext);
wait.until(ExpectedConditions.elementToBeClickable(el));
final WebElement wel = driver.findElement(el);
wel.click();
}
/**
*
* @param driver
* @param text
* @return
*/
public boolean pageContainsText(final WebDriver driver, final String text){
final String xpathExpression = "//*[contains(text(),'" + text + "')]";
final List<WebElement> list = driver.findElements(By.xpath(xpathExpression));
return list.size() > 0;
}

Load Testing web application with Selenium and TestNG

I’ve used Selenium for while to do verification tests of web applications, recently I discovered a very simple way to use it with TestNG and Maven to do some performance testing. TestNG allows for the use of annotations to allow multi-threading and iterations.

pom.xml:

<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.44.0</version>
<scope>test</scope>
</dependency>
<dependencies>

And as for a simple test to get started with… scripting of steps is available online or could be in a future blog post.

/*
* COPYRIGHT. none
*/
package com.example.selenium;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
/**
* Simple test example for Selenium
*/
public class SeleniumTest {

private static final Logger LOGGER = LoggerFactory.getLogger(SeleniumTest.class);
/**
* TODO Un-comment or change if needed to set your local path!
*/
@BeforeClass
public void oneTimeSetUp() {
System.out.println(“————————————– init —————————————-“);
//System.setProperty(“webdriver.firefox.bin”,”C:\\path\\to\\firefox.exe”);
}
/**
* NOTE: uses TestNG – behaves differently than JUnit
*/
@Test(invocationCount = 1, threadPoolSize = 5)
public void testLoadApp() {

final String fn = “testLoadApp”;
final String baseUrl = “http://www.giantgeek.com/index.php”;
LOGGER.debug(“[START] Thread Id: {} is started!”, Thread.currentThread().getId());

WebDriver driver = null;
final long start = System.currentTimeMillis();
try{
driver = (WebDriver)new FirefoxDriver();
driver.get(baseUrl);

final String actual = driver.getTitle();
LOGGER.debug(“Page Title is {}”, actual);
final String expected = “GIANTGEEK.COM”;
Assert.assertEquals(actual,expected);
//perform whatever actions, like login, submit form or navigation


}catch(final WebDriverException ex){
LOGGER.warn(fn+":WebDriverException:{}",ex);
}catch(final Exception ex){
LOGGER.warn(fn+":Exception:{}",ex);
}
finally {
final long elapsed = System.currentTimeMillis() - start;
LOGGER.debug("[END] Thread Id: {}, elapsed={}", Thread.currentThread().getId(),elapsed);
if(driver != null){
driver.quit();
}
}
}
}

WARNING: Selenium Tests MAY fail if the browser used for testing is updated in the Operating System. Updating the pom.xml to a newer release usually helps!

REFERENCES:

Read XML Properties in Java

Once in a while you need to externalize some configuration without the overhead of a complete framework, here’s a simple method to read an XML formatted property file in java. In most cases, it’s a performance advantage to wrap this up in a Singleton pattern, but that’s a different topic altogether.


private getAttributes() {
final String filename = "example.properties";
final InputStream input = getClass().getClassLoader().getResourceAsStream(filename);
if(input==null){
System.err.println("Cannot find properties:"+ filename);
}
final java.util.Properties props = new java.util.Properties();
try {
props.loadFromXML(input);
hostprop = props.getProperty("hostname",null);
userprop = props.getProperty("username",null);
pswdprop = props.getProperty("password",null);
}
catch(final Exception e){
System.err.println("Error occurred while reading properties file:"+ input);
e.printStackTrace();
}
finally{
try {
input.close();
}
catch(final java.io.IOException ex){
ex.printStackTrace();
}
}
}

The matching file would resemble…

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<entry key="hostname">localhost</entry>
<entry key="username">example</entry>
<entry key="password">example</entry>
</properties>

Maven build script for replacement of text in web.xml (and others)

Automated replacement of BUILD_LABEL token in web.xml <description> with Maven. For JAR’s the replacement is commented out, but can be any file.

NOTE: This proves to be rather difficult to do because of the way that Maven copies resources as it’s building the WAR. The most reliable manner I’ve found (so far) is below, it works by making a .tmp copy of the web.xml in a different path and then later uses it in the WAR.


<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.3</version>
<configuration>
<quiet>false</quiet>
</configuration>
<executions>
<execution>
<id>replaceBuildLabel</id>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
<configuration>
<file>${basedir}/src/main/webapp/WEB-INF/web.xml</file>
<outputFile>${project.build.directory}/web.xml.tmp</outputFile>
<replacements>
<replacement>
<token>BUILD_LABEL</token>
<value>Maven-${maven.build.timestamp}</value>
</replacement>
</replacements>
<regex>false</regex>
<quiet>false</quiet>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.5</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
<webXml>${project.build.directory}/web.xml.tmp</webXml>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
<manifestEntries>
<url>${project.url}</url>
<Build-Label>${maven.build.timestamp}</Build-Label>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>

Most importantly, you will want to have this token in the web.xml file for replacement, the description line is best used for this as such:

<description>ExampleWAR [BUILD_LABEL]</description>

during the build, that value would be replaced to something like:

<description>ExampleWAR [Maven-20141015-1700]</description>

REFERENCES:

Ant build script for replacement of text in web.xml (and others)

Automated replacement of BUILD_LABEL in web.xml <description> with Ant. For JAR’s the replacement is commented out, but can be any file


<replace file="${webapp.dir}/WEB-INF/web.xml" token="BUILD_LABEL" value="Ant-${DSTAMP}-${TSTAMP}" />
<war destfile="${jar.dir}/${ant.project.name}.war" webxml="${webapp.dir}/WEB-INF/web.xml" compress="true">

Most importantly, you will want to have this token in the web.xml file for replacement, the description line is best used for this as such:

<description>ExampleWAR [BUILD_LABEL]</description>

during the build, that value would be replaced to something like:

<description>ExampleWAR [Ant-20141015-1700]</description>

REFERENCES:

jboss-web.xml

If you support code for multiple java application servers, you might eventually encounter a file named:


/webapp/WEB-INF/jboss-web.xml

JBoss uses this file to control the path of the web application, whereas Tomcat generally uses the filename of the WAR itself.

Usually, the contents are pretty sparse, you might consider adding one to your projects should you ever wish to deploy them on JBoss:


<jboss-web>
<context-root>example</context-root>
</jboss-web>

NOTE: There are several other attributes that can find their way into this file for JBoss, notably security configuration, like JAAS.

WARNING: Unfortunately, I’ve tried to add a simple DOCTYPE jboss-web and XML preamble to this, file to make it validate, but the server (JBoss 5.1.x) fails to recognize them.

Eclipse FileSync plugin

I’ve done a lot of front-end java coding over my career,  one particularly annoying aspect is the wait for a build (compile-deploy) cycle in my local developement servers to view or test a small change.  One particularly useful tool that I’ve been using for some time is a FileSync plugin for Eclipse.  It is useful as you can “map” folders from your Eclipse project to a path on your local filesystem, as such the individual files are automatically copied to your server installation.  I’ve personally used this approache with JBoss, Tomcat and WebSphere, but there is no reason that it should not work for other servers.

Java temporary file directory path

I’ve recently resurrected some old java code that I’d written back when I primarily used Windows instead of Ubuntu for development. In some of that legacy code, the temporary file paths were hardcoded, to make things more modern and portable, The following line is recommended to get the Operating System values regardless of where it is installed and ran. The file separator “slash” can also be determined in this manner.


private static final String TMPDIR = System.getProperty("java.io.tmpdir") + java.io.File.separatorChar;