Java User-Agent detector and caching

It’s often important for a server side application to understand the client platform. There are two common methods used for this.

  1. On the client itself, “capabilities” can be tested.
  2. Unfortunately, the server cannot easily test these, and as such must usually rely upon the HTTP Header information, notably “User-Agent”.

Example User-agent might typically look like this for a common desktop browser, developers can usually determine the platform without a lot of work.

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)"

Determining robots and mobile platforms, unfortunately is a lot more difficult due to the variations. Libraries as those described below simplify this work as standard Java Objects expose the attributes that are commonly expected.

With Maven, the dependencies are all resolved with the following POM addition:

<dependency>
<groupid>net.sf.uadetector</groupid>
<artifactid>uadetector-resources</artifactid>
<version>2014.10</version>
</dependency>


/* Get an UserAgentStringParser and analyze the requesting client */
final UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();
final ReadableUserAgent agent = parser.parse(request.getHeader("User-Agent"));

out.append("You're a '");
out.append(agent.getName());
out.append("' on '");
out.append(agent.getOperatingSystem().getName());
out.append("'.");

As indicated on the website documentation, running this query for each request uses valuable server resources, it’s best to cache the responses to minimize the impact!

http://uadetector.sourceforge.net/usage.html#improve_performance

NOTE: the website caching example is hard to copy-paste, here’s a cleaner copy.


/*
* COPYRIGHT.
*/
package com.example.cache;

import java.util.concurrent.TimeUnit;
import net.sf.uadetector.ReadableUserAgent;
import net.sf.uadetector.UserAgentStringParser;
import net.sf.uadetector.service.UADetectorServiceFactory;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;

/**
* Caching User Agent parser
* @see http://uadetector.sourceforge.net/usage.html#improve_performance
* @author Scott Fredrickson [skotfred]
* @since 2015jan28
* @version 2015jan28
*/
public final class CachedUserAgentStringParser implements UserAgentStringParser {

private final UserAgentStringParser parser = UADetectorServiceFactory.getCachingAndUpdatingParser();
private static final int CACHE_MAX_SIZE = 100;
private static final int CACHE_MAX_HOURS = 2;
/**
* Limited to 100 elements for 2 hours!
*/
private final Cache<String , ReadableUserAgent> cache = CacheBuilder.newBuilder().maximumSize(CACHE_MAX_SIZE).expireAfterWrite(CACHE_MAX_HOURS, TimeUnit.HOURS).build();

/**
* @return {@code String}
*/
@Override
public String getDataVersion() {
return parser.getDataVersion();
}
/**
* @param userAgentString {@code String}
* @return {@link ReadableUserAgent}
*/
@Override
public ReadableUserAgent parse(final String userAgentString) {
ReadableUserAgent result = cache.getIfPresent(userAgentString);
if (result == null) {
result = parser.parse(userAgentString);
cache.put(userAgentString, result);
}
return result;
}
/**
*
*/
@Override
public void shutdown() {
parser.shutdown();
}
}

REFERENCES:

Maven build script for replacement of text in web.xml (and others)

Automated replacement of BUILD_LABEL token in web.xml <description> with Maven. For JAR’s the replacement is commented out, but can be any file.

NOTE: This proves to be rather difficult to do because of the way that Maven copies resources as it’s building the WAR. The most reliable manner I’ve found (so far) is below, it works by making a .tmp copy of the web.xml in a different path and then later uses it in the WAR.


<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.3</version>
<configuration>
<quiet>false</quiet>
</configuration>
<executions>
<execution>
<id>replaceBuildLabel</id>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
<configuration>
<file>${basedir}/src/main/webapp/WEB-INF/web.xml</file>
<outputFile>${project.build.directory}/web.xml.tmp</outputFile>
<replacements>
<replacement>
<token>BUILD_LABEL</token>
<value>Maven-${maven.build.timestamp}</value>
</replacement>
</replacements>
<regex>false</regex>
<quiet>false</quiet>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.5</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
<webXml>${project.build.directory}/web.xml.tmp</webXml>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
<manifestEntries>
<url>${project.url}</url>
<Build-Label>${maven.build.timestamp}</Build-Label>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>

Most importantly, you will want to have this token in the web.xml file for replacement, the description line is best used for this as such:

<description>ExampleWAR [BUILD_LABEL]</description>

during the build, that value would be replaced to something like:

<description>ExampleWAR [Maven-20141015-1700]</description>

REFERENCES:

Install Subversion Server on Ubuntu

Subversion is a commonly used central version control system for software development. There are currently still a large number of organizations that rely upon it, many have since moved on to Git.

  1. sudo apt-get install apache2 apache2-utils
  2. sudo apt-get install subversion subversion-tools libapache2-svn
  3. sudo mkdir /home/svn
  4. svnadmin create /home/svn/test
  5. Create a group for subversion users:
    sudo groupadd subversion
  6. sudo adduser USERNAME
  7. Add a user to the group:
    sudo useradd -G USERNAME subversion
  8. sudo chown -R www-data:subversion /home/svn/test
  9. sudo chmod -R g+rws /home/svn/test
  10. sudo a2enmod dav_svn
  11. To create/clobber a new file for the first user:
    sudo htpasswd -c /etc/apache2/.htpasswd YOURUSER
  12. To add additional users:
    sudo htpasswd /etc/apache2/.htpasswd YOURUSER
    (repeat for new users without the -c as that creates/clobbers the file)
  13. sudo vi /etc/apache2/sites-available/000-default.conf
    Then add to the bottom:
    (NOTE1: the LimitExcept can be enabled to allow anonymous access):
    (NOTE2: the LimitXMLRequestBody can be uncomment to allow large commits)

    <Location /svn>
    DAV svn
    SVNParentPath /home/svn
    AuthType Basic
    AuthName "Subversion Repository"
    # AuthUserFile /etc/svn-auth
    AuthUserFile /etc/apache2/.htpasswd
    #LimitXMLRequestBody 0
    #<LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
    #</LimitExcept>
    </Location>
  14. sudo service apache2 reload
  15. sudo service apache2 restart

    NOTE: At this point you should be able to browse and do a remote checkout of the code from another machine….

    http://YOUR-IP-OR-HOSTNAME/svn
    and
    svn co http://YOUR-IP-OR-HOSTNAME/svn/test --username YOURUSER --password YOURPASS

  16. sudo vi /etc/init/svnserve.conf
    Add the following:

    # svnserve - Subversion server
    description "Subversion server"
    start on (local-filesystems and net-device-up IFACE=lo and started udev-finish)
    stop on runlevel [06]
    chdir /home/svn
    respawn
    respawn limit 2 3600
    exec /usr/bin/svnserve --foreground --daemon --config-file /home/svn/repos/conf/svnserve.conf --root /home/svn/repos/
  17. Then execute:
    sudo initctl start svnserve
  18. Back on the client side…
    Create a new folder inside your user folder:
    cd ~/test
  19. Check out the project into this folder:
    svn checkout http://YOUR-IP-OR-HOSTNAME/svn/test
  20. Let us just add a new HTML index file to the folder:
    vi index.html
  21. Add it to version control:
    svn add index.html
    Commit the new file:
    svn commit -m "commit message"
    Update:
    svn up
  22. That should cover most cases for you…

REFERENCES:

MKS IntegrityClient integration with Eclipse IDE

While the intgration plugins for most SCM products are done rather simply within Eclipse, MKS(now PTC) IntegrityClient requires a few manual steps.

Caution, the plugin has always been a bit “buggy” but nothing too annoying for daily use. I’ve personally used it since IBM WSAD 5.1(Eclipse 3.x based) up to and including the most current Eclipse Luna (4.4) release.

  1. Go to your MKS installation path, then find the “integrations” folder as in the example below:
    C:\MKS\IntegrityClient\integrations\IBM\eclipse_3.2\eclipse
  2. That folder SHOULD have two folders (features/plugins) as well as three files (.eclipseextension, artifacts.xml, content.xml)
  3. Copy the entire contents and paste/overlay into your ‘eclipse’ folder (where you should already have folders for ‘features/plugins)
  4. Restart Eclipse
  5. Configure as required.

Security through obscurity – hiding your server version information

I’ve recently spent a lot of time reviewing the OWASP documentation, and (like many corporations) realized that I’d neglected to keep up with this configuration item.

By sharing the exact version of each piece of server software you are using, “hackers” are able to quickly identify unpatched systems and their known vulnerabilities.

To make their work harder, there are a few simple steps that the server admin can take to remove this information from the HTTP Headers and error pages.

Apache HTTPd:

  1. sudo vi /etc/apache2/conf-enabled/security.conf
  2. Add:

    ServerTokens ProductOnly
    ServerSignature Off
  3. If using virtual hosts, add the following to each one:
    ServerSignature Off
  4. sudo service apache2 restart

Apache Tomcat:

  1. vi /opt/tomcat7/conf/server.xml
  2. Find the <Connector > entry and add:
    server="Apache"
  3. cd /opt/tomcat7/lib
  4. mkdir -p org/apache/catalina/util
  5. vi /opt/tomcat7/lib/org/apache/catalina/util/ServerInfo.properties
    server.info=Apache Tomcat
  6. sudo service tomcat7 restart

PHP “X-Powered-By: PHP/5.x.x-1ubuntuX.X”

  1. sudo vi /etc/php5/apache2/php.ini
    expose_php = Off
  2. sudo service apache2 restart

REFERENCES:

Prevent Firefox browser auto update

Often you find a neeed to keep an old copy of Firefox around for testing or to use a specific plugin (Example: Selenium). In these cases it can often prove problematic to allow the browser to auto-update. Here are some simple steps to prevent this behavior.

Enter “about:config” into the Firefox URL bar, then change the following values. You can click on them to toggle.

app.update.auto = false
app.update.enabled = false

Alternately, on Windows you can edit the config file at: %APPDATA%\Mozilla\Firefox\Profiles\.default\prefs.js

REFERENCES:

JavaScript language attribute

Occasionally I’ve stumbled upon legacy javascript code that is used to determine javascript support by the visiting users. This often proves comical, because they are many times wasting time making checks for some “VERY OLD” browsers indeed! Here’s a rundown of the versions of javascript as well as their release dates and some common browser versions that implemented them.

  • JavaScript 1.0 (March 1996) = Navigator 2.0 / MSIE 3.0
  • JavaScript 1.1 (August 1996) = Navigator 3.0
  • JavaScript 1.2 (June 1997) = Navigator 4.0-4.05
  • JavaScript 1.3 (October 1998) = Navigator 4.06-4.7x / MSIE 4.0
  • JavaScript 1.4 = Netscape Server
  • JavaScript 1.5 (November 2000) = Navigator 6.0 / Firefox 1.0 / MSIE 5.5 – 8.0 / Safari 3.0-5 / Chrome 1.0-10.x / Opera 6.0
  • JavaScript 1.6 (November 2005) = Firefox 1.5
  • JavaScript 1.7 (October 2006) = Firefox 2.0
  • JavaScript 1.8 (June 2008) = Firefox 3.0 / Opera 11.50
  • JavaScript 1.8.1 = Firefox 3.5
  • JavaScript 1.8.2 (June 2009) = Firefox 3.6
  • JavaScript 1.8.5 (July 2010) = Firefox 4.0 / MSIE 9.0 / Opera 11.60

The language attribute has long been deprecated and should generally be avoided, it’s original purpose was to support other scripting languages, notably VBScript, or particular JavaScript versions. Modern conventions rely on specifying the MIME type instead via the ‘type’ attribute.

<SCRIPT LANGUAGE="JavaScript"> is now <script type="text/javascript">

<SCRIPT LANGUAGE="JavaScript1.1"> is now <script type="text/javascript1.1">

<SCRIPT LANGUAGE="VBScript"> is now <script type="text/vbscript">

<SCRIPT LANGUAGE="TCL"> is now <script type="text/tcl">

REFERENCES:

Old versions of software

There eventually comes a time when you’re not satisfied with the “latest” version of some software, often this is due to increased resources required to run on older machines. Sometimes it can be due to ‘free’ software becoming nag-ware or no longer free.

Here are a few websites that I have found useful when looking for such older software.

Obvious Warning – older software may have security concerns and known bugs… as such, be careful when using them!

Cheers!

Version Control comments

When working on large, multi-group projects I’ve found that it often helps to have information about the ‘version’ of an asset written into the source file (HTML, JSP, PHP, CSS, JS, or other ‘text’ formats).

This is easily accomplished with most version control packages and is done automatically if the following are added inside of an appropriate comment section in the file. The below examples are Java based, but can be easily adopted to any file type.

This is especially helpful when reviewing JavaDoc, and crucial for deployed text files such as CSS and JS as it makes debugging them much easier.

These work with CVS, Subversion, Serena Changeman DS, MKS Source Integrity and a variety of other products.

Raw Source:

/*
$Id: $
$Author: $
$Revision: $
$Date: $

<pre>
$Log: $
</pre>
*/

At check-in becomes:

/*
$Id: project.readme 1.1 2007/07/30 10:42:39CDT Scott dev $
$Author: Scott $
$Revision: 1.1 $
$Date: 2007/07/30 10:42:39CDT $

<pre>
$Log: project.readme $
Revision 1.1 2007/07/30 10:42:39CDT Scott
Scott. test
</pre>
*/

Cheers!