Java User-Agent detector and caching

It’s often important for a server side application to understand the client platform. There are two common methods used for this.

  1. On the client itself, “capabilities” can be tested.
  2. Unfortunately, the server cannot easily test these, and as such must usually rely upon the HTTP Header information, notably “User-Agent”.

Example User-agent might typically look like this for a common desktop browser, developers can usually determine the platform without a lot of work.

"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)"

Determining robots and mobile platforms, unfortunately is a lot more difficult due to the variations. Libraries as those described below simplify this work as standard Java Objects expose the attributes that are commonly expected.

With Maven, the dependencies are all resolved with the following POM addition:

<dependency>
<groupid>net.sf.uadetector</groupid>
<artifactid>uadetector-resources</artifactid>
<version>2014.10</version>
</dependency>


/* Get an UserAgentStringParser and analyze the requesting client */
final UserAgentStringParser parser = UADetectorServiceFactory.getResourceModuleParser();
final ReadableUserAgent agent = parser.parse(request.getHeader("User-Agent"));

out.append("You're a '");
out.append(agent.getName());
out.append("' on '");
out.append(agent.getOperatingSystem().getName());
out.append("'.");

As indicated on the website documentation, running this query for each request uses valuable server resources, it’s best to cache the responses to minimize the impact!

http://uadetector.sourceforge.net/usage.html#improve_performance

NOTE: the website caching example is hard to copy-paste, here’s a cleaner copy.


/*
* COPYRIGHT.
*/
package com.example.cache;

import java.util.concurrent.TimeUnit;
import net.sf.uadetector.ReadableUserAgent;
import net.sf.uadetector.UserAgentStringParser;
import net.sf.uadetector.service.UADetectorServiceFactory;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;

/**
* Caching User Agent parser
* @see http://uadetector.sourceforge.net/usage.html#improve_performance
* @author Scott Fredrickson [skotfred]
* @since 2015jan28
* @version 2015jan28
*/
public final class CachedUserAgentStringParser implements UserAgentStringParser {

private final UserAgentStringParser parser = UADetectorServiceFactory.getCachingAndUpdatingParser();
private static final int CACHE_MAX_SIZE = 100;
private static final int CACHE_MAX_HOURS = 2;
/**
* Limited to 100 elements for 2 hours!
*/
private final Cache<String , ReadableUserAgent> cache = CacheBuilder.newBuilder().maximumSize(CACHE_MAX_SIZE).expireAfterWrite(CACHE_MAX_HOURS, TimeUnit.HOURS).build();

/**
* @return {@code String}
*/
@Override
public String getDataVersion() {
return parser.getDataVersion();
}
/**
* @param userAgentString {@code String}
* @return {@link ReadableUserAgent}
*/
@Override
public ReadableUserAgent parse(final String userAgentString) {
ReadableUserAgent result = cache.getIfPresent(userAgentString);
if (result == null) {
result = parser.parse(userAgentString);
cache.put(userAgentString, result);
}
return result;
}
/**
*
*/
@Override
public void shutdown() {
parser.shutdown();
}
}

REFERENCES:

Java temporary file directory path

I’ve recently resurrected some old java code that I’d written back when I primarily used Windows instead of Ubuntu for development. In some of that legacy code, the temporary file paths were hardcoded, to make things more modern and portable, The following line is recommended to get the Operating System values regardless of where it is installed and ran. The file separator “slash” can also be determined in this manner.


private static final String TMPDIR = System.getProperty("java.io.tmpdir") + java.io.File.separatorChar;

Install Splunk Universal Forwarder on Ubuntu

After a while it can get tedious to access and review server logs via the command line. There are several tools available that can provide the same information in a graphical manner. Recently I’ve migrated to Splunk as there are both Enterprise and Free versions available.

  1. Of course, you’ll need a Splunk server installed first, as the forwarder is really just another (lighter) instance that will forward the log information to a central location.
  2. Download the system appropriate installer from:
    http://www.splunk.com/download/universalforwarder
  3. Check to see if you are running 32 or 64 bit OS.uname -aIf you see i686 you are 32 bit, if x86_64 you are 64 bit!
  4. Download, you’ll likely need a different version:sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-intel.deb
    or
    sudo dpkg -i splunkforwarder-6.1.3-220630-linux-2.6-amd64.deb
  5. Enable auto-start on reboot:cd /opt/splunkforwarder/bin/

sudo ./splunk enable boot-start

    1. Start the server:sudo service splunk start
    2. Set the password:

      The default ‘admin‘ password is ‘changeme‘ so we need to change it immediately to do anything else, or we will see errors in future steps.

      sudo /opt/splunkforwarder/bin/splunk edit user admin -password YOUR_NEW_PASSWORD -auth admin:changeme

    3. Set the server:sudo /opt/splunkforwarder/bin/splunk add forward-server YOUR_SERVER_ADDRESS:9997

      NOTE: if you get prompted for a splunk username/password you likely skipped the above step. Remember – the forwarder is a new ‘light’ installation of the server and as such has it’s own users!

    4. Enable some monitors on the box:Some common services and log locations to get you started…
      Apache2 HTTPd
      sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/apache2 -index main -sourcetype Apache2
      Tomcat7
      sudo /opt/splunkforwarder/bin/splunk add monitor /opt/tomcat7/logs -index main -sourcetype Tomcat7
      MySQL
      sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mysql -index main -sourcetype MySQL
      Postfix (SMTP)
      sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/mail.log -index main -sourcetype Postfix
      Squid3 (Proxy)
      sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/access.log -index main -sourcetype Squid3
      sudo /opt/splunkforwarder/bin/splunk add monitor /var/log/squid/cache.log -index main -sourcetype Squid3

      SonarQube
      sudo /opt/splunkforwarder/bin/splunk add monitor /opt/sonar/logs -index main -sourcetype Sonar
      PM2
      sudo /opt/splunkforwarder/bin/splunk add monitor /home/{user}/.pm2/logs -index main -sourcetype PM2
      NPM
      sudo /opt/splunkforwarder/bin/splunk add monitor /home/scott/.npm/_logs -index main -sourcetype NPM
  1. (OPTIONAL) Verify configuration by opening file at the following:sudo su
    vi /opt/splunkforwarder/etc/apps/search/local/inputs.conf
    exit
  2. You now should be able to log into your server and see new data flowing from the forwarder.

    NOTE: this requires you to enable ‘receiving’ of data on the port specified above, usually 9997.

REFERENCES:

Viewing Hidden Files and Folders on Apple OS/X

Most operating systems make this rather trivial to expose, Apple seems to have made it just a tiny bit more difficult… as such, I provide the simple steps here for my own memory as well as your benefit.

Open Terminal…

  • Launch Terminal, located at /Applications/Utilities/.
  • Type or copy/paste the following commands. Press the return key after you enter each line.
    defaults write com.apple.finder AppleShowAllFiles TRUE
    killall Finder

WARNING: Be particularly careful about the files you modify or delete, you could impact your system in very critical ways… there is a reason they are ‘hidden’, most often it is to keep less-technical users from breaking things 🙂

Return hidden files to their usual state.

Open Terminal…

  • Launch Terminal, located at /Applications/Utilities/.
  • Type or copy/paste the following commands. Press the return key after you enter each line.
    defaults write com.apple.finder AppleShowAllFiles FALSE
    killall Finder

Cheers

CSS2 Colors

CSS Level 2 defines several additional color names that represent the special system-specific colors used by the operating system. These names should look look very familiar to prior developers of “Fat Client” software, primarily VisualBASIC and PowerBuilder.

NOTE: These work with MSIE 5.0+ and Mozilla/Netscape 5.0+, prior browsers “try” to interpret these colors as hexadecimal RGB (Red/Green/Blue) equivalents, resulting in a huge mess.

The colors shown below will be mapped from your current operating system settings and as such MAY vary from computer to computer!

CSS 2 Color Name Example (Using Background-color) Description
ActiveBorder Active window border.
ActiveCaption Active window caption.
AppWorkspace Background color of multiple document interface.
Background Desktop background.
ButtonFace Face color for three-dimensional display elements.
ButtonHighlight Dark shadow for three-dimensional display elements (for edges facing away from the light source).
ButtonShadow Shadow color for three-dimensional display elements.
ButtonText Text on push buttons.
CaptionText Text in caption, size box, and scrollbar arrow box.
GrayText Grayed (disabled) text. This color is set to #000 if the current display driver does not support a solid gray color.
Highlight Item(s) selected in a control.
HighlightText Text of item(s) selected in a control.
InactiveBorder Inactive window border.
InactiveCaption Inactive window caption.
InactiveCaptionText Color of text in an inactive caption.
InfoBackground Background color for tooltip controls.
InfoText Text color for tooltip controls.
Menu Menu background.
MenuText Text in menus.
Scrollbar Scroll bar gray area.
ThreeDDarkShadow Dark shadow for three-dimensional display elements.
ThreeDFace Face color for three-dimensional display elements.
ThreeDHighlight Highlight color for three-dimensional display elements.
ThreeDLightShadow Light color for three-dimensional display elements (for edges facing the light source).
ThreeDShadow Dark shadow for three-dimensional display elements.
Window Window background.
WindowFrame Window frame.
WindowText Text in windows.

Cheers!