Sonatype Nexus2 Repository Manager OSS

To allow for repeatable, faster builds in a continuous build environment, it’s often a good idea to use a central repository to cache common assets and prevent the need to download assets from the internet for each build. Using Nexus allows for those transfers to occur over your local network for previously downloaded assets.

You can download the WAR from:
http://www.sonatype.org/downloads/nexus-latest.war

And install on your Java application server, such as Apache Tomcat, via normal means.

If you are using Maven, you’ll need to make appropriate changes in (/.m2/settings.xml) to direct your builds to use Nexus.

Jenkins and other build automation tools will require similar changes.

REFERENCES:

Selenium HtmlUnit driver separated in 2.53.0

I’ve been a user of Selenium testing for several years, though I noticed that some classes related to the HtmlUnit WebDriver were missing after upgrading from 2.52.0 to 2.53.0. After some research, I discovered that it is now a separate dependency allowing for a separate release cycle. Additionally, if you don’t use this (relatively generic) webdriver, you will no longer need to have it in your binaries.

Here’s all you need to do to add it to your Maven projects for testing.

In your pom.xml file:

<properties>
<selenium.version>2.53.0</selenium.version>
<htmlunitdriver.version>2.20</htmlunitdriver.version>
</properties>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>${selenium.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>htmlunit-driver</artifactId>
<version>${htmlunitdriver.version}</version>
<scope>test</scope>
</dependency>
</dependencies>

REFERENCES:

Java Dependency Vulnerability scanning with Maven victims-enforcer

One of the OWASP guidelines for secure applications is to not use components with known vulnerabilities. Unfortunately it can be a very difficult and time consuming task to keep up with these manually, automation can save you countless hours!

See https://www.owasp.org/index.php/Top_10_2013-A9-Using_Components_with_Known_Vulnerabilities.

NOTE: victims-enforcer can be used in conjunction with the OWASP dependency scanner. I have only found it to be problematic in ‘tycho’ builds.


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4.1</version>
<dependencies>
<dependency>
<groupId>com.redhat.victims</groupId>
<artifactId>enforce-victims-rule</artifactId>
<version>1.3.4</version>
<type>jar</type>
</dependency>
</dependencies>
<executions>
<execution>
<id>enforce-victims-rule</id>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<rule implementation="com.redhat.victims.VictimsRule">
<!--
Check the project's dependencies against the database using
name and version. The default mode for this is 'warning'.

Valid options are:

disabled: Rule is still run but only INFO level messages and no errors.
warning : Rule will spit out a warning message but doesn't result in a failure.
fatal : Rule will spit out an error message and fail the build.
-->
<metadata>warning</metadata>

<!--
Check the project's dependencies against the database using
the SHA-512 checksum of the artifact. The default is fatal.

Valid options are:

disabled: Rule is still run but only INFO level messages and no errors.
warning : Rule will spit out a warning message but doesn't result in a failure.
fatal : Rule will spit out an error message and fail the build.
-->
<fingerprint>fatal</fingerprint>

<!--
Disables the synchronization mechanism. By default the rule will
attempt to update the database for each build.

Valid options are:

auto : Automatically update the database entries on each build.
daily : Update the database entries once per day.
weekly: Update the database entries once per week.
offline : Disable the synchronization mechanism.
-->
<updates>daily</updates><!-- was: auto -->

</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>

Vulnerability database is sourced from: https://victi.ms with backing from RedHat.

REFERENCES:

OWASP Dependency Vulnerability Scanning of Java JARs with Maven

One of the OWASP guidelines for secure applications is to not use components with known vulnerabilities. Unfortunately it can be a very difficult and time consuming task to keep up with these manually, automation can save you countless hours!

See https://www.owasp.org/index.php/Top_10_2013-A9-Using_Components_with_Known_Vulnerabilities.

NOTE: OWASP dependency scanner can be used in conjunction with the victims-enforcer.

Add to your projects pom.xml:

<plugin>
<groupId>org.owasp</groupId>
<artifactId>dependency-check-maven</artifactId>
<version>1.3.4</version>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>

Each time you build, the plug-in will verify the assets against the list of known vulnerable libraries and report them in your output.

Vulnerability database is populated from: https://nvd.nist.gov.

NOTES:

  1. The example above is a very simple implementation, see the documentation for additional functions.
  2. The first use of the plug-in can take a long time as the vulnerability library must be installed locally before initial use.
  3. Similar functionality is available for Ant builds, if desired.

REFERENCES:

RetireJS javascript libary vulnerability scanning with Maven

It’s important to note that even though your site is using a vulnerable library, that does not necessarily mean your site is vulnerable. It depends on whether and how your site exercises
the vulnerable code. That said, it’s better to be safe than sorry.

I identified this method of using the asset after reading the instructions for the Burp/Gulp scanner from h3xstream after the following section caught my eye:
https://github.com/h3xstream/burp-retire-js#maven-plugin-, it contained a small reference to Maven and even showed output but no configuration for use. A couple of attempts later I came up with the following:

Add to pom.xml:

<build>
<plugins>
<plugin>
<groupId>com.h3xstream.retirejs</groupId>
<artifactId>retirejs-maven-plugin</artifactId>
<version>2.1.0</version>
<executions>
<execution>
<id>scanProjectJavascript</id>
<phase>install</phase>
<goals>
<goal>scan</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

After adding this to your pom.xml, the console output for each build will contain information regarding each vulnerable JavaScript library.

One small problem exists in the current version, use behind corporate firewalls can often be blocked, resulting in an error in the console and use of an older version of the vulnerability library to be used in scans.

Error example:

[ERROR] Exception while loading the repository (Most likely unable to access the internet) java.net.UnknownHostException: raw.githubusercontent.com

See the following for updates:
https://github.com/h3xstream/burp-retire-js/issues/8

See https://www.owasp.org/index.php/Top_10_2013-A9-Using_Components_with_Known_Vulnerabilities.

REFERENCES:

Code signing of java applets – using Maven

To sign your java assets during the maven build process, you can add the following to the pom.xml to make use of the values we established in the keystore creation step.

WARNING: for security and maintainability purposes, you should define the ‘configuration’ items in your local ‘settings.xml’ file instead of in the pom.xml as is done here for example only!


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<id>sign</id>
<goals>
<goal>sign</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<alias>selfsigned</alias><!-- ${project.name} -->
<keystore>selfsignkeys.store</keystore><!-- NOTE: you can also specify an absolute path here -->
<storepass>123456</storepass>
<keypass>123456</keypass>
</configuration>
</plugin>

REFERENCES:

Using Ant to parse and download Maven pom.xml dependencies

I’ve migrated most of my projects to Maven, but occasionally have some developers that prefer to use Ant in their development environments. One problem that I used to have with Ant was that it required all dependencies to be checked into the SCM repository for each project. I recently found an Ant plugin that allows for it to read the Maven pom.xml and download the required dependencies, thus making projects MUCH easier to maintain! the steps are very simple.

Maven – pom.xml

  • Make sure that you have your dependencies (nexus?) setup and tested here.

Maven – global settings.xml

  • Make sure that your repositories are correctly configured.

Ant – build.xml (very minimal, I usually add as a step in existing scripts vs. using as standalone)

  • (example):

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <!DOCTYPE project>
    <project name="example" basedir="." default="dependencies" xmlns:artifact="antlib:org.apache.maven.artifact.ant">
    <taskdef uri="antlib:org.apache.maven.artifact.ant" classpath="ant/maven-ant-tasks-2.1.3.jar" />
    <target name="dependencies">
    <echo message="--- getting dependencies from maven pom.xml ---" />
    <artifact:pom id="pom" file="pom.xml" /><!-- settingsFile="settings.xml" -->
    <artifact:dependencies filesetId="test.dependencies" pomRefId="pom" useScope="test" />
    <copy todir="${antlib.dir}">
    <fileset refid="test.dependencies" />
    <mapper type="flatten" />
    </copy>
    </target>
    </project>
     
  • Make sure that you put the JAR (maven-ant-tasks-2.1.3.jar) in the proper place…

Executing:


  • ant dependencies

If everything is working well, you can now purge most of the JAR’s that reside inside your web projects as the Ant build process can retrieve them based on values in the Maven pom.xml file.

REFERENCES:
http://maven.apache.org/ant-tasks/examples/dependencies.html
http://maven.apache.org/ant-tasks/
http://search.maven.org/#artifactdetails%7Corg.apache.maven%7Cmaven-ant-tasks%7C2.1.3%7Cjar

Load Testing web application with Selenium and TestNG

I’ve used Selenium for while to do verification tests of web applications, recently I discovered a very simple way to use it with TestNG and Maven to do some performance testing. TestNG allows for the use of annotations to allow multi-threading and iterations.

pom.xml:

<dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.8.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.44.0</version>
<scope>test</scope>
</dependency>
<dependencies>

And as for a simple test to get started with… scripting of steps is available online or could be in a future blog post.

/*
* COPYRIGHT. none
*/
package com.example.selenium;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
/**
* Simple test example for Selenium
*/
public class SeleniumTest {

private static final Logger LOGGER = LoggerFactory.getLogger(SeleniumTest.class);
/**
* TODO Un-comment or change if needed to set your local path!
*/
@BeforeClass
public void oneTimeSetUp() {
System.out.println("-------------------------------------- init ----------------------------------------");
//System.setProperty("webdriver.firefox.bin","C:\\path\\to\\firefox.exe");
}
/**
* NOTE: uses TestNG - behaves differently than JUnit
*/
@Test(invocationCount = 1, threadPoolSize = 5)
public void testLoadApp() {

final String fn = "testLoadApp";
final String baseUrl = "http://www.giantgeek.com/index.php";
LOGGER.debug("[START] Thread Id: {} is started!", Thread.currentThread().getId());

WebDriver driver = null;
final long start = System.currentTimeMillis();
try{
driver = (WebDriver)new FirefoxDriver();
driver.get(baseUrl);

final String actual = driver.getTitle();
LOGGER.debug("Page Title is {}", actual);
final String expected = "GIANTGEEK.COM";
Assert.assertEquals(actual,expected);
//perform whatever actions, like login, submit form or navigation

}catch(final WebDriverException ex){
LOGGER.warn(fn+":WebDriverException:{}",ex);
}catch(final Exception ex){
LOGGER.warn(fn+":Exception:{}",ex);
}
finally {
final long elapsed = System.currentTimeMillis() - start;
LOGGER.debug("[END] Thread Id: {}, elapsed={}", Thread.currentThread().getId(),elapsed);
if(driver != null){
driver.quit();
}
}
}
}

WARNING: Selenium Tests MAY fail if the browser used for testing is updated in the Operating System. Updating the pom.xml to a newer release usually helps!

REFERENCES:

Maven build script for replacement of text in web.xml (and others)

Automated replacement of BUILD_LABEL token in web.xml <description> with Maven. For JAR’s the replacement is commented out, but can be any file.

NOTE: This proves to be rather difficult to do because of the way that Maven copies resources as it’s building the WAR. The most reliable manner I’ve found (so far) is below, it works by making a .tmp copy of the web.xml in a different path and then later uses it in the WAR.


<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.3</version>
<configuration>
<quiet>false</quiet>
</configuration>
<executions>
<execution>
<id>replaceBuildLabel</id>
<phase>prepare-package</phase>
<goals>
<goal>replace</goal>
</goals>
<configuration>
<file>${basedir}/src/main/webapp/WEB-INF/web.xml</file>
<outputFile>${project.build.directory}/web.xml.tmp</outputFile>
<replacements>
<replacement>
<token>BUILD_LABEL</token>
<value>Maven-${maven.build.timestamp}</value>
</replacement>
</replacements>
<regex>false</regex>
<quiet>false</quiet>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.5</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
<webXml>${project.build.directory}/web.xml.tmp</webXml>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
<manifestEntries>
<url>${project.url}</url>
<Build-Label>${maven.build.timestamp}</Build-Label>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>

Most importantly, you will want to have this token in the web.xml file for replacement, the description line is best used for this as such:

<description>ExampleWAR [BUILD_LABEL]</description>

during the build, that value would be replaced to something like:

<description>ExampleWAR [Maven-20141015-1700]</description>

REFERENCES:

Maven/Ant echoproperties at build time

Once you have started using automation tools for continuous builds, you often find edge cases where your builds have minor variations due to the environments on which the projects have been built. To isolate these, it is often useful to have the build tool output a snapshot of it’s properties at the time the project was built. Thankfully, Ant and Maven make this easy to implement, required additions to your config files are below for each tool.

Maven: (pom.xml)

....
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<phase>install</phase>
<configuration>
<target>
<echoproperties />
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>

NOTE: ‘target’ is preferred over ‘tasks’ in newer versions of the plugin, it was deprecated in 1.5.

Ant: (build.xml)

...
<echoproperties />
...

REFERENCES: