This document describes how to integrate and use the Surrogate framework in a project to achieve better unit testing.
The example unit tests can be found in the surrogate-example folder in the source distribution. Even if the application to test is very simple, it is realistic with respect to common problematic test patterns.
The diagram shows how the Surrogate framework typically integrates in a unit test environment:
| Component | Description |
|---|---|
| CustomerFinder | This main method of this class parses an external file to find the customer name. In addition, the method checks the time used to perform the call and throws an Exception if the call exceeded 2 second. |
| CustomerFinderTestCase |
This is a standard JUnit test class,
When testing the CustomerFinder, we mock the
corresponding java.io.FileReader to return arbitrary values
using a mock object. In addition, we use the
SurrogateManager to mock the calls to System.currentTimeMillis.
|
| MockFileReader |
Hand-written mock object for java.io.FileReader
|
| MockMethod |
Used to mock to mock the calls to System.currentTimeMillis.
|
| SurrogateManager | All mock objects are registered with the manager. The Manager keeps track of the mock objects and mock methods and uses this when asked for a mock object for a particular interface, class, method or constructor. |
| ASPECTS |
The ASPECT classes are woven into the classes under compilation.
They define pointcuts and advices executing
where we want to substitute "real" objects or methods with their mock
counterpart. The Aspects use the SurrogateManager to check
if a mock has been registered for the corresponding pointcut.
|
A listing of the CustomerFinder class follows:
package net.sf.surrogate.example;
import java.io.BufferedReader;
import java.io.FileReader;
/**
* Demo of "difficult" class to test. Uses System.currentTimeMillis and
* instantiates a FileReader with hard-coded name to a remote disk...
*
*/
public class CustomerFinder {
private static final CustomerFinder theInstance = new CustomerFinder();
public static final CustomerFinder getInstance() {
return theInstance;
}
/**
* Gets a Customer identified by the <code>customer id</code>. by
* searching the "D:/production/customer_name.log" for the key
* <pre>
* id = name
* </pre>
* @param customerId
* the customer id.
* @return the Customer name, <code>null</code> if not found
* @throws Exception
* If error when parsing the file format
* @throws Exception
* if the read operation took more than 2 seconds.
*/
public String getCustomerById(String customerId) throws Exception {
long start = System.currentTimeMillis();
String customerName = null;
BufferedReader in = new BufferedReader(new FileReader("D:/production/customer_name.log"));
String line;
while ((line = in.readLine()) != null) {
if (line.startsWith(customerId)) {
int separator = line.indexOf("=");
if (separator < 0) {
throw new Exception("Invalid format:" + line);
}
customerName = line.substring(separator + 1);
break;
}
}
if (customerName == null) {
throw new Exception("Customer " + customerId + " no found");
}
in.close();
long end = System.currentTimeMillis();
if (end - start > 2000) {
throw new Exception("Call took more than 2 seconds");
}
return customerName;
}
}
Obviously, the main problem in testing the CustomerFinder class lies in that it explicitly calls
the static System.currentTimeMillis and that it tries to read from a file on a remote disk using
the java.io.FileReader.
If we test the class "as-is" it is very difficult to behave that the method behaves as expected since
it hard to simulate that a call takes more than 2 seconds. Also the hardcoded path to
the log file might not exist on our local machine and a real call to FileReader would probably
fail.
We hence need to be able to mock the System.currentTimeMillis calls and to replace the
new FileReader() call with mock implementations.
Every project using the Surrogate framework must create at least one Aspect extending
the SurrogateCalls aspect.
Normally, the aspect is put in a separate aspects folder under the
src or test directory to separate it from production files and standard java test files.
The aspect must only implement one "pointcut", defining all the points in the code
where it is possible to inject mocks. An example is:
// Source file aspects/ExampleProjectCalls.aj
aspect ExampleProjectCalls extends SurrogateCalls {
/**
* Implementation of the mock pointcut
*/
protected pointcut mockPointcut() :
(
call (java.io.*Reader.new(..)) ||
call(* java.lang.System.currentTimeMillis())
);
}
The mockPointcut in the example picks out every joinpoint that is a call to System.currentTimeMillis
or any instantiation of any "Reader" in the java.io package. E.g, the following calls is possible to mock:
import java.io.*;
...
long now = System.currentTimeMillis();
FileReader fileReader = new FileReader("c:/tmp");
StringReader stringReader = new StringReader("1=test");
..
The main idea behind Surrogate is that the unit tests have completely control of which mock objects
or mock methods to activate/deactivate. Also, you are free to use any mock object implementation.
In the example, we use "hand-crafted" mock implementation of the FileReader.
However, there exists several libraries for generating mock objects for a given interface or class. See for example the
EasyMock home page.
The unit tests register mock objects or MockMethod with the SurrogateManager as shown in the code below:
public void testGetCustomerById() throws Exception {
SurrogateManager mm = SurrogateManager.getInstance();
mm.reset();
MockFileReader mockFileReader = new MockFileReader();
// Use Surrogate to mock System.currentTimeMillis
MockMethod mockTime = mm.addMockMethod(new MockMethod(System.class,"currentTimeMillis"));
// Use Surrogate to mock java.io.FileReader
mm.addMock(mockFileReader);
CustomerFinder customerFinder = CustomerFinder.getInstance();
// Test normal operation
mockFileReader.setReturnValue("1=Customer 1");
mockTime.addReturnValue(new Long(2000));
mockTime.addReturnValue(new Long(4000));
mockTime.setExpectedCalls(2);
String returnValue = customerFinder.getCustomerById("1");
assertEquals("Customer 1",returnValue);
mockTime.verify();
// Test invalid file format
mockFileReader.setReturnValue("2");
mockTime.reset();
mockTime.addReturnValue(new Long(4000));
try {
customerFinder.getCustomerById("2");
fail("Should get an exception");
} catch (Exception e) {
assertEquals("Invalid format:2",e.getMessage());
}
// Test that an exception is thrown if called took more than 2 seconds
mockFileReader.setReturnValue("3=Customer 3");
mockTime.reset();
mockTime.addReturnValue(new Long(2000));
mockTime.addReturnValue(new Long(4001));
mockTime.setExpectedCalls(2);
try {
customerFinder.getCustomerById("3");
fail("Should get an exception because of timeout");
} catch (Exception e) {
assertEquals("Call took more than 2 seconds",e.getMessage());
mockTime.verify();
}
}
The project must have been setup with Surrogate weaving enabled as described in the Getting started Guide
This section describes how to specify the aspect pointcuts defining where in the
code it is possible to insert a mock object or mock method using the SurrogateManager.
Coding the AspectJ aspects for the test cases are generally not required for all developers. Most projects use some kind of design patterns so that aspects often can be re-used across the project or subprojects. For developers who will be setting up the project for use with the Surrogate framework, it is highly recommended to take a look in the AspectJ Programming Manual
The mockPointcut is defined as an abstract pointcut in the
SurrogateCalls aspect, so you will not need to code
any calls to the manager inside the AspectJ advice. There might however be situations where you want to provide
a "default" stub implementation for a particular class or method - unless it is mocked by the unit tests. In this case,
you could easily create a new advice for this particular pointcut. This case is discussed later.
It is important to note that even if you define a loads of pointcuts for mocking, and weave the java classes with the aspects, the code will still work as normal, abeit a bit slower. The unit tests are responsible for turning on an off mocks as required. This is a strong feature because one unit test need to mock a class, while another might want to use the "real" class.
One question which might arise is why Surrogate does not provide a default pointcut implementation, covering all of the "mockable" pointcuts. Such a pointcut would be:
aspect ExampleProjectCalls extends SurrogateCalls {
/**
* NOT RECOMMENDED !
*/
protected pointcut mockPointcut() :
(
call (*.new(..)) ||
call(* *(..))
);
}
There are several reasons why we have chosen not to define such a "default" mockPointcut:
java.lang.String would be surrounded by Aspect specific byte code. For extremely small
projects, this could probably work, but for larger projects, the weaving time would take long time.
SurrogateCalls
advice would have to check if it was a mock object registered for every call or "new" statement.
Object
and you have registered any mock object, that method would be mocked, since all classes
(also Mock Objects) are assignable from Object. This is probably not what you wanted.
Experience have shown that maintaining the mockPoincut even on large projects is not a big issue, so
Surrogate sticks to this approach.
Generally, the two AspectJ pointcut types to use in the Surrogate framework are the call and execution pointcuts, often combined with the within pointcut to restrict the pointcut scope to as set of classes. There is sometimes a bit confusion on the differences between the call and execution pointcuts and the AspectJ Programming Manual contains a section for this.
In a project environment where Surrogate is used, it will most likely be the case that you weave your project classes with the Surrogate aspects but let thirdparty jars and the java System libraries alone. As a result, an "execution" pointcut will not have any effect if the corresponding class file is not woven. As an example,
execution(* java.lang.System.currentTimeMillis()) // PROBABLY ERROR
execution (com.mycompany.MyClass.new(..)) // PROBABLY ERROR
call (com.mycompany.MyClass.new(..)) // CORRECT
TODO. This section describes how to use the AOP "within" clause to speed up the weaving
When introducing unit tests in a large system, it may often be the case that you want to provide some "default behavior" stubs so that other developers can concentrate about writing unit test of their own code, not mocking all sort of "uninteresting" behavior.
A good example on this are logging services. These are normally called from all places in the code and there might even exists several log service classes in the system.
If every unit test as part of the setup should register stubs to mock all "helper" services, the testing code would be difficult to
read and hard to maintain. One solution could be to register the stub in a common base class setUp but this presents
some new problems. The first problem is that many unit tests do inherit the same base class. The second problem is
that there might be some special unit tests actually wanting to control the mocking of the "helper" service.
The Surrogate framework gives a quite neat solution to this problem. By defining special pointcuts for stub-candidate joinpoints, you can give the unit test the choice to use a mock. If no mock has been registered, the default stub behavior is executed in place of the real code.
An example implementation is:
aspect ExampleProjectCalls extends SurrogateCalls {
/**
* Implementation of the mock pointcut
*/
protected pointcut mockPointcut() :
(
call (java.io.*Reader.new(..)) ||
call(* java.lang.System.currentTimeMillis())
);
/**
* Special pointcut for default stub behavior of call to logging services
*/
pointcut callingLoggingService() :
execution(static void com.mycompany..Logging*.log*(..)) ;
Object around() : callingLoggingService() {
try
{
SurrogateManager.MockExecutor e =
SurrogateManager.getInstance().getMockExecutor(thisJoinPoint);
return e != null ? e.execute(thisJoinPoint.getArgs()) : defaultLogBehaviour();
} catch (Throwable t) {
ExceptionThrower.throwThrowable(t);
return null;
}
}
Object defaultLogBehaviour() {
return null; // return value never used - void
}
}
The "callingLoggingServices" pointcut picks out all joinpoints that are executions of a "Logging" class method starting with "log" and returning void. When the corresponding advice executes, it asks the SurrogateManager if there has been any unit tests registering a mock for the joinpoint. If so, that mock is allowed to execute. Otherwise the default "stub" behavior is executed, i.e. nothing in this case.
The Surrogate framework lets you define two types of mocks, a normal mock object and the so-called
MockMethod.
The Surrogate definition of a "mock object" is simply a class implementing an interface or extending another class.
In this case, the "mock object" can behave as a surrogate for the interface or class.
Both of the types have their advantages and disadvantages. Consider for example the following mockPointcut:
call (* java.util.Calendar.getInstance(..))
public static Calendar getInstance();
public static Calendar getInstance(TimeZone zone);
public static Calendar getInstance(Locale aLocale);
public static Calendar getInstance(TimeZone zone,Locale aLocale);
mockPointcut it uses the following
rules:
mockPointcut is a method or constructor, the method or constructor
signature is matched against all registered MockMethods. If a match is found, the MockMethod
is allowed to execute with the same parameters as supplied to the original method
mockPointcut is a method or constructor, if the
declared return value of the method or signature of the constructor class matches any
of the registered "mock objects", the first "mock object" is returned. The match algorithm
used is simply the java isAssignableFrom reflection method.
I.e. when any of the methods above execute, since all the methods have return values,
we have the change to "short-circuit" the method by two means.
To take public static Calendar getInstance(TimeZone zone) as
an example, we could short-circuit the method and return a Calendar "mock object" in the following ways:
// MockMethod solution
SurrogateManager surrogateManager = SurrogateManager.getInstance();
surrogateManager.reset();
Calendar mockCalendar = new Calendar();
mockCalendar.setTimeInMillis(2000L);
MockMethod mockGetCalendarInstance = new MockMethod(Calendar.class,"getInstance",new Class[] { TimeZone.class });
surrogateManager.addMockMethod(mockGetCalendarInstance);
mockGetCalendarInstance.addReturnValue(mockCalendar);
...
// "mock object" solution
SurrogateManager surrogateManager = SurrogateManager.getInstance();
surrogateManager.reset();
Calendar mockCalendar = new Calendar();
mockCalendar.setTimeInMillis(2000L);
surrogateManager.addMock(mockCalendar);
...
The following table summarizes the pro's and con's of the methods:
| Typical signature/usage | mock objects | MockMethod |
|---|---|---|
public Object getSomeObject(String s);
MyService service = (MyService)theClass.getSomeObject("test");
| Not recommended! The return value is "Object" so all "mock objects" will in fact be able to represent it. - something which make maintenance more difficult. In addition, it will not be possible to validate the input arguments to the method. | Recommended since it is possible to specify the exact mock object to return to the caller as well as checking input arguments. |
public MyService getSomeService();
MyService service = theClass.getSomeService();
| Recommended since the return value is very unlikely to be implemented by several mock objects. And we do not have any input arguments to validate. | More complex, More lines to implement functionality. |
public MyService();
MyService service = new MyService();
| Recommended since the return value is very unlikely to be implemented by several mock objects. And we do not have any input arguments to validate. | More lines to implement functionality. |
final class MyService ...
public MyService();
MyService service = new MyService();
| Impossible to mock, since MyService is final ... | Impossible to mock MyService. But can create MockMethods for the MyService methods and constructors we use ... |
Since Surrogate makes use of the AspectJ technology, it is a bit more complicated to integrate it in your standard java project environment than JUnit and EasyMock. In addition to putting the Surrogate jars in your project test-classpath, you will also need to introduce an extra step in the build process. Namely the "weaving" of java classes with Surrogate advices before running the unit tests.
An alternative to this extra build step is "loadtime-weaving" - supported by AspectJ. In his case, you would run a "weaving-classloader" when running the unit tests. We will not describe this methodology in details in this document - since we have never used it.
Even if the Surrogate-woven classes in theory does not make any harm before explicitly mocking (except from slowing down the system), it is strongly recommended to keep them far away from a production system.
The way of guaranteeing this separation is to weave the Surrogate(d) classes to a special class folder and then run the unit tests with this folder first on the classpath. A typical folder structure is illustrated below:
src
java -- contains "production" source
test -- contains unit tests
aspects -- contains Surrogate aspects
lib -- contains 3rd party libraries
test-lib -- contains 3rd party libraries for testing
target
classes -- production classes
woven-classes -- production classes woven with AspectJ/Surrogate
test-classes -- compiled unit tests, - not woven
A typical classpath would in this case be "target/woven-classes:target/test-classes:target:classes:lib/....."
There will sometimes be the case where a third-party class need to be woven, although this is seldom necessary. In most cases,all the calls to the third-party class can be mocked away with MockMethods, even if the class itself cannot be mocked. However, if the class contains a "static" block with some failing references, the java classloader will fail to load the class, even before any objects are instantiated. In this case, you might have to create an aspect which short-circuit the static initializer block of the class.
Surrogate and similar predecessor frameworks have been used on a large and monolithic J2EE system, with teams up to 30-40 members and several thousand classes. For these kind of systems, it is important that the weaving step is relatively fast because most developers prefer to do a fast code-compile-test cycle. On small systems (say 100 classes or separated sub-systems), the weaving process is generally very fast, so that no special optimization is necessary.
A challenge when using AspectJ technology compared to standard java is that changing one Aspect source file possibly invalidates all the previously woven classes. This is due to the cross-cutting nature of AOP. In most projects, the Surrogate aspect files will normally not be changed very often, so a full "re-weave" will seldom be required. However, if the project is doing massive refactoring, it might be necessary to do a full "re-weave" more often, since some old classes in the "target/classes" folder might contain invalid references. I.e. when the AspectJ compiler tries to weave those classes, you would get an error.
Another alternative is to use the AspectJ support for the Ant "CompilerAdapter". In this case, you would use the AspectJ front-end compiler to both compile java and weave with the Surrogate aspects thus avoiding the problem with "old invalid" classes. The example folder contains such a target. The disadvantage with this approach is that you would have to call the "compile" target before integration testing/packaging , because classes are not put in the standard "target/classes" folder.
The Ant example script in the surrogate-example folder is optimized for a very quick code-compile-test cycle. This is described in the corresponding Ant subsection.
The ant "build.xml" script in the surrogate-example folder shows how to integrate surrogate/AspectJ in the build process.
The Surrogate targets are:
| Ant target | Description |
|---|---|
| compile | Compiles the "real" java classes in the "src" directory and put them in "target/classes" |
| compile-for-test | Alternative compilation using the AspectJ front-end java compiler. In this case, the "real" classes in the "src" directory are compiled directly to the "woven-classes". |
| weave-for-test | Weaves the classes in the "target/classes" folder with the aspects in the "src/aspects" folder and places them in "target/woven-classes". In order to speed the process, the following rules are used: If any of the files in the src/aspects folder have changed since the last weaving, all classes must be re-woven". Otherwise, only the classes changed (by the java compiler) since the last weaving will be recompiled. This is achieved by copying them to a temporary folder, then weaving them back to target/woven-classes" - because the AspectJ ajc compiler only operates on folders/jars when weaving classes. |
| compile-tests | compiles the unit tests and place them in the target/test-classes folder |
| test | Runs all the unit tests in the target/test-classes folder,with the target/woven-classes first on the classpath. |
This section covers some typically encountered problems as reported by pre-Surrogate users.
If you have added a mock object or MockMethod in the unit test and it does not seem to behave correctly, the first thing to do is to rerun the test with the -Dsurrogate.debug=true" option. In Ant and Maven, typically:
ant -Dsurrogate.debug=true test
maven -Dsurrogate.debug=true jar
By redirecting the debug output to a file, and search for the "surrogate:added" string it should be quite easy to find the "ID" of the mock and to find out when the mock was added, removed and if a joinpoint matched the mock in this period. The expected "joinpoint ID" is typically found by searching the file for the name of the class-to-mock or method-to-mock.
If no joinpoint ID call seem to match the place in the code where you expected your mock to replace the "real" stuff, it has normally one of the following causes:
System.out.println statementmockJoinPoint at
the corresponding code. In this case, add a new pointcut to the mockPointCut in
the pointcut definition file.
If a joinpoint call seem to match the place in the code where you expected your mock to replace the "real" stuff, but the SurrogateManager returns a "null" mock or one of your other mocks, check the method signature of the joinpoint and compare it with the mock name. Overloaded methods have different signatures and one MockMethod can only match one signature.
If, for instance your "mock ID" is
public java.lang.String com.mycompany.MyClass.getCustomer(java.lang.String,java.lang.String)
execution(String com.mycompany.MyClass.getCustomer(String, String,Integer))
This is probably due to that "old" mocks are hanging around in the system between unit test runs.
You normally create and registers your mock objects in the TestCase setUp method,
in the TestCase constructor, or in the actual test method. Now since the setUp and actually also the TestCase constructor
are called before each TestCase run, new instances of the mock objects replace the old ones in each test.
But what happens if the "class-under-test" stores a static reference to the mock object? In this case, the
first test would load the class, and the mock object would be stored in the static block. But the class would
not be reloaded in subsequent tests and the static reference would now point to a mock object which has been
removed from the SurrogateManager.
One way to get around this is that the JUnit test declares a private static reference to the mock object.
and never re-creates the mock object. In this way, the "class-under-test" and the JUnit test would always refer to
the same mock.
This is probably because of downcasting of a class returned from a method. A good example
is the JNDI PortableRemoteObject.narrow method with signature:
public static Object narrow(Object narrowFrom,
Class narrowTo)
throws ClassCastException;
CustomerServiceHome home =
(CustomerServiceHome) PortableRemoteObject.narrow(objRef, CustomerServiceHome.class);
java.lang.Object, so if you have a mockPoincut
on the call to this method, any of your mock objects may be returned - and you get a
ClassCastException when trying to cast to the home interface.
The solution to this is to register a
MockMethod in place of e.g.
the narrow method. Since the MockMethod have precedence, you can
explicitly set up the return values of the method to return the correct mock object.