• Home   /
• Archive by category "1"

## Junit Test Case Before Class Assignments

### Introduction to Unit Testing Framework

The various type of software testings include:

• Unit Test: Test individual component/class in isolation.
• Integration Test: Test a group of associated components/classes.
• Acceptance Test (or Functional Test): operate on a fully integrated system, testing against the user interface (e.g., HTML for browser or XML/JSON for web services).
• Regression Test: Tests to ensure the a change (such as enhancement, patches or configuration change) does not break the system or introduce new faults.

Unit Testing is concerned about testing individual programs/classes to ascertain that each program/class runs as per specification. Prior to the arrival of "unit testing framework", programmers tends to write test expressions which print to the console or a trace file (the amount of output is sometimes controlled by a trace-level or debug-level flag). This approach is not satisfactory because it requires human judgment to analyze the results produced. Too many print statements cause the dreaded Scroll Blindness.

JDK 1.4 provides an assertion feature (read Assertion), which enables you to test (or assert) your assumptions about your program logic (such as pre-conditions, post-conditions, and invariants). Nonetheless, assertion is primitive compared with the unit testing framework.

With a proper Unit Testing framework, you can automate the entire unit testing process. Your job becomes designing proper test cases to excite the program. Furthermore, the unit testing process can be integrated into the build process. In this case, the build process not only checks for syntax errors, but semantic errors as well.

#### Extreme Programming

Extreme programming (@ www.xprogramming.com) advocates "write test first, before writing codes".

#### xUnit Unit Testing Framework

xUnit is the family name given to a group of unit testing frameworks that share the same architecture, such as JUnit (for Java), NUnit (for .NET), CppUnit (for C++), PHPUnit (for PHP) and many others.

The xUnit architecture has these common components:

• Test case / Test suites:
• Test fixture:
• Test runner:
• Test result formatter:
• Assertions:

### JUnit

JUnit (@ http://junit.org/) is an open-source Java Unit Testing Framework designed by Kent Beck, Erich Gamma. It is the de facto standard for Java Unit Testing. JUnit is not included in JDK, but included in most of the IDEs such as Eclipse and NetBeans.

#### Installing and Using JUnit

Using JUnit: To use the JUnit, include JUnit jar-files "" and "" in your .

#### Using JUnit under Eclipse

Include JUnit Library in your Java Project: Create a new Java project ⇒ right-click on the project ⇒ Properties ⇒ Java Build Path ⇒ "Libraries" tab ⇒ Add Library ⇒ JUnit ⇒ In "JUnit library version", choose "JUnit 4" ⇒ In "current location" use the eclipse's JUnit or your own download. [Alternatively, when you create a new test case or test suite (as describe below), Eclipse will prompt you to include the JUnit library.]

Create Test case (or Test Suite): To create a new JUnit test case (or test suite, which contains many test cases) ⇒ File ⇒ Others ⇒ Java ⇒ JUnit ⇒ JUnit test case (or JUnit test suite).

Run Test case (or Test Suite): To run a test case (or test suite), right-click the file ⇒ Run As ⇒ JUnit Test.

### JUnit 4

There are two versions of JUnit, version 3 and version 4, which are radically different. JUnit 4 uses the annotation feature (since JDK 1.5) to streamline the process and drop the strict naming conventions of test methods.

#### Getting Started with an Example

Suppose that we wish to carry out unit testing on the following Java program, which uses methods to perform arithmetic operations on two integers. Take note that divide throws an for divisor of zero.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 public class Calculator { public static int add(int number1, int number2) { return number1 + number2; } public static int sub(int number1, int number2) { return number1 - number2; } public static int mul(int number1, int number2) { return number1 * number2; } public static int divInt(int number1, int number2) { if (number2 == 0) { throw new IllegalArgumentException("Cannot divide by 0!"); } return number1 / number2; } public static double divReal(int number1, int number2) { if (number2 == 0) { throw new IllegalArgumentException("Cannot divide by 0!"); } return (double) number1 / number2; } }
##### First Test Case

Let's do it under Eclipse.

1. Create a new Eclipse Java project called "".
2. Create a new class called "" under "" folder, with the above program code.
3. Create a new folder called "" for storing test scripts ⇒ Right-click on the project ⇒ New ⇒ Folder ⇒ In folder name, enter "". Make "" a source folder by right-click on "" ⇒ Build Path ⇒ Use as source folder.
4. Create the first test case called "" ⇒ Right-click on folder "" ⇒ New ⇒ Other ⇒ Java ⇒ JUnit ⇒ JUnit Test Case ⇒ New JUnit 4 test ⇒ In Name, enter "". Enter the following codes:
5. To run the test case, right-click on the file ⇒ Run as ⇒ JUnit Test. The test result is shown in the JUnit panel. 4 tests were run and all succeeded. Study the test results.
6. Try modify one of the test to force a test failure and observe the test result, e.g., @Test public void testAddPass() { assertEquals("error in add()", 0, Calculator.add(1, 2)); ..... }
##### Explanation
• A test case contains a number of tests, marked with annotation of "". Each of the test is run independently from the other tests.
• Inside the test method, we can use static methods (in class ) to assert the expected and actual test outcomes, such as: public static void assertEquals([String message,] long expected, long actual) public static void assertEquals([String message,] double expected, double actual, double epsilon) public static void assertEquals([String message,] Object expected, Object actual) public static void assertNotEquals(.....) public static void assertSame([String message,] Object expected, Object actual) public static void assertNotSame(.....) public static void assertTrue([String message,] boolean condition) public static void assertFalse([String message,] boolean condition) public static void assertNull([String message,] Object object) public static void assertNotNull(......) public static void assertArrayEquals([String message,], int[] expecteds, int[] actuals) public static void assertArrayEquals([String message,], byte[] expecteds, byte[] actuals) public static void assertArrayEquals([String message,], char[] expecteds, char[] actuals) public static void assertArrayEquals([String message,], long[] expecteds, long[] actuals) public static void assertArrayEquals([String message,], byte[] expecteds, byte[] actuals) public static void assertArrayEquals([String message,], short[] expecteds, short[] actuals) public static void assertArrayEquals([String message,], Object[] expecteds, Object[] actuals) public static <T> void assertThat([String message,], T actual, org.hamcrest.Matcher<T> matcher)
##### Run Test Standalone via Test Runner

To run your test standalone (outside Eclipse), you could write a Test Runner as follows:

 1 2 3 4 5 6 7 8 9 10 11 12 13 import org.junit.runner.JUnitCore; import org.junit.runner.Result; import org.junit.runner.notification.Failure; public class RunTestStandalone { public static void main(String[] args) { Result result = JUnitCore.runClasses(AddSubTest.class); for (Failure failure : result.getFailures()) { System.out.println(failure.toString()); } System.out.println(result.wasSuccessful()); } }

You can include more than one test cases using .

##### Run Test in Command-line

JUnit also provides a console version of test-runner called org.junit.runner.JUnitCore for you to run the tests in command-line, with the following syntax:

$java org.junit.runner.JUnitCore TestClass1 [TestClass2 ...] 1. Copy all your classes into one folder (for simplicity). 2. Set the to include the JUnit jar-files:$ export CLASSPATH=.:$CLASSPATH:/path/to/junit/junit-4.11.jar:/path/to/junit/hamcrest-core-1.3.jar > set CLASSPATH=.;%CLASSPATH%;x:\path\to\junit\junit-4.11.jar;x:\path\to\junit\hamcrest-core-1.3.jar 3. Compile all the source files:$ cd /path/to/source-files $javac Calculator.java AddSubTest.java 4. Run the test:$ java org.junit.runner.JUnitCore AddSubTest JUnit version 4.11 .... Time: 0.006 OK (4 tests)
##### Second Test Case

Let's write another test case to test the divide methods, which throw an Exception for divisor of zero. Furthermore, the method returns a double which cannot be compared with absolute precision.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 import static org.junit.Assert.*; import org.junit.Test; public class DivTest { @Test public void testDivIntPass() { assertEquals("error in divInt()", 3, Calculator.divInt(9, 3)); assertEquals("error in divInt()", 0, Calculator.divInt(1, 9)); } @Test public void testDivIntFail() { assertNotEquals("error in divInt()", 1, Calculator.divInt(9, 3)); } @Test(expected = IllegalArgumentException.class) public void testDivIntByZero() { Calculator.divInt(9, 0); } @Test public void testDivRealPass() { assertEquals("error in divInt()", 0.333333, Calculator.divReal(1, 3), 1e-6); assertEquals("error in divInt()", 0.111111, Calculator.divReal(1, 9), 1e-6); } @Test(expected = IllegalArgumentException.class) public void testDivRealByZero() { Calculator.divReal(9, 0); } }

Run the test and observe the test result. Change 's expected value from 0.333333 to 0.333330 and check the test result.

##### Explanation
• To test for exception, use annotation with attribute .
• To compare doubles, use a tolerance (epsilon) as shown.
##### First Test Suite

A test suite comprises many test cases.

To create a test suite under Eclipse ⇒ right-click on the folder ⇒ New ⇒ other ⇒ Java ⇒ JUnit ⇒ JUnit Test Suite ⇒ In Name, enter "" ⇒ Select test cases to be included - AddSubTest and DivTest.

The following test script will be created:

 1 2 3 4 5 6 7 8 9 import org.junit.runner.RunWith; import org.junit.runners.Suite; import org.junit.runners.Suite.SuiteClasses; @RunWith(Suite.class) @SuiteClasses({ AddSubTest.class, DivTest.class }) public class AllTests { }

Take note that the test suite class is marked by annotation and with an empty class body.

To run the test suite ⇒ right-click on the file ⇒ Run as ⇒ JUnit Test. Observe the test results produced.

You can also run the test suite via Test Runner , just like running test cases (as described earlier).

#### Testing Java Classes By Example

Instead of testing static methods in a Java class, let's carry out unit test on a proper self-contained and encapsulated Java class with its own private variables and public operations.

Suppose that we have a class called that represents a number, and capable of performing arithmetic operations.

Again, we shall work under Eclipse.

1. Create a Java project called ""
2. Create a new Java class called "", as follow:
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 public class MyNumber { int number; public MyNumber() { this.number = 0; } public MyNumber(int number) { this.number = number; } public int getNumber() { return number; } public void setNumber(int number) { this.number = number; } public MyNumber add(MyNumber rhs) { this.number += rhs.number; return this; } public MyNumber div(MyNumber rhs) { if (rhs.number == 0) throw new IllegalArgumentException("Cannot divide by 0!"); this.number /= rhs.number; return this; } }
3. Create a new source folder called "" for storing test scripts. Make it a source folder by right-click ⇒ Build Path ⇒ Use as source folder.
4. Create the first test case called MyNumberTest (under "" folder), as follows:
 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 import static org.junit.Assert.*; import org.junit.After; import org.junit.Before; import org.junit.Test; public class MyNumberTest { private MyNumber number1, number2; @Before public void setUp() throws Exception { System.out.println("Run @Before"); number1 = new MyNumber(11); number2 = new MyNumber(22); } @After public void tearDown() throws Exception { System.out.println("Run @After"); } @Test public void testGetterSetter() { System.out.println("Run @Test testGetterSetter"); int value = 33; number1.setNumber(value); assertEquals("error in getter/setter", value, number1.getNumber()); } @Test public void testAdd() { System.out.println("Run @Test testAdd"); assertEquals("error in add()", 33, number1.add(number2).getNumber()); assertEquals("error in add()", 55, number1.add(number2).getNumber()); } @Test public void testDiv() { System.out.println("Run @Test testDiv"); assertEquals("error in div()", 2, number2.div(number1).getNumber()); assertEquals("error in div()", 0, number2.div(number1).getNumber()); } @Test(expected = IllegalArgumentException.class) public void testDivByZero() { System.out.println("Run @Test testDivByZero"); number2.setNumber(0); number1.div(number2); } }
5. Run the test and observe the result. Modify some lines to make the test fails and observe the result.

The output, used for illustrating the sequence of operations, is as follows:

Run @Before Run @Test testDivByZero Run @After Run @Before Run @Test testAdd Run @After Run @Before Run @Test testDiv Run @After Run @Before Run @Test testGetterSetter Run @After
##### Test Fixtures, @Before and @After

A test fixtures is a fixed state of a set of objects used as a baseline for running tests. The purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable.

In JUnit 4, fixtures are setup via the and annotations.

• The annotated method (known as ) will be run before EACH test method (annotated with ) to set up the fixtures.
• The annotation method (known as tearDown()) will be run after EACH test.

We typically declare text fixtures as private instance variables; initialize via annotated method; and clean-up via annotation method. Each test method runs on its own set of text fixtures with the same initial states. This ensures isolation between the test methods.

##### @BeforeClass and @AfterClass

Beside the and , there is also and .

• The annotated method will be run once before any test, which can be used to perform one-time initialization tasks such as setting up database connection.
• The annotated method will be run once after all tests, which can be used to perform housekeeping tasks such as closing database connection.

#### JUnit 4's Annotations

JUnit 4 defines the following six annotations in package .

AnnotationDescription
The annotated method is to be run as a test method.
The annotated method is to be run before EACH of the test method.
The annotated method is to be run after EACH of the test method.
The annotated method is to be run ONCE before any of the test method.
The annotated method is to be run ONCE after all the test methods.
Ignore (don't run) the test method. This is a convenient way to mark out a test method (e.g. after some code changes that fail this test.)
[TODO]
##### Example of @Ignore
@Ignore("Under Construction") @Test public void someTest() { ...... }

#### JUnit - Exceptions Test

To test if the code throws a desired exception, use annotation , as illustrated in the previous example.

#### JUnit - Timing Test

To handle or test timeout, use annotation . For example,

 1 2 3 4 5 6 7 8 import org.junit.Test; public class TimeoutTest { @Test(timeout=1000) public void test() { while (true) {} } }
java.lang.Exception: test timed out after 1000 milliseconds ......

#### JUnit - Parameterized Test

JUnit 4 introduces parameterized test which allows you to run the same test over and over again using different values. To use parameterized test:

1. Annotate the test class with .
2. Create a public static method annotated with that returns a list () as test data set.
3. Create a public constructor that takes its input from the method to setup the test fixtures defined as instance variables. The constructor will be run before EACH test.
4. Create your tests case(s) using the instance variables as the source of the test data.

For example,

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 import static org.junit.Assert.*; import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; import org.junit.runners.Parameterized; import org.junit.runners.Parameterized.Parameters; import java.util.Arrays; @RunWith(Parameterized.class) public class MyNumberParameterizedTest { private MyNumber number1, number2; private int result; @Parameters public static Iterable data() { System.out.println("Run @Parameters"); return Arrays.asList(new Object[][] { { new MyNumber(1), new MyNumber(2), 3 }, { new MyNumber(-1), new MyNumber(-2), -3 }, { new MyNumber(3), new MyNumber(-3), 0 }, }); } public MyNumberParameterizedTest(MyNumber number1, MyNumber number2, int result) { System.out.println("Run constructor"); this.number1 = number1; this.number2 = number2; this.result = result; System.out.println("-- number1=" + number1.getNumber() + " number2=" + number2.getNumber() + " result=" + result); } @Before public void setUp() throws Exception { System.out.println("Run @Before"); System.out.println("-- number1=" + number1.getNumber() + " number2=" + number2.getNumber()); } @Test public void test() { System.out.println("Run @Test"); assertEquals(result, number1.add(number2).getNumber()); } @After public void tearDown() throws Exception { } }
Run @Parameters Run constructor -- number1=1 number2=2 result=3 Run @Before -- number1=1 number2=2 Run @Test Run constructor -- number1=-1 number2=-2 result=-3 Run @Before -- number1=-1 number2=-2 Run @Test Run constructor -- number1=3 number2=-3 result=0 Run @Before -- number1=3 number2=-3 Run @Test

The output trace suggests that method is run once. For EACH test, the constructor is run first, followed by , and methods.

#### Another Example

import org.junit.*; import java.util.ArrayList; import org.junit.runner.Result; public class ArrayListTest { private ArrayList<String> lst; @Before public void init() throws Exception { lst = new ArrayList<String>(); lst.add("alpha"); lst.add("beta"); } @Test public void insertTest() { Assert.assertEquals("wrong size", 2, lst.size()); lst.add(1, "charlie"); Assert.assertEquals("wrong size", 3, lst.size()); Assert.assertEquals("wrong element", "alpha", lst.get(0)); Assert.assertEquals("wrong element", "charlie", lst.get(1)); Assert.assertEquals("wrong element", "beta", lst.get(2)); } @Test public void replaceTest() { Assert.assertEquals("wrong size", 2, lst.size()); lst.set(1, "charlie"); Assert.assertEquals("wrong size", 2, lst.size()); Assert.assertEquals("wrong element", "alpha", lst.get(0)); Assert.assertEquals("wrong element", "charlie", lst.get(1)); } public static void main(String[] args) { Result r = org.junit.runner.JUnitCore.runClasses(ArrayListTest.class); System.out.println(r.wasSuccessful()); } }

To run the test, you can either include a method as above, or use the command-line.

#### JUnit Package org.junit

The core package for JUnit 4 is , which is simple and elegantly designed.

• class: contains methods , , , , , , , , .
• class: contains methods , , , .
• : mark the method as a test method.
• : The test is expected to trigger this exception.
• : Treat the test as fail, if it exceeds the specified milliseconds.
• and : mark the method as to be run before and after EACH test method, for initializing and cleaning-up test fixtures.
• and : mark the method as to be run before and after ONCE for the class.
• : ignore this test method (annotated with ).
• : [TODO]

### JUnit 3.8 (deprecated?)

JUnit 3.8, which uses strict naming convention to denote various entities, is probably deprecated. I suggest that you move to JUnit 4, which is more intuitive by using annotation.

#### Let's begin with an Example

Below is a Java program to be tested. Note that there is a logical error in the program.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 public class Grade { public static char getLetterGrade(int mark) { assert (mark >= 0 && mark <= 100) : "mark is out-of-range: " + mark; if (mark >= 75) { return 'A'; } else if (mark >= 60) { return 'B'; } else if (mark > 50) { return 'C'; } else { return 'F'; } } }

The unit-test program (using JUnit framework) is as follows. Black-box test cases are set up to test typical values as well as boundary values.

Compile and execute the program (with JUnit JAR file included in the ) as follows. Note that one of the unit-test cases catches the logical error.

#### JUnit Terminology

• Class : A class that contains test methods should derive from this superclass. Each can include many test methods.
• Test Methods: A test methods must be named . This is because JUnit uses the reflection mechanism to find and run these methods. Inside a test method, you can use a variation of the method (e.g., , , ) to compare the expected and actual results.
• Test Fixture: The set of objects that a test method operates on. You declare these objects as a private variable, and initialize them by overriding the or via the constructor. You can perform clean-up operations by overriding . Each test method runs on its own instance with its own set of text fixtures. This ensures isolation between the test methods.
• Class : You can combine many (e.g., written by different person) into a test suite, and run them at once.
• Class : for running the or .

#### Writing Tests

Step 1: Extend a subclass of :

import junit.framework.*; public class JUnit38DemoArrayList extends TestCase { public JUnit38DemoArrayList(String name) { super(name); } }

Step 2: If two or more test methods use a common set of test objects (called test fixtures), declare the test fixtures as instance variables. For example, suppose we are testing the class .

private ArrayList<String> lst;

Step 3: Initialize the text fixture. You can override or use the constructor. Each test method runs on its own instance. This provides isolation between test methods. Each test method invoke the constructor to construct an instance of the , followed by , run the steps coded inside the test method, and the before exiting. The test methods may run concurrently. For example, let's initialize our test fixture with two elements.

protected void setUp() throws Exception { lst = new ArrayList<String>(); lst.add("alpha"); lst.add("beta"); } protected void tearDown() throws Exception { super.tearDown(); }

Step 4: Write the test methods for this . All the test methods must be named , as JUnit uses reflection to find these test methods. For example,

public void testInsert() { assertEquals("wrong size", 2, lst.size()); lst.add(1, "charlie"); assertEquals("wrong size", 3, lst.size()); assertEquals("wrong element", "alpha", lst.get(0)); assertEquals("wrong element", "charlie", lst.get(1)); assertEquals("wrong element", "beta", lst.get(2)); } public void testReplace() { assertEquals("wrong size", 2, lst.size()); lst.set(1, "charlie"); assertEquals("wrong size", 2, lst.size()); assertEquals("wrong element", "alpha", lst.get(0)); assertEquals("wrong element", "charlie", lst.get(1)); }

Step 5: You can now run the , using JUnit-provided . There are two versions of : text-based , and GUI-based . To use the text-based , you could include a method as follows:

public static void main(String[] args) { junit.textui.TestRunner.run(JUnit38DemoArrayList.class); }

The expected outputs are:

.. Time: 0.001 OK (2 tests)

You can also invoke the from command-line:

> java junit.textui.TestRunner JUnit38DemoArrayList

You can invoke the GUI-based from command-line:

> java junit.swingui.TestRunner JUnit38DemoArrayList

Step 6: If there are many s (could be written by different people), you can put them together into a and run all the s at once. To do so, in each of the s, create a static method to extract all the test methods as follows:

public static Test suite() { return new TestSuite(JUnit38DemoArrayList.class); }

Next, write a class to include all the s into a .

import java.framework.*; public class AllTests { public static void main(String[] args) { junit.textui.TestRunner.run(suite()); } public static Test suite() { TestSuite suite = new TestSuite(); suite.addTest(JUnit38DemoArrayList.suite()); //suite.addTest(OtherTestCase1.suite()); //suite.addTest(OtherTestCase2.suite()); return suite; } }

### Automating Unit Testing with ANT or Maven

[TODO] To tidy up.

Apache's ANT is the de facto standard for automated building of Java applications (similar to Unix's "" utility). You can download ANT from ant.apache.org (download the ZIP version, and unzip it to a directory of your choice).

We shall use ANT to automate building and testing. First, create a configuration file called as follows:

<?xml version="1.0"?> <!-- to save as "build.xml" --> <project name="Black-Box Unit Test Demo" default="run" basedir="."> <!-- build all classes in this directory --> <!-- To run this: use "ant build" --> <!-- need to include junit.jar in the classpath --> <target name="build"> <javac srcdir="${basedir}"/> <echo message="Build done" /> </target> <!-- Test and build all files --> <!-- To run this: use "ant" (default) or "ant run" --> <target name="run" depends="build"> <java taskname="Test" classname="GradeTestCase" fork="true" failonerror="true" /> <echo message="Unit Test done" /> </target> <!-- delete all class files --> <!-- To run this: use "ant clean" --> <target name="clean"> <delete> <fileset dir="${basedir}" includes="*.class" /> </delete> <echo message="clean done" /> </target> </project>

To build using the above build file, run "". (By default, it executes "", which is depends on "", "" get executed to compile the program, then "" get expected to run the testing. To run only the compilation, use "". To run only the cleanup, use "".)

prompt> ant Buildfile: build.xml build: [javac] Compiling 4 source files [echo] Build done run: [Test] ..F [Test] Time: 0.005 [Test] There was 1 failure: [Test] 1) testBoundaries(GradeTestCase)junit.framework.AssertionFailedError: expected:<C> but was:<F> [Test] at GradeTestCase.testBoundaries(GradeTestCase.java:23) [Test] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [Test] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [Test] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [Test] at GradeTestCase.main(GradeTestCase.java:34) [Test] [Test] FAILURES!!! [Test] Tests run: 2, Failures: 1, Errors: 0 [Test] [echo] Unit Test done

[TODO] to be continued...

### Unit Testing Best Practices

#### Writing Test Cases

How to test a program to ensure correctly? There are two techniques: white-box testing and black-box testing. White-box testing inspect the program codes and test the program logic. Black-box testing does not inspect the program codes, but looking at the input-output, treating the program as a black box.

For black-box testing, the most common approach is to partition the inputs, and design test cases for each input partition. The test cases could test on a typical input value as well as the boundaries.

For example, the following program converts a given mark (0-100) to a letter grade ('A' to 'F'). There is a logical error in the program.

Try to run the above tests to find the logical error. Take note that does not accept as arguments, but upcast to . That is, the output show the 's numeric code.

#### Unit Testing Best Practices (From JUnit FAQ)

The followings are extracted from JUnit FAQ:

1. When should the tests be written?
Tests should be written before the code. Good tests tell you how to best design the system for its intended use. They also prevent tendencies to over-build the software. When all the tests pass, you know you're done. Whenever a customer reports a bug, first write the necessary unit test(s) to expose the bug(s) and fix them. This make it almost impossible for the same bug to resurface later.
2. Do I have to write a test for everything?
No, just test things that could reasonably break. Don't write tests that turn out to be testing the operating system or environment or the compiler. For example, public class AClass { int x; public AClass(int x) { this.x = x; } int getX() { return x; } void setX() { this.x = x; } } A test that testing is merely testing for , i,e, testing the compiler! This can't break unless the compiler or the interpreter break!
3. How often should I run my tests?
Run unit test as often as possible, ideally every time the code is changed. Run all your acceptance, integration, stress, and unit tests at least once per day (for your nightly-built).

### TestNG

TestNG (Test Next Generation) (@ http://testng.org/) is a testing framework inspired from JUnit and NUnit (the xUnit family), but introduces new functionalities like dependency testing, grouping concept to make testing easier and more powerful.

TestNG is designed to cover all types of tests: unit, integration, functional, and etc.

#### Installing TestNG

To install TestNG Eclipse Plug-in ⇒ Launch Eclipse ⇒ Help ⇒ Install New Software ⇒ In Work with, enter http://beust.com/eclipse ⇒ Add ⇒ Select TestNG.

Using TestNG: To use TestNG, include the jar-files in the .

In Eclipse, right-click on the project ⇒ Add Library ⇒ TestNG.

API Documentation: The TestNG API documentation is available @ http://testng.org/javadocs/.

#### Getting Started with TestNG with Example

I shall assume that you are familiar with JUnit 4.

Let's use TestNG (instead of JUnit) to test the class written in the earlier section.

In Eclipse, right-click on the project ⇒ New ⇒ TestNG ⇒ TestNG class.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 import static org.testng.Assert.*; import org.testng.annotations.*; public class MyNumberTestNGTest { private MyNumber number1, number2; @BeforeClass public void oneTimeSetUp() { System.out.println("@BeforeClass - oneTimeSetUp"); } @AfterClass public void oneTimeTearDown() { System.out.println("@AfterClass - oneTimeTearDown"); } @BeforeMethod public void setUp() { number1 = new MyNumber(); number2 = new MyNumber(); System.out.println("@BeforeMethod - setUp before each test"); } @AfterMethod public void tearDown() { System.out.println("@AfterMethod - tearDown before each test"); } @Test public void testAdd() { System.out.println("@Test - testAdd"); number1.setNumber(1); number2.setNumber(2); assertEquals(number1.add(number2).getNumber(), 3); } @Test(expectedExceptions = IllegalArgumentException.class) public void testDiv() { System.out.println("@Test - testDiv with exception"); number1.setNumber(1); number2.setNumber(0); number1.div(number2); } }

To run the test case under Eclipse, right-click on the file ⇒ Run as ⇒ TestNG Test.

@BeforeClass - oneTimeSetUp @BeforeMethod - setUp before each test @Test - testAdd @AfterMethod - tearDown before each test @BeforeMethod - setUp before each test @Test - testDiv with exception @AfterMethod - tearDown before each test @AfterClass - oneTimeTearDown PASSED: testAdd PASSED: testDiv =============================================== Default test Tests run: 2, Failures: 0, Skips: 0 ===============================================

As seen from the output, the annotated method is run ONCE for one-time setup; the is run ONCE for one-time tear down. The and (called and in JUnit 4) are run before and after EACH .

So far, everything is similar to JUnit 4, except some name changes.

#### Running TestNG Test Cases

There are several ways to run test case for TestNG:

• With an XML description file.
• With ANT build tool.
##### Via TestNG XML Description File

TestNG (compared with JUnit) introduces an XML description to describe test suite/test cases to provide more flexibility in running tests.

Prepare the following XML Description file (says ""), which describes a test suite comprising of test cases. Each test case comprises of many Java classes.

 1 2 3 4 5 6 7 8 9

You can run the test suite in command-line:

// Set CLASSPATH to include the TestNG jar-file // For Linux/Mac with bash shell $export CLASSPATH=.:$CLASSPATH:/path/to/testng-6.8/testng-6.8.jar // For Windows > set CLASSPATH=.;%CLASSPATH%;x:\path\to\testng-6.8\testng-6.8.jar // Compile the Java test classes $javac MyNumberTestNGTest.java // Run the test thru XML description file$ java org.testng.TestNG testing.xml @BeforeClass - oneTimeSetUp @BeforeMethod - setUp before each test @Test - testAdd @AfterMethod - tearDown before each test @BeforeMethod - setUp before each test @Test - testDiv with exception @AfterMethod - tearDown before each test @AfterClass - oneTimeTearDown =============================================== MyNumberTestSuite Total tests run: 2, Failures: 0, Skips: 0 ===============================================

In Eclipse, to run a suite description file ⇒ Run ⇒ Run Configurations ⇒ Suite ⇒ Select the desired XML description file.

The XML description file has this syntax:

1. The root tag is <suite>.
2. The <suite> tag can contain one or more <test> tags.
3. The <test> tag can contain one or more <classes> tags.
4. The <classes> tag can contain one or more <method> tags.

[TODO]

#### TestNG's Annotations

NameDescription
Mark a method (or class) as a test method (or class).
Run ONCE before and after all tests in this suite.
Run ONCE before and after all tests in this class.
Run before and after EACH @Test method.
[TODO]
[TODO]
Mark this test method is to get its parameters from the XML description file.
Mark the method, which return an , as data source for a test method.
[TODO]
[TODO]

#### TestNG - Exception Test

Mark the test method that is expected to throw an exception with as seen in the above example.

JUnit 4 uses annotation .

#### TestNG - Ignore Test

To ignore a test, mark it with annotation .

JUnit 4 uses an dedicated annotation to override the annotation.

#### TestNG - Timing Test

To set a timeout (milliseconds) for a test, use annotation (exactly the same as JUnit 4).

#### TestNG - Parameterized Test

##### Via @Parameters and the XML Description File <parameter> Tag

The test class is as follows:

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 import static org.testng.Assert.*; import org.testng.annotations.*; public class TestNGParameterizedTest1 { private MyNumber number1 = new MyNumber(); private MyNumber number2 = new MyNumber(); @Test @Parameters(value={"value1", "value2", "result"}) public void testAdd(int value1, int value2, int result) { System.out.println("value1=" + value1 + " value2=" + value2 + " result=" + result); number1.setNumber(value1); number2.setNumber(value2); assertEquals(number1.add(number2).getNumber(), result); } }

Mark the parameterized test method with annotation , where xxx is a String[]. The values will be passed into the arguments of the method in the same order.

The parameters are fed from the XML description file with the <parameter> tag. For example,

 1 2 3 4 5 6 7 8 9 10 11 12
##### Via the @DataProvider

The annotation can only used to pass simple type (such as and ). To pass objects, you need to use annotation.

For example,

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import static org.testng.Assert.*; import org.testng.annotations.*; public class TestNGParameterizedTest2 { @Test(dataProvider = "testAddDataProvider") public void testAdd(MyNumber number1, MyNumber number2, int result) { System.out.println("number1=" + number1.getNumber() + " number2=" + number2.getNumber() + " result=" + result); assertEquals(number1.add(number2).getNumber(), result); } @DataProvider(name = "testAddDataProvider") public Object[][] parameterIntTestProvider() { return new Object[][]{ {new MyNumber(11), new MyNumber(22), 33}, {new MyNumber(111), new MyNumber(222), 333}, {new MyNumber(1111), new MyNumber(2222), 3333} }; } }
number1=11 number2=22 result=33 number1=111 number2=222 result=333 number1=1111 number2=2222 result=3333 PASSED: testAdd(MyNumber@1e53fc13, MyNumber@1bca52f3, 33) PASSED: testAdd(MyNumber@74b1896c, MyNumber@33b54d4e, 333) PASSED: testAdd(MyNumber@15e19d13, MyNumber@f0f559e, 3333) =============================================== Default test Tests run: 3, Failures: 0, Skips: 0 ===============================================

#### TestNG - Method Dependency Test

TestNG (compared with JUnit) introduces test dependency. For example,

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import static org.testng.Assert.*; import org.testng.annotations.*; public class TestNGDependsTest { @Test public void method1() { System.out.println("@Test: method1"); assertTrue(true); } @Test(dependsOnMethods={"method1"}) public void method2() { System.out.println("@Test: method2"); assertTrue(true); } @Test(dependsOnMethods={"method1","method2"}) public void method3() { System.out.println("@Test: method3"); } }
@Test: method1 @Test: method2 @Test: method3 PASSED: method1 PASSED: method2 PASSED: method3 =============================================== Default test Tests run: 3, Failures: 0, Skips: 0 ===============================================

In , if we change to to fail the test, and will not be run, but marked as skip (instead of fail as in JUnit 4), as shown in the following outputs:

@Test: method1 FAILED: method1 java.lang.AssertionError: expected [true] but found [false] SKIPPED: method2 SKIPPED: method3 =============================================== Default test Tests run: 3, Failures: 1, Skips: 2 ===============================================

#### TestNG - Group Test and Dependency

Each test method can be assigned to one or more groups. We can select one or more groups to test via XML description file. For example,

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 import org.testng.annotations.*; public class TestNGGroupTest { @Test(groups = {"init"}) public void method1() { System.out.println("@Test: method1"); } @Test(groups = {"init", "post-init"}) public void method2() { System.out.println("@Test: method2"); } @Test(groups = {"main"}) public void method3() { System.out.println("@Test: method3"); } }

The XML description file to run methods in group "init" only.

 1 2 3 4 5 6 7 8 9 10 11 12 13 14
##### Dependency on Groups

Instead of specifying dependency on individual method names as in the previous section, we can place related method (e.g., init methods) in groups, and specifying dependency on groups of methods. For example,

 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 import static org.testng.Assert.*; import org.testng.annotations.*; public class TestNGGroupDependsTest { @Test(groups = {"init"}) public void method1() { System.out.println("@Test: method1"); assertTrue(true); } @Test(groups = {"init", "post-init"}) public void method2() { System.out.println("@Test: method2"); } @Test(groups = {"main"}, dependsOnGroups={"init"}) public void method3() { System.out.println("@Test: method3"); } }

If we use in , will be run. However, if we use in , will be skipped.

REFERENCES & RESOURCES

How do I write and run a simple test?
1. Create a class:

package junitfaq; import org.junit.*; import static org.junit.Assert.*; import java.util.*; public class SimpleTest {
2. Write a test method (annotated with ) that asserts expected results on the object under test:

@Test public void testEmptyCollection() { Collection collection = new ArrayList(); assertTrue(collection.isEmpty()); }
3. If you are running your JUnit 4 tests with a JUnit 3.x runner, write a method that uses the class to create a suite containing all of your test methods:

public static junit.framework.Test suite() { return new junit.framework.JUnit4TestAdapter(SimpleTest.class); }
4. Although writing a method to run the test is much less important with the advent of IDE runners, it's still possible:

public static void main(String args[]) { org.junit.runner.JUnitCore.main("junitfaq.SimpleTest"); } }
5. Run the test:

• To run the test from the console, type:

java org.junit.runner.JUnitCore junitfaq.SimpleTest
• To run the test with the test runner used in , type:

The passing test results in the following textual output:

[top]

How do I use a test fixture?

(Submitted by: Jeff Nielsen)

A test fixture is useful if you have two or more tests for a common set of objects. Using a test fixture avoids duplicating the code necessary to initialize (and cleanup) the common objects.

Tests can use the objects (variables) in a test fixture, with each test invoking different methods on objects in the fixture and asserting different expected results. Each test runs in its own test fixture to isolate tests from the changes made by other tests. That is, tests don't share the state of objects in the test fixture. Because the tests are isolated, they can be run in any order.

To create a test fixture, declare instance variables for the common objects. Initialize these objects in a method annotated with . The JUnit framework automatically invokes any methods before each test is run.

The following example shows a test fixture with a common object.

package junitfaq; import org.junit.*; import static org.junit.Assert.*; import java.util.*; public class SimpleTest { private Collection<Object> collection; @Before public void setUp() { collection = new ArrayList<Object>(); } @Test public void testEmptyCollection() { assertTrue(collection.isEmpty()); } @Test public void testOneItemCollection() { collection.add("itemA"); assertEquals(1, collection.size()); } }

Given this test, the methods might execute in the following order:

setUp() testEmptyCollection() setUp() testOneItemCollection()

The ordering of test-method invocations is not guaranteed, so might be executed before . But it doesn't matter, because each method gets its own instance of the .

Although JUnit provides a new instance of the fixture objects for each test method, if you allocate any external resources in a method, you should release them after the test runs by annotating a method with . The JUnit framework automatically invokes any methods after each test is run. For example:

package junitfaq; import org.junit.*; import static org.junit.Assert.*; import java.io.*; public class OutputTest { private File output; @Before public void createOutputFile() { output = new File(...); } @After public void deleteOutputFile() { output.delete(); } @Test public void testSomethingWithFile() { ... } }

With this test, the methods will execute in the following order:

createOutputFile() testSomethingWithFile() deleteOutputFile()

[top]

How do I test a method that doesn't return anything?

(Submitted by: Dave Astels)

Often if a method doesn't return a value, it will have some side effect. Actually, if it doesn't return a value AND doesn't have a side effect, it isn't doing anything.

There may be a way to verify that the side effect actually occurred as expected. For example, consider the method in the Collection classes. There are ways of verifying that the side effect happened (i.e. the object was added). You can check the size and assert that it is what is expected:

@Test public void testCollectionAdd() { Collection collection = new ArrayList(); assertEquals(0, collection.size()); collection.add("itemA"); assertEquals(1, collection.size()); collection.add("itemB"); assertEquals(2, collection.size()); }

Another approach is to make use of MockObjects.

A related issue is to design for testing. For example, if you have a method that is meant to output to a file, don't pass in a filename, or even a . Instead, pass in a . That way you can pass in a to capture the output for testing purposes. Then you can add a method (e.g. ) to encapsulate the creation.

[top]

Under what conditions should I test get() and set() methods?

Unit tests are intended to alleviate fear that something might break. If you think a or method could reasonably break, or has in fact contributed to a defect, then by all means write a test.

In short, test until you're confident. What you choose to test is subjective, based on your experiences and confidence level. Remember to be practical and maximize your testing investment.

Refer also to "How simple is 'too simple to break'?".

[top]

Under what conditions should I not test get() and set() methods?

(Submitted by: J. B. Rainsberger)

Most of the time, get/set methods just can't break, and if they can't break, then why test them? While it is usually better to test more, there is a definite curve of diminishing returns on test effort versus "code coverage". Remember the maxim: "Test until fear turns to boredom."

Assume that the method only does "return x;" and that the method only does "this.x = x;". If you write this test:

@Test public void testGetSetX() { setX(23); assertEquals(23, getX()); }

then you are testing the equivalent of the following:

@Test public void testGetSetX() { x = 23; assertEquals(23, x); }

or, if you prefer,

@Test public void testGetSetX() { assertEquals(23, 23); }

At this point, you are testing the Java compiler, or possibly the interpreter, and not your component or application. There is generally no need for you to do Java's testing for them.

If you are concerned about whether a property has already been set at the point you wish to call , then you want to test the constructor, and not the method. This kind of test is especially useful if you have multiple constructors:

@Test public void testCreate() { assertEquals(23, new MyClass(23).getX()); }

[top]

How do I write a test that passes when an expected exception is thrown?

Add the optional attribute to the annotation. The following is an example test that passes when the expected is raised:

@Test(expected=IndexOutOfBoundsException.class) public void testIndexOutOfBoundsException() { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0); }

[top]

How do I write a test that fails when an unexpected exception is thrown?

Declare the exception in the clause of the test method and don't catch the exception within the test method. Uncaught exceptions will cause the test to fail with an error.

The following is an example test that fails when the is raised:

@Test public void testIndexOutOfBoundsExceptionNotRaised() throws IndexOutOfBoundsException { ArrayList emptyList = new ArrayList(); Object o = emptyList.get(0); }

[top]

How do I test protected methods?

Place your tests in the same package as the classes under test.

Refer to "Where should I put my test files?" for examples of how to organize tests for protected method access.

[top]

How do I test private methods?

Testing private methods may be an indication that those methods should be moved into another class to promote reusability.

But if you must...

If you are using JDK 1.3 or higher, you can use reflection to subvert the access control mechanism with the aid of the PrivilegedAccessor. For details on how to use it, read this article.

If you are using JDK 1.6 or higher and you annotate your tests with @Test, you can use Dp4j to inject reflection in your test methods. For details on how to use it, see this test script.

[top]

Why does JUnit only report the first failure in a single test?

(Submitted by: J. B. Rainsberger)

Reporting multiple failures in a single test is generally a sign that the test does too much, compared to what a unit test ought to do. Usually this means either that the test is really a functional/acceptance/customer test or, if it is a unit test, then it is too big a unit test.

JUnit is designed to work best with a number of small tests. It executes each test within a separate instance of the test class. It reports failure on each test. Shared setup code is most natural when sharing between tests. This is a design decision that permeates JUnit, and when you decide to report multiple failures per test, you begin to fight against JUnit. This is not recommended.

Long tests are a design smell and indicate the likelihood of a design problem. Kent Beck is fond of saying in this case that "there is an opportunity to learn something about your design." We would like to see a pattern language develop around these problems, but it has not yet been written down.

Finally, note that a single test with multiple assertions is isomorphic to a test case with multiple tests:

One test method, three assertions:

public class MyTestCase { @Test public void testSomething() { // Set up for the test, manipulating local variables assertTrue(condition1); assertTrue(condition2); assertTrue(condition3); } }

Three test methods, one assertion each:

public class MyTestCase { // Local variables become instance variables @Before public void setUp() { // Set up for the test, manipulating instance variables } @Test public void testCondition1() { assertTrue(condition1); } @Test public void testCondition2() { assertTrue(condition2); } @Test public void testCondition3() { assertTrue(condition3); } }

The resulting tests use JUnit's natural execution and reporting mechanism and, failure in one test does not affect the execution of the other tests. You generally want exactly one test to fail for any given bug, if you can manage it.

[top]

In Java 1.4, 'assert' is a keyword. Won't this conflict with JUnit's assert() method?

JUnit 3.7 deprecated and replaced it with , which works exactly the same way.

JUnit 4 is compatible with the keyword. If you run with the JVM switch, assertions that fail will be reported by JUnit.

[top]

How do I test things that must be run in a J2EE container (e.g. servlets, EJBs)?

Refactoring J2EE components to delegate functionality to other objects that don't have to be run in a J2EE container will improve the design and testability of the software.

Cactus is an open source JUnit extension that can be used to test J2EE components in their natural environment.

[top]

Do I need to write a test class for every class I need to test?

(Submitted by: J. B. Rainsberger)

No. It is a convention to start with one test class per class under test, but it is not necessary.

Test classes only provide a way to organize tests, nothing more. Generally you will start with one test class per class under test, but then you may find that a small group of tests belong together with their own common test fixture.[1] In this case, you may move those tests to a new test class. This is a simple object-oriented refactoring: separating responsibilities of an object that does too much.

Another point to consider is that the is the smallest execution unit in JUnit: you cannot execute anything smaller than a TestSuite at one time without changing source code. In this case, you probably do not want to put tests in the same test class unless they somehow "belong together". If you have two groups of tests that you think you'd like to execute separately from one another, it is wise to place them in separate test classes.

[1] A test fixture is a common set of test data and collaborating objects shared by many tests. Generally they are implemented as instance variables in the test class.

[top]

Is there a basic template I can use to create a test?

(Submitted by: Eric Armstrong)

The following templates are a good starting point. Copy/paste and edit these templates to suit your coding style.

SampleTest is a basic test template:

import org.junit.*; import static org.junit.Assert.*; public class SampleTest { private java.util.List emptyList; /** * Sets up the test fixture. * (Called before every test case method.) */ @Before public void setUp() { emptyList = new java.util.ArrayList(); } /** * Tears down the test fixture. * (Called after every test case method.) */ @After public void tearDown() { emptyList = null; } @Test public void testSomeBehavior() { assertEquals("Empty list should have 0 elements", 0, emptyList.size()); } @Test(expected=IndexOutOfBoundsException.class) public void testForException() { Object o = emptyList.get(0); } }

[top]

How do I write a test for an abstract class?

Refer to http://c2.com/cgi/wiki?AbstractTestCases.

[top]

When are tests garbage collected?

(Submitted by: Timothy Wall and Kent Beck)

By design, the tree of Test instances is built in one pass, then the tests are executed in a second pass. The test runner holds strong references to all Test instances for the duration of the test execution. This means that for a very long test run with many Test instances, none of the tests may be garbage collected until the end of the entire test run.

Therefore, if you allocate external or limited resources in a test, you are responsible for freeing those resources. Explicitly setting an object to in the method, for example, allows it to be garbage collected before the end of the entire test run.

[top]