The Java Explorer

Tips and insights on Java

  • Subscribe

  • If you find this blog useful, please enter your email address to subscribe and receive notifications of new posts by email.

    Join 37 other followers

Posts Tagged ‘junit’

A utility for multithreaded unit tests

Posted by Eyal Schneider on July 13, 2010

Classes designed to run in a multithreaded environment are difficult to test. In a standard unit test, the test space may be already huge, since we try to cover all combinations of the object’s state and legal operations on that state. For a class designed to run in a multithreaded environment, the test space increases dramatically, due to the possible interleaving of the operations performed by the different threads on the object.

Beside the theoretic challenge, we have a practical one, that arises when simulating multiple caller threads in the test class, and trying to perform assertions in these threads. This can be demonstrated by the following numeric counter use case.  Suppose that we want to implement a thread safe counter. We will provide two implementations, so we start by defining the counter interface, consisting of a single atomic operation:


public interface Counter {
    public int incrementAndGet();
}

Following are two implementations. The first works fine only for a single threaded scenario:


public class DumbCounter implements Counter {
    private int counter;
    @Override
    public int incrementAndGet() {
        return ++counter;
    }
}

The second one is based on AtomicInteger, so it is completely thread safe:

public class AtomicIntCounter implements Counter {
    private AtomicInteger atomicInt = new AtomicInteger();
    @Override
    public int incrementAndGet() {
        return atomicInt.incrementAndGet();
    }
}

Now we want to test these implementations using JUnit. The following test simulates 5 threads operating on the same counter. Each thread tries to increment the counter 100000 times, so we expect each thread to see strictly increasing values of the counter:

 import static org.junit.Assert.assertTrue;

import java.util.concurrent.CountDownLatch;

import org.junit.Before;
import org.junit.Test;
public class TestCounters {
    private static final int THREADS_COUNT = 5;
    private Counter counter;

    @Before
    public void resetCounter() {
        counter = new DumbCounter();
    }

    @Test
    public void test() throws InterruptedException {
        final CountDownLatch latch = new CountDownLatch(THREADS_COUNT);
        for(int i = 0;i < THREADS_COUNT; i++) {
            new Thread(new Runnable(){

                @Override
                public void run() {
                    try{
                        int last = counter.incrementAndGet();
                        for (int i = 0; i < 1000000; i++) {
                            int value = counter.incrementAndGet();
                            assertTrue (value > last);
                            last = value;
                        }
                    } finally {
                        latch.countDown();
                    }
                }
            }).start();
        }

        latch.await();
    }

}

The CountDownLatch is used for waiting for all threads to terminate, before leaving the test method. The method resetCounter() is used for initializing the counter before any test method is being executed. It also chooses the particular implementation we want to test. In this case we test the non-thread-safe DumbCounter, so we expect to see the test fail. However, when we run the test we observe a strange behavior:  JUnit reports that the test passed successfully, but we see the following error messages printed to the error stream by each one of the 5 threads:

Exception in thread “Thread-{N}” java.lang.AssertionError:
at org.junit.Assert.fail(Assert.java:71)
at org.junit.Assert.assertTrue(Assert.java:34)
at org.junit.Assert.assertTrue(Assert.java:43)
at jnunit.TestCounters$1.run(TestCounters.java:31)
at java.lang.Thread.run(Thread.java:619)

The reason for this behavior is the usage of assertions outside of JUnit’s main thread. JUnit detects assertion failures by catching exceptions of type AssertionError thrown in its main thread. The framework is completely unaware of assertion errors being thrown in new threads spawned by the test methods. Consequently, these exceptions simply terminate the threads where they are thrown, and activate the default uncaught exception handler, which prints their details to the error stream. The main thread detects no errors, so it reports success.

In order to solve this situation, we need some asynchronous test runner, that delegates any unchecked exceptions to the main thread. Here is a possible solution : 


public class AsynchTester{
    private Thread thread;
    private volatile Error error;
    private volatile RuntimeException runtimeExc;

    public AsynchTester(final Runnable runnable) {
        thread = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    runnable.run();
                } catch (Error e) {
                    error = e;
                } catch (RuntimeException e) {
                    runtimeExc = e;
                }
            }
        });
    }

    public void start() {
        thread.start();
    }

    public void test() throws InterruptedException {
        thread.join();
        if (error != null)
            throw error;
        if (runtimeExc != null)
            throw runtimeExc;
    }
}

The constructor receives the task to be executed, and start() triggers the asynchronous execution. The method test() is used for waiting the task termination, and throwing unchecked exceptions if needed. Any RuntimeException or Error thrown by the test thread will be re-thrown by the test() method, in order to reflect the actual termination status of the inner thread.

Now we can re-write the test() method of our test class as follows:

@Test
public void test() throws InterruptedException {
    AsynchTester[] testers = new AsynchTester[THREADS_COUNT];
    for(int i = 0;i < THREADS_COUNT; i++) {
        testers[i] = new AsynchTester(new Runnable() {

            @Override
            public void run() {
                int last = counter.incrementAndGet();
                for (int i = 0; i < 1000000; i++) {
                    int value = counter.incrementAndGet();
                    assertTrue (value > last);
                    last = value;
                }
            }
        });
        testers[i].start();
    }

    for(AsynchTester tester : testers)
        tester.test();
}

The method structure is very similar to the original one, and it’s quite easy to synchronize the threads termination now. When testing DumbCounter using this code, we finally get the right behavior: the test fails, and points us to the position in code where the assertion failed.

It is important to note that the solution presented here addresses only one aspect of multithreaded tests, and I chose it because it is both easy to implement and uses a nice technique. However, there are some available test frameworks on the web, suitable for concurrent tests, that cover other aspects of the problem. See for example the GroboUtils JUnit extension, and also ConTest, which uses an interesting approach for increasing the coverage of multithreaded tests.

Posted in java, Testing | Tagged: , , , , , , , | 3 Comments »