Julia Language Unit Testing Writing a Test Set

Example

if VERSION ≥ v"0.5.0-dev+7720"
    using Base.Test
else
    using BaseTestNext
    const Test = BaseTestNext
end

It is helpful to group related @tests together in a test set. In addition to clearer test organization, test sets offer better output and more customizability.

To define a test set, simply wrap any number of @tests with a @testset block:

@testset "+" begin
    @test 1 + 1 == 2
    @test 2 + 2 == 4
end

@testset "*" begin
    @test 1 * 1 == 1
    @test 2 * 2 == 4
end

Running these test sets prints the following output:

Test Summary: | Pass  Total
  +           |    2      2

Test Summary: | Pass  Total
  *           |    2      2

Even if a test set contains a failing test, the entire test set will be run to completion, and the failures will be recorded and reported:

@testset "-" begin
    @test 1 - 1 == 0
    @test 2 - 2 == 1
    @test 3 - () == 3
    @test 4 - 4 == 0
end

Running this test set results in

-: Test Failed
  Expression: 2 - 2 == 1
   Evaluated: 0 == 1
 in record(::Base.Test.DefaultTestSet, ::Base.Test.Fail) at ./test.jl:428
    ...
-: Error During Test
  Test threw an exception of type MethodError
  Expression: 3 - () == 3
  MethodError: no method matching -(::Int64, ::Tuple{})
    ...
Test Summary: | Pass  Fail  Error  Total
  -           |    2     1      1      4
ERROR: Some tests did not pass: 2 passed, 1 failed, 1 errored, 0 broken.
    ...

Test sets can be nested, allowing for arbitrarily deep organization

@testset "Int" begin
    @testset "+" begin
        @test 1 + 1 == 2
        @test 2 + 2 == 4
    end
    @testset "-" begin
        @test 1 - 1 == 0
    end
end

If the tests pass, then this will only show the results for the outermost test set:

Test Summary: | Pass  Total
  Int         |    3      3

But if the tests fail, then a drill-down into the exact test set and test causing the failure is reported.

The @testset macro can be used with a for loop to create many test sets at once:

@testset for i in 1:5
    @test 2i == i + i
    @test i^2 == i * i
    @test i ÷ i == 1
end

which reports

Test Summary: | Pass  Total
  i = 1       |    3      3
Test Summary: | Pass  Total
  i = 2       |    3      3
Test Summary: | Pass  Total
  i = 3       |    3      3
Test Summary: | Pass  Total
  i = 4       |    3      3
Test Summary: | Pass  Total
  i = 5       |    3      3

A common structure is to have outer test sets test components or types. Within these outer test sets, inner test sets test behaviour. For instance, suppose we created a type UniversalSet with a singleton instance that contains everything. Before we even implement the type, we can use test-driven development principles and implement the tests:

@testset "UniversalSet" begin
    U = UniversalSet.instance
    @testset "egal/equal" begin
        @test U === U
        @test U == U
    end

    @testset "in" begin
        @test 1 in U
        @test "Hello World" in U
        @test Int in U
        @test U in U
    end

    @testset "subset" begin
        @test Set() ⊆ U
        @test Set(["Hello World"]) ⊆ U
        @test Set(1:10) ⊆ U
        @test Set([:a, 2.0, "w", Set()]) ⊆ U
        @test U ⊆ U
    end
end

We can then start implementing our functionality until it passes our tests. The first step is to define the type:

immutable UniversalSet <: Base.AbstractSet end

Only two of our tests pass right now. We can implement in:

immutable UniversalSet <: Base.AbstractSet end
Base.in(x, ::UniversalSet) = true

This also makes some of our subset tests to pass. However, the issubset (⊆) fallback doesn't work for UniversalSet, because the fallback tries to iterate over elements, which we can't do. We can simply define a specialization that makes issubset return true for any set:

immutable UniversalSet <: Base.AbstractSet end
Base.in(x, ::UniversalSet) = true
Base.issubset(x::Base.AbstractSet, ::UniversalSet) = true

And now, all our tests pass!

PDF - Download Julia Language for free