Property-Based Testing with jqwik: A Practical Guide
A guide how to write property-based tests in Java: theory and practice
Introduction
As software grows more complex, traditional example-based testing often fails. It can’t find edge cases and unexpected behaviors. Property-based testing (PBT) is a powerful alternative. It generates test cases based on properties that your code should meet without any manual input. This article will explore property-based testing using jqwik. It’s a modern property-based testing library for Java.
What is property — based testing?
Property-based testing is a method. Instead of writing specific test cases, you define properties. Your code must meet them in all cases. The testing framework then generates many test cases to verify these properties.
For example, instead of writing
@Test
void testReverse() {
assertEquals("cba", StringUtils.reverse("abc"));
}
You would define a property:
@Property
void reverseStringTwiceShouldGiveOriginalString(@ForAll String input) {
assertEquals(input, StringUtils.reverse(StringUtils.reverse(input)));
}When to apply property — based testing
Before diving into property-based testing, we must know when it will help the most. Like mutation testing, property-based testing has limited use with low test coverage. It might not be worth the investment in some cases.
Property-based testing is best when you have a solid foundation of unit tests with 80% coverage or higher. This baseline ensures you’ve handled the basic scenarios and edge cases in your code. Property-based testing helps you find edge cases your tests might have missed.
Here are scenarios where property-based testing proves particularly valuable:
Code that runs algorithms or transforms data has functions. They should have properties that hold true for any input.
For instance, property-based testing is great for:
- sorting algorithms
- mathematical computations
- string manipulations
Property-based testing can check invariants in data structures, like lists and trees. It can also check custom collections. These invariants should hold true across all states and transitions.
Stateless business logic: use pure functions. They should transform data without side effects. The lack of state makes it easier to reason about properties and generate more meaningful test cases.
Yet, property-based testing might not be the best fit for:
UI components: user interfaces need scenario-based testing, not property verification.
Heavy I/O operations: code that mainly handles databases, file systems, or network calls might perform better with traditional integration tests.
Simple CRUD operations: basic create, read, update, delete. They often don’t have complex properties to verify beyond basic validation.
Remember that you should use property-based testing to complement, not replace, your existing test suite. It’s an extra tool in your testing arsenal that becomes more powerful as your codebase matures and your test coverage improves.
Getting starter with jqwik
First, add jqwik to your project dependencies:
<dependency>
<groupId>net.jqwik</groupId>
<artifactId>jqwik</artifactId>
<version>1.7.3</version>
<scope>test</scope>
</dependency>Basic property example
Let’s start with a simple example: testing a calculator’s addition function.
class CalculatorProperties {
private Calculator calculator = new Calculator();
@Property
void additionShouldBeCommutative(@ForAll int a, @ForAll int b) {
assertEquals(
calculator.add(a, b),
calculator.add(b, a)
);
}
@Property
void additionWithZeroIsIdentity(@ForAll int a) {
assertEquals(a, calculator.add(a, 0));
assertEquals(a, calculator.add(0, a));
}
}Configuring generated values
You can configure how jqwik generates values.
@Property
void absoluteValueIsAlwaysPositive(
@ForAll @IntRange(min = -1000, max = 1000) int number
) {
assertTrue(Math.abs(number) >= 0);
}
@Property
void stringConcatenationLength(
@ForAll @StringLength(min = 0, max = 50) String str1,
@ForAll @StringLength(min = 0, max = 50) String str2
) {
assertEquals(
str1.length() + str2.length(),
(str1 + str2).length()
);
}Custom generators
Sometimes you need to generate complex domain objects. jqwik provides ways to create custom generators:
class UserProperties {
@Provide
Arbitrary<User> users() {
Arbitrary<String> names = Arbitraries.strings()
.withCharRange('a', 'z')
.ofMinLength(2).ofMaxLength(20);
Arbitrary<Integer> ages = Arbitraries.integers()
.between(18, 100);
return Combinators.combine(names, ages)
.as((name, age) -> new User(name, age));
}
@Property
void userAgeShouldBeValid(@ForAll("users") User user) {
assertTrue(user.getAge() >= 18);
assertTrue(user.getAge() <= 100);
}
}Assumptions and filtering
Sometimes you need to test properties that only hold under certain conditions:
@Property
void squareRootOfSquareIsOriginalNumberForPositives(
@ForAll @Positive double number
) {
double square = number * number;
double sqrt = Math.sqrt(square);
assertEquals(number, sqrt, 0.01);
}
@Property
void divisionProperties(@ForAll int a, @ForAll int b) {
Assume.that(b != 0);
int result = a / b;
int remainder = a % b;
assertEquals(a, (b * result) + remainder);
}Stateful testing
jqwik also supports testing stateful systems by generating sequences of actions:
class StackProperties {
@Property
void stackProperties(@ForAll("stackActions") List<StackAction> actions) {
Stack<Integer> stack = new Stack<>();
actions.forEach(action -> action.run(stack));
}
@Provide
Arbitrary<List<StackAction>> stackActions() {
Arbitrary<StackAction> pushAction = Arbitraries.integers()
.map(i -> new PushAction(i));
Arbitrary<StackAction> popAction = Arbitraries.constant(new PopAction());
return Arbitraries.sequences(Arbitraries.oneOf(pushAction, popAction));
}
}
interface StackAction {
void run(Stack<Integer> stack);
}
class PushAction implements StackAction {
private final int value;
PushAction(int value) {
this.value = value;
}
@Override
void run(Stack<Integer> stack) {
stack.push(value);
}
}
class PopAction implements StackAction {
@Override
void run(Stack<Integer> stack) {
if (!stack.isEmpty()) {
stack.pop();
}
}
}Common pitfalls
Property-based testing is powerful. But, you may face several challenges. Understanding these issues can help you avoid or fix them. One of the most frequent problems is creating properties that lack simplicity. Remember that a property should be simple enough that you can explain it to a colleague in one sentence. If you find yourself writing long properties with many conditions, it’s a sign to break them into smaller, more focused ones.
Performance can become a concern when working with property-based tests. These tests run many times with different inputs. So, expensive operations or large generation ranges can slow your test suite. During development, use smaller ranges or fewer iterations. Save more testing for your CI pipeline.
Test flakiness is particularly frustrating in property-based testing. A flaky test might pass sometimes and fail other times, even with the same inputs. This usually happens when your properties depend on external state or non-deterministic operations. Always ensure your properties are deterministic. With the same inputs, they should always produce the same results.
Finally, be cautious about adding too many assumptions to your properties. Assumptions help focus your test on valid inputs. But too many constraints can stop the test framework from generating enough good test cases. If you find yourself adding many assumptions to a property, it might be a sign that you need to rethink either your property or your design.
Conclusion
Property-based testing with jqwik offers a powerful way to verify your code’s behavior across a wide range of inputs. It needs a different mindset than example-based testing. But it has great benefits. It improves test coverage and finds edge cases. So it is a valuable addition to your testing toolkit.
Remember that property-based testing complements rather than replaces traditional unit testing. Use both approaches to create a comprehensive test suite that gives you confidence in your code’s correctness.