Test-faster-fix-more

Hultnér Technologies

Hultnér Technologies AB | @ahultner | Blog | Slides, foss-north 2019 | Nordisk Python Community | GitHub


⠠⠵ Test faster, fix more

Property based testing in Python using Hypothesis.

⠠⠵ foss-north, April 8-9, 2019 (Gothenburg)

Test Fast, Fix More – Property based in Python testing with Hypothesis (youtube video comming soon)
Did you ever miss that corner case bug? Maybe it was a negative integer, strange timezone conversion behaviour, off by one error or something entirely else. These subtle bugs are often hard to catch and are easily missed in test cases. You like me have probably ran into plenty of code utilising only happy path testing, only to later discover subtle bugs which are easily fixed once pointed out. This is where property based testing comes into the picture.

In this talk I will focus on a wonderful Python library called Hypothesis but the concepts apply to other languages as well. Hypethesis is based on the same concept as the famous QuickCheck library for Haskell, which in turn have been ported a large number of languages. Hypothesis uses a wide range of input to find edge cases that you could otherwise easily miss, once it finds these cases it narrows down the input to the minimal breaking example to provide failures which are easier to understand.

⠠⠵ Property based testsing/Hypothesis FAQ

What’s the requirements for using Hypothesis?
Install it via pip install hypothesis check out the quickstart guide

Do I need to rewrite all my tests?
Absolutely not, I’d encourage to use property based testing as a complement to your current testing and start experimenting with it on critical parts.

Is this the same thing as a fuzzer?
Not quite but similar concepts, I like to think of property-based testing as structured fuzzing.

⠠⠵ Quick Reference

# Minimal usage example
@given(
    st.integers(), 
    st.integers()
)
def test_ints_are_commutative(x, y):
    assert x + y == y + x
	
# Define special edge cases with @example, 
# especially useful for regression testing
@given(st.integers(), st.integers())
@example(-1, 1)
def test_add(a, b):
    assert add(a, b) == a + b