Test-faster-fix-more

⠠⠵ Test faster, fix more

Property based testing in Python using Hypothesis.

⠠⠵ Quick Reference

# Minimal usage example
@given(
    st.integers(), 
    st.integers()
)
def test_ints_are_commutative(x, y):
    assert x + y == y + x
	
# Define special edge cases with @example, 
# especially useful for regression testing
@given(st.integers(), st.integers())
@example(-1, 1)
def test_add(a, b):
    assert add(a, b) == a + b

⠠⠵ Test & Code Podcast Interview, March 27th, 2020

Property Based Testing in Python with Hypothesis - Alexander Hultnér, Episode 107
Hypothesis is the Python tool used for property based testing. Hypothesis claims to combine “human understanding of your problem domain with machine intelligence to improve the quality of your testing process while spending less time writing tests.”

In this episode Alexander Hultnér introduces us to property based testing in Python with Hypothesis.

Some topics covered:

What is property based testing
Thinking differently for property based testing
Using hypothesis / property based testing in conjunction with normal testing
Failures saved and re-run
What parts of development/testing is best suited for hypothesis / property based testing
Comparing function implementations
Testing against REST APIs that use Open API / Swagger with schemathesis
Changing the number of tests in different test environments
System, integration, end to end, and unit tests

⠠⠵ PyCon Sweden, Oct 31 - Nov 1, 2019 (Stockholm)

Did you ever miss that corner case bug? Maybe it was a negative integer, strange timezone conversion behaviour, off by one error or something entirely else. These subtle bugs are often hard to catch and are easily missed in test cases. You like me have probably ran into plenty of code utilising only happy path testing, only to later discover subtle bugs which are easily fixed once pointed out. This is where property based testing comes into the picture.

In this talk I will focus on a wonderful Python library called Hypothesis but the concepts apply to other languages as well. Hypothesis is based on the same concept as the famous QuickCheck library for Haskell, which in turn have been ported a large number of languages. Hypothesis uses a wide range of input to find edge cases that you could otherwise easily miss, once it finds these cases it narrows down the input to the minimal breaking example to provide failures which are easier to understand.

Audience level: Intermediate

Speaker: Alexander Hultnér

Video
Slides

⠠⠵ foss-north, April 8-9, 2019 (Gothenburg)

Test Fast, Fix More – Property based in Python testing with Hypothesis (youtube video)
Did you ever miss that corner case bug? Maybe it was a negative integer, strange timezone conversion behaviour, off by one error or something entirely else. These subtle bugs are often hard to catch and are easily missed in test cases. You like me have probably ran into plenty of code utilising only happy path testing, only to later discover subtle bugs which are easily fixed once pointed out. This is where property based testing comes into the picture.

In this talk I will focus on a wonderful Python library called Hypothesis but the concepts apply to other languages as well. Hypoethesis is based on the same concept as the famous QuickCheck library for Haskell, which in turn have been ported a large number of languages. Hypothesis uses a wide range of input to find edge cases that you could otherwise easily miss, once it finds these cases it narrows down the input to the minimal breaking example to provide failures which are easier to understand.

⠠⠵ Property based testsing/Hypothesis FAQ

What’s the requirements for using Hypothesis?
Install it via pip install hypothesis check out the quickstart guide

Do I need to rewrite all my tests?
Absolutely not, I’d encourage to use property based testing as a complement to your current testing and start experimenting with it on critical parts.

Is this the same thing as a fuzzer?
Not quite but similar concepts, I like to think of property-based testing as structured fuzzing.