allpairspy

Pairwise test combinations generator

These details have not been verified by PyPI

Project links

GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Project description

AllPairs test combinations generator

AllPairs is an open source test combinations generator written in Python, developed and maintained by MetaCommunications Engineering. The generator allows one to create a set of tests using “pairwise combinations” method, reducing a number of combinations of variables into a lesser set that covers most situations.

For more info on pairwise testing see http://www.pairwise.org.

Features

Produces good enough dataset.
Pythonic, iterator-style enumeration interface.
Allows to filter out “invalid” combinations during search for the next combination.
Goes beyond pairs! If/when required can generate n-wise combinations.

Get Started

Basic Usage

Sample Code:

from allpairspy import AllPairs

parameters = [
    ["Brand X", "Brand Y"],
    ["98", "NT", "2000", "XP"],
    ["Internal", "Modem"],
    ["Salaried", "Hourly", "Part-Time", "Contr."],
    [6, 10, 15, 30, 60],
]

print("PAIRWISE:")
for i, pairs in enumerate(AllPairs(parameters)):
    print("{:2d}: {}".format(i, pairs))

Output:

PAIRWISE:
 0: ['Brand X', '98', 'Internal', 'Salaried', 6]
 1: ['Brand Y', 'NT', 'Modem', 'Hourly', 6]
 2: ['Brand Y', '2000', 'Internal', 'Part-Time', 10]
 3: ['Brand X', 'XP', 'Modem', 'Contr.', 10]
 4: ['Brand X', '2000', 'Modem', 'Part-Time', 15]
 5: ['Brand Y', 'XP', 'Internal', 'Hourly', 15]
 6: ['Brand Y', '98', 'Modem', 'Salaried', 30]
 7: ['Brand X', 'NT', 'Internal', 'Contr.', 30]
 8: ['Brand X', '98', 'Internal', 'Hourly', 60]
 9: ['Brand Y', '2000', 'Modem', 'Contr.', 60]
10: ['Brand Y', 'NT', 'Modem', 'Salaried', 60]
11: ['Brand Y', 'XP', 'Modem', 'Part-Time', 60]
12: ['Brand Y', '2000', 'Modem', 'Hourly', 30]
13: ['Brand Y', '98', 'Modem', 'Contr.', 15]
14: ['Brand Y', 'XP', 'Modem', 'Salaried', 15]
15: ['Brand Y', 'NT', 'Modem', 'Part-Time', 15]
16: ['Brand Y', 'XP', 'Modem', 'Part-Time', 30]
17: ['Brand Y', '98', 'Modem', 'Part-Time', 6]
18: ['Brand Y', '2000', 'Modem', 'Salaried', 6]
19: ['Brand Y', '98', 'Modem', 'Salaried', 10]
20: ['Brand Y', 'XP', 'Modem', 'Contr.', 6]
21: ['Brand Y', 'NT', 'Modem', 'Hourly', 10]

Filtering

You can restrict pairs by setting a filtering function to filter_func at AllPairs constructor.

Sample Code:

from allpairspy import AllPairs

def is_valid_combination(row):
    """
    This is a filtering function. Filtering functions should return True
    if combination is valid and False otherwise.

    Test row that is passed here can be incomplete.
    To prevent search for unnecessary items filtering function
    is executed with found subset of data to validate it.
    """

    n = len(row)

    if n > 1:
        # Brand Y does not support Windows 98
        if "98" == row[1] and "Brand Y" == row[0]:
            return False

        # Brand X does not work with XP
        if "XP" == row[1] and "Brand X" == row[0]:
            return False

    if n > 4:
        # Contractors are billed in 30 min increments
        if "Contr." == row[3] and row[4] < 30:
            return False

    return True

parameters = [
    ["Brand X", "Brand Y"],
    ["98", "NT", "2000", "XP"],
    ["Internal", "Modem"],
    ["Salaried", "Hourly", "Part-Time", "Contr."],
    [6, 10, 15, 30, 60]
]

print("PAIRWISE:")
for i, pairs in enumerate(AllPairs(parameters, filter_func=is_valid_combination)):
    print("{:2d}: {}".format(i, pairs))

Output:

PAIRWISE:
 0: ['Brand X', '98', 'Internal', 'Salaried', 6]
 1: ['Brand Y', 'NT', 'Modem', 'Hourly', 6]
 2: ['Brand Y', '2000', 'Internal', 'Part-Time', 10]
 3: ['Brand X', '2000', 'Modem', 'Contr.', 30]
 4: ['Brand X', 'NT', 'Internal', 'Contr.', 60]
 5: ['Brand Y', 'XP', 'Modem', 'Salaried', 60]
 6: ['Brand X', '98', 'Modem', 'Part-Time', 15]
 7: ['Brand Y', 'XP', 'Internal', 'Hourly', 15]
 8: ['Brand Y', 'NT', 'Internal', 'Part-Time', 30]
 9: ['Brand X', '2000', 'Modem', 'Hourly', 10]
10: ['Brand Y', 'XP', 'Modem', 'Contr.', 30]
11: ['Brand Y', '2000', 'Modem', 'Salaried', 15]
12: ['Brand Y', 'NT', 'Modem', 'Salaried', 10]
13: ['Brand Y', 'XP', 'Modem', 'Part-Time', 6]
14: ['Brand Y', '2000', 'Modem', 'Contr.', 60]

Data Source: OrderedDict

You can use collections.OrderedDict instance as an argument for AllPairs constructor. Pairs will be returned as collections.namedtuple instances.

Sample Code:

from collections import OrderedDict
from allpairspy import AllPairs

parameters = OrderedDict({
    "brand": ["Brand X", "Brand Y"],
    "os": ["98", "NT", "2000", "XP"],
    "minute": [15, 30, 60],
})

print("PAIRWISE:")
for i, pairs in enumerate(AllPairs(parameters)):
    print("{:2d}: {}".format(i, pairs))

Sample Code:

PAIRWISE:
 0: Pairs(brand='Brand X', os='98', minute=15)
 1: Pairs(brand='Brand Y', os='NT', minute=15)
 2: Pairs(brand='Brand Y', os='2000', minute=30)
 3: Pairs(brand='Brand X', os='XP', minute=30)
 4: Pairs(brand='Brand X', os='2000', minute=60)
 5: Pairs(brand='Brand Y', os='XP', minute=60)
 6: Pairs(brand='Brand Y', os='98', minute=60)
 7: Pairs(brand='Brand X', os='NT', minute=60)
 8: Pairs(brand='Brand X', os='NT', minute=30)
 9: Pairs(brand='Brand X', os='98', minute=30)
10: Pairs(brand='Brand X', os='XP', minute=15)
11: Pairs(brand='Brand X', os='2000', minute=15)

Parameterized testing pairwise by using pytest

Parameterized testing: value matrix

Sample Code:

import pytest
from allpairspy import AllPairs

def function_to_be_tested(brand, operating_system, minute) -> bool:
    # do something
    return True

class TestParameterized(object):
    @pytest.mark.parametrize(["brand", "operating_system", "minute"], [
        values for values in AllPairs([
            ["Brand X", "Brand Y"],
            ["98", "NT", "2000", "XP"],
            [10, 15, 30, 60]
        ])
    ])
    def test(self, brand, operating_system, minute):
        assert function_to_be_tested(brand, operating_system, minute)

Output:

$ py.test test_parameterize.py -v
============================= test session starts ==============================
...
collected 16 items

test_parameterize.py::TestParameterized::test[Brand X-98-10] PASSED      [  6%]
test_parameterize.py::TestParameterized::test[Brand Y-NT-10] PASSED      [ 12%]
test_parameterize.py::TestParameterized::test[Brand Y-2000-15] PASSED    [ 18%]
test_parameterize.py::TestParameterized::test[Brand X-XP-15] PASSED      [ 25%]
test_parameterize.py::TestParameterized::test[Brand X-2000-30] PASSED    [ 31%]
test_parameterize.py::TestParameterized::test[Brand Y-XP-30] PASSED      [ 37%]
test_parameterize.py::TestParameterized::test[Brand Y-98-60] PASSED      [ 43%]
test_parameterize.py::TestParameterized::test[Brand X-NT-60] PASSED      [ 50%]
test_parameterize.py::TestParameterized::test[Brand X-NT-30] PASSED      [ 56%]
test_parameterize.py::TestParameterized::test[Brand X-98-30] PASSED      [ 62%]
test_parameterize.py::TestParameterized::test[Brand X-XP-60] PASSED      [ 68%]
test_parameterize.py::TestParameterized::test[Brand X-2000-60] PASSED    [ 75%]
test_parameterize.py::TestParameterized::test[Brand X-2000-10] PASSED    [ 81%]
test_parameterize.py::TestParameterized::test[Brand X-XP-10] PASSED      [ 87%]
test_parameterize.py::TestParameterized::test[Brand X-98-15] PASSED      [ 93%]
test_parameterize.py::TestParameterized::test[Brand X-NT-15] PASSED      [100%]

Parameterized testing: OrderedDict

Sample Code:

import pytest
from allpairspy import AllPairs

def function_to_be_tested(brand, operating_system, minute) -> bool:
    # do something
    return True

class TestParameterized(object):
    @pytest.mark.parametrize(
        ["pair"],
        [
            [pair]
            for pair in AllPairs(
                OrderedDict(
                    {
                        "brand": ["Brand X", "Brand Y"],
                        "operating_system": ["98", "NT", "2000", "XP"],
                        "minute": [10, 15, 30, 60],
                    }
                )
            )
        ],
    )
    def test(self, pair):
        assert function_to_be_tested(pair.brand, pair.operating_system, pair.minute)

Other Examples

Other examples could be found in examples directory.

Installation

Installation: pip

pip install allpairspy

Installation: apt

You can install the package by apt via a Personal Package Archive (PPA):

sudo add-apt-repository ppa:thombashi/ppa
sudo apt update
sudo apt install python3-allpairspy

Known issues

Not optimal - there are tools that can create smaller set covering all the pairs. However, they are missing some other important features and/or do not integrate well with Python.
Lousy written filtering function may lead to full permutation of parameters.
Version 2.0 has become slower (a side-effect of introducing ability to produce n-wise combinations).

Dependencies

Python 3.7+ no external dependencies.

Project details

These details have not been verified by PyPI

Project links

GitHub Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

Release history Release notifications | RSS feed

This version

2.5.1

Jul 8, 2023

2.5.0

May 11, 2019

2.4.3

Jan 3, 2019

2.4.2

Dec 29, 2018

2.4.1

Aug 19, 2018

2.4.0

Nov 11, 2017

2.3.1

Oct 21, 2017

2.3.0

May 20, 2017

2.2.0

May 14, 2017

2.1.0

Jan 21, 2017

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

allpairspy-2.5.1.tar.gz (14.3 kB view hashes)

Uploaded Jul 8, 2023 Source

Built Distribution

allpairspy-2.5.1-py3-none-any.whl (9.0 kB view hashes)

Uploaded Jul 8, 2023 Python 3

Hashes for allpairspy-2.5.1.tar.gz

Hashes for allpairspy-2.5.1.tar.gz
Algorithm	Hash digest
SHA256	`f69d31a3b56eee119d1ec6063e9c732dd44fbba352ef738cb22d9699fc4009fe`
MD5	`3a57a1abc7c031d57339f1f2a0b66350`
BLAKE2b-256	`ea9ea5a536c22f56cdbbb0236889cff33915e20fb93a0526887367b38e91122e`

Hashes for allpairspy-2.5.1-py3-none-any.whl

Hashes for allpairspy-2.5.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`3f97cbac2bbee86f4bfceffd0fd91dca740a120c78e9368af2bad81a2b3426e2`
MD5	`3542bc84d238031b932e4eff85701cfa`
BLAKE2b-256	`5a1afedf0f2ff77ec65e599112b2d5402fee5c166d9378b51760b696bd8cde1d`