Project 02 - Python test generator

About the Project

This project has the goal of writing a search based automated test generator for Python. It is part of the Knowledge Search & Extraction - 2023 course from the Università della Svizzera italiana.

In this repository, you can find the following files:

• benchmark/ folder: which contains the benchmark of functions under test to be instrumented

Note: Feel free to modify this file according to the project's necessities.

Environment setup

To install the required dependencies the Python version manager pyenv must be installed and in $PATH.

To set up a Python 3.11 virtualenv to execute parts 1, 2, and 3 of the project run:

deactivate || true  # deactivate existing environment
pyenv install -s 3.11
pyenv shell 3.11
python3.11 -m venv env

source env/bin/activate
pip3.11 install -r requirements.txt

To set up Python 3.7 (last version supported by mut.py) to execute part 4 of the project run:

deactivate || true  # deactivate existing environment
pyenv install -s 3.7
pyenv shell 3.7
python3.7 -m venv env37
source env37/bin/activate

pip3.7 install MutPy==0.6.1
pip3.7 install -r requirements.txt

Instrumentation (Part 1)

To generate the instrumented code for all the files in the benchmark run the command:

# Reset Python to latest (system) version
deactivate || true
pyenv shell 3.11
source env/bin/activate

python3.11 ./instrument.py

The generated files are created in the directory instrumented. Each file name matches the file name of the corresponding source file in benchmark.

Test case generation (Part 2 and Part 3)

To generate test cases for all files in the benchmark run the command:

# Reset Python to latest (system) version
deactivate || true
pyenv shell 3.11
source env/bin/activate

python3.11 ./genetic.py

The test suite is created in the directory tests. One test file is generated for each file present in the benchmark directory. Run the command with the -h options for more details on partial generation.

The test suite can be then executed over the benchmark code with the command:

# Reset Python to latest (system) version
deactivate || true
pyenv shell 3.11
source env/bin/activate

python3.11 -m unittest discover tests

Mutation testing (Part 4)

To run mut.py use Python 3.7 and run:

# Reset Python to 3.7 version
deactivate || true
pyenv shell 3.7
source env37/bin/activate

python3.7 muttest.py