63 lines
1.8 KiB
Markdown
63 lines
1.8 KiB
Markdown
# Information Modelling & Analysis: Project 2
|
|
|
|
Student: Claudio Maggioni
|
|
|
|
Please follow the instructions provided in the project slides
|
|
and consider the submission instructions available on iCorsi.
|
|
|
|
For your convencience, I the following resources are available in the `resources` folder:
|
|
- **defects4j-checkout-closure-1f**: The output of the command `defects4j checkout -p Closure -v 1f -w ...`
|
|
- **modified_classes** The list of buggy classes in: `framework/projects/Closure/modified_classes/`
|
|
|
|
## Setup
|
|
|
|
To install the required libraries run:
|
|
|
|
```shell
|
|
python3 -m venv .env
|
|
source .env/bin/activate
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Data pre-processing
|
|
|
|
To extract and label the feature vectors run:
|
|
|
|
```shell
|
|
python3 ./extract_feature_vectors.py
|
|
python3 ./label_feature_vectors.py
|
|
```
|
|
|
|
The labeled feature vectors are stored in file
|
|
`./metrics/feature_vectors_labeled.csv` from the repository root.
|
|
|
|
## Training
|
|
|
|
To train the classifiers with the grid search procedure defined in the report
|
|
to later extract the optimal combination of hyperparameters run:
|
|
|
|
```shell
|
|
python3 ./train_classifiers.py
|
|
```
|
|
|
|
and answer `y` to run again training when prompted. Answering `n` simply
|
|
computes again data about the best hyperparameter configuration from the metrics
|
|
produced by a previous training.
|
|
|
|
Raw cross validation training metrics are stored in `./models/models.csv`. The
|
|
optimal hyperparameter configurations found are stored in `./models/best.csv`.
|
|
|
|
## Evaluation
|
|
|
|
To run the 20-times 5-fold cross validation procedure delete the file
|
|
`./models/evaluation.csv` and run:
|
|
|
|
```shell
|
|
python3 ./evaluate_classifiers.py
|
|
```
|
|
|
|
Raw data from the repeated cross validation procedure is stored in
|
|
`./models/evaluation.csv`. P-values for each metric of each classifier pair are
|
|
stored in `./models/model_stats.csv`.
|
|
|