62 lines
2.7 KiB
Markdown
62 lines
2.7 KiB
Markdown
---
|
|
title: Software Quality & Testing -- Graded Assignment 12
|
|
author: Claudio Maggioni
|
|
geometry: margin=2.5cm,bottom=3cm
|
|
---
|
|
|
|
# Exercise 1 and 2
|
|
|
|
The necessary edits have been performed in the test class `src/test/java/comments/CommentsTest.java`.
|
|
|
|
# Exercise 3
|
|
|
|
I use EvoMaster to generate test cases and find the bug in the source code. I run
|
|
EvoMaster with the following command from the project root folder `comments-api`:
|
|
|
|
```shell
|
|
java --add-opens java.base/java.net=ALL-UNNAMED \
|
|
--add-opens java.base/java.util=ALL-UNNAMED \
|
|
-jar ./testing-tools/evomaster.jar --blackBox true \
|
|
--bbSwaggerUrl http://localhost:8080/docs/swagger.yaml \
|
|
--outputFormat JAVA_JUNIT_4 --maxTime 5m
|
|
```
|
|
|
|
EvoMaster correctly finds a bug in the `GET /comments` route,
|
|
causing a 500 error when the `limit` query parameter is set to `0` and
|
|
the `offset` query parameter is not given.
|
|
|
|
The generated tests are included in the directory `src/test/java/em`.
|
|
|
|
# Exercise 4
|
|
|
|
To run Randoop without a test oracle I run the following commands from the project root folder:
|
|
|
|
```shell
|
|
echo "comments.repository.MapCriticsRepository" > file.txt
|
|
java -classpath testing-tools/randoop-4.3.2/randoop-all-4.3.2.jar:target/classes \
|
|
randoop.main.Main gentests --classlist=file.txt --time-limit=120
|
|
```
|
|
|
|
To run Randoop with the test oracle `jdoctor-oracles.json` I run the following commands from the project root folder:
|
|
|
|
```shell
|
|
echo "comments.repository.MapCriticsRepository" > file.txt
|
|
java -classpath testing-tools/randoop-4.3.2/randoop-all-4.3.2.jar:target/classes \
|
|
randoop.main.Main gentests --classlist=file.txt --time-limit=120 \
|
|
--specifications=jdoctor-oracles.json
|
|
```
|
|
|
|
The generated tests without the oracle are included in the directory `src/test/java/randoopWithoutOracle`, while
|
|
the ones generated with the oracle are included in the directory `src/test/java/randoopWithOracle`.
|
|
|
|
The difference between the test generated with or without oracle lies in the assertions over the
|
|
`MapCriticsRepository.addComment(Comment)` method. The oracle file `jdoctor-oracles.json` specifies that this method
|
|
should throw a `IllegalArgumentException` when its parameter is null. In the tests without oracle, this exception throwing
|
|
is treated as a possible bug and tests that cause it are placed in `ErrorTest*` classes since by not expecting
|
|
the exception they fail. However, in the tests generated with oracles the exception throwing behaviour is correctly tested,
|
|
thus making tests fail when no exception is thrown and the parameter is indeed `null`. Even with the oracle the tests still fail,
|
|
since a `NullPointerException` is thrown instead of the `IllegalArgumentException` expected in the oracle (and the
|
|
documentation of the method), thus uncovering a new bug in `MapCriticsRepository`.
|
|
|
|
|