Mirrors/servo
1
0
Fork 0
mirror of https://github.com/servo/servo.git synced 2025-07-02 04:53:39 +01:00
Code Issues Projects Releases Packages Wiki Activity

Added more information in Mutation Test Readme

Browse source
This commit is contained in:
Sandeep Hegde 2017-11-08 12:58:46 -05:00
parent d24d6ac8ef
commit 84f694dc93
2 changed files with 19 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

28
python/servo/mutation/README.md
View file

@ -5,7 +5,7 @@ The motivation for mutation testing is to test the breadth coverage of tests for
For more info refer [Wiki page](https://en.wikipedia.org/wiki/Mutation_testing).
In this project, mutation testing is used to test the coverage of WPT for Servo's browsing engine.
Here Mutation testing is used to test the coverage of WPT for Servo's browsing engine.
### Mutation Strategy
This version of mutation testing consists of a Python script that finds random uses of && in Servo's code base and replaces them by ||. The expectation from the WPT tests is to catch this mutation and result in failures when executed on the corresponding code base.
@ -16,7 +16,7 @@ The mutation test aims to run only tests which are concerned with the mutant. Th
#### test_mapping.json
The file test_mapping.json is used to map the source code to their corresponding WPT tests. The user must maintain a updated version of this file in the path where mutation testing needs to be performed. Additionally, the test_mapping.json will only consist of maps of source codes that are present in the current directory. Hence, each folder will have a unique test_mapping.json file. Any source code files that may be present in a path but are not mapped to a WPT in test_mapping.json will not be covered for mutation testing.
### Sample test_mapping.json format
#### Sample test_mapping.json format
A sample of test_mapping.json is as shown below:
```
@ -32,14 +32,14 @@ A sample of test_mapping.json is as shown below:
Please ensure that each folder that requires a mutant to be generated consists of test_mapping.json file so that the script can function as expected.
### Basic Execution Flow
The implementation of mutation testing is as follows:
1. The script is called from the command line, it searches through the path entered by user for test_mapping.json.
2. If found, it reads the json file and parses one componenent of source file to generate mutants.
3. The corresponding WPT tests are run for each mutant and the test results are logged.
4. Once all WPT are run for the first source file, the mutation continues for other source files mentioned in the json file and runs their corresponding WPT tests.
5. Once it has completed executing mutation testing for the entered path, it repeats the above procedure for sub-paths present inside the entered path.
If we want to run mutation test for a source path then there should be test_mapping.json in that path and all the subdirectories which has source files.
Eg: There should be test mapping in following folders if we run mutation test on 'components/script' path.
* components/script/test_mapping.json
* components/script/dom/test_mapping.json
* components/script/task_source/test_mapping.json
* components/script/dom/bindings/test_mapping.json
* ...
### Running Mutation test
The mutation tests can be run by running the below command from the servo directory on the command line interface:
@ -52,4 +52,12 @@ Eg. `python python/servo/mutation/init.py components/script/dom`
The CI script for running mutation testing is present in /etc/ci folder. It can be called by executing the below command from the CLI:
`python /etc/ci/mutation_test.py`
`python /etc/ci/mutation_test.py`
### Execution Flow
1. The script is called from the command line, it searches for test_mapping.json in the path entered by user.
2. If found, it reads the json file and parses it, gets source file to tests mapping.
3. If the source file does not have any local changes then it is mutated at a random line.
4. The corresponding WPT tests are run for this mutant and the test results are logged.
5. Once all WPT are run for the first source file, the mutation continues for other source files mentioned in the json file and runs their corresponding WPT tests.
6. Once it has completed executing mutation testing for the entered path, it repeats the above procedure for sub-paths present inside the entered path.

2
python/servo/mutation/test.py
View file

@ -61,4 +61,4 @@ def mutation_test(file_name, tests):
subprocess.call('git checkout {0}'.format(file_name), shell=True)
else:
print "Cannot mutate {0}".format(file_name)
print "-----------------------------------------------------------------\n"
print "-"*80 + "\n"