Commit 529e7714 authored by Celine Noirot's avatar Celine Noirot
Browse files

Merge branch 'master' into 'dev'

# Conflicts:
#   README.md
#   docs/README.md
#   docs/installation.md
#   docs/usage.md
#   functional_tests/README.md
#   functional_tests/functions.py
#   functional_tests/launch_example.sh
#   functional_tests/main.py
#   main.nf
#   nextflow.config
parents 2b23244c 54210901
Pipeline #48015 skipped with stage
...@@ -45,7 +45,7 @@ A report html file is generated at the end of the workflow with [MultiQC](https: ...@@ -45,7 +45,7 @@ A report html file is generated at the end of the workflow with [MultiQC](https:
The pipeline is built using [Nextflow,](https://www.nextflow.io/docs/latest/index.html#) a bioinformatics workflow tool to run tasks across multiple compute infrastructures in a very portable manner. The pipeline is built using [Nextflow,](https://www.nextflow.io/docs/latest/index.html#) a bioinformatics workflow tool to run tasks across multiple compute infrastructures in a very portable manner.
Two [Singularity](https://sylabs.io/docs/) containers are available making installation trivial and results highly reproducible. Three [Singularity](https://sylabs.io/docs/) containers are available making installation trivial and results highly reproducible.
## Documentation ## Documentation
......
...@@ -71,6 +71,20 @@ two files (`metagwgs.sif` and `eggnog_mapper.sif`) must have been downloaded. ...@@ -71,6 +71,20 @@ two files (`metagwgs.sif` and `eggnog_mapper.sif`) must have been downloaded.
At the end of the build, two files (`metagwgs.sif` and `eggnog_mapper.sif`) must have been generated. At the end of the build, two files (`metagwgs.sif` and `eggnog_mapper.sif`) must have been generated.
**WARNING:** to ensure Nextflow can find the _.sif_ files, we encourage you to change the _nextflow.config_ file in metagWGS to contain these lines:
```
process {
container = '<PATH>/metagwgs.sif'
withLabel: eggnog {
container = '<PATH>/eggnog_mapper.sif'
}
withLabel: mosdepth {
container = '<PATH>/mosdepth.sif'
}
}
```
Where \<PATH\> leads to the directory where the singularity images are built/downloaded.
## V. Use metagWGS ## V. Use metagWGS
See the [Usage](https://forgemia.inra.fr/genotoul-bioinfo/metagwgs/-/blob/master/docs/usage.md) documentation to know how to run metagWGS. See the [Usage](https://forgemia.inra.fr/genotoul-bioinfo/metagwgs/-/blob/master/docs/usage.md) documentation to know how to run metagWGS.
...@@ -62,9 +62,9 @@ python $METAG_PATH/functional_tests/main.py -step 07_taxo_affi -exp_dir $DATASET ...@@ -62,9 +62,9 @@ python $METAG_PATH/functional_tests/main.py -step 07_taxo_affi -exp_dir $DATASET
## III. Output ## III. Output
A ft_\[step\].log file is created for each step of metagwgs. It contains information about each test performed on given files. A `ft_[STEP].log` file is created for each step of metagwgs. It contains information about each test performed on given files.
Exemple with ft_01_clean_qc.log: Exemple with `ft_01_clean_qc.log`:
``` ```
Expected directory: metagwgs-test-datasets/output/01_clean_qc Expected directory: metagwgs-test-datasets/output/01_clean_qc
...@@ -103,7 +103,7 @@ Not tested: 0 ...@@ -103,7 +103,7 @@ Not tested: 0
If a test resulted in 'Failed' instead of 'Passed', the stdout is printed in log. If a test resulted in 'Failed' instead of 'Passed', the stdout is printed in log.
Sometimes, files are not tested because present in exp_dir but not in obs_dir. Then a log ft_\[step\].not_tested is created containing names of missing files. In 02_assembly, there are two possible assembly programs that can be used: metaspades and megahit, resulting in this .not_tested log file. Not tested files are not counted in missed count. Sometimes, files are not tested because present in _exp_dir_ but not in _obs_dir_. Then a log `ft_[STEP].not_tested` is created containing names of missing files. In **02_assembly**, there are two possible assembly programs that can be used: _metaspades_ and _megahit_, resulting in this `.not_tested log` file. Not tested files are not counted in missed count.
### Test methods ### Test methods
...@@ -113,14 +113,23 @@ Sometimes, files are not tested because present in exp_dir but not in obs_dir. T ...@@ -113,14 +113,23 @@ Sometimes, files are not tested because present in exp_dir but not in obs_dir. T
sort_diff: simple bash difference between two files sort_diff: simple bash difference between two files
`diff <(sort exp_path) <(sort obs_path)` `diff <(sort exp_path) <(sort obs_path)`
zdiff: simple bash difference between two gzipped files - **diff**: simple bash difference between two files
`zdiff exp_path obs_path`
`diff exp_path obs_path`
- **zdiff**: simple bash difference between two gzipped files
`zdiff exp_path obs_path`
- **no_header_diff**: remove the headers of .annotations and .seed_orthologs files
`diff <(grep -w "^?#" exp_path) <(grep -w "^?#" obs_path)`
- **cut_diff**: exception for cutadapt.log file
no_header_diff: remove the headers of .annotations and .seed_orthologs files `diff <(tail -n+6 exp_path) <(tail -n+6 obs_path)`
`diff <(grep -w "^?#" exp_path) <(grep -w "^?#" obs_path)`
cut_diff: exception for cutadapt.log file - **not_empty**: in python, check if file is empty
`diff <(tail -n+6 exp_path) <(tail -n+6 obs_path)`
not_empty: in python, check if file is empty not_empty: in python, check if file is empty
`test = path.getsize(obs_path) > 0` `test = path.getsize(obs_path) > 0`
......
...@@ -43,6 +43,7 @@ except ImportError as error: ...@@ -43,6 +43,7 @@ except ImportError as error:
# -obs_dir ../../functional_tests/observed/results \ # -obs_dir ../../functional_tests/observed/results \
# --script ../../functional_tests/test.sh # --script ../../functional_tests/test.sh
# --verbose # --verbose
def parse_arguments(): def parse_arguments():
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
......
...@@ -21,7 +21,7 @@ __status__ = 'dev' ...@@ -21,7 +21,7 @@ __status__ = 'dev'
# Usage # Usage
## cd [work_directory] ## cd [work_directory]
## python metagwgs/functional_tests/main.py -step [step] -exp_dir ./test_expected_logs -obs_dir ./[work_directory]/results [optional: --script ./[work_directory]/launch_nextflow.sh] [optional: --verbose] ## python metagwgs/functional_tests/main.py -step [step] -exp_dir ./test_expected_logs -obs_dir ./[work_directory]/results [optional: --script ./[work_directory]/launch_example.sh] [optional: --verbose]
try: try:
import sys import sys
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment