Commit 8af503e5 authored by MARTIN Pierre's avatar MARTIN Pierre
Browse files

updated README for validation + added launch_example.sh

parent adb2133e
......@@ -2,63 +2,36 @@
## I. Pre-requisites
1. Install metagwgs as describe here:[installation doc](../docs/installation.md)
2. Get datasets : two datasets are currently available for these functional tests,
- ones in source code : [here](../test))
- thoses from [nf-core/mag pipeline](https://github.com/nf-core/test-datasets/tree/mag/test_data)
#TODO : lien direct sur les x fichier, ou sinon indiquer quels fichier telecharger/copier depuis test_data
```
wget
1. Install metagwgs as described here: [installation doc](../docs/installation.md)
2. Get datasets: two datasets are currently available for these functional tests at `https://forgemia.inra.fr/genotoul-bioinfo/metagwgs-test-datasets.git`
Replace "\<dataset\>" with either "small" or "mag":
```
git clone --branch <dataset> git@forgemia.inra.fr:genotoul-bioinfo/metagwgs-test-datasets.git
or
3. Download the expected results from [link-to-test-datasets]. #TODO
wget https://forgemia.inra.fr/genotoul-bioinfo/metagwgs-test-datasets/-/archive/<dataset>/metagwgs-test-datasets-<dataset>.tar.gz
```
## II. Run functional tests
Each step of metagwgs produces a series of files. We want to be able to determine if the modifications we perform on metagwgs have an impact on any of these files (presence, contents, format, ...). You'll find more info about how the files are tested at the end of this page.
#TODO Mettre l'usage du script
There is two way to launch functionnal test :
- by providing result of a pipeline already exectuted
- by providing a script which launch the nextflow pipeline (see example) #TODO mettre un fichier de soumission d'exemple dans les sources
### Launch from pipeline already exectuted
If you have already launched metagwgs [see metagwgs README and usage] on test data:
```
cd project-directory
python [work_dir]/metaG/metagwgs/functional_tests/main.py -step 07_taxo_affi -exp_dir [work_dir]/test_expected_logs -obs_dir ./results
```
### Launch with script
Exemples below use the slurm job manager and launch all 7 steps of metagwgs to ensure all parts of main.nf work as intended.
1. Create a new directory (project-directory) containing a shell script to be used by functional tests:
To launch functional tests, you need to be located at the root of the folder where you want to perform the tests. There are two ways to launch functionnal tests (testing all steps to 07_taxo_affi):
- by providing the results folder of a pipeline already exectuted
```
#!/bin/bash
sbatch -W -p workq -J metagwgs --mem=6G \
--wrap="module load bioinfo/Nextflow-v21.04.1 ; module load system/singularity-3.7.3 ; nextflow run -profile test_genotoul_workq [work_dir]/metaG/metagwgs/main.nf --step '01_clean_qc,02_assembly,03_filtering,04_structural_annot,05_alignment,06_func_annot,07_taxo_affi' --reads '../metagwgs/test/*_{R1,R2}.fastq.gz' --host_fasta '[work_dir]/human_ref/Homo_sapiens.GRCh38_chr21.fa' --host_bwa_index '[work_dir]/human_ref/Homo_sapiens.GRCh38_chr21.fa.{amb,ann,bwt,pac,sa}' --kaiju_db_dir '/bank/kaijudb/kaijudb_refseq_2020-05-25' --taxonomy_dir '[work_dir]/taxonomy' --eggnog_mapper_db_dir '/bank/eggnog-mapper/eggnog-mapper-2.0.4-rf1/data' --min_contigs_cpm 1000 --diamond_bank "[work_dir]/refseq_bacteria_2021-05-20/refseq_bacteria.dmnd" -with-report -with-timeline -with-trace -with-dag -resume"
cd test_folder
python <metagwgs-src>/functional_tests/main.py -step 07_taxo_affi -exp_dir metagwgs-test-datasets-small/output -obs_dir ./results
```
*In this exemple, [work_dir] = "/home/pmartin2/work"*
*"--min_contigs_cpm 1000" is mandatory to have the same results as exp_dir for step 03_filtering*
2. Run functionnal test by providing the script :
- by providing a script which will launch the nextflow pipeline [see example](./launch_example.sh) (this example is designed for the "small" dataset with --min_contigs_cpm>1000, using slurm)
```
cd project-directory
python [work_dir]/metaG/metagwgs/functional_tests/main.py -step 07_taxo_affi -exp_dir [work_dir]/test_expected_logs -obs_dir ./results --script launch_07_taxo_affi.sh
mkdir test_folder
cd test_folder
cp <metagwgs-src>/functional_tests/launch_example.sh ./
python <metagwgs-src>/functional_tests/main.py -step 07_taxo_affi -exp_dir metagwgs-test-datasets-small/output -obs_dir ./results --script launch_example.sh
```
## III. Output
A ft_\[step\].log file is created for each step of metagwgs. It contains information about each test performed on given files.
......@@ -72,13 +45,13 @@ Observed directory: /work/pmartin2/metaG/refac_09_13.2/results/01_clean_qc
------------------------------------------------------------------------------
File: 01_1_cleaned_reads/cleaned_a_R1.fastq.gz
File: 01_1_cleaned_reads/cleaned_c_R1.fastq.gz
Test method: zdiff
Test result: Passed
------------------------------------------------------------------------------
File: 01_1_cleaned_reads/cleaned_a_R2.fastq.gz
File: 01_1_cleaned_reads/cleaned_c_R2.fastq.gz
Test method: zdiff
Test result: Passed
......
#!/bin/bash
sbatch -W -p workq -J functional_test --mem=6G \
--wrap="module load bioinfo/Nextflow-v21.04.1 ; module load system/singularity-3.7.3 ; nextflow run -profile test_genotoul_workq <metagwgs-src>/main.nf --step '01_clean_qc,02_assembly,03_filtering,04_structural_annot,05_alignment,06_func_annot,07_taxo_affi' --reads 'metagwgs-test-datasets-small/input/*_{R1,R2}.fastq.gz' --host_fasta 'metagwgs-test-datasets-small/input/host/Homo_sapiens.GRCh38_chr21.fa' --host_bwa_index 'metagwgs-test-datasets-small/input/host/Homo_sapiens.GRCh38_chr21.fa.{amb,ann,bwt,pac,sa}' --min_contigs_cpm 1000 --kaiju_db_dir '<bank>/kaijudb_refseq_2020-05-25' --taxonomy_dir '<bank>/taxonomy' --eggnog_mapper_db_dir '<bank>/eggnog-mapper-2.0.4-rf1/data' --diamond_bank '<bank>/refseq_bacteria_2021-05-20/refseq_bacteria.dmnd' -with-report -with-timeline -with-trace -with-dag"
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment