Commit 8af503e5 authored by MARTIN Pierre's avatar MARTIN Pierre
Browse files

updated README for validation + added launch_example.sh

parent adb2133e
...@@ -2,63 +2,36 @@ ...@@ -2,63 +2,36 @@
## I. Pre-requisites ## I. Pre-requisites
1. Install metagwgs as describe here:[installation doc](../docs/installation.md) 1. Install metagwgs as described here: [installation doc](../docs/installation.md)
2. Get datasets : two datasets are currently available for these functional tests, 2. Get datasets: two datasets are currently available for these functional tests at `https://forgemia.inra.fr/genotoul-bioinfo/metagwgs-test-datasets.git`
- ones in source code : [here](../test))
- thoses from [nf-core/mag pipeline](https://github.com/nf-core/test-datasets/tree/mag/test_data) Replace "\<dataset\>" with either "small" or "mag":
#TODO : lien direct sur les x fichier, ou sinon indiquer quels fichier telecharger/copier depuis test_data
```
wget
``` ```
git clone --branch <dataset> git@forgemia.inra.fr:genotoul-bioinfo/metagwgs-test-datasets.git
or
3. Download the expected results from [link-to-test-datasets]. #TODO wget https://forgemia.inra.fr/genotoul-bioinfo/metagwgs-test-datasets/-/archive/<dataset>/metagwgs-test-datasets-<dataset>.tar.gz
```
## II. Run functional tests ## II. Run functional tests
Each step of metagwgs produces a series of files. We want to be able to determine if the modifications we perform on metagwgs have an impact on any of these files (presence, contents, format, ...). You'll find more info about how the files are tested at the end of this page. Each step of metagwgs produces a series of files. We want to be able to determine if the modifications we perform on metagwgs have an impact on any of these files (presence, contents, format, ...). You'll find more info about how the files are tested at the end of this page.
#TODO Mettre l'usage du script To launch functional tests, you need to be located at the root of the folder where you want to perform the tests. There are two ways to launch functionnal tests (testing all steps to 07_taxo_affi):
- by providing the results folder of a pipeline already exectuted
There is two way to launch functionnal test :
- by providing result of a pipeline already exectuted
- by providing a script which launch the nextflow pipeline (see example) #TODO mettre un fichier de soumission d'exemple dans les sources
### Launch from pipeline already exectuted
If you have already launched metagwgs [see metagwgs README and usage] on test data:
```
cd project-directory
python [work_dir]/metaG/metagwgs/functional_tests/main.py -step 07_taxo_affi -exp_dir [work_dir]/test_expected_logs -obs_dir ./results
```
### Launch with script
Exemples below use the slurm job manager and launch all 7 steps of metagwgs to ensure all parts of main.nf work as intended.
1. Create a new directory (project-directory) containing a shell script to be used by functional tests:
``` ```
#!/bin/bash cd test_folder
python <metagwgs-src>/functional_tests/main.py -step 07_taxo_affi -exp_dir metagwgs-test-datasets-small/output -obs_dir ./results
sbatch -W -p workq -J metagwgs --mem=6G \
--wrap="module load bioinfo/Nextflow-v21.04.1 ; module load system/singularity-3.7.3 ; nextflow run -profile test_genotoul_workq [work_dir]/metaG/metagwgs/main.nf --step '01_clean_qc,02_assembly,03_filtering,04_structural_annot,05_alignment,06_func_annot,07_taxo_affi' --reads '../metagwgs/test/*_{R1,R2}.fastq.gz' --host_fasta '[work_dir]/human_ref/Homo_sapiens.GRCh38_chr21.fa' --host_bwa_index '[work_dir]/human_ref/Homo_sapiens.GRCh38_chr21.fa.{amb,ann,bwt,pac,sa}' --kaiju_db_dir '/bank/kaijudb/kaijudb_refseq_2020-05-25' --taxonomy_dir '[work_dir]/taxonomy' --eggnog_mapper_db_dir '/bank/eggnog-mapper/eggnog-mapper-2.0.4-rf1/data' --min_contigs_cpm 1000 --diamond_bank "[work_dir]/refseq_bacteria_2021-05-20/refseq_bacteria.dmnd" -with-report -with-timeline -with-trace -with-dag -resume"
``` ```
- by providing a script which will launch the nextflow pipeline [see example](./launch_example.sh) (this example is designed for the "small" dataset with --min_contigs_cpm>1000, using slurm)
*In this exemple, [work_dir] = "/home/pmartin2/work"*
*"--min_contigs_cpm 1000" is mandatory to have the same results as exp_dir for step 03_filtering*
2. Run functionnal test by providing the script :
``` ```
cd project-directory mkdir test_folder
python [work_dir]/metaG/metagwgs/functional_tests/main.py -step 07_taxo_affi -exp_dir [work_dir]/test_expected_logs -obs_dir ./results --script launch_07_taxo_affi.sh cd test_folder
cp <metagwgs-src>/functional_tests/launch_example.sh ./
python <metagwgs-src>/functional_tests/main.py -step 07_taxo_affi -exp_dir metagwgs-test-datasets-small/output -obs_dir ./results --script launch_example.sh
``` ```
## III. Output ## III. Output
A ft_\[step\].log file is created for each step of metagwgs. It contains information about each test performed on given files. A ft_\[step\].log file is created for each step of metagwgs. It contains information about each test performed on given files.
...@@ -72,13 +45,13 @@ Observed directory: /work/pmartin2/metaG/refac_09_13.2/results/01_clean_qc ...@@ -72,13 +45,13 @@ Observed directory: /work/pmartin2/metaG/refac_09_13.2/results/01_clean_qc
------------------------------------------------------------------------------ ------------------------------------------------------------------------------
File: 01_1_cleaned_reads/cleaned_a_R1.fastq.gz File: 01_1_cleaned_reads/cleaned_c_R1.fastq.gz
Test method: zdiff Test method: zdiff
Test result: Passed Test result: Passed
------------------------------------------------------------------------------ ------------------------------------------------------------------------------
File: 01_1_cleaned_reads/cleaned_a_R2.fastq.gz File: 01_1_cleaned_reads/cleaned_c_R2.fastq.gz
Test method: zdiff Test method: zdiff
Test result: Passed Test result: Passed
......
#!/bin/bash
sbatch -W -p workq -J functional_test --mem=6G \
--wrap="module load bioinfo/Nextflow-v21.04.1 ; module load system/singularity-3.7.3 ; nextflow run -profile test_genotoul_workq <metagwgs-src>/main.nf --step '01_clean_qc,02_assembly,03_filtering,04_structural_annot,05_alignment,06_func_annot,07_taxo_affi' --reads 'metagwgs-test-datasets-small/input/*_{R1,R2}.fastq.gz' --host_fasta 'metagwgs-test-datasets-small/input/host/Homo_sapiens.GRCh38_chr21.fa' --host_bwa_index 'metagwgs-test-datasets-small/input/host/Homo_sapiens.GRCh38_chr21.fa.{amb,ann,bwt,pac,sa}' --min_contigs_cpm 1000 --kaiju_db_dir '<bank>/kaijudb_refseq_2020-05-25' --taxonomy_dir '<bank>/taxonomy' --eggnog_mapper_db_dir '<bank>/eggnog-mapper-2.0.4-rf1/data' --diamond_bank '<bank>/refseq_bacteria_2021-05-20/refseq_bacteria.dmnd' -with-report -with-timeline -with-trace -with-dag"
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment