allow to run jflow on genobioinfo
genologin is out, try to run jflow on genobioinfo :
- copy ng6-slurm:/usr/local/bioinfo/src/nG6 (ln de ng6-V3.4.4) from production to genobioinfo:/work/project/PlaGe/sbsuser/ng6-V3.4.4
- create /work/project/PlaGe/sbsuser/jflowSoftware to store software needed by jflow and workflows
- compile/install last version of cctools (7.7). Install it in genobioinfo:/work/project/PlaGe/sbsuser/cctools (first compiled on esitoul-rocky-dev). The current version used on genologin fails on compiling on new infrastructure and compiled version copied on genobioinfo fails to run.
- create ng6 database on esitoul-rocky-dev from a backup (/save/ng6/backup_db)
- update /work/project/PlaGe/sbsuser/ng6-V3.4.4/application.properties for output folders (use /work/user/ng6/xxx)
run addrun workflow on test data failed with python3.6, at pickle.load stage....different tests showed the jflow code only works with python 3.4. Hard to find source code and compile it....I copied if from ng6-slurm scp /tools/python/3.4.3 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser and using it, the pipeline succeeded
/work/project/PlaGe/sbsuser/Python-3.4.3/bin/python3.4 /work/project/PlaGe/sbsuser/ng6-V3.4.4/bin/ng6_cli.py addrun --project-name REPDEV --name GGTestPython3.4 --description makeflow7 --admin-login CTD --date 16/11/2023 --data-nature gDNA --sequencer aviti --species bos --type lane1 --sample type=se sample-id='V' read1='workflows/illumina_qc/data/casava_directory_test/Project_Demo_project/Sample_Demo/Demo_NoIndex_L001_R2_001.fastq.gz'
To run illumina_qc workflow, we need specific version of some software and to update application.properties with new software location (but with same version) in /work/project/PlaGe/sbsuser/ng6-V3.4.4 Depending on software, a compilation was required or a simple copy worked :
1038 scp -r /usr/local/bioinfo/src/FastQC/FastQC_v0.11.2 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1040 scp -r /usr/local/bioinfo/src/fastq_illumina_filter/fastq_illumina_filter-0.1.tar.gz sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1045 scp -r /usr/local/bioinfo/src/FLASH/FLASH-1.2.6 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1047 scp -r /usr/local/bioinfo/src/FASTX-Toolkit/fastx_toolkit-0.0.14 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1049 scp -r /usr/local/bioinfo/src/STAR-2.7.10a_alpha_220314 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1050 scp -r /usr/local/bioinfo/src/STAR/STAR-2.7.10a_alpha_220314 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1059 scp -r /usr/local/bioinfo/src/STAR/STAR-2.7.10a_alpha_220314 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1060 scp -r /tools/java/jre1.8.0_45 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1062 scp -r /usr/local/bioinfo/src/TrimGalore/0.4.5.zip sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1063 scp -r /usr/local/bioinfo/src/Bismark/bismark_v0.19.0.tar.gz sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1064 scp -r /usr/local/bioinfo/src/bowtie/bowtie2-2.3.4.1-linux-x86_64.zip sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1069 scp -r /usr/local/bioinfo/src/cctools/cctools-6.0.14-x86_64-redhat7 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1079 scp -r /usr/local/bioinfo/src/picard-tools/picard-2.18.2 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
1084 scp -r /tools/R/R-3.4.3 sbsuser@genobioinfo.toulouse.inrae.fr:/work/project/PlaGe/sbsuser/jflowSoftware/
I didn't managed to compile STAR, nor R-3.4.3
- stop here for STAR
- I used last version of R R-4.3.2 > compilation OK > test OK...but had to add 'Rpath = /work/project/PlaGe/sbsuser/jflowSoftware/R-4.3.2/bin/' and modify inserstssizes.py to modify PATH (Rscript is called by picard for CollectMetrics component to generate pdf file)