ng6 issueshttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues2021-05-04T11:49:20+02:00https://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/194The Downloadsymlink workflow stays stuck when ng6 ssh user has a rsa passphrase2021-05-04T11:49:20+02:00Romain ThervilleThe Downloadsymlink workflow stays stuck when ng6 ssh user has a rsa passphrase**Symptoms:**
- Creating symbolic links through the web interface results in an infinite loading.
- The associated workflow doesn't end.
- The workflow debug returns the following :
Traceback (most recent call last):
File "/work/ng6/j...**Symptoms:**
- Creating symbolic links through the web interface results in an infinite loading.
- The associated workflow doesn't end.
- The workflow debug returns the following :
Traceback (most recent call last):
File "/work/ng6/jflow/work/downloadsymlink/wf008708/.working/f85edccc0a/_Stash/0/0/0/w0000000", line 86, in <module>
outputs = create_symbolik_link(*new_args)
File "/work/ng6/jflow/work/downloadsymlink/wf008708/.working/f85edccc0a/_Stash/0/0/0/w0000000", line 79, in create_symbolik_link
s.cmd('bash %s'%user_script)
File "/usr/local/bioinfo/src/ng6_sources/ng6-V3.2.7.4/src/weaver/../ng6/utils.py", line 568, in cmd
return self.ssh_results(pid, f)
File "/usr/local/bioinfo/src/ng6_sources/ng6-V3.2.7.4/src/weaver/../ng6/utils.py", line 561, in ssh_results
got = self._read(f)
File "/usr/local/bioinfo/src/ng6_sources/ng6-V3.2.7.4/src/weaver/../ng6/utils.py", line 538, in _read
x = os.read(f, 1024)
**Cause of the problem:**
When lauching an SSH connection with the ng6 user (Python : s = SSH(username, password, server_params[0]) ), a passphrase defined in /home/ng6/.ssh/id_rsa is asked.V4 - MaintenanceRomain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/188[IlluminaWF]Hiding some analyses per defaut2020-07-15T10:02:05+02:00Audrey Gibert[IlluminaWF]Hiding some analyses per defautBy default, there is only the "log files" analysis that is hidden :
![image](/uploads/6603fbe0bddccccce2c13a16675d8079/image.png)
We should also hide
* Subset read files
* alignment
* alignmentStats
* InsertSizes
*...By default, there is only the "log files" analysis that is hidden :
![image](/uploads/6603fbe0bddccccce2c13a16675d8079/image.png)
We should also hide
* Subset read files
* alignment
* alignmentStats
* InsertSizes
* DemultiplexStatsV4 - MaintenanceAudrey GibertAudrey Gibert2021-07-01https://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/183WF AddAnalysis : rsync errors in error output2020-10-02T19:33:12+02:00Audrey GibertWF AddAnalysis : rsync errors in error outputWhen there is a lot of files to be sent to NG6 with the Addanalysis workflow, the rsync command seems to have some trouble.
I had 6 file groups to send, only 1/6 worked without error message.
The error message looks like this :
```bash...When there is a lot of files to be sent to NG6 with the Addanalysis workflow, the rsync command seems to have some trouble.
I had 6 file groups to send, only 1/6 worked without error message.
The error message looks like this :
```bash
rsync: sender failed to remove PL1_Pool2_11_BD20_2017.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_11_BD20_2017.rem.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_11_BD20_2017.rem.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.rem.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.rem.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.rem.1.fq.gz: No such file or directory (2)
...
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.1.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_6_BD73_2017.rem.2.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.2.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.1.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.rem.1.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.2.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.rem.2.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.rem.1.fq.gz: Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.rem.2.fq.gz: Not a directory (20)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
```
Md5sum is the same between the files to send, and the files in NG6, so no rush in here. But maybe this feature can be improved..V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/181make filenames in NG6 uniq2020-05-15T07:36:11+02:00Gerald Salingerald.salin@inra.frmake filenames in NG6 uniqin NG6 FAQ, for downloading :
On a Windows workstation (Command lines), copy/paste this list in a file "listOfFiles" (B**e aware that if the same filename is found in different url, the local file will be overriden**). Then you can down...in NG6 FAQ, for downloading :
On a Windows workstation (Command lines), copy/paste this list in a file "listOfFiles" (B**e aware that if the same filename is found in different url, the local file will be overriden**). Then you can download curl for Windows (https://curl.haxx.se/download.html), extract the archive, and run this following command in a windows prompt
should not happen.
prefix the filename so that it is uniq (add the name of the parent folder which is uniq? the run Id + analysis Id?)V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/180Analysis sort by data in project view2020-05-05T14:40:13+02:00Celine NoirotAnalysis sort by data in project viewUnable to sort project analysis by date
![image](/uploads/f37bbc5558f1d7655864dd3a3ef086cb/image.png)Unable to sort project analysis by date
![image](/uploads/f37bbc5558f1d7655864dd3a3ef086cb/image.png)V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/152Tests a faire lors d'une mise en prod2020-07-09T14:07:54+02:00Celine NoirotTests a faire lors d'une mise en prodJ'aimerai que l'on liste l'ensemble des element a verifier lors d'une mise en prod pour eviter au mieux les bugs
tester :
* [ ] le download
* [ ] create symblink
* [ ] add project
* [ ] add run
* [ ] add analysis
* [ ] add raw files
* [...J'aimerai que l'on liste l'ensemble des element a verifier lors d'une mise en prod pour eviter au mieux les bugs
tester :
* [ ] le download
* [ ] create symblink
* [ ] add project
* [ ] add run
* [ ] add analysis
* [ ] add raw files
* [ ] vue projet
* [ ] vue run/downloads
* [ ] vue analyses/downloads
* [ ] verifier le application.properties est a jours en fonction des nouveaux dev
* [ ] verifier que le template dans typo3 est a jours en fonction des nouveaux devV4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/136Fail to update a user's role on a project2020-06-11T17:08:53+02:00Romain ThervilleFail to update a user's role on a projectOn nG6 (prod), the user 513 was both member and admin on a project.
We should check if another user is concerned, and find why there were 2 rows in 'fe_rights' linking the user to the project.
Kanboard link : https://genomique.genotoul...On nG6 (prod), the user 513 was both member and admin on a project.
We should check if another user is concerned, and find why there were 2 rows in 'fe_rights' linking the user to the project.
Kanboard link : https://genomique.genotoul.fr/kanboard/?controller=TaskViewController&action=show&project_id=18&task_id=1132V4 - MaintenanceRomain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/125Differences between projet's hidden parameter and its run's hidden parameter2020-06-11T17:09:11+02:00Romain ThervilleDifferences between projet's hidden parameter and its run's hidden parameterSometimes (see exemple), hidden projects have visible analyses. Sometimes they do not.
http://ng6-test.toulouse.inra.fr
Project **FlaxCoreCol**
![2019-09-05_172449](/uploads/82d77c3e33ce35a60eb412286fa9d529/2019-09-05_172449.png)
Proj...Sometimes (see exemple), hidden projects have visible analyses. Sometimes they do not.
http://ng6-test.toulouse.inra.fr
Project **FlaxCoreCol**
![2019-09-05_172449](/uploads/82d77c3e33ce35a60eb412286fa9d529/2019-09-05_172449.png)
Project **FlaxCoreCol** Run **FlaxCoreCol-3** (recent)
![2019-09-05_172520](/uploads/bfbc0dea5ea36649347228321404ebd5/2019-09-05_172520.png)
Project **FlaxCoreCol** Run **FlaxCoreCol-4** (older)
![2019-09-05_172649](/uploads/05b5faaf00dc4f2f9d7ef39d0d8caca9/2019-09-05_172649.png)
(At least the icons are missing, but a check in the database is necessary.)V4 - MaintenanceRomain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/109ftp server architecture2019-08-30T09:27:13+02:00Celine Noirotftp server architectureKeep/generate run/analysis architecture for a project when ftp server will be available.Keep/generate run/analysis architecture for a project when ftp server will be available.V4 - MaintenanceGerald Salingerald.salin@inra.frGerald Salingerald.salin@inra.frhttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/108export metadata2019-08-30T09:27:13+02:00Celine Noirotexport metadataadd a functionality to export all metadata of project, run, analysis.add a functionality to export all metadata of project, run, analysis.V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/105sort order not numeric in contaminationsearch table2019-10-28T15:44:50+01:00Gerald Salingerald.salin@inra.frsort order not numeric in contaminationsearch table![image](/uploads/c5da447e601b3750890b5432d22e18a3/image.png)![image](/uploads/c5da447e601b3750890b5432d22e18a3/image.png)V4 - MaintenanceAudrey GibertAudrey Giberthttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/91[methylseq] do not store files for analyses Alignment and Metylation extractor2018-11-06T17:50:17+01:00Gerald Salingerald.salin@inra.fr[methylseq] do not store files for analyses Alignment and Metylation extractorthese data are too huge and for no use after the quality pipeline is overthese data are too huge and for no use after the quality pipeline is overV4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/71clean pipeline outputs if succeeded2018-09-06T15:09:01+02:00Gerald Salingerald.salin@inra.frclean pipeline outputs if succeededallow the deletion of files in the worflow output dir to save dik spaceallow the deletion of files in the worflow output dir to save dik spaceV4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/55cutadapt output changed from 1.3 to 18 and 1.142018-08-13T19:10:53+02:00Gerald Salingerald.salin@inra.frcutadapt output changed from 1.3 to 18 and 1.14cutadpat component post-proccess failed :
the output no longer contains "Processed reads"
with cutadapt V1.3
`cutadapt version 1.3
Command line parameters: -b CTGTCTCTT -b ATACACATCT -b AGATCTAT -b AAGAGACAG --error-rate 0.1 --minimum-l...cutadpat component post-proccess failed :
the output no longer contains "Processed reads"
with cutadapt V1.3
`cutadapt version 1.3
Command line parameters: -b CTGTCTCTT -b ATACACATCT -b AGATCTAT -b AAGAGACAG --error-rate 0.1 --minimum-length 20 --times 4 --paired-output /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_cut.fastq --paired-output /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_cut.fastq -o /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R2_cut.fastq /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R2_tmp.fastq /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_tmp.fastq
Maximum error rate: 10.00%
No. of adapters: 4
Processed reads: 4344
Processed bases: 434400 bp (0.4 Mbp)
Trimmed reads: 2106 (48.5%)
Trimmed bases: 113988 bp (0.1 Mbp) (26.24% of total)
Too short reads: 377 (8.7% of processed reads)
Too long reads: 0 (0.0% of processed reads)
Total time: 0.28 s
Time per read: 0.064 ms
`
with cutadapt 18. and 1.14 :
`This is cutadapt 1.8.3 with Python 2.7.2
Command line parameters: -b CTGTCTCTT -b ATACACATCT -b AGATCTAT -b AAGAGACAG --error-rate 0.1 --minimum-length 20 --times 4 --paired-output /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_cut.fastq --paired-output /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_cut.fastq -o /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R2_cut.fastq /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R2_tmp.fastq /work/ng6-test/jflow_gsalin/work/illumina_matepair/wf000073/CutAdapt_default/B_CAGATC_L004_R1_tmp.fastq
Trimming 4 adapters with at most 10.0% errors in paired-end legacy mode ...
Finished in 0.18 s (41 us/read; 1.45 M reads/minute).
=== Summary ===
Total read pairs processed: 4,344
Read 1 with adapter: 3,610 (83.1%)
Read 2 with adapter: 0 (0.0%)
Pairs that were too short: 377 (8.7%)
Pairs written (passing filters): 3,967 (91.3%)
Total basepairs processed: 814,063 bp
Read 1: 434,400 bp
Read 2: 379,663 bp
Total written (filtered): 686,031 bp (84.3%)
Read 1: 342,864 bp
Read 2: 343,167 bp
`V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/52unable to use the modules functionality of jflow for insertsizes.py component2018-08-13T10:57:08+02:00Gerald Salingerald.salin@inra.frunable to use the modules functionality of jflow for insertsizes.py componentthe process of this module call a self.add_python_execution(inserts_metrics...)
the modules specified in application.properties for InsertSizes component is not managed in jflow as it is a PythonFunction
for the moment we have to put th...the process of this module call a self.add_python_execution(inserts_metrics...)
the modules specified in application.properties for InsertSizes component is not managed in jflow as it is a PythonFunction
for the moment we have to put this module load (module load system/R-3.4.3) in the bashrc fileV4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/46weaver : make patch for function.py2019-08-30T09:23:58+02:00Celine Noirotweaver : make patch for function.pyTODO
l214TODO
l214V4 - MaintenanceCeline NoirotCeline Noirothttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/37Update documentation with rules and modules2018-09-07T12:03:51+02:00Celine NoirotUpdate documentation with rules and modulesBy upgrading jflow, new functionnalities have been added.
The documentation should also be upgrade.By upgrading jflow, new functionnalities have been added.
The documentation should also be upgrade.V4 - Maintenancehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/33correct source indent (tabs/white space) for all files2019-08-29T14:11:18+02:00Gerald Salingerald.salin@inra.frcorrect source indent (tabs/white space) for all filesV4 - MaintenanceRomain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/32add manual for purging data2018-08-02T14:57:31+02:00Gerald Salingerald.salin@inra.fradd manual for purging dataV4 - MaintenanceClaire KuchlyClaire Kuchly