ng6 issueshttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues2019-08-30T13:22:58+02:00https://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/104loadin icon go on display while an error is raised2019-08-30T13:22:58+02:00Roxane Boyerloadin icon go on display while an error is raisedwhile downloading data with 'symlink', even if an error is raised the message is not displayed and the user has to wait :
![image](/uploads/fb5a7e6c8e7dac37eaac89bd231ce4da/image.png)while downloading data with 'symlink', even if an error is raised the message is not displayed and the user has to wait :
![image](/uploads/fb5a7e6c8e7dac37eaac89bd231ce4da/image.png)V3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/116Custom message for purged data2019-08-30T13:23:46+02:00Romain ThervilleCustom message for purged dataWe need another message, replacing "Results folder not synchronized yet... " when the data has been purged.We need another message, replacing "Results folder not synchronized yet... " when the data has been purged.V3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/110handle several data space2019-09-06T17:11:30+02:00Celine Noirothandle several data spaceFor seqOccIn project need to handle several project space.
To do:
* [x] handle different space when add run/analyse/project ...
* [x] handle different retention date thanks to space_id
* [x] handle spaces in download plugin
Validatio...For seqOccIn project need to handle several project space.
To do:
* [x] handle different space when add run/analyse/project ...
* [x] handle different retention date thanks to space_id
* [x] handle spaces in download plugin
Validation to be checked :
* [x] addproject with default space
* [x] addproject with specific space
* [x] addrun to a project with a specific space -> run data are in space ?
* [x] addanalysis to a project with a specific space -> analysis data are in space ?
* [x] addanalysis to a run with a specific space -> analysis data are in space ?V3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/107handle limit storage period/date depending on storage space parameter2019-08-30T17:25:07+02:00Celine Noirothandle limit storage period/date depending on storage space parameterData storage period depend on kind of project (eg seqoccin), this should be a workflow parameter.
To ensure the user do not set period value, the period has to be set in application.properties.
- For rented space validate how to set en...Data storage period depend on kind of project (eg seqoccin), this should be a workflow parameter.
To ensure the user do not set period value, the period has to be set in application.properties.
- For rented space validate how to set end of storage date
![image](/uploads/dcb383c6dea62c2d9ededd61a85fe433/image.png)
- end of storage date could depend on a date or a period depending on storage space name
```
[space_default]
retention_period = 3m
path=/ng6
[space_default_old]
retention_period = 2y
path=/ng6
[space_seqoc]
# pour un espace donnee : deux parametre exclusif retention_date != retention_period
retention_date=12/03/2021
#retention_period = 2year
path=/seqoc
```
- Add retention_time parameter in application.properties.example
V3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/106Configuring different storage space2019-09-06T17:11:30+02:00Celine NoirotConfiguring different storage spaceAdd a workflow parameter to store raw data and processed results in different storage space depending on project (ng6 or seqoccin or labs ?)
Warn:
- Quality control must be keep even if storage space is deleted, must be store in in ng6 ...Add a workflow parameter to store raw data and processed results in different storage space depending on project (ng6 or seqoccin or labs ?)
Warn:
- Quality control must be keep even if storage space is deleted, must be store in in ng6 space ?
- Think to handle data purge (do not delete if it's in rented space), check administration purge.
Code :
- web interface:
- storage info is stored in application.properties and db
- file eg: run_view.tmpl use relative path : `{$link=(('fileadmin'|cat:$runs[key($runs)].directory)|cat:'/')|cat:$file}`
- statistics storage space: check and define how to handle those spaces
- jflow code :
- application.properties will contain new section for storage space defined py project/lab name
- the storage space name is linked to a project
- functions get_save_directory() ... has to be adapted
- db must contain storage space name
- filesystem
- web interface: create symbolic link in fileadmin directory on different spaces
- adding new storage space, create symbolic link (look at how is done in jflow for [storage] section
- only project table has new attribute storage spaceV3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/86downloadsymlink, special characters2019-09-27T09:11:39+02:00Gerald Salingerald.salin@inra.frdownloadsymlink, special characterssome special characters are included in run names (e.g. /, (, )), introducing a bug in the download step (at least creating symlinks)some special characters are included in run names (e.g. /, (, )), introducing a bug in the download step (at least creating symlinks)V3.2.5Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/117Add duplication stat to alignmentStat analyze2019-08-30T13:23:44+02:00Claire KuchlyAdd duplication stat to alignmentStat analyzeV3.2.5Natacha FaivreNatacha Faivrehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/158On nG6, trying a project space migration results in a wf that stays "started"2020-06-11T17:02:37+02:00Romain ThervilleOn nG6, trying a project space migration results in a wf that stays "started"Since the last update, I have tried validating the migration feature in production environment, and a bug occured.
* The workflow stays started:
![Sans_titre](/uploads/607ad112d43bc1d57610f4308af1fda2/Sans_titre.png)
* The directorie...Since the last update, I have tried validating the migration feature in production environment, and a bug occured.
* The workflow stays started:
![Sans_titre](/uploads/607ad112d43bc1d57610f4308af1fda2/Sans_titre.png)
* The directories generated by Makeflow have "root" as Owner and group:
```
[root@ng6-slurm switchspaceid]# pwd
/work/ng6/jflow/work/switchspaceid
[root@ng6-slurm switchspaceid]# ls -lt
total 6
drwxr-x--x 4 root root 4096 17 janv. 09:05 wf004643
drwxr-x--x 4 root root 4096 16 janv. 16:11 wf004638
drwxr-x--x 4 ng6 ng6 4096 15 janv. 17:28 wf004616
drwxr-x--x 4 ng6 ng6 4096 10 janv. 14:48 wf004542
drwxr-x--x 4 ng6 ng6 4096 24 déc. 13:48 wf004448
drwxr-x--x 4 ng6 ng6 4096 24 déc. 13:39 wf004447
drwxr-x--x 4 ng6 ng6 4096 24 déc. 13:35 wf004446
drwxr-x--x 4 ng6 ng6 4096 24 déc. 13:22 wf004445
drwxr-x--x 4 ng6 ng6 4096 23 déc. 15:57 wf004440
drwxr-x--x 4 ng6 ng6 4096 11 déc. 16:45 wf004238
drwxr-x--x 4 ng6 ng6 4096 11 déc. 16:38 wf004237
```
* No ".stdout" log file, and not much more infos in ng6.log:
```
01-17 09:05 MoveProject.process DEBUG Begin MoveProject Component.
01-17 09:05 MoveProject.process DEBUG self.space_id=seqoccin
01-17 09:05 MoveProject.process DEBUG self.project_id=1518
01-17 09:05 MoveProject.process DEBUG stdout = /work/ng6/jflow/work/switchspaceid/wf004643/MoveProject_default/MoveProject.stdout
```V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/157Workflow addproject introuvable2020-06-11T17:12:34+02:00Romain ThervilleWorkflow addproject introuvableManque 1 import dans /workflows/addproject/__init__.py
Chargement infini en tentant addproject depuis l'interface web, et addproject est le seul workflow qui n'apparait pas dans get_available_workflows.
Plus d'infos dans le commit à sui...Manque 1 import dans /workflows/addproject/__init__.py
Chargement infini en tentant addproject depuis l'interface web, et addproject est le seul workflow qui n'apparait pas dans get_available_workflows.
Plus d'infos dans le commit à suivre.V3.2.6Romain ThervilleRomain Therville2020-01-16https://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/155Warning on hidden column while create a run, analyse or projet2020-01-10T10:24:43+01:00Celine NoirotWarning on hidden column while create a run, analyse or projet`
ng6-test@ng6-test: /work/ng6-test/jflow_rtherville $python3.4 /save/ng6-test/src/ng6-git-rtherville/bin/ng6_cli.py addanalysis --admin-login cnoirot --run-id 15165 --data-file /work/ng6-test/jflow_rtherville/test.txt --analysis-name "A...`
ng6-test@ng6-test: /work/ng6-test/jflow_rtherville $python3.4 /save/ng6-test/src/ng6-git-rtherville/bin/ng6_cli.py addanalysis --admin-login cnoirot --run-id 15165 --data-file /work/ng6-test/jflow_rtherville/test.txt --analysis-name "Analyse de run" --analysis-description "68 + 20 Manech" --analysis-software "GATK..." --analysis-software-options "-" --analysis-software-version "GATK:3.7 - Picard:2.1.1 - Bwa:0.7.15 - Samtools:1.3.1 - FastQC:v0.11.2 - Bcftools:1.3.1"
parsing Makeflow...
checking Makeflow for consistency...
Makeflow has 1 rules.
starting workflow....
submitting job: /work/ng6-test/jflow_rtherville/work/addanalysis/wf000499/.working/153171b866/_Stash/0/0/0/w0000000 none /work/ng6-test/jflow_rtherville/work/addanalysis/wf000499/BasicAnalysis_default/analysis.cfg /work/ng6-test/jflow_rtherville/tmp/tmp22_7y9dk.dump
submitted job 16508
**/save/ng6-test/src/ng6-git-rtherville/src/weaver/../pymysql/cursors.py:146: Warning: Incorrect integer value: 'True' for column 'hidden' at row 1
result = self._query(query)
**
job 16508 completed
nothing left to do.
sending incremental file list
./
test.txt
sent 166 bytes received 34 bytes 400.00 bytes/sec
total size is 61 speedup is 0.30
###########
# Results #
###########
> V3.2.6Celine NoirotCeline Noirothttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/153The DownloadSymlink workflow bugs when an analysis is an ONT Demultiplex2020-01-10T10:24:16+01:00Romain ThervilleThe DownloadSymlink workflow bugs when an analysis is an ONT Demultiplex**When does it happen?**
From the "download" page, when selecting an entire project that contains a demultiplex ONT analysis.
![2019-12-16_172314](/uploads/d98631c574a688f57a243b4984156f8d/2019-12-16_172314.png)
It's okay when the run ...**When does it happen?**
From the "download" page, when selecting an entire project that contains a demultiplex ONT analysis.
![2019-12-16_172314](/uploads/d98631c574a688f57a243b4984156f8d/2019-12-16_172314.png)
It's okay when the run and analyzes are deployed and selected:
![2019-12-16_172422](/uploads/b95a120a14968d72406ad2d2d27e7f77/2019-12-16_172422.png)
**Why?**
In utils.py:
*12-16 13:19 Utils DEBUG get_directories_structure_and_content. src_directories = /save/ng6///data/run/fb3cdcb00, /save/ng6///data/analyze/e3df5acbb, /save/ng6///data/analyze/2753b0457, /save/ng6///data/run/b2e7de531, /save/ng6///data/analyze/ae6d2ec97, /save/ng6///data/analyze/cfec28f91
12-16 13:19 Utils DEBUG get_directories_structure_and_content. dest_directories = /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/RawData, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/Analyse_RUNStats.64606, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/Analyse_DemultiplexONT.64608, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/RawData, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/Analyse_DemultiplexONT.64568, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/Analyse_RUNStats.64567, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT3-Climares-mtplx6-2-PAD92409.16250/Analyse_DemultiplexONT.64608, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT3-Climares-mtplx6-2-PAD92409.16250/Analyse_RUNStats.64606, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT2-Climares-mtplx6-PAD92409.16245/Analyse_DemultiplexONT.64568, /work/lipm/gouzy/Project_CLIMARES.482/Run_ONT2-Climares-mtplx6-PAD92409.16245/Analyse_RUNStats.64567*
(The destination tab contains 2 versions of the same runs and analyzes, one with '-' separatores, one with '_'.)
This may be fixed with an update to the download page's javascript call:
http://ng6-test.toulouse.inra.fr:8080//run_workflow?username=ng6-test&password=aCYLKVdYzI8AaTcB%25C2%25A4%25C3%2596%25C3%25B3i%2540%25C3%2598%2515%25C3%25B6%2522&user_directory=/work/ng6-test/jflow_rtherville/&admin_login=CTD&data_id=15032&run_id=15032&analysis_id=57729::-::57730&workflow_class=DownloadSymlink&callback=jQuery111105034437105204775_1576513683378&_=1576513683383
(By removing the analyzes that have no directory.)
The workflow tries the following treatment:
ln -s /save/ng6/data/run/fb3cdcb00 /work/ng6/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/RawData
ln -s /save/ng6/data/analyze/e3df5acbb /work/ng6/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/Analyse_RUNStats.64606
ln -s /save/ng6/data/analyze/2753b0457 /work/ng6/Project_CLIMARES.482/Run_ONT3_Climares_mtplx6_2_PAD92409.16250/Analyse_DemultiplexONT.64608
ln -s /save/ng6/data/run/b2e7de531 /work/ng6/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/RawData
ln -s /save/ng6/data/analyze/ae6d2ec97 /work/ng6/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/Analyse_DemultiplexONT.64568
ln -s /save/ng6/data/analyze/cfec28f91 /work/ng6/Project_CLIMARES.482/Run_ONT2_Climares_mtplx6_PAD92409.16245/Analyse_RUNStats.64567
But fails, BECAUSE /save/ng6/data/analyze/2753b0457 AND /save/ng6/data/analyze/ae6d2ec97 DO NOT EXIST!V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/150The "send mail" footer covers 10 columns instead of 11.2020-06-11T17:05:13+02:00Romain ThervilleThe "send mail" footer covers 10 columns instead of 11.In Administration > Mail obsolete project, the last table line has a bad display.
`<th colspan="10"> should be replaced with <th colspan="11">.`In Administration > Mail obsolete project, the last table line has a bad display.
`<th colspan="10"> should be replaced with <th colspan="11">.`V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/121purge email configuration2019-10-24T09:38:56+02:00Gerald Salingerald.salin@inra.frpurge email configurationcheck the recipient field, I think there were multiple copies of support.ng6 in the last purge session.check the recipient field, I think there were multiple copies of support.ng6 in the last purge session.V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/27Project view : hidden run and analyses are counted2019-09-18T17:27:36+02:00Celine NoirotProject view : hidden run and analyses are countedIn data overview hidden project are counted
![image](/uploads/14b9494bc2aedd913f56bf7bf8c0c025/image.png)In data overview hidden project are counted
![image](/uploads/14b9494bc2aedd913f56bf7bf8c0c025/image.png)V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/8Metacharacter "&" in password creates a bug2019-09-12T09:27:13+02:00Audrey GibertMetacharacter "&" in password creates a bugWhen "&" is in the password, the registered password stops just before the "&" :
Example:
if "hgff&ld" is written,
the registered password in the database is "hgff"When "&" is in the password, the registered password stops just before the "&" :
Example:
if "hgff&ld" is written,
the registered password in the database is "hgff"V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/137The static retention period given in the purge mail has to be computed2020-01-27T17:01:35+01:00Romain ThervilleThe static retention period given in the purge mail has to be computed"Dear user,
The two years data storage period of part or whole Accord-Bio project (1 run(s) and 7 analyse(s)) is ended."
Now that the retention is not a constant anymore, the purge mail text should be updated to give the right retention..."Dear user,
The two years data storage period of part or whole Accord-Bio project (1 run(s) and 7 analyse(s)) is ended."
Now that the retention is not a constant anymore, the purge mail text should be updated to give the right retention period.V3.2.6Romain ThervilleRomain Therville2019-11-21https://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/128Discard some email adress from purge email.2019-09-20T10:27:40+02:00Celine NoirotDiscard some email adress from purge email.Add a list a emails address to discard from purge email.
To be configure as "email_from"
Eg:
plugin.tx_nG6_pi6.email_to_discard = ctd@inra.fr,ng6-support@inra.fr
Remove thoses email from the list of destinator of the emailAdd a list a emails address to discard from purge email.
To be configure as "email_from"
Eg:
plugin.tx_nG6_pi6.email_to_discard = ctd@inra.fr,ng6-support@inra.fr
Remove thoses email from the list of destinator of the emailV3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/127Purge email should be send to all the users of the project.2019-09-20T11:37:40+02:00Celine NoirotPurge email should be send to all the users of the project.From now, email to alert for a purge has to be sent to all the member, administrator and manager of the project.From now, email to alert for a purge has to be sent to all the member, administrator and manager of the project.V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/31add link and name to runs and analyses and projects in the purge email2019-09-10T11:58:20+02:00Gerald Salingerald.salin@inra.fradd link and name to runs and analyses and projects in the purge emaile.g.
Files impacted of:
- runs names: 314,315,316,317,735,723,1292,1414,1423,1530,1974,1983
- analyses ids: 1149,1151,1154,1150,1152,1153,1155,1159,1160,1156,1157,1158,3650,3652,3662,3634,3633,3623,3621,3636,3635,5835,5840,5852,5858,585...e.g.
Files impacted of:
- runs names: 314,315,316,317,735,723,1292,1414,1423,1530,1974,1983
- analyses ids: 1149,1151,1154,1150,1152,1153,1155,1159,1160,1156,1157,1158,3650,3652,3662,3634,3633,3623,3621,3636,3635,5835,5840,5852,5858,5855,5845,6309,6310,6311,6313,6314,6320,6321,6382,6381,6380,6379,6378,6383,6758,6757,6759,6763,8532,8539,8544,8547,8559,8568,8571,8582,158V3.2.6Romain ThervilleRomain Thervillehttps://forgemia.inra.fr/genotoul-bioinfo/ng6/-/issues/169FAQ style is ugly2020-02-11T14:32:56+01:00Celine NoirotFAQ style is uglyDo it less uglyDo it less uglyV3.2.6Celine NoirotCeline Noirot