GitLab

When there is a lot of files to be sent to NG6 with the Addanalysis workflow, the rsync command seems to have some trouble.

I had 6 file groups to send, only 1/6 worked without error message.

The error message looks like this :

rsync: sender failed to remove PL1_Pool2_11_BD20_2017.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_11_BD20_2017.rem.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_11_BD20_2017.rem.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.rem.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_12_BK28_2018.rem.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.1.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.2.fq.gz: No such file or directory (2)
rsync: sender failed to remove PL1_Pool2_14_A69_2013.rem.1.fq.gz: No such file or directory (2)
...
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.1.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_6_BD73_2017.rem.2.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.2.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.1.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.rem.1.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.2.fq.gz: Not a directory (20)
rsync: send_files failed to open "/work/ng6/jflow/work/data/analyze/efa9850d7/PL1_Pool2_9_BE53_2017.rem.2.fq.gz": Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.rem.1.fq.gz: Not a directory (20)
rsync: sender failed to remove PL1_Pool2_7_BK70_2_2018.rem.2.fq.gz: Not a directory (20)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]

Md5sum is the same between the files to send, and the files in NG6, so no rush in here. But maybe this feature can be improved..