Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to create directory/file when path has space: Exit 141 #698

Closed
llrs opened this issue Sep 9, 2024 · 4 comments
Closed

Failed to create directory/file when path has space: Exit 141 #698

llrs opened this issue Sep 9, 2024 · 4 comments

Comments

@llrs
Copy link

llrs commented Sep 9, 2024

Many thanks for maintaining Bismark it is a great tool.

I have also encountered error 141 while using bismark_methylation_extractor:

I used this command:

~/bin/Bismark-0.24.2/bismark_methylation_extractor --ignore_r2 10 --ignore 10 --gzip -p --comprehensive --merge_non_CpG --parallel 5 P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam -o out/Seagate\ Hub/methyl/ && date && notify-send "Finished"

And here what it was printed on screen:

 *** Bismark methylation extractor version v0.24.2 ***

Output will be written into the directory: out/Seagate Hub/methyl/
Setting option '--no_overlap' since this is (normally) the right thing to do for paired-end data

Summarising Bismark methylation extractor parameters:
===============================================================
Bismark paired-end SAM format specified (default)
Number of cores to be used: 5
First 10 bp will be disregarded when processing the methylation call string of Read 1
First 10 bp will be disregarded when processing the methylation call string of Read 2
Strand-specific outputs will be skipped. Separate output files for cytosines in CpG, CHG and CHH context will be generated
Merge CHG and CHH context to non-CpG context specified
Output path specified as: out/Seagate Hub/methyl/

Checking file >>P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam<< for signs of file truncation...


Now testing Bismark result file >P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam< for positional sorting (which would be bad...)	...passed!
Writing result file containing methylation information for C in CpG context to out/CpG_context_P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.txt.gz
sh: 1: cannot create out/Seagate: Permission denied
Writing result file containing methylation information for C in any other context to out/Non_CpG_context_P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.txt.gz
sh: 1: cannot create out/Seagate: Permission denied

Now reading in Bismark result file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam


Now reading in Bismark result file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam


Now reading in Bismark result file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam

Now reading in Bismark result file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam

Now reading in Bismark result file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam

skipping SAM header line:	@HD	VN:1.0	SO:queryname	SS:queryname:natural
skipping SAM header line:	@SQ	SN:1	LN:248956422
skipping SAM header line:	@SQ	SN:10	LN:133797422
skipping SAM header line:	@SQ	SN:11	LN:135086622
skipping SAM header line:	@SQ	SN:12	LN:133275309
skipping SAM header line:	@SQ	SN:13	LN:114364328
skipping SAM header line:	@SQ	SN:14	LN:107043718
skipping SAM header line:	@SQ	SN:15	LN:101991189
skipping SAM header line:	@SQ	SN:16	LN:90338345
skipping SAM header line:	@SQ	SN:17	LN:83257441
skipping SAM header line:	@SQ	SN:18	LN:80373285
skipping SAM header line:	@SQ	SN:19	LN:58617616
skipping SAM header line:	@SQ	SN:2	LN:242193529
skipping SAM header line:	@SQ	SN:20	LN:64444167
skipping SAM header line:	@SQ	SN:21	LN:46709983
skipping SAM header line:	@SQ	SN:22	LN:50818468
skipping SAM header line:	@SQ	SN:3	LN:198295559
skipping SAM header line:	@SQ	SN:4	LN:190214555
skipping SAM header line:	@SQ	SN:5	LN:181538259
skipping SAM header line:	@SQ	SN:6	LN:170805979
skipping SAM header line:	@SQ	SN:7	LN:159345973
skipping SAM header line:	@SQ	SN:8	LN:145138636
skipping SAM header line:	@SQ	SN:9	LN:138394717
skipping SAM header line:	@SQ	SN:MT	LN:16569
skipping SAM header line:	@SQ	SN:X	LN:156040895
skipping SAM header line:	@SQ	SN:Y	LN:57227415
skipping SAM header line:	@SQ	SN:KI270728.1	LN:1872759
skipping SAM header line:	@SQ	SN:KI270727.1	LN:448248
skipping SAM header line:	@SQ	SN:KI270442.1	LN:392061
skipping SAM header line:	@SQ	SN:KI270729.1	LN:280839
skipping SAM header line:	@SQ	SN:GL000225.1	LN:211173
skipping SAM header line:	@SQ	SN:KI270743.1	LN:210658
skipping SAM header line:	@SQ	SN:GL000008.2	LN:209709
skipping SAM header line:	@SQ	SN:GL000009.2	LN:201709
skipping SAM header line:	@SQ	SN:KI270747.1	LN:198735
skipping SAM header line:	@SQ	SN:KI270722.1	LN:194050
skipping SAM header line:	@SQ	SN:GL000194.1	LN:191469
skipping SAM header line:	@SQ	SN:KI270742.1	LN:186739
skipping SAM header line:	@SQ	SN:GL000205.2	LN:185591
skipping SAM header line:	@SQ	SN:GL000195.1	LN:182896
skipping SAM header line:	@SQ	SN:KI270736.1	LN:181920
skipping SAM header line:	@SQ	SN:KI270733.1	LN:179772
skipping SAM header line:	@SQ	SN:GL000224.1	LN:179693
skipping SAM header line:	@SQ	SN:GL000219.1	LN:179198
skipping SAM header line:	@SQ	SN:KI270719.1	LN:176845
skipping SAM header line:	@SQ	SN:GL000216.2	LN:176608
skipping SAM header line:	@SQ	SN:KI270712.1	LN:176043
skipping SAM header line:	@SQ	SN:KI270706.1	LN:175055
skipping SAM header line:	@SQ	SN:KI270725.1	LN:172810
skipping SAM header line:	@SQ	SN:KI270744.1	LN:168472
skipping SAM header line:	@SQ	SN:KI270734.1	LN:165050
skipping SAM header line:	@SQ	SN:GL000213.1	LN:164239
skipping SAM header line:	@SQ	SN:GL000220.1	LN:161802
skipping SAM header line:	@SQ	SN:KI270715.1	LN:161471
skipping SAM header line:	@SQ	SN:GL000218.1	LN:161147
skipping SAM header line:	@SQ	SN:KI270749.1	LN:158759
skipping SAM header line:	@SQ	SN:KI270741.1	LN:157432
skipping SAM header line:	@SQ	SN:GL000221.1	LN:155397
skipping SAM header line:	@SQ	SN:KI270716.1	LN:153799
skipping SAM header line:	@SQ	SN:KI270731.1	LN:150754
skipping SAM header line:	@SQ	SN:KI270751.1	LN:150742
skipping SAM header line:	@SQ	SN:KI270750.1	LN:148850
skipping SAM header line:	@SQ	SN:KI270519.1	LN:138126
skipping SAM header line:	@SQ	SN:GL000214.1	LN:137718
skipping SAM header line:	@SQ	SN:KI270708.1	LN:127682
skipping SAM header line:	@SQ	SN:KI270730.1	LN:112551
skipping SAM header line:	@SQ	SN:KI270438.1	LN:112505
skipping SAM header line:	@SQ	SN:KI270737.1	LN:103838
skipping SAM header line:	@SQ	SN:KI270721.1	LN:100316
skipping SAM header line:	@SQ	SN:KI270738.1	LN:99375
skipping SAM header line:	@SQ	SN:KI270748.1	LN:93321
skipping SAM header line:	@SQ	SN:KI270435.1	LN:92983
skipping SAM header line:	@SQ	SN:GL000208.1	LN:92689
skipping SAM header line:	@SQ	SN:KI270538.1	LN:91309
skipping SAM header line:	@SQ	SN:KI270756.1	LN:79590
skipping SAM header line:	@SQ	SN:KI270739.1	LN:73985
skipping SAM header line:	@SQ	SN:KI270757.1	LN:71251
skipping SAM header line:	@SQ	SN:KI270709.1	LN:66860
skipping SAM header line:	@SQ	SN:KI270746.1	LN:66486
skipping SAM header line:	@SQ	SN:KI270753.1	LN:62944
skipping SAM header line:	@SQ	SN:KI270589.1	LN:44474
skipping SAM header line:	@SQ	SN:KI270726.1	LN:43739
skipping SAM header line:	@SQ	SN:KI270735.1	LN:42811
skipping SAM header line:	@SQ	SN:KI270711.1	LN:42210
skipping SAM header line:	@SQ	SN:KI270745.1	LN:41891
skipping SAM header line:	@SQ	SN:KI270714.1	LN:41717
skipping SAM header line:	@SQ	SN:KI270732.1	LN:41543
skipping SAM header line:	@SQ	SN:KI270713.1	LN:40745
skipping SAM header line:	@SQ	SN:KI270754.1	LN:40191
skipping SAM header line:	@SQ	SN:KI270710.1	LN:40176
skipping SAM header line:	@SQ	SN:KI270717.1	LN:40062
skipping SAM header line:	@SQ	SN:KI270724.1	LN:39555
skipping SAM header line:	@SQ	SN:KI270720.1	LN:39050
skipping SAM header line:	@SQ	SN:KI270723.1	LN:38115
skipping SAM header line:	@SQ	SN:KI270718.1	LN:38054
skipping SAM header line:	@SQ	SN:KI270317.1	LN:37690
skipping SAM header line:	@SQ	SN:KI270740.1	LN:37240
skipping SAM header line:	@SQ	SN:KI270755.1	LN:36723
skipping SAM header line:	@SQ	SN:KI270707.1	LN:32032
skipping SAM header line:	@SQ	SN:KI270579.1	LN:31033
skipping SAM header line:	@SQ	SN:KI270752.1	LN:27745
skipping SAM header line:	@SQ	SN:KI270512.1	LN:22689
skipping SAM header line:	@SQ	SN:KI270322.1	LN:21476
skipping SAM header line:	@SQ	SN:GL000226.1	LN:15008
skipping SAM header line:	@SQ	SN:KI270311.1	LN:12399
skipping SAM header line:	@SQ	SN:KI270366.1	LN:8320
skipping SAM header line:	@SQ	SN:KI270511.1	LN:8127
skipping SAM header line:	@SQ	SN:KI270448.1	LN:7992
skipping SAM header line:	@SQ	SN:KI270521.1	LN:7642
skipping SAM header line:	@SQ	SN:KI270581.1	LN:7046
skipping SAM header line:	@SQ	SN:KI270582.1	LN:6504
skipping SAM header line:	@SQ	SN:KI270515.1	LN:6361
skipping SAM header line:	@SQ	SN:KI270588.1	LN:6158
skipping SAM header line:	@SQ	SN:KI270591.1	LN:5796
skipping SAM header line:	@SQ	SN:KI270522.1	LN:5674
skipping SAM header line:	@SQ	SN:KI270507.1	LN:5353
skipping SAM header line:	@SQ	SN:KI270590.1	LN:4685
skipping SAM header line:	@SQ	SN:KI270584.1	LN:4513
skipping SAM header line:	@SQ	SN:KI270320.1	LN:4416
skipping SAM header line:	@SQ	SN:KI270382.1	LN:4215
skipping SAM header line:	@SQ	SN:KI270468.1	LN:4055
skipping SAM header line:	@SQ	SN:KI270467.1	LN:3920
skipping SAM header line:	@SQ	SN:KI270362.1	LN:3530
skipping SAM header line:	@SQ	SN:KI270517.1	LN:3253
skipping SAM header line:	@SQ	SN:KI270593.1	LN:3041
skipping SAM header line:	@SQ	SN:KI270528.1	LN:2983
skipping SAM header line:	@SQ	SN:KI270587.1	LN:2969
skipping SAM header line:	@SQ	SN:KI270364.1	LN:2855
skipping SAM header line:	@SQ	SN:KI270371.1	LN:2805
skipping SAM header line:	@SQ	SN:KI270333.1	LN:2699
skipping SAM header line:	@SQ	SN:KI270374.1	LN:2656
skipping SAM header line:	@SQ	SN:KI270411.1	LN:2646
skipping SAM header line:	@SQ	SN:KI270414.1	LN:2489
skipping SAM header line:	@SQ	SN:KI270510.1	LN:2415
skipping SAM header line:	@SQ	SN:KI270390.1	LN:2387
skipping SAM header line:	@SQ	SN:KI270375.1	LN:2378
skipping SAM header line:	@SQ	SN:KI270420.1	LN:2321
skipping SAM header line:	@SQ	SN:KI270509.1	LN:2318
skipping SAM header line:	@SQ	SN:KI270315.1	LN:2276
skipping SAM header line:	@SQ	SN:KI270302.1	LN:2274
skipping SAM header line:	@SQ	SN:KI270518.1	LN:2186
skipping SAM header line:	@SQ	SN:KI270530.1	LN:2168
skipping SAM header line:	@SQ	SN:KI270304.1	LN:2165
skipping SAM header line:	@SQ	SN:KI270418.1	LN:2145
skipping SAM header line:	@SQ	SN:KI270424.1	LN:2140
skipping SAM header line:	@SQ	SN:KI270417.1	LN:2043
skipping SAM header line:	@SQ	SN:KI270508.1	LN:1951
skipping SAM header line:	@SQ	SN:KI270303.1	LN:1942
skipping SAM header line:	@SQ	SN:KI270381.1	LN:1930
skipping SAM header line:	@SQ	SN:KI270529.1	LN:1899
skipping SAM header line:	@SQ	SN:KI270425.1	LN:1884
skipping SAM header line:	@SQ	SN:KI270396.1	LN:1880
skipping SAM header line:	@SQ	SN:KI270363.1	LN:1803
skipping SAM header line:	@SQ	SN:KI270386.1	LN:1788
skipping SAM header line:	@SQ	SN:KI270465.1	LN:1774
skipping SAM header line:	@SQ	SN:KI270383.1	LN:1750
skipping SAM header line:	@SQ	SN:KI270384.1	LN:1658
skipping SAM header line:	@SQ	SN:KI270330.1	LN:1652
skipping SAM header line:	@SQ	SN:KI270372.1	LN:1650
skipping SAM header line:	@SQ	SN:KI270548.1	LN:1599
skipping SAM header line:	@SQ	SN:KI270580.1	LN:1553
skipping SAM header line:	@SQ	SN:KI270387.1	LN:1537
skipping SAM header line:	@SQ	SN:KI270391.1	LN:1484
skipping SAM header line:	@SQ	SN:KI270305.1	LN:1472
skipping SAM header line:	@SQ	SN:KI270373.1	LN:1451
skipping SAM header line:	@SQ	SN:KI270422.1	LN:1445
skipping SAM header line:	@SQ	SN:KI270316.1	LN:1444
skipping SAM header line:	@SQ	SN:KI270340.1	LN:1428
skipping SAM header line:	@SQ	SN:KI270338.1	LN:1428
skipping SAM header line:	@SQ	SN:KI270583.1	LN:1400
skipping SAM header line:	@SQ	SN:KI270334.1	LN:1368
skipping SAM header line:	@SQ	SN:KI270429.1	LN:1361
skipping SAM header line:	@SQ	SN:KI270393.1	LN:1308
skipping SAM header line:	@SQ	SN:KI270516.1	LN:1300
skipping SAM header line:	@SQ	SN:KI270389.1	LN:1298
skipping SAM header line:	@SQ	SN:KI270466.1	LN:1233
skipping SAM header line:	@SQ	SN:KI270388.1	LN:1216
skipping SAM header line:	@SQ	SN:KI270544.1	LN:1202
skipping SAM header line:	@SQ	SN:KI270310.1	LN:1201
skipping SAM header line:	@SQ	SN:KI270412.1	LN:1179
skipping SAM header line:	@SQ	SN:KI270395.1	LN:1143
skipping SAM header line:	@SQ	SN:KI270376.1	LN:1136
skipping SAM header line:	@SQ	SN:KI270337.1	LN:1121
skipping SAM header line:	@SQ	SN:KI270335.1	LN:1048
skipping SAM header line:	@SQ	SN:KI270378.1	LN:1048
skipping SAM header line:	@SQ	SN:KI270379.1	LN:1045
skipping SAM header line:	@SQ	SN:KI270329.1	LN:1040
skipping SAM header line:	@SQ	SN:KI270419.1	LN:1029
skipping SAM header line:	@SQ	SN:KI270336.1	LN:1026
skipping SAM header line:	@SQ	SN:KI270312.1	LN:998
skipping SAM header line:	@SQ	SN:KI270539.1	LN:993
skipping SAM header line:	@SQ	SN:KI270385.1	LN:990
skipping SAM header line:	@SQ	SN:KI270423.1	LN:981
skipping SAM header line:	@SQ	SN:KI270392.1	LN:971
skipping SAM header line:	@SQ	SN:KI270394.1	LN:970
skipping SAM header line:	@PG	ID:Bismark	VN:v0.24.2	CL:"bismark --bowtie2 /usr/bin/bowtie2 -p 2 --multicore 5 --genome /home/data/genome/human/ENSEMBL/109/ -1 P4_gDNA_PBMCs_1.fastq.gz -2 P4_gDNA_PBMCs_2.fastq.gz"
skipping SAM header line:	@PG	ID:samtools	PN:samtools	PP:Bismark	VN:1.19	CL:/home/[email protected]/bin/samtools view -bSh -
skipping SAM header line:	@PG	ID:samtools.1	PN:samtools	PP:samtools	VN:1.19	CL:/home/[email protected]/bin/samtools view -h P4_gDNA_PBMCs_1.fastq.gz.temp.1_bismark_bt2_pe.bam
skipping SAM header line:	@PG	ID:samtools.2	PN:samtools	PP:samtools.1	VN:1.19	CL:/home/[email protected]/bin/samtools view -bSh -
skipping SAM header line:	@PG	ID:samtools.3	PN:samtools	PP:samtools.2	VN:1.19	CL:samtools sort -@ 5 -o P4_gDNA_PBMCs_1_bismark_bt2_pe_sort.bam P4_gDNA_PBMCs_1_bismark_bt2_pe.bam
skipping SAM header line:	@PG	ID:samtools.4	PN:samtools	PP:samtools.3	VN:1.19	CL:samtools sort -@ 7 -n -o P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam P4_gDNA_PBMCs_1_bismark_bt2_pe_sort.bam
skipping SAM header line:	@PG	ID:samtools.5	PN:samtools	PP:samtools.4	VN:1.19	CL:/home/[email protected]/bin/samtools view -h P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam

It did generate a report but appears truncated:

Here is the file P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn_splitting_report.txt generated:

P4_gDNA_PBMCs_1_bismark_bt2_pe_sortn.bam

Parameters used to extract methylation information:
Bismark Extractor Version: v0.24.2
Bismark result file: paired-end (SAM format)
Ignoring first 10 bp of Read 1
Ignoring first 10 bp of Read 2
Output specified: comprehensive
No overlapping methylation calls specified
Methylation in CHG and CHH context will be merged into "non-CpG context" output

Versions

#samtools version

samtools --version
samtools 1.19
Using htslib 1.19

bowtie2 version

bowtie2 --version
/home/[email protected]/bin/bowtie2-2.5.1-linux-x86_64/bowtie2-align-s version 2.5.1
64-bit
Built on 0ba86a911637
Wed Jan 18 03:20:56 UTC 2023
Compiler: gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC) 
Options: -O3 -msse2 -funroll-loops -g3 -g -O2 -fvisibility=hidden -I/hbb_exe_gc_hardened/include -ffunction-sections -fdata-sections -fstack-protector -D_FORTIFY_SOURCE=2 -fPIE -std=c++11 -DPOPCNT_CAPABILITY -DNO_SPINLOCK -DWITH_QUEUELOCK=1 -DWITH_ZSTD
Sizeof {int, long, long long, void*, size_t, off_t}: {4, 8, 8, 8, 8, 8}

Bismark

 ~/bin/Bismark-0.24.2/bismark --version


          Bismark - Bisulfite Mapper and Methylation Caller.

                       Bismark Version: v0.24.2
        Copyright 2010-23 Felix Krueger, Altos Bioinformatics  
                https://github.com/FelixKrueger/Bismark

Upon further investigation I think this is a kernel error: " it is sent a SIGPIPE signal from the kernel and it exits with a status of 141." With an additional comment:

Conventionally, an exit status N greater than 128 indicates the program was terminated by signal N - 128. Since SIGPIPE is signal 13, 141 - 128 = 13 indicates your program was ended by a SIGPIPE.

I think this is because bismark attempts to create a directory when there are spaces in the path:
See the log message: sh: 1: cannot create out/Seagate: Permission denied that doesn't have the full path out/Seagate Hub/methyl path.

I'll soon test the same command with a path without spaces. This might help other users like at #607

@llrs
Copy link
Author

llrs commented Sep 9, 2024

Yes, it works if the path doesn't have spaces. I'm sorry, my perl is a little bit rusty and I don't know if I will be able to find how to identify where a patch might be needed.

@FelixKrueger
Copy link
Owner

Thanks for reporting all the details on this. I am not surprised that spaces within paths lead to undesired behaviour, undoubtedly there will be further issues with special characters, slashes, interpunction and so on.

Rather than catering for all sorts of edge cases, couldn't one just use a folder names that don't require escaping \?

@llrs
Copy link
Author

llrs commented Sep 9, 2024

Yes, I renamed the external drive that had spaces on it. But it would also be nice if bismark could handle this, or provide an informative error if it detects one of those special cases. Feel free to close the issue as you see fit.

@FelixKrueger
Copy link
Owner

Thanks @llrs, I think catching these issues during submission is a good idea. I will add it to my list of things that are nice-to-have.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants