Pipeline not progressing past checkBam

[RMann24]

Hi there,
I'm having problems getting RedDog to progress any further than Task=RedDog.checkBam.
All tasks up to this point are classified as "up-to-date". I have run ‘errorcheck.txt’ but it did not identify any issues.
There are "alignBowtiePE.Success" files in the /temp/success/ folder but no "checkBam.pbs" scripts in the /log/ folder.
Any help would be appreciated.

Rachel

[d-j-e]

Hi Rachel,

Can you check if the bams are actually being created? Check in the output/temp subfolders - unfortunately, the way the samtools part of the command works is to produce the bam file then populate it. So the pipeline is tricked into thinking the stage has successfully completed, even when it hasn't. Hence checkBam was added to the pipe - this simply checks that the bam file is at least a reasonable size.

BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces a series of tmp files to launch each job - are these being produced (in the reddog folder itself)?

Can you also try run the pipe again so I can see the error?
To capture the run just do:
rubra RedDog --config your_config --style run > run.txt

when you hit return, then hit 'y' and return and the pipe will run sending the output to the run.txt file.

I'm working today and tomorrow before taking a break until early January, so hopefully we can fix this asap...
cheers,
David

[d-j-e]

OOPS - didn't mean to close it!!!

[RMann24]

Thanks very much for your help David. The >run.txt only recorded the
initial info so I've just cut and past the info from the terminal into the
file. I should have mentioned that I am working at AgriBio and RedDog is
installed on our server here.

Rachel

Dr Rachel Mann | Research Scientist | Microbiology | Biosciences Research
Division
Department of Economic Development, Jobs, Transport and Resources
AgriBio, Centre for AgriBioscience, 5 Ring Road, Bundoora VIC 3086
T: +61 3 9032 7280 | E: rachel.mann@ecodev.vic.gov.au
W: Wednesday, Thursday and Friday

From: d-j-e notifications@github.com
To: katholt/RedDog RedDog@noreply.github.com,
Cc: RMann24 rachel.mann@ecodev.vic.gov.au
Date: 22/12/2015 03:07 PM
Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Hi Rachel,
Can you check if the bams are actually being created? Check in the
output/temp subfolders - unfortunately, the way the samtools part of the
command works is to produce the bam file then populate it. So the pipeline
is tricked into thinking the stage has successfully completed, even when
it hasn't. Hence checkBam was added to the pipe - this simply checks that
the bam file is at least a reasonable size.
BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces
a series of tmp files to launch each job - are these being produced (in
the reddog folder itself)?
Can you also try run the pipe again so I can see the error?
To capture the run just do:
rubra RedDog --config your_config --style run > run.txt
when you hit return, then hit 'y' and return and the pipe will run sending
the output to the run.txt file.
I'm working today and tomorrow before taking a break until early January,
so hopefully we can fix this asap...
cheers,
David
—
Reply to this email directly or view it on GitHub.

---

Department of Economic Development, Jobs, Transport and Resources, Government of
Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential
information. If you are not the intended recipient, you may not distribute or
reproduce this e-mail or the attachments. If you have received this message in
error, please notify us by return email.

---

RedDog V1beta.5 - phylogeny run

Copyright (c) 2015, David Edwards, Bernie Pope, Kat Holt
All rights reserved. (see README.txt for more details)

Mapping: Bowtie2 V2.2.3
Preset Option: --sensitive-local
3 replicon(s) in GenBank reference GCA_000022865.1_ASM2286v1_genomic
3 replicon(s) to be reported
78 sequence pair(s) to be mapped

Output folder:
/group/pathogens/Ba/Ba-redog/

Start Pipeline? (y/n)
Starting pipeline...
2291 jobs to be executed in total
2209 jobs left to execute

[rp50@dev3 RedDog]$ rubra RedDog --config RedDog_config_ba --style run > run.txt
y
Traceback (most recent call last):
File "/usr/local/bin/rubra", line 9, in
load_entry_point('Rubra==0.1.5', 'console_scripts', 'rubra')()
File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/rubra.py", line 66, in main
gnu_make_maximal_rebuild_mode=rebuildMode)
File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 2680, in pipeline_run
raise errt
ruffus.ruffus_exceptions.RethrownJobError:

Exceptions running jobs for

'def RedDog.checkBam(...):'

Original exceptions:

Exception #1
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC023.bam, ARC023.alignBowtiePE.Success] -> [ARC023.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #2
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[A0174.bam, A0174.alignBowtiePE.Success] -> [A0174.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #3
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC025.bam, ARC025.alignBowtiePE.Success] -> [ARC025.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #4
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC009.bam, ARC009.alignBowtiePE.Success] -> [ARC009.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #5
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC013.bam, ARC013.alignBowtiePE.Success] -> [ARC013.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #6
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[4599.bam, 4599.alignBowtiePE.Success] -> [4599.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #7
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC079.bam, ARC079.alignBowtiePE.Success] -> [ARC079.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #8
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC096.bam, ARC096.alignBowtiePE.Success] -> [ARC096.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #9
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC083.bam, ARC083.alignBowtiePE.Success] -> [ARC083.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #10
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC027.bam, ARC027.alignBowtiePE.Success] -> [ARC027.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #11
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC095.bam, ARC095.alignBowtiePE.Success] -> [ARC095.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #12
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC137.bam, ARC137.alignBowtiePE.Success] -> [ARC137.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #13
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC127.bam, ARC127.alignBowtiePE.Success] -> [ARC127.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #14
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC061.bam, ARC061.alignBowtiePE.Success] -> [ARC061.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #15
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC106.bam, ARC106.alignBowtiePE.Success] -> [ARC106.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #16
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC028.bam, ARC028.alignBowtiePE.Success] -> [ARC028.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #17
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC086.bam, ARC086.alignBowtiePE.Success] -> [ARC086.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #18
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC120.bam, ARC120.alignBowtiePE.Success] -> [ARC120.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #19
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC026.bam, ARC026.alignBowtiePE.Success] -> [ARC026.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #20
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC112.bam, ARC112.alignBowtiePE.Success] -> [ARC112.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #21
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC073.bam, ARC073.alignBowtiePE.Success] -> [ARC073.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #22
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC108.bam, ARC108.alignBowtiePE.Success] -> [ARC108.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #23
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC049.bam, ARC049.alignBowtiePE.Success] -> [ARC049.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #24
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC094.bam, ARC094.alignBowtiePE.Success] -> [ARC094.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #25
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC021.bam, ARC021.alignBowtiePE.Success] -> [ARC021.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #26
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC075.bam, ARC075.alignBowtiePE.Success] -> [ARC075.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #27
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC077.bam, ARC077.alignBowtiePE.Success] -> [ARC077.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #28
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC115.bam, ARC115.alignBowtiePE.Success] -> [ARC115.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #29
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC135.bam, ARC135.alignBowtiePE.Success] -> [ARC135.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #30
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC125.bam, ARC125.alignBowtiePE.Success] -> [ARC125.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #31
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC062.bam, ARC062.alignBowtiePE.Success] -> [ARC062.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #32
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC011.bam, ARC011.alignBowtiePE.Success] -> [ARC011.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #33
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC063.bam, ARC063.alignBowtiePE.Success] -> [ARC063.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #34
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC047.bam, ARC047.alignBowtiePE.Success] -> [ARC047.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #35
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC117.bam, ARC117.alignBowtiePE.Success] -> [ARC117.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #36
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC060.bam, ARC060.alignBowtiePE.Success] -> [ARC060.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #37
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC074.bam, ARC074.alignBowtiePE.Success] -> [ARC074.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #38
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC092.bam, ARC092.alignBowtiePE.Success] -> [ARC092.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #39
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC111.bam, ARC111.alignBowtiePE.Success] -> [ARC111.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #40
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC059.bam, ARC059.alignBowtiePE.Success] -> [ARC059.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #41
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC113.bam, ARC113.alignBowtiePE.Success] -> [ARC113.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #42
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC056.bam, ARC056.alignBowtiePE.Success] -> [ARC056.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #43
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC114.bam, ARC114.alignBowtiePE.Success] -> [ARC114.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #44
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC118.bam, ARC118.alignBowtiePE.Success] -> [ARC118.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #45
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC110.bam, ARC110.alignBowtiePE.Success] -> [ARC110.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #46
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC082.bam, ARC082.alignBowtiePE.Success] -> [ARC082.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #47
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC076.bam, ARC076.alignBowtiePE.Success] -> [ARC076.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #48
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC116.bam, ARC116.alignBowtiePE.Success] -> [ARC116.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #49
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC134.bam, ARC134.alignBowtiePE.Success] -> [ARC134.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #50
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC103.bam, ARC103.alignBowtiePE.Success] -> [ARC103.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found

[RMann24]

Sorry David, forgot to mention that .bam files have been produced in all
of the /temp subfolders

Dr Rachel Mann | Research Scientist | Microbiology | Biosciences Research
Division
Department of Economic Development, Jobs, Transport and Resources
AgriBio, Centre for AgriBioscience, 5 Ring Road, Bundoora VIC 3086
T: +61 3 9032 7280 | E: rachel.mann@ecodev.vic.gov.au
W: Wednesday, Thursday and Friday

From: d-j-e notifications@github.com
To: katholt/RedDog RedDog@noreply.github.com,
Cc: RMann24 rachel.mann@ecodev.vic.gov.au
Date: 22/12/2015 03:07 PM
Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Hi Rachel,
Can you check if the bams are actually being created? Check in the
output/temp subfolders - unfortunately, the way the samtools part of the
command works is to produce the bam file then populate it. So the pipeline
is tricked into thinking the stage has successfully completed, even when
it hasn't. Hence checkBam was added to the pipe - this simply checks that
the bam file is at least a reasonable size.
BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces
a series of tmp files to launch each job - are these being produced (in
the reddog folder itself)?
Can you also try run the pipe again so I can see the error?
To capture the run just do:
rubra RedDog --config your_config --style run > run.txt
when you hit return, then hit 'y' and return and the pipe will run sending
the output to the run.txt file.
I'm working today and tomorrow before taking a break until early January,
so hopefully we can fix this asap...
cheers,
David
—
Reply to this email directly or view it on GitHub.

---

Department of Economic Development, Jobs, Transport and Resources, Government of
Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential
information. If you are not the intended recipient, you may not distribute or
reproduce this e-mail or the attachments. If you have received this message in
error, please notify us by return email.

---

[d-j-e]

Oh dear, this appears to be a bug in the execution of a Rubra command - Bernie, one of the coauthors of RedDog, is also the main collaborator of Rubra so I'll have to check with him exactly what is going on... And I'm not sure if he's on holidays yet.

However, I'd also like for you to run the small test data described in the manual to see if the same thing happens with that data set if you have the time atm.

[RMann24]

Hi David,
apologies in the delay in getting back to you.
I have run the test set (see attached). It did not progress past making
the output directory.
The pipeline seems to intermittently progress to different points, even
with the same data sets. We'd appreciate any help, there are a few groups
here that would really like to use the pipeline,

kind regards,

Rachel

From: d-j-e notifications@github.com
To: katholt/RedDog RedDog@noreply.github.com,
Cc: RMann24 rachel.mann@ecodev.vic.gov.au
Date: 22/12/2015 04:17 PM
Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Oh dear, this appears to be a bug in the execution of a Rubra command -
Bernie, one of the coauthors of RedDog, is also the main collaborator of
Rubra so I'll have to check with him exactly what is going on... And I'm
not sure if he's on holidays yet.
However, I'd also like for you to run the small test data described in the
manual to see if the same thing happens with that data set if you have the
time atm.
—
Reply to this email directly or view it on GitHub.

---

Department of Economic Development, Jobs, Transport and Resources, Government of
Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential
information. If you are not the intended recipient, you may not distribute or
reproduce this e-mail or the attachments. If you have received this message in
error, please notify us by return email.

---

RedDog V1beta.8 - phylogeny run

Copyright (c) 2016 David Edwards, Bernie Pope, Kat Holt
All rights reserved. (see README.txt for more details)

Mapping: Bowtie2 V2.2.3
Preset Option: --sensitive-local
2 replicon(s) in GenBank reference reference
2 replicon(s) to be reported
3 sequence pair(s) to be mapped

Output folder:
/group/pathogens/Rachel/reddog_test_output/

Start Pipeline? (y/n)
Starting pipeline...
93 jobs to be executed in total

Traceback (most recent call last):
File "/usr/local/bin/rubra", line 9, in
load_entry_point('Rubra==0.1.5', 'console_scripts', 'rubra')()
File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/rubra .py", line 66, in main
gnu_make_maximal_rebuild_mode=rebuildMode)
File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 2680, in pi peline_run
raise errt
ruffus.ruffus_exceptions.RethrownJobError:

Exceptions running jobs for

'def RedDog.makeDir(...):'

Original exception:

Exception #1
exceptions.IOError([Errno 2] No such file or directory: '/group/pathogens/Ra                                       chel/reddog_test_output/temp/success/dir.makeDir.Success'):
for RedDog.makeDir.Job = [False -> dir.makeDir.Success]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in                                        run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_clea                                       nup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in                                        job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 950, in makeDir
    runStageCheck('makeDir', flagFile, outPrefix, full_sequence_list_string)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/u                                       tils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
IOError: [Errno 2] No such file or directory: '/group/pathogens/Rachel/reddo                                       g_test_output/temp/success/dir.makeDir.Success'

[d-j-e]

How are you editing the config file? - can you send me a copy to davide at unimelb dot edu dot au ?

[d-j-e]

Hi Rachel,
I did notice a couple of things with the config file you are using - The config files are both for version 1b5, but you are now running version 1b8 - try changing to the new version of config file (you may have to get Jason's help to include his "local hacks").
I also noticed a typo that may not be helping...
the following bit ends with a comma (this list will be slightly different to the one for v1b5, because we have turned back to Fasttree rather than raxml (you might have to install fasttree as well...):

}
stageDefaults = {
    "distributed": True,
    "walltime": "01:00:00",
    "memInGB": 4,
    "queue": 'batch',
    "modules": [
        "bwa",
        "samtools",
        "bcftools",
        "gcc",
        "bowtie2",
        "eautils",


    ]
}

The final entry in the modules list should not have a comma ie.

       "bowtie2",
        "eautils"
    ]
}

Have a go at those changes and let me know how it goes... If you are really keen to install this and still have problems, I may have to volunteer to visit you all (Did my undergrad at La Trobe many years ago...).