Biowulf at the NIH
RSS Feed
Bam2Fastq on Biowulf

BAM-format files are used to store alignment information and unaligned reads from next-generation sequencing machines. This tool is intended to extract raw sequences (with qualities) from a BAM file.

The environment variable(s) need to be set properly first. The easiest way to do this is by using the modules commands as in the example below.

[user@biowulf]$ module avail bam2fastq
----------------------------- /usr/local/Modules/3.2.9/modulefiles --------------------------
bam2fastq/1.1.0

[user@biowulf]$ module load bam2fastq

[user@biowulf]$ module list
Currently Loaded Modulefiles:
  1) bam2fastq/1.1.0

[user@biowulf]$ module unload bam2fastq

[user@biowulf]$ module load bam2fastq/1.1.0

[user@biowulf]$ module list
Currently Loaded Modulefiles:
  1) bam2fastq/1.1.0

[user@biowulf]$ module show bam2fastq
-------------------------------------------------------------------
/usr/local/Modules/3.2.9/modulefiles/bam2fastq/1.1.0:

module-whatis    Sets up bam2fastq 1.1.0 
prepend-path     PATH /usr/local/bam2fastq 
-------------------------------------------------------------------

 

Submitting a single batch job

1. Create a batch script along the following lines.

#!/bin/bash
# This file is runbam2fastq
#
#PBS -N bam2fastq
#PBS -m be
#PBS -k oe

module load bam2fastq
cd /data/$USER/somewhereWithInputFile
bam2fastq --aligned -o myfile.fastq  myfile.bam

2. Submit the script using the 'qsub' command on Biowulf

[user@biowulf]$ qsub -l nodes=1 /data/$USER/runbam2fastq

Useful commands:

freen: see http://biowulf.nih.gov/user_guide.html#freen

qstat: search for 'qstat' on http://biowulf.nih.gov/user_guide.html for it's usage.

jobload: search for 'jobload' on http://biowulf.nih.gov/user_guide.html for it's usage.

 

Submitting a swarm of jobs

Using the 'swarm' utility, one can submit many jobs to the cluster to run concurrently.

Set up a swarm command file (eg /data/username/cmdfile). Here is a sample file:

bam2fastq /data/$USER/run1/myfile.bam
bam2fastq /data/$USER/run2/myfile.bam
bam2fastq /data/$USER/run3/myfile.bam
[.....]

 

There is one flag of swarm that is required: '-f'. However, bam2fastq may require that input files contain the hash or pound character '#'. If this is the case, please also include the swarm option --no-comment. Otherwise, swarm will interpret anything following the '#' character as a comment and will delete it.

By default, each line of the commands above will be executed on '1' processor core of a node and uses 1GB of memory. If this is not what you want, you will need to specify '-g' flags when you submit the job on biowulf.

Say if each line of the commands above also will need to use 10gb of memory instead of the default 1gb of memory, make sure swarm understands this by including '-g 10' flag:

[user@biowulf]$ swarm -g 10 --module bam2fastq -f cmdfile

--module: setup bam2fastq environmental variables for each swarm command

--no-comment: ignore '#' character

For more information regarding running swarm, see swarm.html

 

Running an interactive job

User may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

[user@biowulf] $ qsub -I -l nodes=1
qsub: waiting for job 2236960.biobos to start
qsub: job 2236960.biobos ready

[user@pXXX]$ cd /data/$USER/myruns [user@pXXX]$ bam2fastq myfile.bam........... [user@pXXX] exit qsub: job 2236960.biobos completed [user@biowulf]$

User may add property of node in the qsub command to request specific interactive node. For example, if you need a node with 24gb of memory to run job interactively, do this:

[user@biowulf]$ qsub -I -l nodes=1:g24:c16

 

Documentation

http://www.hudsonalpha.org/gsl/software/bam2fastq.php