Biowulf at the NIH
RSS Feed
RSEQtools on Biowulf

A modular framework to analyze RNA-Seq data using compact and anonymized data summaries.

Programs Location

/usr/local/rseqtools/mrf

The following environmental variable need to be set correctly first:

For bash users:

$ export PATH=/usr/local/rseqtools/mrf:$PATH

For tcsh/csh users:

% set path=(/usr/local/rseqtools/mrf ${path})
Submitting a single RSEQtools batch job

1. Create a script file. The file will contain the lines similar to the lines below. Modify the path of location before running. Remember, the $PATH environmental variables have to be set correctly first. See http://helix.nih.gov/Applications/rseqtools.html for command examples.

#!/bin/bash
# This file is rseqtoolsscript
#
#PBS -N rseqtools
#PBS -m be
#PBS -k oe

export PATH=/usr/local/rseqtools/mrf:$PATH
cd /data/user/somewhereWithInputfile
.....
.....
cut -f1,2,3,4,5,8,9,10 knownGene.txt > knownGene.interval
export PATH=/usr/local/ucsc/bin/x86_64:$PATH
/usr/local/rseqtools/mrf/mergeTranscripts knownIsoforms.txt knownGene.interval compositeModel > knownGene_composite.interval
/usr/local/rseqtools/mrf/createSpliceJunctionLibrary /data/genome/gbdb/hg18/hg18.2bit knownGene.interval 20 > knownGene_2x20_spliceJunctions.fa
/usr/local/bowtie/bowtie-build -f /spin/db/genome/hg18/chr_all.fa
/usr/local/bowtie/bowtie-build -f knownGene_2x20_spliceJunctions.fa knownGene_2x20_spliceJunctions
....
....

3. Submit the script using the 'qsub' command on Biowulf, e.g. Note, user is recommend to run benchmarks to determine what kind of node is suitable for his/her jobs.

qsub -l nodes=1 /data/username/theScriptFileAbove

Useful commands:

freen: see http://biowulf.nih.gov/user_guide.html#freen

qstat: search for 'qstat' on http://biowulf.nih.gov/user_guide.html for it's usage.

jobload: search for 'jobload' on http://biowulf.nih.gov/user_guide.html for it's usage.

Submitting a swarm of rseqtools jobs

Using the 'swarm' utility, one can submit many jobs to the cluster to run concurrently.

Set up a swarm command file (eg /data/username/cmdfile). Here is a sample file:

export PATH=/usr/local/rseqtools/mrf:$PATH ;\
cd /data/user/run1/; \
/usr/local/rseqtools/mrf/mergeTranscripts knownIsoforms.txt knownGene.interval compositeModel > knownGene_composite.interval;\
export PATH=/usr/local/ucsc/bin/x86_64:$PATH;\ /usr/local/rseqtools/mrf/createSpliceJunctionLibrary /data/genome/gbdb/hg18/hg18.2bit knownGene.interval 20 > knownGene_2x20_spliceJunctions.fa;\

export PATH=/usr/local/rseqtools/mrf:$PATH ;\
cd /data/user/run2/; \
mergeTranscripts knownIsoforms.txt knownGene.interval compositeModel > knownGene_composite.interval;\
/usr/local/rseqtools/mrf/createSpliceJunctionLibrary /data/genome/gbdb/hg18/hg18.2bit knownGene.interval 20 > knownGene_2x20_spliceJunctions.fa;\

.........

export PATH=/usr/local/rseqtools/mrf:$PATH ;\
cd /data/user/run10/; \
/usr/local/rseqtools/mrf/mergeTranscripts knownIsoforms.txt knownGene.interval compositeModel > knownGene_composite.interval;\
/usr/local/rseqtools/mrf/createSpliceJunctionLibrary /data/genome/gbdb/hg18/hg18.2bit knownGene.interval 20 > knownGene_2x20_spliceJunctions.fa;\

There are one flag of swarm that's required '-f' and two other flags of swarm user most possibly needs to specify when submit a swarm job: '-t' and '-g'.

-f: the swarm command file name above (required)
-t: number of processors per node to use for each line of the commands in the swarm file above.(optional)
-g: GB of memory needed for each line of the commands in the swarm file above.(optional)

By default, each line of the commands above will be executed on '1' processor core of a node and uses 1GB of memory. If this is not what you want, you will need to specify '-t' and '-g' flags when you submit the job on biowulf.

Say if each line of the commands above also will need to use 10gb of memory instead of the default 1gb of memory, make sure swarm understands this by including '-g 10' flag:

biowulf> $ swarm -g 10 -f cmdfile

For more information regarding running swarm, see swarm.html

Running an interactive rseqtools job

User may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

biowulf% qsub -I -l nodes=1
qsub: waiting for job 2236960.biobos to start
qsub: job 2236960.biobos ready

[user@p4]$ cd /data/user/myruns
[user@p4]$ export PATH=/usr/local/rseqtools/mrf:$PATH/
[user@p4]$ cd /data/userID/rseqtools/run1
[user@p4]$ mergeTranscripts knownIsoforms.txt knownGene.interval compositeModel > knownGene_composite.interval;\
[user@p4]$ /usr/local/rseqtools/mrf/createSpliceJunctionLibrary /data/genome/gbdb/hg18/hg18.2bit knownGene.interval 20 > knownGene_2x20_spliceJunctions.fa;\
[user@p4]$ ...........
[user@p4] exit
qsub: job 2236960.biobos completed
[user@biowulf ~]$

User may add property of node in the qsub command to request specific interactive node. For example, if you need a node with 24gb of memory to run job interactively, do this:

biowulf% qsub -I -l nodes=1:g24

Documentation

http://archive.gersteinlab.org/proj/rnaseq/rseqtools