Biowulf at the NIH
RSS Feed
cgatools on Biowulf

Cgatools provide tools for downstream analysis of Complete Genomics data. The focus is to provide command line utilities. The general areas of functionality include genome comparison, format conversion, and reference tools.

The easiest way to add the cgatools executable to your path is by using the modules commands as in the example below:

helix% module avail cgatools

------------- /usr/local/Modules/3.2.9/modulefiles ----------------
cgatools/1.4.0.15 cgatools/1.5.0.31 cgatools/1.6.0.43

helix% module load cgatools

helix% module list
Currently Loaded Modulefiles:
  1) cgatools/1.6.0.43

helix% module unload cgatools/1.6.0.43

helix% module load cgatools/1.5.0.31

helix% module list
Currently Loaded Modulefiles:
  1) cgatools/1.5.0.31

Submitting a single cgatools batch job

1. Create a script file similar to the one below:

#!/bin/bash
# This file is bfast
#
#PBS -N cgatools
#PBS -m be
#PBS -k oe

module load cgatools

cd /data/user/somewhereWithInputfile

cgatools fasta2crr --input build36.fa.bz2 --output build36.crr
cgatools listcrr --reference build36.crr --output listcrr.out

3. Submit the script using the 'qsub' command on Biowulf.

qsub -l nodes=1 /data/username/theScriptFileAbove

This will submit the job to a node with at least 1 GB of memory and 2 cores. If you need more memory for this job, you can specify a node with more memory. e.g. if your job needs 8 GB of memory, submit with:

qsub -l nodes=1:g8 /data/username/theScriptFileAbove

Use 'freen' to see available types of nodes.

Submitting a swarm of cgatools jobs

Using the 'swarm' utility, one can submit many jobs to the cluster to run concurrently.

Set up a swarm command file (eg /data/username/cmdfile). Here is a sample file:

calldiff -reference myfile.crr --variantA file1a.var --variantB file1b.var
calldiff -reference myfile.crr --variantA file2a.var --variantB file2b.var
calldiff -reference myfile.crr --variantA file3a.var --variantB file3b.var
[...]

Submit this swarm with

swarm -f cmdfile --module cgatools

By default, each line of the commands above will be executed on '1' processor core of a node and uses 1GB of memory. If each of your cgatools commands above requires more than 1 GB of memory, you should specify the memory required by using the '-g #' flag to swarm, where # is the number of Gigabytes required by one command line.

Say if each line of the commands above also will need to use 10gb of memory instead of the default 1gb of memory, make sure swarm understands this by including '-g 10' flag:

biowulf> $ swarm -g 10 -f cmdfile

For more information regarding running swarm, see swarm.html

Running an interactive cgatools job

Users may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

biowulf% qsub -I -l nodes=1
qsub: waiting for job 2236960.biobos to start
qsub: job 2236960.biobos ready

[user@p4]$ cd /data/user/myruns
[user@p4]$ module load cgatools [user@p4]$ cd /data/userID/cgatools/run1
[user@p4]$ cgatools fasta2crr --input build36.fa.bz2 --output build36.crr
[user@p4]$ cgatools listcrr --reference build36.crr --output listcrr.out
[user@p4] exit
qsub: job 2236960.biobos completed
[user@biowulf ~]$

User may add property of node in the qsub command to request specific interactive node. For example, if you need a node with 24gb of memory to run job interactively, do this:

biowulf% qsub -I -l nodes=1:g24

Documentation

http://cgatools.sourceforge.net/docs/1.3.0/