Biowulf at the NIH
RSS Feed
Scythe on Biowulf

Scythe uses a Naive Bayesian approach to classify contaminant substrings in sequence reads. It considers quality information, which can make it robust in picking out 3'-end adapters, which often include poor quality bases.

Most next generation sequencing reads have deteriorating quality towards the 3'-end. It's common for a quality-based trimmer to be employed before mapping, assemblies, and analysis to remove these poor quality bases. However, quality-based trimming could remove bases that are helpful in identifying (and removing) 3'-end adapter contaminants. Thus, it is recommended you run Scythebefore quality-based trimming, as part of a read quality control pipeline.

The Bayesian approach Scythe uses compares two likelihood models: the probability of seeing the matches in a sequence given contamination, and not given contamination. Given that the read is contaminated, the probability of seeing a certain number of matches and mistmatches is a function of the quality of the sequence. Given the read is not contaminated (and is thus assumed to be random sequence), the probability of seeing a certain number of matches and mismatches is chance. The posterior is calculated across both these likelihood models, and the class (contaminated or not contaminated) with the maximum posterior probability is the class selected.

 

The environment variable(s) need to be set properly first. The easiest way to do this is by using the modules commands as in the example below.

$ module avail scythe
-------------------- /usr/local/Modules/3.2.9/modulefiles ----------------------
scythe/0.981

$ module load scythe

$ module list
Currently Loaded Modulefiles:
1) scythe/0.981 $ module unload scythe $ module load scythe/0.981 $ module show scythe ------------------------------------------------------------------- /usr/local/Modules/3.2.9/modulefiles/scythe/0.981: module-whatis Sets up scythe 0.981 prepend-path PATH /usr/local/apps/scythe/0.981 -----------------------------------------------------------------

 

Submitting a Single Batch Job

1. Create a script file alone the following lines:

#!/bin/bash
# This file is FileName
#
#PBS -N RunName
#PBS -m be
#PBS -k oe

module load scythe
scythe -a adapter_file.fasta -o trimmed_sequences.fasta sequences.fasta

3. Submit the script using the 'qsub' command on Biowulf.

qsub -l nodes=1 /data/username/theScriptFileAbove

 

Submitting a Swarm of Jobs

Using the 'swarm' utility, one can submit many jobs to the cluster to run concurrently.

Set up a swarm command file (eg /data/username/cmdfile). Here is a sample file:

module load scythe; cd /data/$USER/Dir1; scythe -a adapter_file.fasta -o trimmed_sequences.fasta sequences.fastq
module load scythe; cd /data/$USER/Dir2; scythe -a adapter_file.fasta -o trimmed_sequences.fasta sequences.fastq
module load scythe; cd /data/$USER/Dir3; scythe -a adapter_file.fasta -o trimmed_sequences.fasta sequences.fastq
[.....]

By default, each line of the commands above will be executed on '1' processor core of a node and uses 1GB of memory. If this is not what you want, you will need to specify '-g' flags when you submit the job on biowulf.

Say if each line of the commands above also will need to use 10gb of memory instead of the default 1gb of memory, make sure swarm understands this by including '-g 10' flag:

biowulf> $ swarm -g 10 -f cmdfileAbove

For more information regarding running swarm, see swarm.html

 

Running an Interactive Job

User may need to run jobs interactively sometimes. Such jobs should not be run on the Biowulf login node. Instead allocate an interactive node as described below, and run the interactive job there.

$ qsub -I -l nodes=1
qsub: waiting for job 2236960.biobos to start
qsub: job 2236960.biobos ready 
      
$ module load scythe
$ cd /data/user/somewhereWithInputfile
$ scythe -a adapter_file.fasta -o trimmed_sequences.fasta sequences.fastq
$ ...........
$ exit
qsub: job 2236960.biobos completed 

User may add property of node in the qsub command to request specific interactive node. For example, if you need a node with 8gb of memory to run job interactively, do this:

$ qsub -I -l nodes=1:g8

Documentation

https://github.com/vsbuffalo/scythe