Parallel Computing with Matlab on Biowulf
The Matlab Distributed Computing Toolbox and Distributed Computing Engine
enables you to develop distributed and parallel MATLAB applications and execute them
on the cluster. We have 10 Parallel Computing toolbox licenses and 16 Distributed Computing
engines. The licenses expire on June 1, 2012.
There is now a trial installation of Distributed Matlab on the Biowulf cluster. We will consider purchasing this product if there is sufficient interest from the Biowulf user community. The number of licenses will depend on demonstrated interest from users. Therefore, if you expect to use Distributed Matlab on the Biowulf cluster, please let us know by sending email to staff@helix.nih.gov.
Parallel Distributed Matlab includes functions based on the Message Passing Interface (MPI) standard that supports explicit communication, enabling you to develop parallel applications in MATLAB. The products also support parallel for loops and global array semantics via distributed arrays for creating parallel applications without explicit message passing.
Creating a Distributed Array job
See the Matlab documentation on distributed arrays.
- Create a a distributed array Matlab script.
Sample Distributed Matlab script (replace the italicized text with your own appropriate directory path)
------------------ file dce_rand.m --------------------------------- abc = findResource('scheduler','type','generic'); abc.DataLocation = '/data/user/mydir'; abc.ClusterMatlabRoot = matlabroot; abc.HasSharedFilesystem = true; abc.ClusterOsType = 'unix'; abc.SubmitFcn = @distributedSubmitFnc; abc.GetJobStateFcn = @getJobStateFcn; abc.DestroyJobFcn = @destroyJobFcn; get(abc) a=abc.createJob; a.createTask(@rand,1,{4,4}); a.createTask(@rand,1,{4,4}); a.createTask(@rand,1,{4,4}); a.createTask(@rand,1,{4,4}); a.submit; waitForState(a,'finished'); results=getAllOutputArguments(a); results{1:4} exit --------------------------------------------------------------- - Create a batch script. Sample script
---------------------------- file matlabdce.bat---------------- #!/bin/tcsh # #PBS -N Matlab #PBS -m be #PBS -k oe date cd /data/user/mydir matlab -nodisplay -r dce_rand ----------------------------------------------------------------
Note that the Matlab script is called 'dce_rand.m', but the '.m' extension should be left out when using the '-r filename' switch in the Matlab command line.
The alternate command-line syntax 'matlab -nodisplay < dce_rand.m' is not recommended, as it may cause type-ahead buffer warnings. - Submit the batch script:
qsub -l nodes=1 matlabdce.bat
Testing a Distributed Matlab job
If the main Matlab job is not compute-intensive, it could be run interactively on the Biowulf head node or an interactive batch session. Sample session: (user input in bold)
[user@biowulf ~]$ matlab -nodisplay -r dce_rand
< M A T L A B >
Copyright 1984-2005 The MathWorks, Inc.
Version 7.1.0.183 (R14) Service Pack 3
August 02, 2005
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
>> >> >> >> >> >> Type: 'generic'
DataLocation: '/home/user/'
HasSharedFilesystem: 1
Jobs: [1x1 distcomp.simplejob]
ClusterMatlabRoot: matlabroot
MatlabCommandToRun: [1x58 char]
SubmitFcn: @pbsSubmitFunc
Configuration: ''
>> >> >> >> >> >> Submitting job
126748[].biobos
>> >> >>
ans =
0.9501 0.8913 0.8214 0.9218
0.2311 0.7621 0.4447 0.7382
0.6068 0.4565 0.6154 0.1763
0.4860 0.0185 0.7919 0.4057
ans =
0.9501 0.8913 0.8214 0.9218
0.2311 0.7621 0.4447 0.7382
0.6068 0.4565 0.6154 0.1763
0.4860 0.0185 0.7919 0.4057
ans =
0.9501 0.8913 0.8214 0.9218
0.2311 0.7621 0.4447 0.7382
0.6068 0.4565 0.6154 0.1763
0.4860 0.0185 0.7919 0.4057
ans =
0.9501 0.8913 0.8214 0.9218
0.2311 0.7621 0.4447 0.7382
0.6068 0.4565 0.6154 0.1763
0.4860 0.0185 0.7919 0.4057
>> [user@biowulf ~]$
|
Monitoring a Distributed Array job
A distributed array job submits a single PBS job array. This job can be monitored with the qstat command.
[susanc@biowulf ~]$ qstat -u susanc
biobos:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
204183[].biobos susanc norm matlab_Job -- 1 1 -- -- B --
The state 'B' indicates that the job is an array job, with at least one subjob
in Run state. To see the status of all the subjobs in this job, use 'qstat -t'.
[susanc@biowulf ~]$ qstat -t -u susanc
biobos:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
204183[1].biobo susanc norm matlab_Job -- 1 1 -- -- R 00:01
204183[2].biobo susanc norm matlab_Job -- 1 1 -- -- R 00:01
204183[3].biobo susanc norm matlab_Job -- 1 1 -- -- X --
204183[4].biobo susanc norm matlab_Job -- 1 1 -- -- X --
204183[5].biobo susanc norm matlab_Job -- 1 1 -- -- X --
204183[6].biobo susanc norm matlab_Job -- 1 1 -- -- R 00:01
204183[7].biobo susanc norm matlab_Job -- 1 1 -- -- R 00:01
204183[8].biobo susanc norm matlab_Job -- 1 1 -- -- Q --
204183[9].biobo susanc norm matlab_Job -- 1 1 -- -- Q --
204183[10].biob susanc norm matlab_Job -- 1 1 -- -- Q --
In this job array, 4 subjobs are running ('R' state), 3 have exited ('X'), and 3 are still queued ('Q').
A Parallel (MPI) Job
- Sample Matlab script (replace the italicized text with your own appropriate directory path).
---------------------file dce_mpi.m---------------------------------------- pjob = findResource('scheduler', 'type','generic') pjob.ClusterMatlabRoot = matlabroot pjob.DataLocation = '/data/user/mydir' pjob.HasSharedFilesystem = true; pjob.ClusterOsType = 'unix'; pjob.ParallelSubmitFcn = @parallelSubmitFcn; pjob.GetJobStateFcn = @getJobStateFcn; pjob.DestroyJobFcn = @destroyJobFcn; MaxNumWorkers = 7 j2 = pjob.createParallelJob('Max', MaxNumWorkers ) j2.createTask(@rand,1,{4,4}) j2.submit j2.waitForState results = j2.getAllOutputArguments results{1:7} quit --------------------------------------------------------------------------- - Run this job on the command line:
[user@biowulf ~] matlab -nodisplay -r dce_mpi
This job has 7 workers. It will therefore be submitted to 4 nodes by the Matlab program. When the workers have completed the tasks, the results from the 7 workers will be displayed on the terminal window. - Or create a batch script and submit it to the batch system. Sample batch script:
---------------------------- file matlabdce.bat---------------- #!/bin/tcsh # #PBS -N Matlab #PBS -m be #PBS -k oe date cd /data/user/mydir matlab -nodisplay -r dce_mpi ----------------------------------------------------------------
Submit with:qsub -l nodes=1 matlabdce.bat
The main Distributed Matlab job will run on one node, which will then submit the Matlab MPI job to 4 nodes.
Documentation
Matlab Distributed Computing Engine
Distributed Computing Toolbox
Run Matlab in Batch mode on Unix Machines
Matlab's Search Results on Batch
Batch Queuing System on Biowulf
Matlab Help
Matlab FAQ
The Mathworks Homepage
Biowulf home page | Helix Systems | NIH
