Lab Cluster Usage

How to use the SGE queue

Introduction

Dear All,

The queue for the lab is ready. For a maximal efficiency it is important you use the queuing system to launch a job on a remote machine. If you connect to a node by ssh and launch something there is a risk that "qsub" send another one and then the machine WILL FREEZE.

How it works:? 1 - Just put you commend into a bash script like the following (called launch_T_coffee.sh):

---

#!/bin/bash source t_coffee -in sequence.fasta -mode psicoffee -output color_html

---

2- Then launch you script with the following command line: qsub -q ws_cn@italy,ws_cn@fiji -cwd -o $PWD -e $PWD launch_T_coffee.sh

STDOUT will go a file called launch_T_coffee.sh.oJOBNUMBER STDERR will go a file called launch_T_coffee.sh.eJOBNUMBER

-o and -e are used to declare the repertory where will be created the output and error files.

YOU HAVE TO ALWAYS DECLARE A LIST OF NODES YOU WANT TO WORK ON. THIS LIST CAN NOT INCLUDE A MACHINE OF SOMEBODY ELSE. Here is the list of the cpu included in the queue:

Machine     # cpus    #cpus lent to the ws_cn queue
------------------------------------------------------------
Argentina      2        0 (Ionas)
Taiwan         8        6 (Jia-Ming)
France         8        6 (Jeff)
Finland        8        6 (Carsten)
Ireland        8        6 (Cedrik)
Hungary        2        0 (Giovanni)
Palestine      8        6 (Paolo)
Cuba           4        2 (Alberto)
Canada         2        0 (Miquel)
Portugal       8        6 (Jose A)
Belgium        4        4
Italy          2        2
Germany        4        4
Fiji           2        2
Scotland       4        4

I suggest you create a variable in your bashrc: export MY_NODES="ws_cn@MYMACHINE,ws_cn@belgium,ws_cn@italy,ws_cn@germany,ws_cn@fiji,ws_cn@scotland"

And then you launch script like this: qsub -q $MY_NODES -cwd -o $PWD -e $PWD launch_T_coffee.sh

Please DO NOT use directly "-q ws_cn" because jobs may go to somebody else machine.

Cheat sheet of qstat command: extract from http://confluence.rcs.griffith.edu.au:8080/display/v20zCluster/SGE+cheat+sheet

qstat                                    current user jobs
qstat -u "*"                             all user jobs
qstat -g c                               show available nodes and load
qstat -f                                 detailed list of machines and job state 
qstat -explain c -j job-id               specific job status
qdel job-id                              delete job
qsub -l h_vmem=### job.sh                mem limit, see queue_conf(5) RESOURCE LIMITS

qstat -f -u "*"
qconf -mc
qsub -w v job.00                         Troubleshoot problems with queue/scheduling