SGE queue configuration

From Research Computing

SGE queues

In SGE 5.3p3, queue were assigned to individual hosts, and had to be combined logically into groups by associating a so-called "complex", such as the "p4 complex"; such complexes had to be requested as resources. Many queues were needed under this system.

The new system, under SGE 6.x, is streamlined. Queues are associated with hostgroups and may have one or more access control lists associated with them.

To show the available queues, use qconf -sql; for example,

flengyel@m254 ~ $ qconf -sql
all.q
instructional.q
p3.q
p4.q
research.q

Each queue is associated with a hostgroup. There are five hostgroups defined on the computational cluster. These may be displayed with the command qconf -shgrp:

flengyel@m254 ~ $ qconf -shgrpl
@allhosts
@instructional
@p3hosts
@p4hosts
@researchosts

The order of the hostgroups in this listing corresponds with the order of queues in the qconf -sql listing. For example, all.q has the hostgroup @allhosts, which contains every host in the computational cluster: m01-m69.

The hosts in a given hostgroup can be listed with the command qconf -shgrp. For example, the hosts in the @p4hosts hostgroup are as follows.

flengyel@m254 ~ $ qconf -shgrp @p4hosts
group_name @p4hosts
hostlist m31.gc.cuny.edu m32.gc.cuny.edu m33.gc.cuny.edu m34.gc.cuny.edu \
         m35.gc.cuny.edu m36.gc.cuny.edu m37.gc.cuny.edu m38.gc.cuny.edu \
         m39.gc.cuny.edu m40.gc.cuny.edu m41.gc.cuny.edu m42.gc.cuny.edu \
         m43.gc.cuny.edu m44.gc.cuny.edu m45.gc.cuny.edu m46.gc.cuny.edu \
         m47.gc.cuny.edu m48.gc.cuny.edu m49.gc.cuny.edu m50.gc.cuny.edu \
         m51.gc.cuny.edu m52.gc.cuny.edu m53.gc.cuny.edu m54.gc.cuny.edu \
         m55.gc.cuny.edu m56.gc.cuny.edu m57.gc.cuny.edu m58.gc.cuny.edu \
         m59.gc.cuny.edu m60.gc.cuny.edu m61.gc.cuny.edu m62.gc.cuny.edu \
         m63.gc.cuny.edu m64.gc.cuny.edu m65.gc.cuny.edu m66.gc.cuny.edu \
         m67.gc.cuny.edu m68.gc.cuny.edu m69.gc.cuny.edu
flengyel@m254 ~ $                                          

To request the use of a queue in an SGE script, use the syntax

#$ -q <queue>

For example,

#$ -q p4.q

(Gaussian users are advised to select this queue--the code we have does not work on the p3's as of 10/8/06.)

Access control lists

Access control lists, also known in SGE parlance as usersets, have been implemented on the computational cluster. ACLs (access control lists) are lists of users. The ACLs defined are as follows.

deadlineusers: granted access to every resource.
research: granted access only to the research cluster.
students: granted access only to the instructional cluster.
grid: granted access to the grid.

Access control lists are used to allow or exclude access to SGE queues. If a username appears in a list of users excluded from access to a resource, the user is denied access even if the username appears in an allow list. In practice the ACLs are used to users access to queues. Each queue has an associated hostgroup, and one of the above ACLs.

Queue ACL assignments

The SGE queues are associated with usersets (named groups of users) as well as hostslists. The notation

<queue list> : ACL 

means that the list <queue list> of queues to the left of the colon is assigned the (comma separated) list of ACLs (access control lists) to the right of the colon. For example,

all.q : deadlineusers

means that the default queue all.q, whose hostgroup, @allhosts (see below), which contains every node in the research and instructional clusters, is assigned the deadlineusers userset. Only deadlineusers have access to this queue.

all.q           : deadlineusers, grid
p4.q            : deadlineusers, research
p3.q            : deadlineusers, research
instructional.q : deadlineusers, students
research.q      : deadlineusers, research