Research Computing:Services
From Research Computing
| Table of contents |
|
|
Intro
This page links to other pages which describe our hardware and software facilities, how to gain access, research computing projects, who we are, and our policies and procedures.
How to gain access
How to apply for a computational cluster account
How to obtain a Research Computing Wiki account
Getting Help
"Who you gonna call? Ghostbusters!"
If you are new to the computational cluster
If you're new to the computational cluster, see the sections on the computational cluster (http://research.gc.cuny.edu/wiki/index.php/Research_Computing:Services#Computational_Cluster) and the section on SUN Grid Engine Documentation (http://research.gc.cuny.edu/wiki/index.php/Research_Computing:Services#SUN_Grid_Engine_Documentation) for information on submitting and monitoring jobs using the SGE job manager.
If you are new to Linux
We can suggest a few online tutorials. The Quick Linux Tutorial (http://www.fsid.cvut.cz/cz/U201/LINUX.HTML) covers basic commands for interacting with a Linux System. A more comprehensive series of tutorials is available from Dr. Bob's Lowfat Linux (http://lowfatlinux.com/) site.
If something isn't working
If an online trouble isn't covered online, write to the Help Desk (mailto:helpdesk@gc.cuny.edu). Please include the following data:
- Your username
- the command you were running
- the job id (if applicable)
- error messages encountered
You can cut and paste error messages into your email message.
IMPORTANT: Please inform the help desk that your request concerns research computing.
Projects
Our projects include development and support of high-performance computational clusters and grids, and open source LAMP technology web sites.
The CUNY Grid Project
LAMP Web Sites
LAMP (http://www.onlamp.com/) is an acronym for Linux (http://www.linux.org/) Apache (http://www.apache.org/) MySQL (http://www.mysql.com/) PHP (http://www.php.net/)/PERL (http://www.perl.org/)/Python (http://www.python.org/). Research computing implemented the Graduate Center's first LAMP (http://www.onlamp.com/) web sites in April, 2005 CE (http://www.radix.net/~dglenn/defs/ce.html). LAMP technology is an open source alternative to proprietary operating systems and applications.
CUR Center for Urban Research
Research computing provides administration and assistance for several Center for Urban Research sites.
Online Analysis of Census Data
Survey Documentation and Analysis of Census Data (http://cur.gc.cuny.edu) The site currently has the NYC Public Urban Microdata Survey for the year 2000 census. US Census data coming soon!
Urban Research Wiki
Urban Research (http://urbanresearch.gc.cuny.edu)
OASIS NYC
Oasis NYC (http://oasisnyc.gc.cuny.edu) Maintained by the CUNY Mapping Service
Research Computing Wiki
Research Computing Web (http://research.gc.cuny.edu). The site you are viewing is implemented with the open source code wiki engine MediaWiki (http://wikipedia.sourceforge.net/).
Research Computing Request Tracker
In February 2006, Research Computing implemented a professional ticketing system on an old 500 MHz DELL 2450 using the freely available open source software Request Tracker v 3.4.5 (http://www.bestpractical.com/rt/), Apache 1.13.34 with mod_perl version 1.29, MySQL version 4.024. and PERL 5.8.7; all running under the free Debian 3.1 Linux operating system (stable release "Sarge"). In April 2008, the Request Tracker was upgraded to version 3.6 and moved to a Red Hat Enterprise Linux server.
Open Source Statistics
Research computing supports the open source statistical computing environment R (see the R Project (http://www.r-project.org/) website). To facilitate the import into the R system of SPSS syntax files for data in text format, we have available PSPP (http://www.gnu.org/software/pspp/) version 0.4.0, an open source clone of SPSS. Although PSPP provides a limited subset of the full SPSS functionality, it enables the conversion of an SPSS syntax file with text data to SPSS binary format, which can then be imported into R using the foreign library. This software is available on monad.gc.cuny.edu.
For further details on importing SPSS syntax data into R under Linux, please see the following link. The example illustrates the process with a dataset obtained from the ICPSR (http://www.icpsr.org).
Importing SPSS syntax data files into R under Linux using PSPP
CUNY Graduate Center Cambridge Structural Database and IsoStar Web
The Cambrdige Structural Database IsoStar system is now available on the web to authorized users at http://isostar.gc.cuny.edu. SSH access to the X Windows program Mogul is also enabled. Please contact the Assistant Director for Research Computing at the CUNY Graduate Center for an account and for assistance. If you already have a cluster account, you may use this to access the IsoStar web.
IsoStar Users
IsoStar users should download the IsoStar client software for local installation from http://isostar.gc.cuny.edu. Install the IsoStar client for your operating system only. Windows users should download the install program and run it. MIME file type associations (for the chemical/x-isostar MIME filetype) will work automatically with Microsoft Internet Explorer 7 and Firefox under Windows. Linux users will have to configure their browsers to recognize the ISTR MIME type so that the run_isostar program will be invoked when an ISTR file is downloaded from IsoStar; see below.
Running the Linux IsoStar Client under Firefox
Unzip the file isostar_client.tar.gz in a convenient location (in this example, /home/mydir). When you download an istar file from an IsoStar web page, your Firefox will prompt you to ask how to handle ISTR (chemical/x-isostar MIME type) files. Firefox will open a dialog box prompting you to tell it what to do with ISTR files. Select Open with and Browse to /home/mydir/isostar/bin/run_isostar. Also, check "Do this automatically for files like this from now on."
Mogul, Mercury and ConQuest Users
Use an SSH client with X-windows tunneling enabled. For example, from the Linux command line, use the command
ssh -X yourusername@isostar.gc.cuny.edu
to gain access. Windows users will need an SSH client with X-Windows tunneling enabled, as well as an X-Windows emulation program. First-time users will be prompted for the site number and the confirmation number; these numbers are displayed in the ssh login screen. The Cambridge Structural Database is installed under /usr/local/cambridge on the isostar.gc.cuny.edu machine. Your path on that machine includes the CSD binaries.
Computational Cluster
The CUNY Graduate Center supports a parallel LINUX for Graduate Center faculty and faculty sponsored research.
Cluster compute nodes
The cluster includes 59 compute nodes, including 20 Intel Q9550 2.83 GHz quad-core compute nodes, equipped with 8 gigabytes of RAM and 500-gigabyte hard drives; 30 Intel E6600 2.4 GHz dual-core compute nodes, equipped with 8 gigabytes of RAM and 250-gigabyte hard drives; and 9 Intel Pentium 4 3.2 GHz single processor compute nodes, equipped with 1 gigabyte of RAM and 80-gigabyte hard drives. Four terabytes of storage is available for user files.
Job submission, monitoring and control
Sun Microsystems Sun Grid Engine (SGE) version 6.0u10.
Cluster head nodes
There are three cluster head nodes: monad.gc.cuny.edu, grid.gc.cuny.edu and neptune.gc.cuny.edu. Access to 32-bit compilers and libraries is available on monad and grid. Access to 64-bit compilers and libraries is available on neptune. Submission of compiled code through the SGE is possibly on any of the cluster head nodes.
NYSGrid access node: cunygrid.gc.cuny.edu
Researchers who wish to submit jobs to the New York State Grid should use this node. All cluster accounts are valid on cunygrid. However, users should apply to research computing for a grid certificate to obtain access to NYSGrid.
Instructional cluster use
Graduate Center instructors may request access to the Linux cluster for their courses.
Research Computing Policies
Operational Policies Our policy is to go public with our policies.
Research Computing policy on guest system administrators
Account policy (http://research.gc.cuny.edu/wiki/index.php/Account_policy)
Installed Software
CPLEX linear, quadratic and integer programming solver
Running ILOG CPLEX on the cluster
To set up your account to run CPLEX, add the following line to your .bash_profile file in your home directory:
export ILOG_LICENSE_FILE=/home/nept/apps64/ILOG/ilm/license.dat
The ILOG CPLEX and AMPL software is installed under
/home/nept/apps64/ILOG/cplex
If you run batch jobs, you will need to add the following directories to the search path (the variable PATH):
/home/nept/apps64/ILOG/ampl/ /home/nept/apps64/ILOG/cplex/bin/x86-64_sles9.0_3.3/
Example:
export ILOG=/home/nept/apps64/ILOG export PATH=$ILOG/cplex/bin/x86-64_sles9.0_3.3/:$ILOG/ampl:$PATH
Example cplex files are located in the following directory on neptune.
/home/nept/apps64/ILOG/cplex/examples/src
Example ampl files are located in the following directory on neptune.
/home/nept/apps64/ILOG/ampl/examples
The examples follow the book AMPL: A modeliing language for mathematical programming.
You may run interactive sessions with qrsh:
qrsh -q quad.q
LEDA libraries
The 64-bit LEDA libraries and source are installed on neptune under /home/nept/apps64/LEDA
Also, the 32-but LEDA libraries are installed on monad under /home/m254/apps/LEDA.
We have both 32-bit and 64-bit systems available--researchers might want to run and compile 32-bit programs on the p4s (which will be upgraded) or on 64-bit machines.
Both locations contain the source code.
PETSc: Portable, Extensible Toolkit for Scientific Computation
From the PETSc website (http://www.mcs.anl.gov/petsc/petsc-as/): PETSc is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. It employs the MPI standard for all message-passing communication.
The 64-bit PETSc libraries are installed on neptune under /home/nept/apps64/petsc.
Emergent neural network simulator
Installed in /home/nept/apps64/emergent.
SUN Grid Engine Documentation
Submitting OpenMPI jobs to the Sun Grid Engine
Submitting MPICH jobs to the Sun Grid Engine
MPICH software installations and programming hacks
Submitting single-threaded Gaussian 03 jobs to the Sun Grid Engine
Instructional cluster: submitting gaussian jobs with gsub
Submitting parallel Gaussian 03/Linda jobs to the Sun Grid Engine
Submitting array jobs to the Sun Grid Engine
System Documentation
SGE mpich parallel execution environment (SGE 5.3p6)
SGE gauss parallel execution environment for Gaussian 03 with Linda (SGE 5.3p6)
SGE queue configuration This is updated and simplified to reflect the upgrade to SGE version 6.07u.
Globus 2.4 configuration Retained for details on configuration of the certificate authority (though this is better covered in the globus 4.x documentation).
Parallel Gaussian, util.so and LD_LIBRARY_PATH
Ganglia installation and configuration
System Administration Notes
Some of our adventures in system administration are recorded here (not including confidential system details).
Amanda configuration notes for the Spectra Logic 2K
A step by step tutorial on AMANDA (http://fedoranews.org/ghenry/amanda/) Note the step on editing /etc/xinetd/amanda.
Amanda cheat sheet (http://www.mast.queensu.ca/~mastop/service.catalogue/backup/cheat-sheet.html) This will be adapted for our site.
Notes on Disaster Recovery A work in progress.
Brute force SSH attacks (http://www.whitedust.net/article/27/Recent%20SSH%20Brute-Force%20Attacks/) Article on a common attack on Linux systems.
Unlocking a Digital UNIX account Worth a small fortune in consulting fees.
Software RAID for RDHCP server
Load balancing by bonding interfaces
Triple boot Knoppix CentOS Windows
External Links
Who we are
See About Research Computing (http://research.gc.cuny.edu/wiki/index.php/Research_Computing:About).
