Computing instructions

From CompBio
Jump to: navigation, search

Contents

Access

Access from the inside

Find a workstation; log on to the console with your username and password.

Access from the outside

Logging in

To connect to our cluster from the outside, you need to log in to our gateway machine using ssh. On Linux or Mac OS X, open a terminal and run

ssh username@abyss.compbio.washington.edu

where username is your username.

On Windows you can use something like PuTTy; just enter the hostname abyss.compbio.washington.edu and click Open:

putty1.png

The connection from outside will work only if your public internet IP address or subnet is allowed by our firewall. For this to happen, you may need to email your friendly sysadm the public internet IP address of your machine/subnet from which you plan to connect. (Your public internet IP address is a sequence of four numbers that does not begin with 10, 172.16 through 172.31, or 192.168. See Wikipedia for more about this. To find your public internet IP address, visit one of the many sites indexed by Google that can answer your question.)

As an alternative to submitting your IP address to the sysadm, you can:

Note that the public internet IP address for abyss.compbio.washington.edu is 128.208.120.81, which you can try if you have problems with using the name. Also, if you don't like the name abyss, you can use limbo.compbio.washington.edu.

When connecting, you'll have to agree to the initial warning informing you about abyss' rsa2 key fingerprint. If you're using PuTTY it will look something like this:

putty2.png

Finally, you'll have to authenticate. If you're using PuTTY, this is the step where you enter your username at the login prompt. Otherwise, you'll jump straight to the password question. Your password will not be shown on the screen. Assuming you authenticate correctly, you should see the abyss prompt. For PuTTY, your terminal window should look something like this:

putty3.png

Because abyss is a gateway machine, and not the place to do real work, you will then need to log into a workstation for further access. At the abyss prompt, run:

ssh username@workstationname.compbio.washington.edu

Substitute workstationname for the actual workstation name. This should log you into the workstation assigned to you generally.

Changing your password

At the prompt of any machine on our network, use the yppasswd program to change your password. THIS IS IMPORTANT: YOU SHOULD DO THIS THE FIRST TIME YOU LOG IN TO CHANGE IT FROM THE DEFAULT!

File transfer

To copy a file from a remote location, on Linux or Mac OS X, you can open a terminal and run:

scp username@abyss.compbio.washington.edu:filename .

where filename is the full (or relative) path of the file.

On Mac OS X you can use a GUI program Fugu and on Windows you can use a GUI program WinSCP. For example, to use WinSCP you'll need to enter hostname abyss.compbio.washington.edu:

winscp1.png

Agree to the remote host's fingerprint:

winscp2.png

And finally you'll be greeted with a two-pane listing of files. The left side lists files on your computer, and the right lists files in your account as seen by abyss. You should be able to drag and drop files to copy them around.

winscp3.png

Please note that you are highly encouraged to become familiar and comfortable with Linux command line tools to do file operations instead of GUI tools. This will save you time and serve you better in the long term.

Computing infrastructure

Workstations

Workstations are divided into two categories: Desktops and Terminals.


Desktops

Desktops have dedicated CPU(s), RAM, hard drives, and peripherals (monitor, keyboard, mouse) only when the user assigned occupies a physical space in our group (i.e., doesn't work from home predominantly). More than one user may be assigned to a desktop, but they are expected to recognize the competition for resources that may occur and resolve contention issues on their own.


Terminals

Terminals don't have dedicated CPUs or hard drives, but do have peripherals so that users may use them to access our computing infrastructure. Users can be assigned to particular terminals, but this is not hard and fast. These users are given dedicated disk access which is what really defines their presence. The reason there is a "class" separation is because terminal users either (i) do not have their primary office space in the group; (ii) research only part-time in the group; (iii) are temporary visitors; or (iii) not very active.

If you are not assigned a dedicated desktop, then you are expected to dominantly use one of these terminals. You may ask a user assigned to a desktop if you can occasionally share the machine for convenience when collaborating with someone in an office, for computational efficiency, etc. Such an arrangement is fine provided all parties are in full knowledge of the resources (mainly CPU and memory) they are competing for.

These rules are not intended to disadvantage anyone and there should be enough computing resources to go around for everyone. As long as you play nice with others, things should be okay. For example, showing up unnannounced and using a random machine to check your email or perform some other low-impact activity is generally acceptable.


Workstation inventory

Here is a list of the desktops in our group, the location, primary purpose and primary users. The primary user designation is served for special long term users. In general, given joining and attrition, assignments are pretty variable and it's your responsibility to use the right desktop. An easy rule of thumb is that the desktop is used by the person with the same username as /maxa/home/username.

<
Desktop IP (10.20.10.N) Location Primary purpose
entropy 8 B70A desktop
emergence 9 B70A terminal
singularity 10 217 desktop
convergence 11 B70A desktop
time 13 B70A desktop
continuity 15 B70A desktop
eternity 16 B70A desktop
cosmos 17 217 desktop
maya 20 217 terminal
zen 1 B70A desktop

Due to human nature, this list may reflect the past instead of the present. To check a user's disk assignment, issue the command echo ~username.

Farm

The farm is where all large computations will be performed. The machines are named fpN (fp1, fp2, ...) and the IP addresses are 10.21.1.N where N corresponds to the number of the farm machine.

Servers

The servers serve data for external (and sometimes internal) use. The machines are named spN and the IP addresses are 10.21.1.N where N corresponds to the number of the server machine. These machines are designed to be identical mirrors of each other at least virtually (and thus the alias sp.compbio.washington.edu can map to any server machine). The current implementation is to have it so that one server can almost seamlessly replace another if one of them fails (i.e., sp2 can replace sp1), but the long term plan is to move dynamic information (mail, user-supplied data) to a network-accessible disk, which will enable all server machines to be treated identically both internally and externally (i.e., the IP addresses can rotate).

The addresses visible to the external world in terms of servers are combio.washington.edu (for e-mail and the main group webpages) and <resource>.compbio.washington.edu where <resource> is a series of services like bioverse, protinfo, dd, software, and data. The special resource abyss is available to users for working remotely from select external networks.

People with accounts on the server machine will not be able to login through the use of a password (you will need to setup a .rhosts file in your home directory on a server machine via NFS). Also, only http and mail access to the server address from the outside world is permitted (and this the only address you should be using to send and receive e-mail from the group). Any data you store on the server machine should be a mirror of what you already have internally, and you should be prepared to lose this data at any time.

Resources

Resources include printers, shared workstations, network storage devices, and have IP addresses of the form 10.20.12.N.


Shared workstations

These are powerful workstations that enable local computations to be performed to benefit both our group and the public use. There is currently one shared workstation: cosmos.compbio.washington.edu. These workstations contain CPU and hard drives for use by the entire group.


Printers

There is one printer: papyrus.compbio.washington.edu. It can be configured using the printtool utility (the queue name is papyrus, and the IP address can be gotten by pinging the host name). Do not set this printer to operate in duplex (double page) mode and then forget to reset it back!

Usability issues

Organisation

Generally, all user space directories will be under /home/<username> on a given disk, and there will only be one user on a disk (to ensure that disks don't get full).

All desktops and shared workstations will have two user space mountpoints, /maxa and /maxb, and a given user directory will be under the /home directory, for example: /maxa/home/<username>. There will also be a /spare mountpoint consisting of a partition from the system disk to be used for emergencies. Do not create directories outside of this system -- it makes accounting and maintenance a pain (and it could result in your files being deleted). /maxa and /maxb can be a combination of single disks, logical volumes, or RAIDs.

Terminals will not have hard drives other than the system drive. That is, the only mountpoint will be a /.

All farm machines will have will have a / and /maxa mountpoints. The /maxa mountpoints on the farm can be used for archival purposes (with directories created using the structure described above).

Servers will have mountpoints going up to /maxf, each consisting of a single disk. /maxg will consist of a single logical volume or a RAID.

Network storage devices will have a hostname associated with it, as well as any number of mountpoints as necessary, following the /maxX convention.


Getting started/installation of additional packages

If you're just getting started, I recommend copying the .tcshrc, .login, .aliases and .emacs files from the ~ram directory.

I recommend installing only RPMs in the system space and all non-RPM executables/packages in your own user space. Anything installed in the system space can disappear at any time (in the event of an upgrade, etc.).

Generally, there is a lot of freedom for users to install any OS of their choice on their machines, though it has to be compatible with Red Hat Linux. Specifically, rsh, rlogin, and finder have to work.


Running programs

Avoid using other people's desktops for your jobs. Run stuff on the farm with a nice value of 19, with no more than one CPU-consuming process per processor. You can check the behaviour of your processes with the top or several other commands. If your processes do not behave this way (i.e., the nice value isn't 19, and/or you have too many processes), then you need to kill them, figure out what the problem is, and then restart your jobs.

Always use the full pathname for the nice command (since there's a shell built-in command with the same name that doesn't quite work the same).


Access to machines

All machines are cross mounted with regards to each other. You can access different machines by cding to /hosts/&;t;machine>. Machine hang-ups and crashes

Given the work we do, it's inevitable that machines will routinely hang or crash. There are several ways to fix it when it happens.

If you have the root password, then you can try to identify the source of a problem and fix it, or you can shut down the machine cleanly (unmounting the disks so it doesn't get checked on reboot) and reboot.

If you don't have the root password, or can't find someone who does, and you need the machine back up soon, just push the button. There is a very very very small chance you'll lose data. But I recommend taking this risk, since this is better than waiting more than a few minutes for someone else to fix it. A reboot without unmounting the disks may be slow if the filesystem will be checked.

If I am around, I'd be happy to help. The other systems people in the building may be able to help if they are not too busy. But in general it's worth pressing the button rather than wait.

Please use the root password extremely carefully. You could potentially destroy all data in our cluster with it. NEVER become root when logged in remotely, even over a secure connection! NEVER cross over to another filesystem when root! If you're ever in doubt, consult someone more knowledgeable, but it is better to be safe than sorry.


Backups and archiving

MAKE BACKUPS OF ALL YOUR DATA--YOU ARE RESPONSIBLE FOR ALL BACKUPS! Most people don't learn the lesson of making backups until they lose months of data (myself included :-). I recommend not learning it in this manner. Space is available on the farm to archive your data. Farm disks should only be used primarily for this purpose.

Archives of your most important data can be burned on machines with a CD or DVD writable drive. I recommend doing this and mailing a copy to someone very far away once in a while.

If you're making CDs of your directories, you should first use mkisofs to convert your files into a ISO image and then issue the appropriate cdrecord command:

mkisofs -r -o image-file directory cdrecord -v speed=4 dev=0,0,0 image-file

If you're just burning audio CDs, you can use this command provided your audio files are all converted to .cdr format:

cdrecord -v speed=4 dev=0,0,0 -audio *.cdr


Moving large amounts of data

Use tar with rsh instead of NFS. If you want to copy files from host1 to host2 and you're logged onto host1, type:

tar -cvf - directory | rsh host2 "cd directory ; tar -xvf - "

or if you're logged onto host2, type:

rsh host1 "tar -cvf - directory" | tar -xvf -


Caveats

Do NOT use Microsoft Windows conventions, filenames, and files. That is, files with extensions like ".exe" or ".htm", and files containing ^M. This may result in data loss and will end up being your headache.

Communication

Do NOT use Microsoft Windows conventions, filenames, and files. That is, files with extensions like ".exe" or ".htm", and files containing ^M. This may result in data loss and will end up being your headache.

You can either use the user@u.washington.edu address or user@compbio. The latter is less reliable than the former, but if you want to use one, write your friendly sysadm@compbio.

The following mailing aliases are defined for communication with everyone in the group depending on the topic. Generally most email should be sent to agroup@compbio unless it is of a personal/local nature (in which case it should go to pgroup@compbio.

  • group@compbio should be used for general group-related issues (both administrative and scientific) that can viewed by both group members and other people who are subscribed to the list (former group members, etc.)
  • agroup@compbio should be used for any issues that people need to be archived (including weekly reports, etc.). See the instructions for using the group archive for further information.
  • pgroup@compbio should be used for private group-related issues. This will include matters related to group meetings, non-group-related issues such as parties and outings, and so on. Members in this list will only include current/active members.
  • rgroupN@compbio is for members who are working with us from a remote location.

Computing conventions

  • Desktop names have to signify some philosophy.
  • Farm machines are of the form: fp1..fpN. fp0 is reserved.
  • Servers are of the form: sp1..spN. sp0 is reserved.
  • NAS machines are of form: nas0..nasN. nas0 is reserved.
  • Individidual disk mount points are of the form maxa, maxb, maxc, etc.
  • On desktops, there can only be the disk mount points maxa and maxb. maxb can be a single disk or a logical volume comprising of more than one disk.
  • Servers can have up to six disks (maxa-f), plus direct attached storage which is in the form dasa, dasb, etc.
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox