Introduction to Boston University’s Scientific Computing Facility Doug Sondak [email protected] Scientific Computing and Visualization

Introduction to Boston University’s Scientific Computing Facility

Doug Sondak

[email protected]

Scientific Computing

and Visualization

Outline

Introduction Hardware Software Account Information Tutorials Some Basic Unix Commands How to Run Batch

Information Services & Technology

2

04/19/23

Introduction

BU’s Scientific Computing Facility (SCF) addresses high-performance computing and visualization needs of the BU community

Large resource requirements CPU time Memory

One group used 3 centuries of CPU time last year!


3

04/19/23

Overview of SCV Scientific Computing and Visualization Computing

high-performance, parallel computers (SCF) support

parallelization optimization tutorials

Visualization support to create graphics, from publication figures to interactive 3D

displays and movies 15’x8’ tiled display for 3D visualization tutorials


4

04/19/23

Current Hardware - BladeCenter

IBM BladeCenter katana currently 272 processors heterogeneous mixture of blades

Intel Xeon and AMD Opteron 2.4-3.0 GHz blades contain 4 or 8 processors May use up to 32 processors at a time

parallel or serial Two blades each have 8 processors and 96 GB memory


5

04/19/23

Current Hardware – Blue Gene

IBM Blue Gene 1024 nodes, each with 2 processors PowerPC 440 processors

700 MHz 32-bit

only runs MPI codes each processor is relatively slow, so you need good scalability to

make it worthwhile May use 1024 processors during the day, all 2048 processors at

night


6

04/19/23

Current Hardware - pSeries

IBM p655 shared-memory machines Power4 processors 9 machines, each with 8 processors

6 machines 1.1 GHz, 16 GB memory 3 machines 1.7 GHz, 8 GB memory

login node is twister


7

04/19/23

Software

All machines run under Unix C, C++, Fortran, Java, Perl, Python, Tcl Matlab

all machines except Blue Gene PCT (Parallel Computing Toolbox)

Maple all machines except Blue Gene

Mathematica all machines except Blue Gene


8

04/19/23

Software (cont’d)

Gauss blade center, pSeries

R all machines except Blue Gene

VTK Paraview Maya


9

04/19/23

Accounts and Policies Accounts and allocations are based on “projects” Must be faculty or research staff to apply for projects

Project PI then adds accounts for students, collaborators, etc. May assign post-doc or admin. staff as “administrative contact”

Apply for project athttp://www.bu.edu/tech/accounts/special/research/accounts/applications/ Click on “Boston University faculty and research staff may apply for a new

project.” If you’re not sure how much time to ask for, 1000 katana hours is considered

modest The form has fields for requesting time on each machine, but the time is

actually all in one pot, and can be spent on any machine This will give you accounts on all machines except for Blue Gene


10

04/19/23

Accounts and Policies (cont’d)

For Blue Gene account, once you are awarded a “regular” account go back to http://www.bu.edu/tech/accounts/special/research/accounts/applications/

There is a paragraph starting with “Blue Gene accounts.” Click on the “SCF User Information Page” link. You will be requested for your user name and password Click on the “Update Personal Information” link


11

04/19/23

Disk Space

The disk space in your home directory is minimal Most project PI’s request a “project” directory

project directory contains disk space allocated to the specific project once your account has been activated go back to

http://www.bu.edu/tech/accounts/special/research/accounts/applications/

and click “request a Project Disk Space allocation.” a request of a few GB will be rubber-stamped large requests (hundreds of GB) will require stronger justification


12

04/19/23

“Buy-In” Nodes

Departments/groups may purchase their own BladeCenter nodes alternative to buying and administering their own machines

Group that owns node(s) gets priority for batch runs can be restricted to specific users if desired

May also have log-in node(s) for interactive use log-in node restricted to specified groups/users


13

04/19/23

Services

SCV has 3 components:1. system administration

excellent administrators! can often accommodate special requests

2. visualization can help with creating visualizations, including animations, for

presentations, publication, and display wall wall has 3D capability


14

04/19/23

Services (cont’d)

3. high-performance computing code parallelization and optimization

MPI OpenMP PCT (Matlab)

help with any large-scale computing issues algorithms coding

There is no charge for SCV services


15

04/19/23

Tutorials

SCV offers tutorials on a variety of computing and visualization topics

If you have interest in an area that we don’t presently cover, let us know!

Current offerings are in three areas Programming High-Performance Computing Visualization


16

04/19/23

Tutorials (cont’d)

Programming Intro to Matlab Intro to Fortran Intro to C/C++

High-Performance Computing Code Tuning (C, Fortran) Tuning Matlab Codes Intro to MPI Parallelization with OpenMP Matlab Parallel Computing Toolbox (PCT)


17

04/19/23

Tutorials (3)

Visualization Intro to Scientific Visualization Scientific Visualization using Paraview Scientific Visualization using VTK Scientific Visualization using Matlab Graphics Programming in C/C++: OpenGL and OpenSceneGraph Graphics and Images for Presentation and Publication


18

04/19/23

Some Basic Unix


19

04/19/23

Login

From a terminal (e.g., Xwin-32, putty)ssh katanathis requests a “secure shell” on katanayou’ll be prompted for your username and password

You’re now logged on to the blade server “katana” You’ll see something like “katana:~ %”

this is a Unix prompt, which is where you type in commands


20

04/19/23

Directories

“directories” are analogous to PC “folders” delimited by / rather than \ as on a PC

when you first log in, current directory is called your “home” directory

directory where you are located at any given time is your “working” directory pwd means “print working directory”

type pwd

to create a subdirectory, use mkdir (make directory) type mkdir sub1, where sub1 is the subdirectory name


21

04/19/23

File Management

ls lists all files and directories in your current directory type ls

you should see your new directory listed

some shorthand directory names:~ home directory

. current directory

.. one level above current directory

cd dirname (change directory) moves you to dirname type cd sub1 type pwd


22

04/19/23

File Management (cont’d)

type ls ~ lists everything in your home directory

type ls .. since you’re one directory below your home directory, this is the same

as ls ~

cp from to will copy a file from is the file name you’re copying from to is either a file name or a directory

if it’s a file name, the copy will be given that name if it’s a directory, the file will retain the old name and be placed in

the specified directory


23

04/19/23

File Management (3)

copy the file /scratch/sondak/emacs_file to your sub1 directory cp /scratch/sondak/emacs_file . we will use it later

a path is a full set of directories leading to a file or directory e.g., when you type pwd, it shows the path to your current directory

file suffixes generally don’t have any special meanings as they do under Windows there are some cases where they do have meaning, such as .m for a

Matlab program


24

04/19/23

Emacs

text files are created with an editor kind of like a fancy “notepad”

two most common Unix editors are vi and emacs since I use emacs, that’s what we’ll use here we’ll introduce some emacs commands using the file

emacs_file two ways to enter commands

menu shortcuts


25

04/19/23

Emacs Exercise make sure you’re in your sub1 directory type emacs emacs_file

the file will appear in an emacs window

try migrating around the file with the arrow keys you can also navigate by clicking on the desired location

click on the 0 in 0.282

to delete a character use Delete button hit Delete 3 times to delete 0.2 type 1.3 you’ve now changed 0.282 to 1.382


26

04/19/23

Emacs Exercise (cont’d) Highlight 0.288 with the mouse

Don’t include spaces before or after Edit -> Cut will delete the highlighted characters

can also use CTL-w

Oh no, we made a mistake! Edit -> Undo undoes the previous command. Try it; 0.288 should reappear. can also use CTL-x u can keep repeating undo, and it will undo last command, second-to-

last command, third-to-last command, etc.


27

04/19/23

Emacs Exercise (3) The point is the location on the left side of the cursor;

i.e., between the character on which the cursor resides and the character to its left.

The mark is similar to the point, but it’s location is set explicitly (i.e., doesn’t move with cursor).


28

04/19/23

Emacs Exercise (4) Suppose we want to delete all the characters between

(inclusively) 0.288 and 0.407. Set the cursor on the 0 in 0.288. To set the mark, CTL-spacebar

The note “Mark set” will appear at the bottom of the screen.

Move the cursor to the right of 0.407 We place it to the right of the 7 rather than on it because the point is

always to the left of the cursor

Edit -> Cut will delete all characters between the mark and the point shortcut is CTL-w


29

04/19/23

Emacs Exercise (5) For now, let’s put the characters back in by using

Edit -> Undo (or CTL-x u) Move the cursor to the start of the current line using

CTL-a (CTL-e moves to end of current line)

Delete (“kill”) the line with CTL-k Another CTL-k deletes the newline character


30

04/19/23

Emacs Exercise (6)

“Meta” key is the Esc key We will use M to refer to the Meta key The Meta key is hit prior to the subsequent key(s), not

simultaneously as with CTL Place the cursor at the top of the file M-> will move the cursor to the bottom of the file (don’t

confuse notation with arrows I’m using for menu notation; this is “Meta greater-than”)

M-< will move it back to the top


31

04/19/23

Emacs Exercise (7)

Whatever we deleted last is available in a buffer Move the cursor to the beginning of the “M1rel” line Edit -> Paste “yanks” the current buffer

can also use CTL-y


32

04/19/23

Emacs Exercise (8)

CTL-h gets you into the Emacs help function at prompt type ? to list types of help I often use “a” (apropos)

Type CTL-h Then type a at the prompt Then type “kill” at the prompt The window will split, listing all Emacs commands that

have something to do with the word kill


33

04/19/23

Emacs Exercise (9)

Type CTL-x 1 to get you back to the un-split screen Split screens can be very handy, especially for cutting

from one file and pasting in another Type CTL-x 2 to split screen horizontally

can open different files in each half of the screen toggle across split with CTL-o (“other”)

CTL-x 0 eliminates the half screen in which the cursor currently resides (try it) this is a zero, not an “oh”


34

04/19/23

Emacs Exercise (10) To save the modified file, File -> Save (current buffer)

can also use CTL-x CTL-s A note will appear at the bottom of the window saying the file has been

saved

To save it under a new name, File -> Save Buffer As… can also use CTL-x CTL-w You’ll be prompted for the name at the bottom of the screen Note that when you get these kinds of prompts, you can edit them using

emacs commands Type a file name and then move back and forth with arrow keys

File -> Exit Emacs (or CTL-x CTL-c) to quit Previous version will appear with ~ suffix


35

04/19/23

Katana Batch System


36

04/19/23

Katana Batch System If your run will take less than 10 minutes of CPU time,

you can run interactively Longer runs are submitted to the batch system

Sun Grid Engine

May use up to 32 processors at a time can be 32 serial runs can be one 32-processor parallel run can be combination

Serial runs are limited to 24 hours Parallel runs are limited to 5 hours


37

04/19/23

Katana Batch System (cont’d) Job Control File (JCF)

this is file you submit to batch system line starting with #! defines Unix shell lines starting with #$ are batch system commands lines starting with # are comments

Here’s a “minimalist” JCF for a serial run:


38

04/19/23

#!/bin/tcsh# run time limit hr:min:sec#$ -l h_rt=2:00:00mycode

Katana Batch System (3) JCF must have execute permission

chmod 755 myjcf

qsub myjcf submits your job to the queue qstat –u username checks status of all your jobs


39

04/19/23

Katana Batch System (4) Create a trivial Matlab code with some output, e.g.,

I called mine “dumbcode.m”

Create a JCF


40

04/19/23

a = 3;disp([‘a = ‘ int2str(a)]);

#!/bin/tcsh# run time limit hr:min:sec#$ -l h_rt=0:02:00matlab –r –nodisplay dumbcode

Katana Batch System (5) Submit to batch system When it’s done 2 files should be in your working

directory dumbcode.e###### dumbcode.o######

where ###### represents a 6-digit number

The .e file contains errors The .o file contains output


41

04/19/23

References General SCV/SCF information

http://www.bu.edu/tech/research/

Blade Server http://www.bu.edu/tech/research/computation/linux-cluster/katana-cluster/

Account Application and Maintenance http://www.bu.edu/tech/accounts/special/research/accounts/applications/

Slides from Live SCV Tutorials http://www.bu.edu/tech/research/training/presentations/list/

Web-based SCV Tutorials http://www.bu.edu/tech/research/training/tutorials/list/

Katana Batch System http://www.bu.edu/tech/research/computation/linux-cluster/katana-cluster/runningjobs/


42

04/19/23

Human Help

If you ever have any questions, feel free to contact us Doug Sondak, [email protected] Kadin Tseng, [email protected]


43

04/19/23

Documents

Introduction to Boston University’s Scientific Computing Facility Doug Sondak [email protected] Scientific Computing and Visualization