Computer Systems Lab Courses
2CS 363 GMU Spring 2005 2
Programming Languages
Sebesta Fig. 1.2
Languages are • an abstraction used by the programmer to express an idea• interface to the underlying computer architecture
CS 363 GMU Spring 2005 3
Why study Programming Languages?
• Increases ability to express ideas in a language– wide variety of programming features
• Improves ability to choose appropriate language– Each language has strengths and weaknesses in term of
expressing ideas
• Improves ability to learn new languages– different paradigms, different features– What does the future of programming languages hold?
• Improves understanding of significance of implementation
• Provides ability to design new languages– Domain specific languages increasingly popular
CS 363 GMU Spring 2005 4
Language Paradigms
• Imperative– Central features are variables, assignment statements, and
iteration– Ex: C, Pascal, Fortran
• Object-oriented– Encapsulate data objects with processing– Inheritance and dynamic type binding– Grew out of imperative languages– Ex: C++, Java
• Functional– Main means of making computations is by applying functions to
given parameters– Ex: LISP, Scheme, Haskell
CS 363 GMU Spring 2005 5
Language Paradigms
• Logic– Declarative Rule-based – implicit control flow– Ex: Prolog
• Dataflow– Declarative Model computation as information flow – implicit control
flow– Inherently parallel
• Event-Driven– Continuous loop with handlers that respond to events generated in
unpredictable order, such as mouse clicks– Often an add-on feature– Ex: Java
• Concurrent– Multiple interacting processes– Often an add-on feature– Ex: Java, High Performance Fortran (HPF), Linda
CS 363 GMU Spring 2005 6
Programming Domains• Scientific applications
– One of the earliest uses of computers– Large number of floating point computations– Long running– Imperative (Fortran, C) and Parallel (High Performance Fortran)
• Business applications– Produce reports, use decimal numbers and characters– Increasingly toward web-centric (Java, Perl, XML-based languages)– Imperative (Cobol) and domain specific (SQL)
• Artificial intelligence– Model human behavior and deduction– Symbol manipulation– Functional (Lisp) and Logical (Prolog)
• Systems programming– Need efficiency because of continuous use– Parallel and event driven– Imperative (C)
• …
CS 363 GMU Spring 2005 7
Influences on Language Design
• Programming methodologies– 1950s and early 1960s: Simple applications; worry
about machine efficiency– Late 1960s: People efficiency became important;
readability, better control structures• Structured programming• Top-down design and step-wise refinement
– Late 1970s: Process-oriented to data-oriented• data abstraction
– Middle 1980s: Object-oriented programming
CS 363 GMU Spring 2005 8
Compilers
Scanner(lexical
analysis)
Parser(syntax
analysis)
CodeOptimizer
SemanticAnalysis
(IC generator)
CodeGenerator
SymbolTable
Sourcelanguage
tokens Syntacticstructure
Syntactic/semanticstructure
Machinelanguage
Computer
Input Data
Output
CS 363 GMU Spring 2005 9
Interpreters
Sourcelanguage Interpreter
Input Data
Output
CS 363 GMU Spring 2005 10
What makes a language successful?
• Expressive Power– Included features impact programmer use
• Ease of use for Novice– Pascal, Basic, Logo
• Ease of Implementation
• Excellent Compilers
• Economics, Patronage, Legacy
11
Comparative Languages
CS 363 GMU Spring 2005 12
FORTRAN - 1957
• FORmula TRANslating systems
• FORTRAN I - 1957
(FORTRAN 0 - 1954 - not implemented)– Designed by John Backus for the new IBM 704,
which had index registers and floating point hardware
– Environment of development:• Computers were small and unreliable• Applications were scientific• No programming methodology or tools• Machine efficiency was most important
CS 363 GMU Spring 2005 13
FORTRAN
• FORTRAN 90 - 1990– Modules– Dynamic arrays– Pointers– Recursion– CASE statement– Parameter type checking
CS 363 GMU Spring 2005 14
FORTRAN
• Contributions– First widely used programming language– Changed the way people interacted with computers– Set the standard for compilers
• Goal: generate machine code similar to that of machine language programmers highly optimized compiler
• Much of the theory of compilers was developed in this compiler.
CS 363 GMU Spring 2005 15
Sample Fortran subroutine defcolor(rgb,nframe) implicit none integer nframe integer ihpixf, jvpixf parameter(ihpixf = 256, jvpixf = 256) ! pixel size character*1 rgb(3,ihpixf,jvpixf) ! RGB data array local integer i, j, idummy do 100 j = 1, jvpixf do 100 i = 1, ihpixf idummy = i*3*nframe + j*2 idummy = mod(idummy,256) ! assuming color depth is 256 (0--255) rgb(1,i,j) = char(idummy) ! red idummy = i*1*nframe + j*3 idummy = mod(idummy,256) rgb(2,i,j) = char(idummy) ! green idummy = i*5*nframe + j*7 idummy = mod(idummy,256) rgb(3,i,j) = char(idummy) ! blue 100 continue return end
CS 363 GMU Spring 2005 16
Sample Fortran real*4 one,eps, ht, tf real*8 one8, eps8 one = 1. one8 = 1. eps = 1. eps8 = 1. ht = 100000 tf = 24. print *, 'Test for precision of real*4, based on
1+eps=1' 10 eps = eps/2. if (one .ne. one+eps) go to 10 eps = 2.*eps print *,' relative precision is ',eps print * print *,'Test for precision of real*4,', + 'based
on 100000+eps=100000' eps = 1. 15 eps = eps/2.
if (ht .ne. ht+eps) go to 15 eps = 2.*eps print *,' relative precision is ',eps print * print *,'Test for precision of real*4,', + 'based
on 24+eps=24' eps = 1. 17 eps = eps/2. if (tf .ne. tf+eps) go to 17 eps = 2.*eps print *,' relative precision is ',eps print * print *, 'Test for precision of real*8, based on
1+eps=1' 20 eps8 = eps8/2. if (one8 .ne. one8+eps8) go to 20 eps8 = 2.*eps8 print *,' relative precision is ',eps8 end
CS 363 GMU Spring 2005 17
LISP - 1959
• LISt Processing language• AI research needed a language that:
– Processed data in lists (rather than arrays)– Allowed symbolic computation (rather than numeric)
• Only two data types: atoms and lists• Syntax is based on lambda calculus• No variables or assignment• Control via recursion and conditional expressions• (A B C D) – apply function A to arguments B C D
CS 363 GMU Spring 2005 18
LISP
• Pioneered functional programming
• First interpreters slow
• COMMON LISP and Scheme are contemporary dialects of LISP
• ML, Miranda, and Haskell are related languages
CS 363 GMU Spring 2005 19
LISP Sample
Problem: remove the first occurrence of atom A from list L
(define (remove L A) (cond ( (null? L) '() ) ( (= (car L) A) (cdr L)) ; Match found! (else (cons (car L)(remove (cdr L) A))) ; keep searching ) )
CS 363 GMU Spring 2005 20
BASIC - 1964
• Beginner’s All-purpose Symbolic Instruction Code
• Designed by Kemeny & Kurtz at Dartmouth• Design Goals:
– Easy to learn and use for non-science students– “pleasant and friendly”– Fast turnaround for homework– Free and private access– User time is more important than computer time
• Current popular dialect: Visual BASIC • First widely used language with time sharing
CS 363 GMU Spring 2005 21
APL and SNOBOL
• Characterized by dynamic typing and dynamic storage allocation
• APL (A Programming Language) 1962– Designed as a hardware description language (at
IBM)– Highly expressive (many operators, for both
scalars and arrays of various dimensions)– Programs are very difficult to read
• SNOBOL(1964)– Designed as a string manipulation language (at
Bell Labs)– Powerful operators for string pattern matching
CS 363 GMU Spring 2005 22
Snobol Example
* Find biggest words and numbers in a test string * BIGP = (*P $ TRY *GT(SIZE(TRY,SIZE(BIG))) $ BIG FAIL STR = 'IN 1964 NFL ATTENDANCE JUMPED TO 4,807,884; ‘ 'AN INCREASE OF 401,810.' P = SPAN('0123456789,') BIG = STR BIGP OUTPUT = 'LONGEST NUMBER IS ' BIG P = SPAN('ABCDEFGHIJKLMNOPQRSTUVWXYZ') BIG = STR BIGP OUTPUT = 'LONGEST WORD IS ' BIG END
CS 363 GMU Spring 2005 23
ALGOL 68 - 1968
• From the continued development of ALGOL 60, but it is not a superset of that language
• Features:– User-defined data structures– Reference types– Dynamic arrays (called flex arrays)
• Contribution:– Had even less usage than ALGOL 60 BUT had
strong influence on subsequent languages, especially Pascal, C, and Ada
CS 363 GMU Spring 2005 24
Important ALGOL Descendants
• Pascal - 1971– Designed by Wirth, who quit the ALGOL 68
committee (didn't like the direction)– Designed for teaching structured
programming– Small, simple, nothing really new– From mid-1970s until the late 1990s, it was
the most widely used language for teaching programming in colleges
CS 363 GMU Spring 2005 25
Pascal Sample
program fibonacci(input, output);type natural 0..maxint;var fnm1,fnm2, fn,n,i: naturalbegin readln(n); if n <= 1 then writeln(n) {F0 = 0 and F1 = 1} else begin {compute Fn} fnm2 := 0; fnm1 := 1; for i := 2 to n do begin fn := fnm1 + fnm2; fnm2 := fnm1; fnm1 := fn; end writeln(fn); endend.
CS 363 GMU Spring 2005 26
Important ALGOL Descendants
• C - 1972– Designed for systems programming (at Bell
Labs by Dennis Richie)– Evolved primarily from B, but also ALGOL 68– Powerful set of operators, but poor type
checking– Initially spread through UNIX
CS 363 GMU Spring 2005 27
C Sample
#include <stdio.h>main() { int fnm1,fnm2, fn,n,i; scanf(“%d”,&n); if (n <= 1) printf(“%d\n”, n) /* F0 = 0 and F1 = 1*/ else { /* compute Fn*/ fnm2 = 0; fnm1 = 1; for (i = 2; i<=n ;++i) { fn = fnm1 + fnm2; fnm2 = fnm1; fnm1 = fn; } printf(“%d\n”,fn); }}
CS 363 GMU Spring 2005 28
Prolog - 1972
• Developed at the University of Aix-Marseille, by Comerauer and Roussel, with some help from Kowalski at the University of Edinburgh
• Applications in AI, DBMS• Based on formal logic• Non-procedural• Can be summarized as being an intelligent
database system that uses an inferencing process to infer the truth of given queries
• Inefficient execution
CS 363 GMU Spring 2005 29
Prolog Sample
parent(X,Y) :- mother(X,Y). parent(X,Y) :- father(X,Y). sibling(X,Y) :- mother(M,X), mother(M,Y),
father(D,X), father(D,Y),X \== Y. haschildren(X) :- parent(X,_). grandparent(X,Y) :-
parent(X,M),parent(M,Y). cousin(X,Y) :-
parent(A,X),parent(B,Y),sibling(A,B).
mother(anne,mary). mother(anne,liz). mother(anne,susan). mother(anne,virginia). mother(elizabeth,russ). mother(mabel,anne). father(james,zelie). father(james,harry). father(james,ned). father(john,will). father(john,russell). father(jim,rachel). father(jim,maggie). father(tim,patrick).
grandparent(anne,Y).
CS 363 GMU Spring 2005 30
Ada - 1983 (began in mid-1970s)
• Huge design effort, involving hundreds of people, much money, and about eight years
• Environment: More than 450 different languages being used for DOD embedded systems (no software reuse and no development tools)
• Named for Ada Lovelace (1815-1851) – first programmer (worked with Charles Babbage).
CS 363 GMU Spring 2005 31
Ada Sample
package ArrayCalc is type Mydata is private; function sum return integer; procedure setval(arg:in integer); private size: constant:= 99; type myarray is array(1..size) of
integer; type Mydata is record val: myarray; sz: integer := 0; end record; v: Mydata; end;
package body ArrayCalc is function sum return integer is temp: integer; -- Body of function sum begin temp := 0; for i in 1..v.sz loop temp := temp + v.val(i); end loop; v.sz:=0; return temp; end sum; procedure setval(arg:in integer) is begin v.sz:= v.sz+1; v.val(v.sz):=arg; end setval; end;
with Text_IO; use Text_IO; with ArrayCalc; use ArrayCalc; procedure main is k, m: integer; begin -- of main get(k); while k>0 loop for j in 1..k loop get(m); put(m,3); setval(m); end loop; new_line; put("SUM ="); put(ArrayCalc.sum,4); new_line; get(k); end loop; end;
CS 363 GMU Spring 2005 32
Smalltalk - 1972-1980
• Developed at Xerox PARC, initially by Alan Kay, later by Adele Goldberg
• First full implementation of an object-oriented language (data abstraction, inheritance, and dynamic type binding)
CS 363 GMU Spring 2005 33
C++ - 1985
• Developed at Bell Labs by Stroustrup
• Evolved from C and SIMULA 67 • Facilities for object-oriented programming, taken
partially from SIMULA 67, were added to C• Also has exception handling• A large and complex language, in part because it
supports both procedural and OO programming• Rapidly grew in popularity, along with OOP• ANSI standard approved in November, 1997
CS 363 GMU Spring 2005 34
C++ Related Languages
• Eiffel - a related language that supports OOP– (Designed by Bertrand Meyer - 1992)– Not directly derived from any other language– Smaller and simpler than C++, but still has most of
the power
• Delphi (Borland)– Pascal plus features to support OOP– More elegant and safer than C++
CS 363 GMU Spring 2005 35
Java (1995)
• Developed at Sun in the early 1990s
• Based on C++– Significantly simplified (does not include struct, union, enum, pointer arithmetic, and half of the assignment coercions of C++)
– Supports only OOP– Has references, but not pointers– Includes support for applets and a form of
concurrency
CS 363 GMU Spring 2005 36
Scripting Languages
• JavaScript– HTML-embedded at client side and executed by the
client (i.e. web browser)– create dynamic HTML documents
• PHP– HTML enabled server side– Interpreted by server when a document in which it is
embedded is requested.– Output of interpretation is HTML that replaces the
PHP
37
Supercomputer Applications
Topics
• Parallel computing and High Performance Computing
• Clusters
• MPI – Message Passing Interface– Passing messages among processes
• OpenGL and computer graphics applications
Parallel Computing Overview
● a high performance parallel computer is a computer that can solve large problems in a much shorter time than a single desktop computer.
● characterized by fast CPUs, large memory, a high speed interconnect, and high speed input/output.
● able to speed up computations; both by making the sequential components run faster and by doing more operations in parallel.
● High performance parallel computers are in demand because there is a need for tremendous computational capabilities in science, engineering, and business.
● There are applications that require gigabytes of memory and gigaflops of performance.
● Today, application scientists are striving for terascale performance, to permit an even larger class of problems to be solved.
● Terascale refers to computers that perform more than one trillion floating-point operations per second, called "teraflops"
● High performance parallel computers are used in a wide variety of disciplines. – Meteorologists use them for the prediction of
tornadoes and thunderstorms – help computational biologists analyze DNA
sequences– used by pharmaceutical companies in the design of
new drugs,
– by oil companies for their seismic exploration,– Wall Street for the analysis of financial markets– NASA uses them for aerospace vehicle design,– the entertainment industry uses them for special
effects in movies and commercials● The common characteristic of all these complex
scientific and business applications is the need to perform computations on large datasets or large equations.
Parallelism in Computer Programs● It used to be thought that computer programs were
sequential in nature● algorithm is defined as the "sequence of steps"
necessary to carry out a computation.● In the first 30 years of computer use, programs
were run sequentially because of this thinking. ● the 1980's saw great successes with parallel
computers.● Dr. Geoffrey Fox published a book entitled Parallel
Computing Works! helped lead to a reversal in thinking.
● Parallel computing is what a computer does when it carries out more than one computation at a time using more than one processor.
● If one processor can perform the arithmetic in time t, then ideally p processors can perform the arithmetic in time t/p.
● The benefit of parallelism is that it allows researchers to do computations on problems they previously were unable to solve.
Comparison of Parallel Computers
The hardware components of parallel computers:
* kinds of processors
* types of memory organization
* flow of control
* interconnection networks
We'll see what is common to these parallel computers, and what makes each one of them unique.
ProcessorsThere are three types of parallel computers:
1. computers with a small number of powerful processors
2. computers with a large number of less powerful processors
3. computers that are medium scale in between the two extremes
A Small Number of Powerful Processors
They are general-purpose computers that perform especially well on applications that have large vector lengths.
The examples of this type of computer are the Cray SV1 and the Fujitsu VPP5000.
A Large Number of Less Powerful Processors
Computers with a large number of less powerful processors, named a Massively Parallel Processor (MPP), typically have thousands of processors.
The processors are usually proprietary and air-cooled. Because of the large number of processors, the distance between the furthest processors can be quite large requiring a sophisticated internal network that allows distant processors to communicate with each other quickly.
Medium Scale Computers
Medium scale computers typically have hundreds of processors. The processor chips are usually not proprietary; rather they are commodity processors like the Pentium III.
These are general-purpose computers that perform well on a wide range of applications.
The most common example of this class is the Linux Cluster.
Trends and Examples
Decade Processor Type Computer Example
1970s Pipelined, Proprietary Cray-1
1980s Massively Parallel, Proprietary Thinking
Machines CM2
1990s Superscalar, RISC, Commodity SGI
Origin2000
2000s CISC, Commodity Workstation
Clusters
Background on MPI
• MPI - Message Passing Interface
– Library standard defined by a committee of vendors, implementers, and parallel programmers
– Used to create parallel SPMD programs based on message passing
• 100% portable: one standard, many implementations
• Available on almost all parallel machines in C and Fortran
• Over 100 advanced routines but 6 basic
Key Concepts of MPI
• Used to create parallel SPMD programs based on message passing
– Normally the same program is running on several different processors
– Processors communicate using message passing
• Typical methodology: start job on n processorsdo i=1 to j each processor does some calculation pass messages between processorend doend job
Summary• MPI is used to create parallel programs based on
message passing
• Usually the same program is run on multiple processors
• The 6 basic calls in MPI are:– MPI_INIT( ierr )– MPI_COMM_RANK( MPI_COMM_WORLD, myid, ierr )– MPI_COMM_SIZE( MPI_COMM_WORLD, numprocs, ierr )– MPI_Send(buffer, count,MPI_INTEGER,destination, tag, MPI_COMM_WORLD, ierr)
– MPI_Recv(buffer, count, MPI_INTEGER,source,tag, MPI_COMM_WORLD, status,ierr)
– call MPI_FINALIZE(ierr)
MPI helloworld.c
#include <mpi.h>main(int argc, char **argv){ int numtasks, rank; MPI_Init(&argc, &argv); MPI_Comm_size(MPI_COMM_WORLD, & numtasks); MPI_Comm_rank(MPI_COMM_WORLD, &rank);
printf("Hello World from process %d of %d\n“,
rank, numtasks);
MPI_Finalize();}
MPI_COMM_WORLD
• MPI_INIT defines a communicator called MPI_COMM_WORLD for every process that calls it.
• All MPI communication calls require a communicator argument
• MPI processes can only communicate if they share a communicator.
• A communicator contains a group which is a list of processes
• Each process has it’s rank within the communicator
• A process can have several communicators
Sample Run and Output
• A Run with 3 Processes:– manjra> mpirun -np 3 -machinefile
machines.list helloworld Hello World from process 0 of 3 Hello World from process 1 of 3 Hello World from process 2 of 3
A Run by default– manjra> helloworld
Hello World from process 0 of 1
Sample Run and Output
• A Run with 6 Processes:– manjra> mpirun -np 6 -machinefile
machines.list helloworld• Hello World from process 0 of 6• Hello World from process 3 of 6• Hello World from process 1 of 6• Hello World from process 5 of 6• Hello World from process 4 of 6• Hello World from process 2 of 6
• Note: Process execution need not be in process number order.
Image ProcessingEdge Detection
Parallel “Game of Life”
..o.........
..o.........
..o.........
.....oo.....
.oo.........
....ooo.ooo.
...ooo......
............
.oo.....o...
........o...
...oo...o...
............
Computer Architecture
Classification of Computer Architectures
1. Basic SISD Architectures: Traditional Machines
1. Zero Address: Stack Approach
2. One Address: Accumulator Approach
3. Two Address: Register/Memory Address Approach
2. Advanced Architectures: RISC Machines
3. SIMD Architectures: Vector Pipelines and Parallelism
4. MIMD Architectures: Distributed Processing (PVM), Hypercube Architecture
Digital Logic Level
1. Logic gates, Boolean Algebra, Circuit Design, Multiplexers
2. Parity, Error correction (Hamming Code), and Data Compression (Huffman Code)
Microprogramming and Microachitectures
1. Microprocessors and Instruction formats
2. Opcode design: Expanding opcodes
3. Target Machine verses hardware considerations
4. Registers and word length
5. Example Mic-1 Microprogram Level Interpreter
Assembly Language level
1. Assembly Language Instruction Sets, Introduction to SPIM
2. Branching and Loops
3. Addressing Techniques
4. Input and Output
5. Program Tuning