Upload
ngolien
View
228
Download
1
Embed Size (px)
Citation preview
University of Tartu, Institute of Computer Science
Introduction to Scientific Computing
MTAT.08.025
Spring 2017
2 Practical information
Lectures: Liivi 2 - 122 MON 10:15Computer classes: Liivi 2 - 205, MON 12:15 Amnir Hadachi <am-
[email protected]>3 ECTS
Lectures: 16h; Computer Classes: 16h; Independent work: 46hFinal grade forms from :
1. Active partitipation at lectures 10%
2. Stand-up quiz(es) 10%
3. Computer class activities 50%
4. Exam 30%
Course homepage (http://courses.cs.ut.ee/2017/isc)
3 Introduction 1.1 Syllabus
1 Introduction1.1 SyllabusLectures:
• Python for Scientific Computing NumPy, SciPy
• Scientific Computing - an Overview
• Floating point numbers, how to deal with roundoff errors
• Large problems in Linear Algebra, condition number
• Memory hierarchies and making use of it
• Numerical integration and differentiation
• Numerical solution of differential and integral equations
• Fast Fourier Transform
4 Introduction 1.1 Syllabus
Computer Classes (preliminary plan)
1. Python for Scientific Computing; Fibonacci numbers; Collatz conjecture
2. Discretization and round-off errors
3. NumPy arrays, matrices; LU-Factorization with Gauss Elimination Method(GEM)
4. UT HPC server; LU-Factorization and GEM on HPC cluster
5. Floating point numbers
6. Fractals
7. Fourier series and Fast Fourier Transform
8. Discrete-time models and ordinary differential equations
5 Introduction 1.2 Literature
1.2 Literature
General Scientific Computing:
1. RH Landau, A First Course in Scientific Computing. Symbolic, Graphic, andNumeric Modeling Using Maple, Java, Mathematica, and Fortran90. PrincentonUniversity Press, 2005.
2. LR Scott, T Clark, B Bagheri. Scientific Parallel Computing. Princenton Uni-versity Press, 2005.
3. MT Heath, Scientific Computing; ISBN: 007112229X, McGraw-Hill Compa-nies, 2001.
4. JW Demmel, Applied Numerical Linear Algebra; ISBN: 0898713897, Societyfor Industrial & Applied Mathematics, Paperback, 1997.
6 Introduction 1.2 Literature
Python for SC:
1. Hans Petter Langtangen, A Primer on Scientific Programming withPython, Springer, 2009. Book webpage (http://vefur.simula.no/intro-programming/).
2. Hans Petter Langtangen, Python Scripting for Computational Science. ThirdEdition, Springer 2008. Web-site for the book (http://folk.uio.no/hpl/scripting/).
3. Neeme Kahusk, Sissejuhatus Pythonisse (http://www.cl.ut.ee/inimesed/nkahusk/sissejuhatus-pythonisse/).
4. Travis E. Oliphant, Guide to NumPy (http://www.tramy.us), TrelgolPublishing 2006.
5. Help documentation inside sage – used during Computer Classes
7 Introduction 1.3 Scripting vs programming
1.3 Scripting vs programming
1.3.1 What is a script?
• Very high-level, often short, programwritten in a high-level scripting language
• Scripting languages:
– Unix shells,
– Tcl,
– Perl,
– Python,
– Ruby,
– Scheme,
– Rexx,
– JavaScript,
– VisualBasic,
– ...
8 Introduction 1.3 Scripting vs programming
1.3.2 Characteristics of a script
• Glue other programs together
• Extensive text processing
• File and directory manipulation
• Often special-purpose code
• Many small interacting scripts may yield a big system
• Perhaps a special-purpose GUI on top
• Portable across Unix, Windows, Mac
• Interpreted program (no compilation+linking)
9 Introduction 1.3 Scripting vs programming
1.3.3 Why not stick to Java, C/C++ or Fortran?
Features of Perl and Python compared with Java, C/C++ and Fortran:
• shorter, more high-level programs
• much faster software development
• more convenient programming
• you feel more productive
• no variable declarations ,but lots of consistency checks at run time
• lots of standardized libraries and tools
10 Introduction 1.4 Scripts yield short code
1.4 Scripts yield short code
Consider reading real numbers from a file, where each line can contain an arbitrarynumber of real numbers:� �1.1 9 5.2
1.762543E-02
0 0.01 0.001 9 3 7�Python solution:� �
F = open(filename, ’r’)
n = F.read().split()�Perl solution:� �
open F, $filename;
$s = join "", <F>;
@n = split ’ ’, $s;�
11 Introduction 1.5 Performance issues
Ruby solution:� �n = IO.readlines(filename).join.split�
...Doing this in C++ or Java1 requires at least a loop, and in Fortran and C quitesome code lines are necessary
1.5 Performance issues
1.5.1 Scripts can be slow
• Perl and Python scripts are first compiled to byte-code
• The byte-code is then interpreted
1True in case of older Java implementations. With Java 8 and Java Streames that come with it, forexample, plain Java 8 code does the same:Files.lines(filePath).flatMap( Pattern.compile("\\s+")::splitAsStream );
12 Introduction 1.5 Performance issues
• Text processing is usually as fast as in C
• Loops over large data structures might be very slow
� �for i in range(len(A)):
A[i] = ...�• Fortran, C and C++ compilers are good at optimizing such loops at compile time
and produce very efficient assembly code (e.g. 100 times faster)
• Fortunately, long loops in scripts can easily be migrated to Fortran or C (orspecial libraries like numpy!)
13 Introduction 1.5 Performance issues
1.5.2 Scripts may be fast enough
Read 100 000 (x,y) data from file and write (x,f(y)) out again
• Pure Python: 4s
• Pure Perl: 3s
• Pure Tcl: 11s
• Pure C (fscanf/fprintf): 1s
• Pure C++ (iostream): 3.6s
• Pure C++ (buffered streams): 2.5s
• Numerical Python modules: 2.2s (!)
• Remark: in practice, 100 000 data points are written and read in binary format,resulting in much smaller differences
14 Introduction 1.5 Performance issues
1.5.3 When scripting is convenient
• The application’s main task is to connect together existing components
• The application includes a graphical user interface
• The application performs extensive string/text manipulation
• The design of the application code is expected to change significantly
• CPU-time intensive parts can be migrated to C/C++ or Fortran
• The application can be made short if it operates heavily on list or hash structures
• The application is supposed to communicate with Web servers
• The application should run without modifications on Unix, Windows, and Mac-intosh computers, also when a GUI is included
15 Introduction 1.5 Performance issues
1.5.4 When to use C, C++, Java, Fortran
• Does the application implement complicated algorithms and data structures?
• Does the application manipulate large datasets so that execution speed iscritical?
• Are the application’s functions well-defined and changing slowly?
• Will type-safe languages be an advantage, e.g., in large development teams?
At this course we will be using python and corresponding libraries for Scientific Com-puting (numpy, scipy through sage)
NOTE: is Python really a scripting language?
• Object-oriented language!
• Cython
• PyPy – jit (just in time) compiler
16 What is Scientific Computing? 2.1 Introduction to Scientific Computing
2 What is Scientific Computing?2.1 Introduction to Scientific Computing
• Scientific computing – subject on crossroads of
• physics, chemistry, [social, engineering,...] sciences
– problems typically translated into
∗ linear algebraic problems
∗ sometimes combinatorial problems
• a computational scientist needs knowledge of some aspects of
– numerical analysis
– linear algebra
– discrete mathematics
17 What is Scientific Computing? 2.1 Introduction to Scientific Computing
• An efficient implementation needs some understanding of
– computer architecture
∗ both on the CPU level
∗ on the level of parallel computing
– some specific skills of software management
Scientific Computing – field of study concerned with constructing math-ematical models and numerical solution techniques and using computersto analyse and solve scientific and engineering problems
• typically – application of computer simulation and other forms of computationto problems in various scientific disciplines.
18 What is Scientific Computing? 2.1 Introduction to Scientific Computing
Main purpose of Scientific Computing:
• mirroring
• predictionof real world processes’
• characteristics
– behaviour
– development
Example of Computational SimulationASTROPHYSICS: what happens with collision of two black holes in the universe?Situation which is
• impossible to observe in the nature,
• test in a lab
• estimate barely theoretically
19 What is Scientific Computing? 2.1 Introduction to Scientific Computing
Computer simulation CAN HELPBut what is needed for simulation?
• adequate mathematical model (Einstein’s general relativity theory)
• algorithm for numerical solution of the equations
• enough big computer for actual realisation of the algorithms
Frequently: need for simulation of situations that could be performed experimantally,but simulation on computers is needed because:
• HIGH COST OF THE REAL EXPERIMENT. Examples:
– car crash-tests
– simulation of gas explosions
– nuclear explosion simulation
– behaviour of ships in Ocean waves
20 What is Scientific Computing? 2.1 Introduction to Scientific Computing
– airplane aerodynamics
– strength calculations in big constructions (for example oil-platforms)
– oil-field simulations
• TIME FACTOR. Some examples:
– Climate change predictions
– Geological development of the Earth (including oil-fields)
– Glacier flow model
– Weather prediction
• SCALE OF THE PROBLEM. Some examples:
– modeling chemical reactions on the molecular level
– development of biological ecosystems
21 What is Scientific Computing? 2.1 Introduction to Scientific Computing
• PROCESSES THAT CAN NOT BE INTERVENED Some examples:
– human heart model
– global economy model
22 What is Scientific Computing? 2.2 Specifics of computational problems
2.2 Specifics of computational problems
Usually, computer simulation consists of:
1. Creation of mathematical model – usually in a form of equations –physical properties and dependencies of the subject
2. Algorithm creation for numerical solution of the equations
3. Application of the algorithms in computer software
4. Using the created software on a computer in a particular simulationprocess
5. Visualizing the results in an understandable way using computer graph-ics, for example
6. Integration of the results and repetition/redesign of arbitrary given stepabove
23 What is Scientific Computing? 2.2 Specifics of computational problems
Most often:
• algorithm
– written down in an intuitive way
– and/or using special modeling software
• computer program written, based on the algorithm
• testing
• iterating
Explorative nature of Scientific Computing!
24 What is Scientific Computing? 2.3 Mathematical model
2.3 Mathematical model
GENERAL STRATEGY:REPLACE A DIFFICULT PROBLEM WITH A MORE SIMPLE ONE
• – which has the same solution
• – or at least approximate solution
– but still reflecting the most important features of the problem
25 What is Scientific Computing? 2.3 Mathematical model
SOME EXAMPLES OF SUCH TECHNIQUES:
• Replacing infinite spaces with finite ones (in maths sense)
• Infinite processes replacement with finite ones
– replacing integrals with finite sums
– derivatives replaced by finite differences
• Replacing differential equations with algebraic equations
• Nonlinear equations replaced by linear equations
• Replacing higher order systems with lower order ones
• Replacing complicated functions with more simple ones (like polynomials)
• Arbitrary structured matrix replacement with more simple structured matrices
26 What is Scientific Computing? 2.3 Mathematical model
AT THIS COURSE WE TRY TO GIVE:
an overview of some methods and analysis for development of reliable andefficient software for Scientific Computing
Reliability means here both the reliability of the software as well as adequatyof the results – how much can one rely on the achieved results:
• Is the solution acceptable at all? is it a real solution? (extraneous solution?,instability of the solution? etc); Does the solution algorithm guarantee a solutionat all?
• How big is the calculated solution’s deviation from the real solution? How wellthe simulation reflects the real world?
Another aspect: software reliability
27 What is Scientific Computing? 2.3 Mathematical model
Efficiency expressed on various levels of the solution process
• speed
• amount of used resources
Resources can be:
– Time
– Cost
– Number of CPU cycles
– Number of processes
– Amount of RAM
– Human labour
28 What is Scientific Computing? 2.3 Mathematical model
General formula:
minapproximation error
timeEven more generally: – minimise time of the solution
Efficient method requires:
(i) good discretisation
(ii) good computer implementation
(iii) depends on computer architecture (processor speed, RAM size, mem-ory bus speed, availability of cache, number of cache levels and otherproperties )