CS 2510Graduate Operating Systems

Jack LangeAssistant Professor

University of Pittsbugh

Course Objectives

• Understand basic OS principals• Understand historical background of OS design• Gain practical experience with current systems– x86 and Linux

• Gain exposure to OS research• Gain experience in reviewing and evaluating

research papers• Gain experience in presenting research papers

Administrivia

• Instructor: Jack Lange– Email: jacklange@cs.pitt.edu– Office: Sennott Square #5407– Office Hours: Monday 2-4PM

• Teaching Assistant: Brian Kocoloski– Email: briankoco@cs.pitt.edu– Office: Sennott Square #6804– Office Hours: TBD

Communication• Course homepage

– http://www.cs.pitt.edu/~jacklange/teaching/cs2510-f12/– Announcements, clarifications, corrections– Additional resources for projects

• Google Group– http://groups.google.com/group/pitt-cs2510-f12– pitt-cs2510-f12@googlegroups.com – Private discussion group

• Open venue for class discussions and questions

– Based on email (Pitt addresses)• Email me if you want to use a different one

Class Details

• Inverse of other graduate courses– Lectures on Tuesdays• Provide background• Supplemental readings assigned (Check Schedule!)

– Presentations on Thursdays• Student presents selected research paper• Other students read and provide a review of the paper

before hand

Grading

• Grading– Final (30%)– Presentations (10%)– Paper reviews (10%)– 4 Projects (50%)

• Late policy– Submit by midnight of the due date– 10% penalty for every day late

Projects

• Work Individually– C is required

• Highly Recommended: OS or having some familiarity with Unix systems programming, preferably in C or C++– All projects are in C– BUILDING software is 50% of the grade of this

class

1-9

Reading papers

• Mandatory readings are assigned throughout semester– Must submit a review for each assigned paper by

midnight on the day before class– Each student must ask 1 relevant question during

each presentation

Reviewing papers• Paper title and its author(s).

• Brief one-line summary.

• At least one paragraph of the most important ideas: perhaps a combination of their motivations, observations, interesting parts of the design, or clever parts of their implementation.

• At least one paragraph of the largest flaws; maybe an experiment was poorly designed or the main idea had a narrow scope applicability. Being able to assess weaknesses as well as strengths is an important skill for this course and beyond.

• At least one last paragraph where you state the relevance of the ideas today, potential future research suggested by the article, etc.

• Reviewing and evaluating research papers is 10% of final grade

Presentations• Presenting research is a critical skill for graduate students

• Large part of the class will be reading, evaluating and presenting research papers

• Each student must do 1– Present paper and answer questions about it– Develop own presentation (Cannot use author’s slides)– Be able to clearly present research ideas and think on your feet– Presentations will be graded (10% of final grade)– Evaluation will be based on presentation quality as well as

technical knowledge

What makes a good presentation

• What is the problem?• Why should people care about the problem?• How do you intend to address it (high level)?• How did you address it (w/ lowlevel details of interesting

components)?• Why is your approach superior to others?• What are the shortcomings of your approach?• Did your approach work?

• Evaluation will be based on how well you answer these questions!

What makes a bad presentation

• Lack of context• Reading from slides• Lots of text or data• Only focusing on technical details• Ignoring high level points

Why learn about Operating Systems?

• Tangible reasons– Build or modify a real operating system– Administer and use system well– Tune application performance

• Intangibles– Intrinsic curiosity

• Understand how much of you computer system works– Gain/apply knowledge in other areas of Computer Science

• Computer architecture and devices• Synchronization in programming languages• Data structures and algorithms• Performance analysis

– Challenge of designing large, complex systems

What is an operating System

• Not easy to define:Users

ApplicationsOperating System

Hardware• OS:

– Everything that isn’t an application or hardware• OS:

– Software that abstract hardware into a useful form for applications– Standard Library– Resource Coordinator

First Function: Standard Library

• Advantages of standard library– Allow applications to reuse common facilities– Make different devices look the same– Provide higher level abstractions

• Challenges– What are the right

abstractions?

Second Function: Resource Coordinator

• Resource: “Anything valuable” – (e.g. CPU, memory, disk, network)

• Advantages of resource coordinator– Virtualize resource so multiple users/applications can share– Protect applications from one another– Provide efficient and fair access to resources

• Challenges– What mechanisms?– What policies?

What Functionality in OS?• No single right answer– Desired functionality depends on outside factors

Users & Applications

OperatingSystem

Computer Architecture

TechnologyChanges

Expectations

• OS must adapt– Change abstractions provided to users– Change algorithms to implement those abstractions– Change low-level implementation to deal with hardware

• Current operating systems driven by its evolution• Two distinct cases in history

– Case 1: Computers are expensive– Case 2: Computers are cheap

History of the OS• Commercial systems (1950s) and HPC systems (early 1990s)

– Enormous and expensive• Goal: Get the system working

– Single operator/programmer/user runs and debugs at a time• OS functionality

– Standard library -> No coordination of resources– Monitor that is always resident; transfer control to programs

OS

User JobMemory

• Problem: Inefficient use of hardware– Performance Metrics

• Throughput and Utilization

Batch Processing• Batch: Group of jobs submitted to machine together

– Operator collects jobs; orders efficiently; runs one at a time

• Role of OS: Same as before• Advantages

– Amortize setup costs over many jobs– Keeps machine busy during a single users idle time

• Disadvantage– User must wait for results until batch collected and submitted

• If bug, receive memory and register dump; submit job again

• Improve system throughput and utilization, but lose interactivity

• Still the prevalent supercomputing model

Spooling• Problem: Mechanical I/O devices much slower than CPU

– Read 17 cards/sec vs. execute 1000s instructions/sec

• Spooling: Overlap I/O (e.g. card reading and line printing) with execution– Read punch cards to disk– Compute (reading and writing to disk)– Write output from disk to printer

• New OS functionality– Buffering, DMA, interrupt handling

• Advantage: Improves throughput and utilization• Disadvantage: Single job must wait during I/O for data

Multiprogrammed Batch Systems

• Spooling provides pool of ready jobs– Keep multiple jobs resident in memory– OS chooses which job to run– When job waits for I/O, switch to another resident job

• New OS functionality– Job scheduling policies– Memory management and protection (virtual memory)

• Advantage: Improves throughput and utilization• Disadvantage: Still not interactive

OS

User Job 1 Memory

User Job 2

User Job 2

History of the OS: Phase 2• Introduction of inexpensive, fast devices

– Keyboards and monitors text editors and interactive debuggers– New set of performance trade-offs

• Goal: True interactivity (via improved response time)

• Time-sharing:Switch between jobs to give appearance of dedicated machine

• Advantage– Users easily submit jobs and get immediate feedback

• New OS functionality– More complex job scheduling, memory management– Concurrency control and synchronization

Personal Computers

• Computers became cheap (late 70s, early 80s)– Dedicated machine per user

• “Improved” functionality from OS– Remove time-sharing of multiple jobs– No protection– No virtual memory– OS becomes subroutine again

• Conclusion: OS functionality changes with hardware and users

OS

User JobMemory

State of current systems• Large

– 100k’s to millions of lines of code– 100-1000 person-years of work

• Complex– Performance is important– Conflicting needs of different users

• Poorly understood– System outlives any of its builders– Cannot remove all bugs– Behavior is hard to predict, tuning is done by guessing

• Current trends– Multiprocessors– Networked systems– Cloud based systems

Distributed System Architecture

Distributed System Architecture

Distributed Memory Architecture

Message Passing• Explicit communication operations

– Specify sender and receiver– Serialize data into a message– Transmit message over communication channel

• Communication Channels– Sockets

• E.g. Ethernet– RDMA (Remote direct memory access)

• E.g. Infiniband– Shared memory

• Becoming more prevalent on many-core architectures

• The future OS architecture?

What we’ll learn

• Understand basic OS principals• Understand historical background of OS design• Gain practical experience with current systems– x86 and Linux

• Gain exposure to OS research• Gain experience in reviewing and evaluating

research papers• Gain experience in presenting research papers

First assignment

• Pick a paper and day for presentation• Send me (jacklange@cs.pitt.edu) an email

with choice of date and email address I should use for communication– Presentation days will be assigned according to

First Come First Serve (FCFS)