Upload
others
View
27
Download
0
Embed Size (px)
Citation preview
Chapter 5: Synchronization ���1
Chapter 5: Synchronization
Start of Lecture: February 3, 2014
���2
Chapter 5: Synchronization
Reminders
• Put up Exercise 2 for Chapters 3-5; due February 12
• Put up Assignment 2; due March 7
• Assignment 2 is significantly harder than Assignment 1, so get working on it early; for this assignment, it really won’t work to wait until a couple of days before, it really need to be weeks before (particularly as its much more code as well)
• Any questions or comments?
���3
Chapter 5: Synchronization
Let’s step back and re-evaluate
• Why are we learning this stuff about operating systems?
• Process control block? So what?
• What am I supposed to get out of my comp sci degree?
• of course, this is subjective
• One of the main goals: be able to understand computing
• all the way from high-level (algorithms, theory of computation) to
• low-level (how instructions represented in binary, how they executed on a CPU with switches, maybe in future quantum computers, etc.)
• with in-between being varying levels of abstraction in implementation and algorithm development
���4
Chapter 5: Synchronization
Most courses try to teach algorithmic thinking and how to implement those solutions
• Operating systems course closer to the low-level side of the understanding
• for some, the specific knowledge will be useful for future projects
• others will stay closer to high-level algorithms side, where much of the low-level is abstracted away, making low-level knowledge less relevant
• The overarching theme, however, is taking a problem and
• Step 1. Abstract away the details to find a higher-level representation and understanding of the problem to simplify finding a solution
• Step 2. Take your high-level, abstract solution and turn in into an implementation (something that runs and finds the “answer”)
���5
Chapter 5: Synchronization
Tackling Daunting Projects
• You have finished Assignment 1, and maybe you spent some time staring at a screen wondering where to begin
• This will be even more true for Assignment 2
• A core skill in Computing Science is to be able to take your plan (and/or a given specification) and convert it into an implementation; each assignment tries to make you better at this
���6
Chapter 5: Synchronization
Main components to a project
• Understand the problem to be solved, or understand requirements for the software application, or …
• Research (i.e. find) or develop algorithms to solve the problem; usually there are many tools available
• e.g. max-flow algorithm to solve airline scheduling
• e.g. design patterns for large software projects
• Concisely write down the high-level design for solving the problem, to crystallize your plan and understanding
• Convert this plan into an implementation
���7
Chapter 5: Synchronization
Stuck on the implementation?
• One issue might be that you have not crystallized your plan or thought about how to divide and conquer
• Think about how your code can be modularized, and possibly write down the sub-solutions that need to be implemented to solve the larger solution
• e.g. dot product thread function (nice sub piece for larger problem)
• Once you have a plan, don’t stare at the screen; start writing!
• “Writing a novel is like driving a car at night. You can see only as far as your headlights, but you can make the whole trip that way” – E. L. Doctorow
���8
Chapter 5: Synchronization
Advice for completing your plan
• Pick low hanging fruit: you might feel immobilized by the larger problem, but there are usually parts you can easily complete e.g. function that sets up signal handler
• Copy-and-paste: re-use code you know works
• some programmers frown on this, but it nots black-and-white; can be a good starting point if you use it correctly
• if you thoroughly understand pasted code and properly simplify and minimize what is copied for your new application, then can speed implementation and save time
• e.g. reading through a file and obtaining lines correctly
• e.g. do non-blocking I/O correctly through sockets
���9
Chapter 5: Synchronization
Advice for completing your plan
• Keep it simple and don’t be too much of a perfectionist
• Don’t design your code for the most general setting, as that gets too complicated and you don’t really know what you’ll need later; likely, you’ll end up scrapping your original implementation and writing it for the new (more general) setting later anyway
• Obviously, don’t hardcode everything either, but do try to avoid grand re-designs and refactoring to make your code general (rabbit hole)
• Small, clear wins in 25 minute bursts
• modularize your code properly so that you can sit down and complete a part, instead of leaving half-way through an overwhelming piece
���10
Chapter 5: Synchronization
Advice for completing your plan
• Proper reconnaissance: use the resources around you
• look at really nicely implemented code to learn how other people factor their code and best practices in a language
• search the internet for similar problems and implementation ideas; letting these ideas percolate will let you better formulate a plan
• pairs programming: discuss problem with other people to brainstorm; when you get stuck, a second pair of eyes helps get out of the hole
• Make a small end-to-end prototype or skeleton that touches all major components after reconnaissance
• laying out your plan, even as a skeleton, is a big achievement
• can tackle unknown, more difficult cases/parts later
���11
Chapter 5: Synchronization
Remember that this is still a Science
• This means more of a focus on problems and algorithms
• its not about learning any specific language; you’ll learn many in your career, because a language is a tool to solve problems
• programming in C is to show you how to manage resources, like memory, that other languages might abstract away; C is the tool for you to have closer control with the hardware, not the end goal
• This means being analytical when evaluating different algorithms
• investigate possible solutions; hypothesize which algorithm(s) more appropriate; then run a controlled (research) experiment with enough samples to make claims about significant differences in accuracy/speed between your implementations. Yes, your stats class is useful (and besides, data is beautiful)
• Maybe surprisingly, scientific exploration is not cut-and-dry; its about discussion and being creative (not just clean analysis)
���12
Chapter 5: Synchronization
Once you’re done undergrad and programming a big project
• You will hopefully have obtained a tool belt of algorithms and solution techniques that will help guide your search towards the right set of algorithms for your problem
• You will hopefully have a better understanding of how to tackle big problems and divide-and-conquer (modularize)
• You will hopefully have cultivated an analytical mind
• You will hopefully have learned (more) how to learn
• But likely you won’t be an expert in any language; you will have to teach yourself and at that time become an expert in the required language (e.g. Objective C for an app)
���13
Chapter 5: Synchronization
Simple Example: Assignment 1
• Needed to implement a function, get_mem_layout, to scan virtual address space
• Needed to test that function by writing test code
• Needed to separate (modularize) get_mem_layout into its own file, to be included in other code that wants to be able to scan its address space
���14
Chapter 5: Synchronization
One possible plan for Assignment 1• Write test code, assuming get_mem_layout is
implemented; separate into header file and make sure everything is compiling correctly with get_mem_layout temporarily a skeleton with no functionality
• Start tackling main components of get_mem_layout
• write skeleton for check_page function to check if memory location is R/W: check_page(char * address) { }
• write skeleton code in get_mem_layout that loops over address space and calls check_page on each address
• write skeleton for function that setups signal handler to catch segmentation faults from check_page
• write skeleton for function that set the values in a chunk
���15
Chapter 5: Synchronization ���16
static void setup_handler() {}!static void check_page(char * address) {}!static void fill_chunk(struct memchunk *chunk) {}!int get_mem_layout { setup_handler(); while (address < MAX_ADDRESS) { /* Determine readable-writable */ check_page(address); /* After checking, we should store info */ fill_chunk(current_chunk); }}
Initial non-compiling layout to start tackling the assignment
Chapter 5: Synchronization
One possible plan for Assignment 1
• Start picking off low-hanging fruit
• Fill in code for setting up a signal handler (copy-paste from sig2.c)
• Fill in code to check if a page is readable or writable (start to understand why you need the signal handler function)
• Fill in code that sets the values in a chunk, like RW (start seeing what global variables you might need)
• Determine PAGE_SIZE so can increment address variable
• Start thinking about more difficult procedure of where to jump after signal is caught, and so the order that functions should be called in the main loop
• But now this difficult part is simpler because other simpler components separated from this task, and you’ve identified the difficult part
���17
Chapter 5: Synchronization
What are your approaches?• Going from high-level abstraction to implementation is
often quite different for each person; in teams, often iterate discussion to brainstorm a plan and implementation
• and in teams often do code reviews of each others code to sync
• What approaches do you use? What did you do to complete Assignment 1?
• Anyone use pairs programming to help find bugs? Some typical bugs I saw were
• using int instead of unsigned long, and getting overflow (infinite loops)
• overwriting memory when checking if could write, and not restoring to the original value if the write was successful
���18
Chapter 5: Synchronization
Video Break: brought to you by a wonderful classmate
���19
Chapter 5: Synchronization
More about threads and modularizing
• Pretend someone asks you to implement a server, which accepts connections and hands them off to threads
• You know about pthreads, but need to think about how you are going to design and modularize your system
• Where do you start?
���20
Chapter 5: Synchronization
What are some of the first things you might think about when approaching this problem?
• How do you write a server to accept connections? Is there some code out there I can look at that I trust?
• What is the core thread functionality? Are there different group of threads doing different things?
• What information are the threads sharing? Will there be issues? I remember something about semaphores for synchronization from that CMPUT 379 class, maybe I should look into those again?
���21
Chapter 5: Synchronization
Demo: server.c
• Let’s look at server.c to start with code we trust
���22
Chapter 5: Synchronization ���23
Chapter 5: Synchronization
Start of Lecture: February 5, 2014
���24
Chapter 5: Synchronization
Reminders
• You should be reading Chapters 4, 5 and 6; we are starting into CPU scheduling today
• It’s a good idea to stay on top of the readings, because the final exam will draw a lot from the book, so you want to know the material
• you’re expected to know the readings in the book, even if I don’t explicitly lecture on it
• Exercise 2 due next Wednesday
���25
Chapter 5: Synchronization
Demo: server.c
• Let’s finish off a few things with server.c
• Friday, Jan. 14, Ankush Roy will discuss select_server.c and how to properly do non-blocking I/O
• due to your understanding of busy waiting versus performing a self-block to context-switch off the processor, this distinction will be much simpler for you (yah for understanding what you’re coding!)
���26
Chapter 5: Synchronization
How to implement a server with fork
• server.c shows how to set up a socket, listen for connections
• server.c gives you some idea of what you need to do when you fork off children to handle connections
• including making sure children are not left as zombies
• server.c only needs to write a fixed buffer (write a response)
• What might child have to do in addition for HTTP server?
• needs to read client request: GET filename HTTP/1.1
• needs to try to locate file and send back a tailored response (either the file or a not found response, or an error, etc.)
• needs to write event to server log
���27
Chapter 5: Synchronization
How to implement a server with threads?
• How might the separated code that handles a new accepted client connection be different for threads?
• Whenever you think threads, you think ease in sharing memory and variables
• Whenever you think sharing memory and variables, you think mutexes and semaphores
���28
Chapter 5: Synchronization
Some best practices in C
• Use strncpy instead of strcpy to prevent buffer overflow
• same is true of using fgets instead of gets, snprintf instead of sprintf
• Use portable functions (probably have to look it up)
• e.g. use C library for system calls, rather than directly calling system call
• Reasonably optimize code, as cannot always know what compiler might make more optimal
���29
char *buff = “Hammer time!”;while (/* Some condition */) { … if(len > strlen(buff)) /* Do some stuff */ …}
char *buff = “Hammer time!”;int buff_len = strlen(buff);while (/* Some condition */) { … if(len > buff_len) /* Do some stuff */ …}
Chapter 5: Synchronization
Some best practices in C!
• Do not typedef structs — obfuscates that the variable is a struct and pollutes namespace
• Generally avoid globals variables; but, otherwise, preface non-extern globals and private functions with static
• not too important, but can prevent someone from incorrectly changing scope
• Use const for function arguments so variable need not be copied (code explicitly states that will not modify it)
���30
static int check_state(const int var) { if (var > 0) return 1; return -1;}
/* This won’t compile! */static int check_state(const int var) { var = 5; return -1;}
Chapter 5: Synchronization
Some best practices in C• Write clear code — don’t obfuscate to get fewer lines
• Headers are for encapsulation — should only contain functions you want exposed to outside world; prototype private functions inside c file
• Use structs for nice encapsulation (like object-oriented programming, can be good for abstraction)
• Handle the case that a header is included multiple times
���31
#ifndef MY_UTILS_H#define MY_UTILS_H/* Header interface for all functions you want to make available */#endif /* MY_UTILS_H */
Chapter 5: Synchronization
Review of last 5 chapters
• Services provided by the OS: system calls
• to run in kernel mode, need to ask kernel to run a system call
• Multi-programming and concurrency: switching between tasks to let all processes make progress
• key notion of process as entity with states (e.g. ready, running, blocked)
• context-switching generally initiated by an interrupt, involves storing and loading state of process and involves scheduling processes
• Multi-processing and parallel computing
• determining how to break down problem in subtasks, with data parallelism (e.g. processing separate parts of data) and task parallelism (e.g. searching over many possible configurations)
���32
Chapter 5: Synchronization
Review of last 5 chapters• Process tree in Linux, children created with fork, with
parents responsible for resource use (etc.) of children
• Inter-process communication: shared memory and message passing (such as signals, sockets)
• with a focus on synchronous (blocking) versus asynchronous (non-blocking) communication
• Threads as a light-weight unit of work, difference with processes created by fork()
• Synchronizing cooperating processes
• mutexes (spin locks) and binary semaphores (blocking wait) for locking and counting semaphores for synchronizing work and resources
• monitors for guaranteed mutual exclusion
���33
Chapter 5: Synchronization
A threads review questions
• Parallel max: finding the max value in an array of size N using M threads.
• Fill in the pthread function; don’t forget to use whatever mutexes you might need to lock critical sections
• Code comments describing what code should be there; left out most of code in main, so mentally fill it in
���34
int max_val;int array[100];pthread_mutex_t mutex1, mutex2; /* or more */ !int pthread_fcn(void *threadarg) { /* What is the required threadarg? */ /* What needs to be locked? */}!int main() { pthread_t threads[10]; pthread_mutex_init(&mutex1, NULL); /* Fill array with random values */ max_val = array[0]; /* init max*/ /* Create threads */ /* * Join on the threads, since have * to wait for them to complete */ printf(“The max value is: %d\n”, max_val); pthread_exit(NULL); }