Kernel Locking Techniques

by Robert Lovepresented by Scott Price

IntroductionLocking can be very tough to implement correctly

Poorly designed locking can result into code that is hard to read and performs poorly.

Locking is needed to synchronize code paths in the kernel.

Critical sections require combinations of concurrent or re-entrance protection and proper ordering of events

Improper designed locking can result in crashes and other oddities.

++i shared is dangerous so all shared memory in the kernel needs to have some type of locking.

two threads can access i at the same time and could increment i once or could increment i twice.

I++ example

No Lock Lock

SMP Locks in Uniprocessor Kernel

Interrupt handlers, preemptible kernel and any code that can block can all create locking issues

If your running SMP machine or not, anyone could use your code. So you have to make sure code is safe on both SMP and unicore

combinations of CONFIG_SMP and CONFIG_PREEMPT in varying locking support

Lock everything appropriate and make sure all situations are covered.

Bad things that Happen When we don’t Lock

Kernel

Complex locking primitives are built off atomic operations. They are the building blocks of kernel locks

Atomic operations are like add and subtract but perform in one uninterruptible operation

two types of Atomic operators exist. Ones that work on integers and ones that work on bits

You cannot pass an atomic type to anything but an atomic operation.

Some architectural limitations do not expect atomic operations to have more then 24 bits

Atomic Operations

Atomic Operations and interruptsWhen we have shared data that is accessed by normal kernel code and Interrupt handler code, we need to have atomic operations to both acquire the lock and disable interrupts at the same time.

If we do not use an atomic operations to both lock and disable interrupts, an interrupt can possibly happen during a locking phase. The interrupt will try to acquire the lock and can possibly deadlock

Besides this issue we can use locks without disabling interrupts

SpinlocksAre a Simple single-holder lock.

If process attempts to acquire a lock and it is not available, the process will keep trying to acquire the lock (spinning) until the lock is available.

Once the lock is acquired you can process the critical region and release the lock

SMP Unicore

SpinlocksSpinlocks should only be used when the lock is not going to be held very long. Or other processes will spin doing nothing.

Do not use spin locks on processes that will go to sleep. Or else a deadlock could occur.

spin_lock_irqsave() and spin_unlock_irqrestore() are atomic operations that acquire the lock and disable interrupts with one atomic operation

we need to both acquire the lock and disable interrupts in one operation or else we could have a race condition

SemaphoresSemaphores are sleeping locks because they can cause a task to sleep on contention instead of spin. They are used when you are going to take hold of a lock for a long time

There is overhead for putting task to sleep and waking task up. This should not be used for locks that are held for a short amount of time.

Semaphores have a queue for how many threads are waiting. If the number is positive, this is the amount of threads that are waiting on the queue.

If negative, the semaphore is unavailable and the absolute value is the usage count

Semaphore

Semaphores manipulate 2 methods up(), down() they increment/decrement the wait queue

up_interruptible() the calling process is added to the wait queue and blocked

down_interruptible() the process obtains the semaphore

up and down increment of semaphores have to atomic operations

Reader/Writer Locks

is safe for multiple threads to read data concurrently. As long as nothing modifies the data

R/W locks allow multiple concurrent readers but only a single writer.

If data is access, it naturally divides into clear reading and writing patterns especially with greater amount of reading time, then writing time. If so R/W locks are preferred

R/W spinlock is called rwlock, similar to spinlock with the exception of separate R/W locking

R/W locks can give appreciable optimization

Attempting to acquire exclusive access while holding reader access will deadlock

reader/writer locks

spinlock version semaphore version

Provide a spinning lock that is very fast to acquire for reading but incredibly slow to acquire for writing

Ideal for situations with many readers and a few writers

Only used for special cases.

Big-Reader Locks

Global kernel lock BKL

Still exist today but lots of work went into removing it for more fine-grained localized locks

a spinning lock that is recursive so two consecutive request will not deadlock the process like a spinlock will do

A process can sleep and enter the scheduler while holding. When a process is holding, if another process enters the scheduler, the lock is dropped so other processes can obtain it.

It exist but shouldn't use other locks are more efficient

Big-Kernel locks

Preemptive ControlLinux kernel is fully preemptible

Allows processes to be preempted by higher priority processes even if there is a process running in the kernel

Preemptible kernel creates synchronization issues of SMP

Kernel preemption is synchronized by SMP locks so most issues are solved writing SMP safe code.

Introduces a few new locking issues

For preemptible situations, we can disable and enable the preemption if needed

Conclusion

SMP reliability and scalability in the linux kernel are improving since SMP was introduce

Future holds better performance

Kernel developers should do their part by writing code that implements smart, sane, proper locking with an eye to both scalability and reliability!!

OR ELSE!

This could happen!!!