Upload
simon-miller
View
248
Download
1
Embed Size (px)
Citation preview
1
Linux Operating System
許 富 皓
2
Memory Addressing -- with the assistance of 江瑞敏 and 許齊顯
3
Entries of Page Global Directory
The content of the first entries of the Page Global Directory that map linear addresses lower than 0xc0000000 (the first 768 entries with PAE disabled, or the first 3 entries with PAE enabled) depends on the specific process.
Conversely, the remaining entries should be the same for all processes and equal to the corresponding entries of the master kernel Page Global Directory.
4
Kernel Page Tables
The kernel maintains a set of page tables for its own use. This set of page tables is rooted at a so-called master ke
rnel Page Global Directory. After system initialization, the set of page tables are never directl
y used by any process or kernel thread. Rather, the highest entries of the master kernel Page Global Direc
tory are the reference model for the corresponding entries of the Page Global Directories of EVERY regular process in the system.
Duplicate the Content of MKPGD
copy_prcess() copy_mm() dump_mm() mm_init()
mm_alloc_pgd()pgd_alloc()pgd_alloc()
pgd_prepopulate_pmd()
5
6
How Kernel Initializes Its Own page tables A two-phase activity:
In the first phase, the kernel creates a limited address space including
the kernel’s code segment the kernel’s data segments the initial page tables 128 KB for some dynamic data structures.
This minimal address space is just large enough to install the kernel in RAM and to initialize its core data structures. .
In the second phase, the kernel takes advantage of all of the existing RAM and sets up the page tables properly.
7
Phase One
The Special Dot Symbol [GNU]
The special symbol `.' refers to the current address that as is assembling into.
Thus, the expression `melvin: .long .' defines melvin to contain its own address.
Assigning a value to . is treated the same as a .org directive. Thus, the expression `.=.+4' is the same as sa
ying `.space 4'.
8
initial_page_table and __brk_base
The provisional Page Global Directory is contained in the initial_page_table variable.
The provisional Page Tables are stored starting from __brk_base.
9
Assumption
CPU architecture is x86_32. vmlinux[wikipedia] size is 7MB.
On Linux systems, vmlinux is a statically linked executable file that contains the Linux kernel in one of the object file formats supported by Linux, which includes ELF, COFF and a.out.
boot loader put linux kernel at physical address 0x01000000. 10
Phase One Mapping Size
In order to map 24 MB of RAM, 6 Page Tables are required.
16 MB (reserved memory)
+ 7 MB (vmlinux size)
+ 1 MB (MAPPING_BEYOND_END)
11
MAPPING_BEYOND_END
Beside mapping the vmlinux, linux kernel will map additional memory for bootmem allocator.
In x86_32 with PAE disable, the value of MAPPING_BEYOND_END is 1MB.
bootmem allocator: When a system is initialized, there is no buddy system and slab
allocator; hence, bootmem allocator is responsible for memory management and memory allocation.
12
:
:
physical address
0x00000000
1 MB: mapping beyond end
7 MB: vmlinux
Physical Address Layout
0x01000000
0x01700000
0x017fffff
16 MB
13
14
initial_page_table in Phase One The objective of this first phase of paging is to allow these 24 MB
of RAM to be easily addressed both in protected mode before and after paging is enabled.
Therefore, the kernel must create a mapping from both the linear addresses 0x00000000 through 0x017fffff and the linear addresses 0xc0000000 through 0xc17fffff into the physical addresses 0x00000000 through 0x017fffff. In other words, the kernel during its first phase of initialization can
address the first 24 MB of RAM by either linear addresses identical to the physical ones or 24 MB worth of linear addresses, starting from 0xc0000000.
physical addresslinear address
0x00000000
Mapping Linear Addresses to Physical Addresses in Phase One (1)
0x017fffff
0xc0000000
0xc17fffff
0xffffffff
24M
24M
0x00000000
0x017fffff
24M
4 K
…
4 K
15
pt
4 K
pgd
physical addresslinear address
0x00000000
Mapping Linear Addresses to Physical Addresses in Phase One (2)
0x017fffff
0xc0000000
0xc17fffff
0xffffffff
24M
24M
0x00000000
0x017fffff
24M
4 K
…
4 K
16
pt
4 K
pgd
17
Contents of initial_page_table in Phase One The Kernel creates the desired mapping by filli
ng all the initial_page_table entries with zeroes, except for entries 0 ~ 5, 0x300 (decimal 768) ~ 0x305 (decimal 773); the latter six entries span all linear addresses between 0xc0000000 and 0xc17fffff.
The 0 ~ 5, 0x300 ~ 0x305 entries are initialized as follows: The address field of entries 0 and 0x300 is set to th
e physical address of __brk_base.
Initialize initial_page_table page_pde_offset = (__PAGE_OFFSET >> 20);
movl $pa(__brk_base), %edi movl $pa(initial_page_table), %edx movl $PTE_IDENT_ATTR, %eax10: leal PDE_IDENT_ATTR(%edi),%ecx /* Create PDE entry */ movl %ecx,(%edx) /* Store identity PDE entry */ movl %ecx, page_pde_offset(%edx) /* Store kernel PDE entry */ addl $4,%edx movl $1024, %ecx11: stosl addl $0x1000,%eax loop 11b /* * End condition: we must map up to the end + MAPPING_BEYOND_END. */ movl $pa(_end) + MAPPING_BEYOND_END + PTE_IDENT_ATTR, %ebp cmpl %ebp,%eax jb 10b addl $__PAGE_OFFSET, %edi movl %edi, pa(_brk_end) shrl $12, %eax movl %eax, pa(max_pfn_mapped) /* Do early initialization of the fixmap area */ movl $pa(initial_pg_fixmap)+PDE_IDENT_ATTR,%eax movl %eax,pa(initial_page_table+0xffc)
0xc00 (=0x300 * 4)
4k
number of entries in PTs.
18
Phase 1: Page Table Layout
entry 0 (4 byte)
entry 1
entry 2
:
entry 5
:
:
entry 768
entry 769
entry 770
:
entry 773
:
entry 1023
initial_pagr_table (pgd)
entry 0 (4 byte)
entry 1
:
entry 1023
4 KB
:
::
4 KB
:
__brk_base (pte) physical address
4 MB
19
entry 0 (4 byte)
entry 1
…
entry 1023
entry 0 (4 byte)
entry 1
…
entry 1023
::
0x00000000
0x017fffff
……
24 MB
After paging is enable, eip’s values use entry 0 to entry 5 of initial_page_table to tranfer into physical addresses.
When executing file kernel/head.S, values of eip are within the range between 0x00000000 and 0x00800000.
Objectives of initial_page_table
88 ENTRY(startup_32) /*protected mode code*/
99 lgdt pa(boot_gdt_descr)
:
211 movl $pa(initial_page_table), %edx
:
390 /* Enable paging */
391 movl $pa(initial_page_table), %eax
392 movl %eax,%cr3
393 movl $CR0_STATE,%eax
394 movl %eax,%cr0
395 ljmp $__BOOT_CS,$1f
396 1:
398 addl $__PAGE_OFFSET, %esp
:
448 lgdt early_gdt_descr
449 lidt idt_descr
:
468 jmp *(initial_code)
:
679 ENTRY(initial_page_table)
680 .fill 1024,4,0
:
718 ENTRY(stack_start)
719 .long init_thread_union+THREAD_SIZE
:
754 boot_gdt_descr:
755 .word __BOOT_DS+7
:
759 idt_descr:
760 .word IDT_ENTRIES*8-1 …
765 ENTRY(early_gdt_descr)
766 .word GDT_ENTRIES*8-1
virtual address
physical address
logical address
||
virtual address
(segment base address =0)
||
physical address
(paging is not enabled yet.)
Paging Unit
eip
Function i386_start_kernel () is inside a pure C program (head32.c); hence, its address is above 0xc0000000;therefore, after this instruction, values of eip will be greater than 0xc0000000.
Before paging is enable (before line 190), eip’s values are equal to physical addresses.
eip
20
21
Enable the Paging Unit The startup_32( ) assembly language function als
o enables the paging unit. This is achieved by loading the physical address of initial_page_table into the cr3 control register and by setting the PG flag of the cr0 control register, as shown in the following equivalent code fragment:
movl $pa(initial_page_table), %eax movl %eax,%cr3 movl $CR0_STATE,%eax movl %eax,%cr0
22
Phase 2
23
How Kernel Initializes Its Own Page Tables --- Phase 2 Finish the Page Global Directory The final mapping provided by the kernel Page Tables mu
st transform virtual addresses starting from 0xc0000000 to physical addresses starting from 0x00000000.
There are two different configurations that will affect the size of the linear mapping region. CONFIG_HIGHMEM CONFIG_NOHIGHMEM
CONFIG_NOHIGHMEM
If CONFIG_NOHIGHMEM is set, the kernel can only access physical memory less than 1024 MB.
There are 2 cases in this configuration: Case 1: RAM size is less than 895 MB.
Why 895 MB? Case 2: RAM size is between 895 MB and 1024 MB.
24
CONFIG_HIGHMEM
If CONFIG_HIGHMEM is set, the kernel can access physical memory larger than 1024 MB.
There are 3 cases in this configuration: Case 1: RAM size is less than 887 MB. Case 2: RAM size is between 887 MB and 4096 MB. Case 3: RAM size is larger than 4096 MB.
25
Assumption
We assume that the kernel is configured as CONFIG_HIGHMEM .
The following three cases will be discussed: Case 1: RAM size is less than 887 MB. Case 2: RAM size is between 887 MB and 4096 MB. Case 3: RAM size is larger than 4096 MB.
P.S.: The operations performed in case 1 and case 2 of configuration CONFIG_NOHIGHMEM are the same as the ones in case 1 and case 2 of configuration CONFIG_HIGHMEM .
26
Phase 2
Case 1:
When RAM Size Is Less Than 887MB
27
Function Call Chain to kernel_physical_mapping_init()
setup_arch()
init_mem_mapping()
init_memory_mapping()
kernel_physical_mapping_init()
setup_arch() writes the physical address of swapper_pg_dir in the cr3 control register using load_cr3(swapper_pg_dir).
28
kernel_physical_mapping_init() Reinitialize swapper_pg_dir Invokes __flush_tlb_all() to invalidate all TLB entrie
s.
29
Function Call to paging_init
• startup_32 start_kernel setup_arch paging_init
• paging_init is no longer in charge of initializing swapper_pg_dir which is one of its major work in Linux versions around 2.6.16.
[1][2][3][4]
[1][2][3][4]
30
paging_init()
• paging_init():– Invoke pagetable_init()– Invokes __flush_tlb_all() to invalidate all TLB entries
• #ifdef CONFIG_HIGHMEM
– pagetable_init() :• Invokes permanent_kmaps_init()
– permanent_kmaps_init() • Invoke page_table_range_init()
• #else
– pagetable_init() :• do nothing.
32
Important Function Call In Phase 2
31
setup_arch()setup_arch()
init_mem_mappinginit_mem_mapping
init_memory_mappinginit_memory_mapping early_ioremap_page_table_range_init
early_ioremap_page_table_range_init
kernel_pysical_mapping_init
kernel_pysical_mapping_init
x86_init.paging.pagetable_init
x86_init.paging.pagetable_init
paging_initpaging_init
pagetable_initpagetable_init
permenent_kmaps_initpermenent_kmaps_init
kmap_initkmap_init
continuous linear mapping others
33
Assumption We assume that the CPU is a recent 80x86 micropr
ocessor supporting 4 MB pages and "global" TLB entries.
Notice that the User/Supervisor flags in all Page Global Directory entries referencing linear addresses above 0xc0000000 are cleared, thus denying processes in User Mode access to the kern
el address space. Notice also that the Page Size flag is set
so that the kernel can address the RAM by making use of large pages.
Linear Address and Physical Address Mapping
3M vmalloc pkmap fixmap
Linear address
4 M 880 M 3M
Physical address
0 0xc0000000
4k mapping
0xff7fe000 0xffc00000 0xfffff000
0xff800000 0xfffa1000
887MB
hole
887MB
34
0xc0400000
4M 880M
0x00000000 0x00400000
4kmapping
4M mapping
35
Clearance of Page Global Directory Entries Created in Phase 1 The identity mapping of the first megabytes
of physical memory (24 MB in our example) built by the startup_32( ) function is required to complete the initialization phase of the kernel.
When this mapping is no longer necessary, the kernel clears the corresponding page table entries.
MKPGD Mapping
swapper_pg_dir
entry 0
:
entry 768 (4 byte)
entry 769 (4 byte)
entry 770
:
entry 989
:
entry 1023
entry 0 (4 byte)
:
entry 1023 (4 byte)
4 KB
:4 KB
4 M
::
4 KB
:
4 KB
:
4 KB
physical memory
4M= 1024x4k
.
.
.
36
pt
34 entries
:
entry 0 (4 byte)
:
entry 767 (4 byte)
:
entry 1023 (4 byte)
pt
:
3M= 768x4k
37
Phase 2
Case 2: When RAM Size Is between 887MB and 4096MB
38
Phase 2 – Case 2
Final kernel page table when RAM size is between 887 MB and 4096 MB : In this case, the RAM CNNNOT be mapped entirely into the kerne
l linear address space, because the address space is only 1GB. Therefore, during the initialization phase Linux only maps a RAM
window having size of 887 MB into the kernel linear address space. If a program needs to address other parts of the existing RAM, so
me other linear address interval (from the 896th MB to the 1st GB) must be mapped to the required RAM.
This implies changing the value of some page table entries.
39
Phase 2 – Case 2 Code
To initialize the Page Global Directory, the kernel uses the same code as in the previous case.
Linear Address and Physical Address Mapping
3M vmalloc pkmap fixmap
Linear address
4 M 880M 3M
Physical address
0 0xc0000000
4k mapping
0xff7fe000 0xffc00000 0xfffff000
0xff800000 0xfffa1000
887MB
hole
887MB
34
0xc0400000
4M 880M
0x00000000 0x00400000
4kmapping
4M mapping
MKPGD Mapping
swapper_pg_dir
entry 0
:
entry 768 (4 byte)
entry 769 (4 byte)
entry 770
:
entry 989
:
entry 1023
entry 0 (4 byte)
:
entry 1023 (4 byte)
4 KB
:4 KB
4 M
::
4 KB
:
4 KB
:
4 KB
:::::::
physical memory
4M= 1024x4k
.
.
.
36
pt
34 entries
:
entry 0 (4 byte)
:
entry 767 (4 byte)
:
entry 1023 (4 byte)
pt
:
3M= 768x4k
42
Phase 2
Case 3:
When RAM Size Is More Than 4096MB
43
Assumption
Assume: The CPU model supports Physical Address E
xtension (PAE). The amount of RAM is larger than 4 GB. The kernel is compiled with PAE support.
44
RAM Mapping Principle
Although PAE handles 36-bit physical addresses, linear addresses are still 32-bit addresses.
As in case 2, Linux maps a 887-MB RAM window into the kernel linear address space; the remaining RAM is left unmapped and handled by dynamic remapping.
Translation Table Layout Notice that all CPU models that support PAE also support large 2 MB
pages and global pages. As in the previous case, whenever possible, Linux uses large pages to reduce the number of page tables .
The first 448 (888/2=444) entries in the Page Middle Directory are filled with the physical address of the first 888 MB of RAM. There are 512 entries, but the last 64 (512-444=68) are reserved for non
contiguous memory allocation.
45
Translation Table Layout
0
1
:
:
:
:
:
443
2M
887
MB
444
445
:
:
511
2M
2M
2M
:
:
68
swapper_pg_dir
empyt_zero_page
pmd
46
4k
:
4k
1M
entry 0 (4 byte)
:
entry 255 (4 byte)
:
entry 511 (4 byte)
pt
:
:::
initial_page_table and swapper_pg_dir
After be initialized, the content of swapper_pg_dir will be copied into initial_page_table.
static inline void clone_pgd_range(pgd_t *dst, pgd_t *src, int count)
{
memcpy(dst, src, count * sizeof(pgd_t));
}
clone_pgd_range(initial_page_table+KERNEL_PGD_BOUNDARY,swapper_pg_dir+KERNEL_PGD_BOUNDARY,KERNEL_PGD_PTRS);
9
47
Fix-Mapped Linear Addresses
48
Usage of Fix-Mapped Linear Addresses
The initial part of the fourth gigabyte of kernel linear addresses maps the physical memory of the system.
However, around 128 MB of linear addresses are always left available because the kernel uses them to implement noncontiguous memory allocation
and fix-mapped linear addresses.
49
Fix-Mapped Linear Addresses vs. Physical Addresses Basically, a fix-mapped linear address is a constant line
ar address like 0xffffc000 whose corresponding physical address can be set up in an arbitrary way. Thus, each fix-mapped linear address maps one page frame of the physical memory.
Fix-mapped linear addresses are conceptually similar to the linear addresses that map the first 888 MB of RAM. However, a fix-mapped linear address can map any physical address.
The mapping established by the linear addresses in the initial portion of the fourth gigabyte is linear Linear address X maps physical address X - PAGE_OFFSET.
50
Data Structure enum fixed_addresses Each fix-mapped linear address is represented by an int
eger index defined in the enum fixed_addresses data structure:
enum fixed_addresses {#ifdef CONFIG_X86_32 FIX_HOLE, FIX_VDSO,#else VSYSCALL_LAST_PAGE, VSYSCALL_FIRST_PAGE=VSYSCALL_LAST_PAGE+((VSYSCALL_END- VSYSCALL_START)>>PAGE_SHIFT)-1, VVAR_PAGE, VSYSCALL_HPET,#endif ...
__end_of_fixed_addresses};
51
How to Obtain the Linear Address Set of a Fix-Mapped Linear Address Fix-mapped linear addresses are placed at the end of the
fourth gigabyte of linear addresses. The fix_to_virt( ) function computes the constant li
near address starting from the index: unsigned long __FIXADDR_TOP = 0xfffff000;#define FIXADDR_TOP ((unsigned long)__FIXADDR_TOP)#define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT))
P.S.: #define PAGE_SHIFT 12
Therefore, fix-mapped linear addresses are supposed to use with kernel paging mechanism that uses 4 KB page frames.
52
the Linear Address Set of a Fix-Mapped Linear Address
4k
4k
4k
:
4k
4k
0xfffff000 0
0xffffe000 1
0xffffd000 2
0xffffc000 3
virtual
address
0xffffffff
53
Associate a Physical Address with a Fix-mapped Linear Address
Macros: set_fixmap(idx,phys) and set_fixmap_nocache(idx,phys): Both functions initialize the Page Table entry c
orresponding to the fix_to_virt(idx) linear address with the physical address phys; however, the second function also sets the PCD flag of the Page Table entry, thus disabling the hardware cache when accessing the data in the page frame .