Dynamic Linker

Dynamic LinkerDynamic LinkerAn OverviewAn Overview

Sanjiv MalikSanjiv Malik

2009/1/26 2

Type of LinkingType of Linking Static Linking : All symbols resolved at the time of linking. FoStatic Linking : All symbols resolved at the time of linking. For r

example, example, gccgcc ––static flag sets the program to be linked in static static flag sets the program to be linked in static fashion.fashion.Disadvantage : Large program sizeDisadvantage : Large program sizeAdvantage : Fast processing Advantage : Fast processing

Dynamic Linking : Symbols are resolved at the time of execution Dynamic Linking : Symbols are resolved at the time of execution of the program. of the program. gccgcc by default links the program dynamically. by default links the program dynamically. The program size is small, but the runtime performance cost is The program size is small, but the runtime performance cost is substantial.substantial.

2009/1/26 3

Example linking processExample linking process

Linker (ld)

Translators

m.c

m.o

Translators

a.c

a.o

p

separately compiled relocatable object files

executable object file (contains code and data for all functions defined in m.c and a.c)

2009/1/26 4

What does a linker do?What does a linker do? Merges object filesMerges object files

merges multiple merges multiple relocatablerelocatable (.o) object files into a single (.o) object files into a single executableexecutable object file object file that can loaded and executed by the loader.that can loaded and executed by the loader.

Resolves external referencesResolves external references as part of the merging process, resolves as part of the merging process, resolves external references.external references.

external referenceexternal reference: reference to a symbol defined in another object file.: reference to a symbol defined in another object file. Relocates symbolsRelocates symbols

relocates relocates symbolssymbols from their relative locations in the .o files to new absolute from their relative locations in the .o files to new absolute positions in the executable.positions in the executable.

updates all references to these symbols to reflect their new posupdates all references to these symbols to reflect their new positions.itions. references can be in either code or datareferences can be in either code or data code: code: a(); /* ref to symbol a */a(); /* ref to symbol a */ data: data: intint **xpxp=&x; /* ref to symbol x */=&x; /* ref to symbol x */

2009/1/26 5

Executable and linkable format Executable and linkable format (ELF)(ELF)

Standard binary format for object filesStandard binary format for object files Derives from AT&T System V UnixDerives from AT&T System V Unix

later adopted by BSD Unix variants and Linuxlater adopted by BSD Unix variants and Linux

One unified format for One unified format for relocatablerelocatable object files (.o), executable object files (.o), executable object files, and shared object files (.so)object files, and shared object files (.so) generic name: ELF binariesgeneric name: ELF binaries

Better support for shared libraries than old a.out formats.Better support for shared libraries than old a.out formats.

2009/1/26 6

ELF object file formatELF object file format Elf headerElf header

magic number, type (.o, exec, .so), machine, byte magic number, type (.o, exec, .so), machine, byte ordering, etc.ordering, etc.

Program header tableProgram header table page size, virtual addresses for memory segments page size, virtual addresses for memory segments

(sections), segment sizes.(sections), segment sizes. .text section.text section

codecode .data section.data section

initialized (static) datainitialized (static) data ..bssbss sectionsection

uninitializeduninitialized (static) data(static) data ““Block Started by SymbolBlock Started by Symbol”” ““Better Save SpaceBetter Save Space”” has section header but occupies no spacehas section header but occupies no space

ELF header

Program header table(required for executables)

.text section

.data section

.bss section

.symtab

.rel.txt

.rel.data

.debug

Section header table(required for relocatables)

0

2009/1/26 7

ELF object file formatELF object file format ..symtabsymtab sectionsection

symbol tablesymbol table procedure and static variable namesprocedure and static variable names section names and locationssection names and locations

..rel.textrel.text sectionsection relocation info for .text sectionrelocation info for .text section addresses of instructions that will need to be addresses of instructions that will need to be

modified in the executablemodified in the executable instructions for modifying.instructions for modifying.

..rel.datarel.data sectionsection relocation info for .data sectionrelocation info for .data section addresses of pointer data that will need to be addresses of pointer data that will need to be

modified in the merged executablemodified in the merged executable ..debug sectiondebug section

info for symbolic debugging (info for symbolic debugging (gccgcc --g)g)

ELF header


.text section

.data section

.bss section

.symtab

.rel.text

.rel.data

.debug


0

2009/1/26 8

Example C programExample C program

int e=7;

int main() {int r = a();exit(0);

}

m.c a.cextern int e;

int *ep=&e;int x=15; int y;

int a() { return *ep+x+y;

}

2009/1/26 9

Merging .o files into an executableMerging .o files into an executable

main()m.o

int *ep = &e

a()a.o

int e = 7

headerssystem code

main()

a()

0system code

int *ep = &e

int e = 7

system data

more system code

int x = 15int y

system data

int x = 15

Relocatable object files Executable object file

.text

.text

.data & .bss

.text

.data

.text

.data

.bss .symtab.debug

.data

uninitialized data .bss

2009/1/26 10

Relocating symbols and resolving Relocating symbols and resolving external referencesexternal references

Symbols are lexical entities that name functions and variables.Symbols are lexical entities that name functions and variables.Each symbol has a Each symbol has a valuevalue (typically a memory address).(typically a memory address).Code consists of symbol Code consists of symbol definitionsdefinitions and and references.references.References can be either References can be either locallocal or or external.external.

Ref to external symbol a

Def oflocal symbol ep

Ref toexternalsymbol e

Defs of local symbols x and y

Def of local symbol e

int e=7;


}

m.c a.cextern int e;



} Ref to external symbol exit(defined in libc.so) Refs of local

symbols e,x,y

Def oflocal symbol a

2009/1/26 11

m.o relocation infom.o relocation infoDisassembly of section .text:

00000000 <main>: 00000000 <main>: 0: 55 pushl %ebp1: 89 e5 movl %esp,%ebp3: e8 fc ff ff ff call 4 <main+0x4>

4: R_386_PC32 a 8: 6a 00 pushl $0x0 a: e8 fc ff ff ff call b <main+0xb>

b: R_386_PC32 exit f: 90 nop

Disassembly of section .data:

00000000 <e>: 0: 07 00 00 00

source: objdump

int e=7;


}

m.c

2009/1/26 12

a.o relocation info (.text)a.o relocation info (.text)a.cextern int e;



}

Disassembly of section .text:

00000000 <a>: 0: 55 pushl %ebp1: 8b 15 00 00 00 movl 0x0,%edx 6: 00

3: R_386_32 ep7: a1 00 00 00 00 movl 0x0,%eax

8: R_386_32 x c: 89 e5 movl %esp,%ebpe: 03 02 addl (%edx),%eax10: 89 ec movl %ebp,%esp12: 03 05 00 00 00 addl 0x0,%eax 17: 00

14: R_386_32 y 18: 5d popl %ebp19: c3 ret

2009/1/26 13

a.o relocation info (.data)a.o relocation info (.data)a.cextern int e;



}


00000000 <ep>: 0: 00 00 00 00

0: R_386_32 e 00000004 <x>: 4: 0f 00 00 00

2009/1/26 14

Executable after relocation and Executable after relocation and external reference resolution (.text)external reference resolution (.text)

08048530 <main>: 8048530: 55 pushl %ebp8048531: 89 e5 movl %esp,%ebp8048533: e8 08 00 00 00 call 8048540 <a> 8048538: 6a 00 pushl $0x0 804853a: e8 35 ff ff ff call 8048474 <_init+0x94> 804853f: 90 nop

08048540 <a>: 8048540: 55 pushl %ebp8048541: 8b 15 1c a0 04 movl 0x804a01c,%edx 8048546: 08 8048547: a1 20 a0 04 08 movl 0x804a020,%eax 804854c: 89 e5 movl %esp,%ebp804854e: 03 02 addl (%edx),%eax8048550: 89 ec movl %ebp,%esp8048552: 03 05 d0 a3 04 addl 0x804a3d0,%eax 8048557: 08 8048558: 5d popl %ebp8048559: c3 ret

2009/1/26 15

Executable after relocation and Executable after relocation and external reference resolution (.data)external reference resolution (.data)


0804a010 <__data_start>: 804a010: 00 00 00 00

0804a014 <p.2>: 804a014: f8 a2 04 08

0804a018 <e>: 804a018: 07 00 00 00

0804a01c <ep>: 804a01c: 18 a0 04 08

0804a020 <x>: 804a020: 0f 00 00 00

int e=7;


}

m.c

a.cextern int e;



}

2009/1/26 16

Strong and weak symbolsStrong and weak symbols

Program symbols are either Program symbols are either strongstrong or or weakweak strong: procedures and initialized strong: procedures and initialized globalsglobals weak: weak: uninitializeduninitialized globalsglobals

int foo=5;

p1() {}

int foo;

p2() {}

p1.c: p2.c:

strong

weak

strong

strong

2009/1/26 17

Static libraries (archives)Static libraries (archives)

Translator

p1.c

p1.o

Translator

p2.c

p2.o libc.a static library (archive) of relocatable object files concatenated into one file.

executable object file (only contains code and data for libc functions that are called from p1.c and p2.c)

Further improves modularity and efficiency by packaging commonly used functions (e.g., C standard library, math library)

Linker selectively includes only the .o files in the archive that are actually needed by the program.

Linker (ld)

p

2009/1/26 18

Creating static librariesCreating static libraries

Translator

atoi.c

atoi.o

Translator

printf.c

printf.o

libc.a

Archiver (ar)

... Translator

random.c

random.o

ar rs libc.a \atoi.o printf.o … random.o

Archiver allows incremental updates: • recompile function that changes and replace .o file in archive.

C standard library

2009/1/26 19

The complete pictureThe complete picture

Translator

m.c

m.o

Translator

a.c

a.o

libc.so

Static Linker (ld)

p

Loader/Dynamic Linker(ld-linux.so)

libwhatever.a

p’

libm.so

2009/1/26 20

Dynamically linked shared Dynamically linked shared libraries libraries

libc.so functions called by m.cand a.c are loaded, linked, and (potentially) shared among processes.

shared libraries of dynamically relocatable object files

Translators(cc1, as)

m.c

m.o

Translators(cc1,as)

a.c

a.o

libc.so

Linker (ld)

p

Loader/Dynamic Linker(ld-linux.so)

libm.so

2009/1/26 21

Solaris specific options for generating Solaris specific options for generating static and dynamic executablesstatic and dynamic executables

2009/1/26 22

Linking process in SolarisLinking process in SolarisIn Solaris/Open Solaris, the linking process is performed in twoIn Solaris/Open Solaris, the linking process is performed in two

steps: steps: Compile time linking is done by the Compile time linking is done by the ““ldld”” tool called the Link tool called the Link

Editor. Editor. The link-editor, ld(1), concatenates and interprets data from one or more input files. These files can be relocatableobjects, shared objects, or archive libraries. From these input files, one output file is created. This file is either a relocatableobject, an executable application, or a shared object.

The link-editor is most commonly invoked as part of the compilation environment.

2009/1/26 23

Dynamic linker in SolarisDynamic linker in Solaris The runtime linker, ld.so.1, processes dynamic executables and

shared objects at runtime, binding the executable and shared objects together to create a runnable process.

During the link-editing of a dynamic executable, a special .interpsection, together with an associated program header, are created. This section contains a path name specifying the program’s interpreter. The default name supplied by the link-editor is the name of the runtime linker: /usr/lib/ld.so.1 for a 32–bit executable and /usr/lib/64/ld.so.1 for a 64–bit executable.

The dynamic linker ld.so.1 is itself an ELF shared library. At The dynamic linker ld.so.1 is itself an ELF shared library. At program startup, the system maps the program startup, the system maps the ld.sold.so to a part of the to a part of the address space and runs its bootstrap code. address space and runs its bootstrap code.

2009/1/26 24

Link Editor FunctionsLink Editor FunctionsFollowing is summary of Link Editor functions: The concatenation of sections of the same characteristics from the input

relocatable objects to form new sections within the output file. The concatenated sections can in turn be associated to output segments.

The processing of symbol table information from both relocatable objects and shared objects to verify and unite references with definitions. The generation of a new symbol table, or tables, within the output file.

The processing of relocation information from the input relocatable objects, and the application of this information to the output file by updating other input sections. In addition, output relocation sections might be generated for use by the runtime linker.

The generation of program headers that describe all the segments that are created.

The generation of dynamic linking information sections if necessary, which provide information such as shared object dependencies and symbol bindings to the runtime linker.

2009/1/26 25

Symbol processing by Link EditorSymbol processing by Link Editor

During input file processing, all local symbols from the input relocatableobjects are passed through to the output file image. All global symbols are accumulated internally within the link-editor. Each global symbol supplied by a relocatable object is searched for within this internal symbol table. If a symbol with the same name has already been encountered from a previous input file, a symbol resolution process is called. This symbol resolution process determines which of the two entries are kept.

On completing input file processing, and providing no fatal symbol resolution errors have occurred, the link-editor determines if any unresolved symbol references remain. Unresolved symbol references can cause the link-edit to terminate.

Finally, the link-editor’s internal symbol table is added to the symbol tables of the image being created.

2009/1/26 26

Example Example $ cat main.c

extern int u_bar;extern int u_foo();int t_bar;int d_bar = 1;d_foo(){return (u_foo(u_bar, t_bar, d_bar));}

$ cc -o main.o -c main.c $ nm -x main.o

[Index] Value Size Type Bind Other Shndx Name...............[8] |0x00000000|0x00000000|NOTY |GLOB |0x0 |UNDEF |u_foo[9] |0x00000000|0x00000040|FUNC |GLOB |0x0 |2 |d_foo[10] |0x00000004|0x00000004|OBJT |GLOB |0x0 |COMMON |t_bar[11] |0x00000000|0x00000000|NOTY |GLOB |0x0 |UNDEF |u_bar[12] |0x00000000|0x00000004|OBJT |GLOB |0x0 |3|d_bar

2009/1/26 27

ELF File processingELF File processing Sections are the smallest

indivisible units that can be processed within an ELF file.

Segments are a collection of sections that represent the smallest individual units that can be mapped to a memory image by the dynamic linker ld.so.

2009/1/26 28

Functions of the Dynamic LinkerFunctions of the Dynamic LinkerThe runtime linker:

Analyzes the executable’s dynamic information section (.dynamic) and determines what dependencies are required.Locates and loads these dependencies, analyzing their dynamic information sections to determine if any additional dependencies are required.Performs any necessary relocations to bind these objects in preparation for process execution.Calls any initialization functions provided by the dependencies.Passes control to the application.Can be called upon during the application’s execution, to perform any delayed function binding.

2009/1/26 29

ELF Parsing by Dynamic LinkerELF Parsing by Dynamic Linker

ELF header


.text section

.data section

.bss section

.symtab

.rel.text

.dynamic

.debug


0

.text segment(r/o)

.data segment(initialized r/w)

.bss segment(uninitialized r/w)

Executable object file for example program p

Process image

0x08048494

init and shared libsegments

0x080483e0

virtual addr

0x0804a010

0x0804a3b0

2009/1/26 30

1. Resolving the Dependencies1. Resolving the Dependencies When linking a dynamic executable, one or more shared objects

are explicitly referenced. These objects are recorded as dependencies within the dynamic executable.

The runtime linker uses this dependency information to locate, and load, the associated objects. These dependencies are processed in the same order as the dependencies were referenced during the link-edit of the executable.

Once all the dynamic executable’s dependencies are loaded, each dependency is inspected, in the order the dependency is loaded, to locate any additional dependencies. This process continues until all dependencies are located and loaded. This technique results in a breadth-first ordering of all dependencies.

2009/1/26 31

1. Resolving the Dependencies1. Resolving the Dependencies The Solaris runtime linker looks in two default locations for dependencies

/lib and /usr/lib. The dependencies of a dynamic executable or shared object can be displayed

using ldd. For example, the file /usr/bin/cat has the following dependencies: $ ldd /usr/bin/cat

libc.so.1 => /lib/libc.so.1libm.so.2 => /lib/libm.so.2

The dependencies recorded in an object can be inspected using dump. Use this command to display the file’s .dynamic section, and look for entries that have a NEEDED tag. $ dump -Lvp progprog:[INDEX] Tag Value[1] NEEDED libfoo.so.1[2] NEEDED libc.so.1[3] RUNPATH /home/me/lib:/home/you/lib.........

2009/1/26 32

1. Resolving the Dependencies1. Resolving the Dependencies The dynamic segment (pointed to by the program header) in the ELThe dynamic segment (pointed to by the program header) in the ELF file F file

contains a pointer to the file's string table (DT_STRTAB) as welcontains a pointer to the file's string table (DT_STRTAB) as well as to the l as to the DT_NEEDED entries, each of which contains the offset in the striDT_NEEDED entries, each of which contains the offset in the string table ng table for the name of a required library. The dynamic linker creates afor the name of a required library. The dynamic linker creates a scope listscope list for for the executable, consisting of libraries to be loaded. the executable, consisting of libraries to be loaded.

For each of the entries in the For each of the entries in the scope listscope list , the linker searches for the file , the linker searches for the file containing the library. Once the file is found, the linker readscontaining the library. Once the file is found, the linker reads the ELF Header the ELF Header to find the program header, which points to the dynamic segment to find the program header, which points to the dynamic segment ..

The linker maps the library to the process address space. From tThe linker maps the library to the process address space. From the dynamic he dynamic segment, it adds the library's symbol table to the chain of segment, it adds the library's symbol table to the chain of symbol tablessymbol tables -- and and if the libraries has further dependencies, it adds those librariif the libraries has further dependencies, it adds those libraries to the list to be es to the list to be loaded and the process is continued. For clarification, note thaloaded and the process is continued. For clarification, note that in fact it t in fact it actually creates a actually creates a structstruct link_maplink_map for each of the library and adds it into a for each of the library and adds it into a global linked list. global linked list.

2009/1/26 33

Symbol table structureSymbol table structure t y p e d e f s t r u c t {

E l f 3 2 _ W o r d s t _ n a m e ;E l f 3 2 _ A d d r s t _ v a l u e ;E l f 3 2 _ W o r d s t _ s i z e ;u n s i g n e d c h a r s t _ i n f o ;u n s i g n e d c h a r s t _ o t h e r ;E l f 3 2 _ H a l f s t _ s h n d x ;

} E l f 3 2 _ S y m ;

2009/1/26 34

Parsing other sections of ELFParsing other sections of ELF For dynamic linking, the Dynamic linker primarily uses two For dynamic linking, the Dynamic linker primarily uses two

processorprocessor--specific tables, the Global Offset Table (GOT) and specific tables, the Global Offset Table (GOT) and the Procedure Linkage Table (PLT). Dynamic linkers support the Procedure Linkage Table (PLT). Dynamic linkers support PIC Code through the GOT in each shared library. PIC Code through the GOT in each shared library.

The GOT contains absolute addresses to all of the static data The GOT contains absolute addresses to all of the static data referenced in the program. Both the executables that use the referenced in the program. Both the executables that use the shared libraries and the shared library itself has a PLT. Similashared libraries and the shared library itself has a PLT. Similar to r to how the GOT redirects any positionhow the GOT redirects any position--independent address independent address calculations to absolute locations, the PLT redirects positioncalculations to absolute locations, the PLT redirects position--independent function calls to absolute locations. independent function calls to absolute locations.

2009/1/26 35

Parsing other sections of ELFParsing other sections of ELFIn the .dynamic section, the important tag types are:In the .dynamic section, the important tag types are:

DT_NEEDEDDT_NEEDED: This element holds the string table offset of a : This element holds the string table offset of a nullnull--terminated string, giving the name of a needed library. The terminated string, giving the name of a needed library. The offset is an index into the table recorded in the DT_STRTAB offset is an index into the table recorded in the DT_STRTAB entry.entry.DT_HASHDT_HASH: This element holds the address of the symbol hash : This element holds the address of the symbol hash table which refers to the symbol table referenced by the table which refers to the symbol table referenced by the DT_SYMTAB element.DT_SYMTAB element.DT_STRTABDT_STRTAB: This element holds the address of the string table.: This element holds the address of the string table.DT_SYMTABDT_SYMTAB: This element holds the address of the symbol : This element holds the address of the symbol table. table.

2009/1/26 36

2. Relocation Processing2. Relocation Processing After the runtime linker has loaded all the dependencies required

by an application, the linker processes each object and performsall necessary relocations.

Relocation is the process of connecting symbolic references withsymbolic definitions. For example, when a program calls a function, the associated call instruction must transfer control to the proper destination address at execution. Relocatable files must have information that describes how to modify their section contents. This information allows executable and shared object files to hold the right information for a process’s program image.

2009/1/26 37

3. Loading segments in memory3. Loading segments in memory The LD_BIND_NOW variable determines the dynamic linking The LD_BIND_NOW variable determines the dynamic linking

behavior. If its set, the dynamic linker evaluates the PLT entribehavior. If its set, the dynamic linker evaluates the PLT entries, es, which is all entries of type R_386_JMP_SLOT, at the load time which is all entries of type R_386_JMP_SLOT, at the load time itself. Otherwise, the dynamic linker does lazy linking of itself. Otherwise, the dynamic linker does lazy linking of procedure addresses and hence the addresses are not bound procedure addresses and hence the addresses are not bound unless the routines are called. unless the routines are called.

2009/1/26 38

4. Delayed Function Binding4. Delayed Function Binding Under delayed function binding or lazy loading model, any dependencies that

are labeled for lazy loading are loaded only when explicitly referenced. By taking advantage of the lazy binding of a function call, the loading of a dependency is delayed until the function is first referenced. As a result, objects that are never referenced are never loaded.

As a practical example (.dynamic), shows libdebug.so.1 is marked for lazy loading. The symbol information section

(.SUNW_syminfo), shows the symbol reference that triggers libdebug.so.1 loading.$ cc -o prog prog.c -L. -zlazyload -ldebug -znolazyload -lelf -R’$ORIGIN’$ elfdump -d progDynamic Section: .dynamicindex tag value[0] POSFLAG_1 0x1 [ LAZY ][1] NEEDED 0x123libdebug.so.1[2] NEEDED 0x131 libelf.so.1[3] NEEDED 0x13d libc.so.1

2009/1/26 39

A look into Solaris runtime linkerA look into Solaris runtime linkerThe File The File dl_runtime.cdl_runtime.c contains the following main routines:contains the following main routines: __dl_fixupdl_fixup [Resolves the PLT Symbols][Resolves the PLT Symbols] __dl_profile_fixupdl_profile_fixup __dl_call_pltexitdl_call_pltexitOther related routines:Other related routines: elf_machine_plt_valueelf_machine_plt_value elf_machine_fixup_pltelf_machine_fixup_plt __dl_lookup_symbol_xdl_lookup_symbol_x

2009/1/26 40

ARM Specific ELF Header SettingsARM Specific ELF Header Settings

For ARM target environment, the values in the ELF header are specifically defined. All other values are as specified in the Tool Interface Standard Portable Formats Specification:

e_machine is set to EM_ARM (defined as 40) e_ident[EI_CLASS] is set to ELFCLASS32 e_ident[EI_DATA] is set to:

ELFDATA2LSB for little-endian targetsELFDATA2MSB for big-endian targets

2009/1/26 41

Special sections in ARM ELF FilesSpecial sections in ARM ELF Files

In Executable ARM ELF, all Executables have at least two Sections, unless the linker has been invoked with -nodebug:

The Symbol Table Section:This Section has the following attributes:sh_name: ".symtab"sh_type: SHT_SYMTABsh_addr: 0 (to indicate it is not part of the image)

The String Table Section:This Section has the following attributes:sh_name: ".strtab"sh_type: SHT_STRTABsh_addr: 0 (to indicate it is not part of the image)

2009/1/26 42

Special Sections in ARM ELFSpecial Sections in ARM ELF Debugging Sections ARM Executable ELF supports three types of debugging

information held in debugging Sections. 1. ASD debugging tables

These provide backwards compatibility with ARM's Symbolic Debugger. ASD debugging information is stored in a single Section in the executable named .asd.

2. DWARF Version 1.03. DWARF Version 2.0

Technology

Dynamic Linker