162306a36Sopenharmony_ci=========================== 262306a36Sopenharmony_ciLivepatch module ELF format 362306a36Sopenharmony_ci=========================== 462306a36Sopenharmony_ci 562306a36Sopenharmony_ciThis document outlines the ELF format requirements that livepatch modules must follow. 662306a36Sopenharmony_ci 762306a36Sopenharmony_ci 862306a36Sopenharmony_ci.. Table of Contents 962306a36Sopenharmony_ci 1062306a36Sopenharmony_ci.. contents:: :local: 1162306a36Sopenharmony_ci 1262306a36Sopenharmony_ci 1362306a36Sopenharmony_ci1. Background and motivation 1462306a36Sopenharmony_ci============================ 1562306a36Sopenharmony_ci 1662306a36Sopenharmony_ciFormerly, livepatch required separate architecture-specific code to write 1762306a36Sopenharmony_cirelocations. However, arch-specific code to write relocations already 1862306a36Sopenharmony_ciexists in the module loader, so this former approach produced redundant 1962306a36Sopenharmony_cicode. So, instead of duplicating code and re-implementing what the module 2062306a36Sopenharmony_ciloader can already do, livepatch leverages existing code in the module 2162306a36Sopenharmony_ciloader to perform the all the arch-specific relocation work. Specifically, 2262306a36Sopenharmony_cilivepatch reuses the apply_relocate_add() function in the module loader to 2362306a36Sopenharmony_ciwrite relocations. The patch module ELF format described in this document 2462306a36Sopenharmony_cienables livepatch to be able to do this. The hope is that this will make 2562306a36Sopenharmony_cilivepatch more easily portable to other architectures and reduce the amount 2662306a36Sopenharmony_ciof arch-specific code required to port livepatch to a particular 2762306a36Sopenharmony_ciarchitecture. 2862306a36Sopenharmony_ci 2962306a36Sopenharmony_ciSince apply_relocate_add() requires access to a module's section header 3062306a36Sopenharmony_citable, symbol table, and relocation section indices, ELF information is 3162306a36Sopenharmony_cipreserved for livepatch modules (see section 5). Livepatch manages its own 3262306a36Sopenharmony_cirelocation sections and symbols, which are described in this document. The 3362306a36Sopenharmony_ciELF constants used to mark livepatch symbols and relocation sections were 3462306a36Sopenharmony_ciselected from OS-specific ranges according to the definitions from glibc. 3562306a36Sopenharmony_ci 3662306a36Sopenharmony_ciWhy does livepatch need to write its own relocations? 3762306a36Sopenharmony_ci----------------------------------------------------- 3862306a36Sopenharmony_ciA typical livepatch module contains patched versions of functions that can 3962306a36Sopenharmony_cireference non-exported global symbols and non-included local symbols. 4062306a36Sopenharmony_ciRelocations referencing these types of symbols cannot be left in as-is 4162306a36Sopenharmony_cisince the kernel module loader cannot resolve them and will therefore 4262306a36Sopenharmony_cireject the livepatch module. Furthermore, we cannot apply relocations that 4362306a36Sopenharmony_ciaffect modules not yet loaded at patch module load time (e.g. a patch to a 4462306a36Sopenharmony_cidriver that is not loaded). Formerly, livepatch solved this problem by 4562306a36Sopenharmony_ciembedding special "dynrela" (dynamic rela) sections in the resulting patch 4662306a36Sopenharmony_cimodule ELF output. Using these dynrela sections, livepatch could resolve 4762306a36Sopenharmony_cisymbols while taking into account its scope and what module the symbol 4862306a36Sopenharmony_cibelongs to, and then manually apply the dynamic relocations. However this 4962306a36Sopenharmony_ciapproach required livepatch to supply arch-specific code in order to write 5062306a36Sopenharmony_cithese relocations. In the new format, livepatch manages its own SHT_RELA 5162306a36Sopenharmony_cirelocation sections in place of dynrela sections, and the symbols that the 5262306a36Sopenharmony_cirelas reference are special livepatch symbols (see section 2 and 3). The 5362306a36Sopenharmony_ciarch-specific livepatch relocation code is replaced by a call to 5462306a36Sopenharmony_ciapply_relocate_add(). 5562306a36Sopenharmony_ci 5662306a36Sopenharmony_ci2. Livepatch modinfo field 5762306a36Sopenharmony_ci========================== 5862306a36Sopenharmony_ci 5962306a36Sopenharmony_ciLivepatch modules are required to have the "livepatch" modinfo attribute. 6062306a36Sopenharmony_ciSee the sample livepatch module in samples/livepatch/ for how this is done. 6162306a36Sopenharmony_ci 6262306a36Sopenharmony_ciLivepatch modules can be identified by users by using the 'modinfo' command 6362306a36Sopenharmony_ciand looking for the presence of the "livepatch" field. This field is also 6462306a36Sopenharmony_ciused by the kernel module loader to identify livepatch modules. 6562306a36Sopenharmony_ci 6662306a36Sopenharmony_ciExample: 6762306a36Sopenharmony_ci-------- 6862306a36Sopenharmony_ci 6962306a36Sopenharmony_ci**Modinfo output:** 7062306a36Sopenharmony_ci 7162306a36Sopenharmony_ci:: 7262306a36Sopenharmony_ci 7362306a36Sopenharmony_ci % modinfo livepatch-meminfo.ko 7462306a36Sopenharmony_ci filename: livepatch-meminfo.ko 7562306a36Sopenharmony_ci livepatch: Y 7662306a36Sopenharmony_ci license: GPL 7762306a36Sopenharmony_ci depends: 7862306a36Sopenharmony_ci vermagic: 4.3.0+ SMP mod_unload 7962306a36Sopenharmony_ci 8062306a36Sopenharmony_ci3. Livepatch relocation sections 8162306a36Sopenharmony_ci================================ 8262306a36Sopenharmony_ci 8362306a36Sopenharmony_ciA livepatch module manages its own ELF relocation sections to apply 8462306a36Sopenharmony_cirelocations to modules as well as to the kernel (vmlinux) at the 8562306a36Sopenharmony_ciappropriate time. For example, if a patch module patches a driver that is 8662306a36Sopenharmony_cinot currently loaded, livepatch will apply the corresponding livepatch 8762306a36Sopenharmony_cirelocation section(s) to the driver once it loads. 8862306a36Sopenharmony_ci 8962306a36Sopenharmony_ciEach "object" (e.g. vmlinux, or a module) within a patch module may have 9062306a36Sopenharmony_cimultiple livepatch relocation sections associated with it (e.g. patches to 9162306a36Sopenharmony_cimultiple functions within the same object). There is a 1-1 correspondence 9262306a36Sopenharmony_cibetween a livepatch relocation section and the target section (usually the 9362306a36Sopenharmony_citext section of a function) to which the relocation(s) apply. It is 9462306a36Sopenharmony_cialso possible for a livepatch module to have no livepatch relocation 9562306a36Sopenharmony_cisections, as in the case of the sample livepatch module (see 9662306a36Sopenharmony_cisamples/livepatch). 9762306a36Sopenharmony_ci 9862306a36Sopenharmony_ciSince ELF information is preserved for livepatch modules (see Section 5), a 9962306a36Sopenharmony_cilivepatch relocation section can be applied simply by passing in the 10062306a36Sopenharmony_ciappropriate section index to apply_relocate_add(), which then uses it to 10162306a36Sopenharmony_ciaccess the relocation section and apply the relocations. 10262306a36Sopenharmony_ci 10362306a36Sopenharmony_ciEvery symbol referenced by a rela in a livepatch relocation section is a 10462306a36Sopenharmony_cilivepatch symbol. These must be resolved before livepatch can call 10562306a36Sopenharmony_ciapply_relocate_add(). See Section 3 for more information. 10662306a36Sopenharmony_ci 10762306a36Sopenharmony_ci3.1 Livepatch relocation section format 10862306a36Sopenharmony_ci======================================= 10962306a36Sopenharmony_ci 11062306a36Sopenharmony_ciLivepatch relocation sections must be marked with the SHF_RELA_LIVEPATCH 11162306a36Sopenharmony_cisection flag. See include/uapi/linux/elf.h for the definition. The module 11262306a36Sopenharmony_ciloader recognizes this flag and will avoid applying those relocation sections 11362306a36Sopenharmony_ciat patch module load time. These sections must also be marked with SHF_ALLOC, 11462306a36Sopenharmony_ciso that the module loader doesn't discard them on module load (i.e. they will 11562306a36Sopenharmony_cibe copied into memory along with the other SHF_ALLOC sections). 11662306a36Sopenharmony_ci 11762306a36Sopenharmony_ciThe name of a livepatch relocation section must conform to the following 11862306a36Sopenharmony_ciformat:: 11962306a36Sopenharmony_ci 12062306a36Sopenharmony_ci .klp.rela.objname.section_name 12162306a36Sopenharmony_ci ^ ^^ ^ ^ ^ 12262306a36Sopenharmony_ci |________||_____| |__________| 12362306a36Sopenharmony_ci [A] [B] [C] 12462306a36Sopenharmony_ci 12562306a36Sopenharmony_ci[A] 12662306a36Sopenharmony_ci The relocation section name is prefixed with the string ".klp.rela." 12762306a36Sopenharmony_ci 12862306a36Sopenharmony_ci[B] 12962306a36Sopenharmony_ci The name of the object (i.e. "vmlinux" or name of module) to 13062306a36Sopenharmony_ci which the relocation section belongs follows immediately after the prefix. 13162306a36Sopenharmony_ci 13262306a36Sopenharmony_ci[C] 13362306a36Sopenharmony_ci The actual name of the section to which this relocation section applies. 13462306a36Sopenharmony_ci 13562306a36Sopenharmony_ciExamples: 13662306a36Sopenharmony_ci--------- 13762306a36Sopenharmony_ci 13862306a36Sopenharmony_ci**Livepatch relocation section names:** 13962306a36Sopenharmony_ci 14062306a36Sopenharmony_ci:: 14162306a36Sopenharmony_ci 14262306a36Sopenharmony_ci .klp.rela.ext4.text.ext4_attr_store 14362306a36Sopenharmony_ci .klp.rela.vmlinux.text.cmdline_proc_show 14462306a36Sopenharmony_ci 14562306a36Sopenharmony_ci**`readelf --sections` output for a patch 14662306a36Sopenharmony_cimodule that patches vmlinux and modules 9p, btrfs, ext4:** 14762306a36Sopenharmony_ci 14862306a36Sopenharmony_ci:: 14962306a36Sopenharmony_ci 15062306a36Sopenharmony_ci Section Headers: 15162306a36Sopenharmony_ci [Nr] Name Type Address Off Size ES Flg Lk Inf Al 15262306a36Sopenharmony_ci [ snip ] 15362306a36Sopenharmony_ci [29] .klp.rela.9p.text.caches.show RELA 0000000000000000 002d58 0000c0 18 AIo 64 9 8 15462306a36Sopenharmony_ci [30] .klp.rela.btrfs.text.btrfs.feature.attr.show RELA 0000000000000000 002e18 000060 18 AIo 64 11 8 15562306a36Sopenharmony_ci [ snip ] 15662306a36Sopenharmony_ci [34] .klp.rela.ext4.text.ext4.attr.store RELA 0000000000000000 002fd8 0000d8 18 AIo 64 13 8 15762306a36Sopenharmony_ci [35] .klp.rela.ext4.text.ext4.attr.show RELA 0000000000000000 0030b0 000150 18 AIo 64 15 8 15862306a36Sopenharmony_ci [36] .klp.rela.vmlinux.text.cmdline.proc.show RELA 0000000000000000 003200 000018 18 AIo 64 17 8 15962306a36Sopenharmony_ci [37] .klp.rela.vmlinux.text.meminfo.proc.show RELA 0000000000000000 003218 0000f0 18 AIo 64 19 8 16062306a36Sopenharmony_ci [ snip ] ^ ^ 16162306a36Sopenharmony_ci | | 16262306a36Sopenharmony_ci [*] [*] 16362306a36Sopenharmony_ci 16462306a36Sopenharmony_ci[*] 16562306a36Sopenharmony_ci Livepatch relocation sections are SHT_RELA sections but with a few special 16662306a36Sopenharmony_ci characteristics. Notice that they are marked SHF_ALLOC ("A") so that they will 16762306a36Sopenharmony_ci not be discarded when the module is loaded into memory, as well as with the 16862306a36Sopenharmony_ci SHF_RELA_LIVEPATCH flag ("o" - for OS-specific). 16962306a36Sopenharmony_ci 17062306a36Sopenharmony_ci**`readelf --relocs` output for a patch module:** 17162306a36Sopenharmony_ci 17262306a36Sopenharmony_ci:: 17362306a36Sopenharmony_ci 17462306a36Sopenharmony_ci Relocation section '.klp.rela.btrfs.text.btrfs_feature_attr_show' at offset 0x2ba0 contains 4 entries: 17562306a36Sopenharmony_ci Offset Info Type Symbol's Value Symbol's Name + Addend 17662306a36Sopenharmony_ci 000000000000001f 0000005e00000002 R_X86_64_PC32 0000000000000000 .klp.sym.vmlinux.printk,0 - 4 17762306a36Sopenharmony_ci 0000000000000028 0000003d0000000b R_X86_64_32S 0000000000000000 .klp.sym.btrfs.btrfs_ktype,0 + 0 17862306a36Sopenharmony_ci 0000000000000036 0000003b00000002 R_X86_64_PC32 0000000000000000 .klp.sym.btrfs.can_modify_feature.isra.3,0 - 4 17962306a36Sopenharmony_ci 000000000000004c 0000004900000002 R_X86_64_PC32 0000000000000000 .klp.sym.vmlinux.snprintf,0 - 4 18062306a36Sopenharmony_ci [ snip ] ^ 18162306a36Sopenharmony_ci | 18262306a36Sopenharmony_ci [*] 18362306a36Sopenharmony_ci 18462306a36Sopenharmony_ci[*] 18562306a36Sopenharmony_ci Every symbol referenced by a relocation is a livepatch symbol. 18662306a36Sopenharmony_ci 18762306a36Sopenharmony_ci4. Livepatch symbols 18862306a36Sopenharmony_ci==================== 18962306a36Sopenharmony_ci 19062306a36Sopenharmony_ciLivepatch symbols are symbols referred to by livepatch relocation sections. 19162306a36Sopenharmony_ciThese are symbols accessed from new versions of functions for patched 19262306a36Sopenharmony_ciobjects, whose addresses cannot be resolved by the module loader (because 19362306a36Sopenharmony_cithey are local or unexported global syms). Since the module loader only 19462306a36Sopenharmony_ciresolves exported syms, and not every symbol referenced by the new patched 19562306a36Sopenharmony_cifunctions is exported, livepatch symbols were introduced. They are used 19662306a36Sopenharmony_cialso in cases where we cannot immediately know the address of a symbol when 19762306a36Sopenharmony_cia patch module loads. For example, this is the case when livepatch patches 19862306a36Sopenharmony_cia module that is not loaded yet. In this case, the relevant livepatch 19962306a36Sopenharmony_cisymbols are resolved simply when the target module loads. In any case, for 20062306a36Sopenharmony_ciany livepatch relocation section, all livepatch symbols referenced by that 20162306a36Sopenharmony_cisection must be resolved before livepatch can call apply_relocate_add() for 20262306a36Sopenharmony_cithat reloc section. 20362306a36Sopenharmony_ci 20462306a36Sopenharmony_ciLivepatch symbols must be marked with SHN_LIVEPATCH so that the module 20562306a36Sopenharmony_ciloader can identify and ignore them. Livepatch modules keep these symbols 20662306a36Sopenharmony_ciin their symbol tables, and the symbol table is made accessible through 20762306a36Sopenharmony_cimodule->symtab. 20862306a36Sopenharmony_ci 20962306a36Sopenharmony_ci4.1 A livepatch module's symbol table 21062306a36Sopenharmony_ci===================================== 21162306a36Sopenharmony_ciNormally, a stripped down copy of a module's symbol table (containing only 21262306a36Sopenharmony_ci"core" symbols) is made available through module->symtab (See layout_symtab() 21362306a36Sopenharmony_ciin kernel/module/kallsyms.c). For livepatch modules, the symbol table copied 21462306a36Sopenharmony_ciinto memory on module load must be exactly the same as the symbol table produced 21562306a36Sopenharmony_ciwhen the patch module was compiled. This is because the relocations in each 21662306a36Sopenharmony_cilivepatch relocation section refer to their respective symbols with their symbol 21762306a36Sopenharmony_ciindices, and the original symbol indices (and thus the symtab ordering) must be 21862306a36Sopenharmony_cipreserved in order for apply_relocate_add() to find the right symbol. 21962306a36Sopenharmony_ci 22062306a36Sopenharmony_ciFor example, take this particular rela from a livepatch module::: 22162306a36Sopenharmony_ci 22262306a36Sopenharmony_ci Relocation section '.klp.rela.btrfs.text.btrfs_feature_attr_show' at offset 0x2ba0 contains 4 entries: 22362306a36Sopenharmony_ci Offset Info Type Symbol's Value Symbol's Name + Addend 22462306a36Sopenharmony_ci 000000000000001f 0000005e00000002 R_X86_64_PC32 0000000000000000 .klp.sym.vmlinux.printk,0 - 4 22562306a36Sopenharmony_ci 22662306a36Sopenharmony_ci This rela refers to the symbol '.klp.sym.vmlinux.printk,0', and the symbol index is encoded 22762306a36Sopenharmony_ci in 'Info'. Here its symbol index is 0x5e, which is 94 in decimal, which refers to the 22862306a36Sopenharmony_ci symbol index 94. 22962306a36Sopenharmony_ci And in this patch module's corresponding symbol table, symbol index 94 refers to that very symbol: 23062306a36Sopenharmony_ci [ snip ] 23162306a36Sopenharmony_ci 94: 0000000000000000 0 NOTYPE GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.printk,0 23262306a36Sopenharmony_ci [ snip ] 23362306a36Sopenharmony_ci 23462306a36Sopenharmony_ci4.2 Livepatch symbol format 23562306a36Sopenharmony_ci=========================== 23662306a36Sopenharmony_ci 23762306a36Sopenharmony_ciLivepatch symbols must have their section index marked as SHN_LIVEPATCH, so 23862306a36Sopenharmony_cithat the module loader can identify them and not attempt to resolve them. 23962306a36Sopenharmony_ciSee include/uapi/linux/elf.h for the actual definitions. 24062306a36Sopenharmony_ci 24162306a36Sopenharmony_ciLivepatch symbol names must conform to the following format:: 24262306a36Sopenharmony_ci 24362306a36Sopenharmony_ci .klp.sym.objname.symbol_name,sympos 24462306a36Sopenharmony_ci ^ ^^ ^ ^ ^ ^ 24562306a36Sopenharmony_ci |_______||_____| |_________| | 24662306a36Sopenharmony_ci [A] [B] [C] [D] 24762306a36Sopenharmony_ci 24862306a36Sopenharmony_ci[A] 24962306a36Sopenharmony_ci The symbol name is prefixed with the string ".klp.sym." 25062306a36Sopenharmony_ci 25162306a36Sopenharmony_ci[B] 25262306a36Sopenharmony_ci The name of the object (i.e. "vmlinux" or name of module) to 25362306a36Sopenharmony_ci which the symbol belongs follows immediately after the prefix. 25462306a36Sopenharmony_ci 25562306a36Sopenharmony_ci[C] 25662306a36Sopenharmony_ci The actual name of the symbol. 25762306a36Sopenharmony_ci 25862306a36Sopenharmony_ci[D] 25962306a36Sopenharmony_ci The position of the symbol in the object (as according to kallsyms) 26062306a36Sopenharmony_ci This is used to differentiate duplicate symbols within the same 26162306a36Sopenharmony_ci object. The symbol position is expressed numerically (0, 1, 2...). 26262306a36Sopenharmony_ci The symbol position of a unique symbol is 0. 26362306a36Sopenharmony_ci 26462306a36Sopenharmony_ciExamples: 26562306a36Sopenharmony_ci--------- 26662306a36Sopenharmony_ci 26762306a36Sopenharmony_ci**Livepatch symbol names:** 26862306a36Sopenharmony_ci 26962306a36Sopenharmony_ci:: 27062306a36Sopenharmony_ci 27162306a36Sopenharmony_ci .klp.sym.vmlinux.snprintf,0 27262306a36Sopenharmony_ci .klp.sym.vmlinux.printk,0 27362306a36Sopenharmony_ci .klp.sym.btrfs.btrfs_ktype,0 27462306a36Sopenharmony_ci 27562306a36Sopenharmony_ci**`readelf --symbols` output for a patch module:** 27662306a36Sopenharmony_ci 27762306a36Sopenharmony_ci:: 27862306a36Sopenharmony_ci 27962306a36Sopenharmony_ci Symbol table '.symtab' contains 127 entries: 28062306a36Sopenharmony_ci Num: Value Size Type Bind Vis Ndx Name 28162306a36Sopenharmony_ci [ snip ] 28262306a36Sopenharmony_ci 73: 0000000000000000 0 NOTYPE GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.snprintf,0 28362306a36Sopenharmony_ci 74: 0000000000000000 0 NOTYPE GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.capable,0 28462306a36Sopenharmony_ci 75: 0000000000000000 0 NOTYPE GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.find_next_bit,0 28562306a36Sopenharmony_ci 76: 0000000000000000 0 NOTYPE GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.si_swapinfo,0 28662306a36Sopenharmony_ci [ snip ] ^ 28762306a36Sopenharmony_ci | 28862306a36Sopenharmony_ci [*] 28962306a36Sopenharmony_ci 29062306a36Sopenharmony_ci[*] 29162306a36Sopenharmony_ci Note that the 'Ndx' (Section index) for these symbols is SHN_LIVEPATCH (0xff20). 29262306a36Sopenharmony_ci "OS" means OS-specific. 29362306a36Sopenharmony_ci 29462306a36Sopenharmony_ci5. Symbol table and ELF section access 29562306a36Sopenharmony_ci====================================== 29662306a36Sopenharmony_ciA livepatch module's symbol table is accessible through module->symtab. 29762306a36Sopenharmony_ci 29862306a36Sopenharmony_ciSince apply_relocate_add() requires access to a module's section headers, 29962306a36Sopenharmony_cisymbol table, and relocation section indices, ELF information is preserved for 30062306a36Sopenharmony_cilivepatch modules and is made accessible by the module loader through 30162306a36Sopenharmony_cimodule->klp_info, which is a :c:type:`klp_modinfo` struct. When a livepatch module 30262306a36Sopenharmony_ciloads, this struct is filled in by the module loader. 303