162306a36Sopenharmony_ci===========================
262306a36Sopenharmony_ciLivepatch module ELF format
362306a36Sopenharmony_ci===========================
462306a36Sopenharmony_ci
562306a36Sopenharmony_ciThis document outlines the ELF format requirements that livepatch modules must follow.
662306a36Sopenharmony_ci
762306a36Sopenharmony_ci
862306a36Sopenharmony_ci.. Table of Contents
962306a36Sopenharmony_ci
1062306a36Sopenharmony_ci.. contents:: :local:
1162306a36Sopenharmony_ci
1262306a36Sopenharmony_ci
1362306a36Sopenharmony_ci1. Background and motivation
1462306a36Sopenharmony_ci============================
1562306a36Sopenharmony_ci
1662306a36Sopenharmony_ciFormerly, livepatch required separate architecture-specific code to write
1762306a36Sopenharmony_cirelocations. However, arch-specific code to write relocations already
1862306a36Sopenharmony_ciexists in the module loader, so this former approach produced redundant
1962306a36Sopenharmony_cicode. So, instead of duplicating code and re-implementing what the module
2062306a36Sopenharmony_ciloader can already do, livepatch leverages existing code in the module
2162306a36Sopenharmony_ciloader to perform the all the arch-specific relocation work. Specifically,
2262306a36Sopenharmony_cilivepatch reuses the apply_relocate_add() function in the module loader to
2362306a36Sopenharmony_ciwrite relocations. The patch module ELF format described in this document
2462306a36Sopenharmony_cienables livepatch to be able to do this. The hope is that this will make
2562306a36Sopenharmony_cilivepatch more easily portable to other architectures and reduce the amount
2662306a36Sopenharmony_ciof arch-specific code required to port livepatch to a particular
2762306a36Sopenharmony_ciarchitecture.
2862306a36Sopenharmony_ci
2962306a36Sopenharmony_ciSince apply_relocate_add() requires access to a module's section header
3062306a36Sopenharmony_citable, symbol table, and relocation section indices, ELF information is
3162306a36Sopenharmony_cipreserved for livepatch modules (see section 5). Livepatch manages its own
3262306a36Sopenharmony_cirelocation sections and symbols, which are described in this document. The
3362306a36Sopenharmony_ciELF constants used to mark livepatch symbols and relocation sections were
3462306a36Sopenharmony_ciselected from OS-specific ranges according to the definitions from glibc.
3562306a36Sopenharmony_ci
3662306a36Sopenharmony_ciWhy does livepatch need to write its own relocations?
3762306a36Sopenharmony_ci-----------------------------------------------------
3862306a36Sopenharmony_ciA typical livepatch module contains patched versions of functions that can
3962306a36Sopenharmony_cireference non-exported global symbols and non-included local symbols.
4062306a36Sopenharmony_ciRelocations referencing these types of symbols cannot be left in as-is
4162306a36Sopenharmony_cisince the kernel module loader cannot resolve them and will therefore
4262306a36Sopenharmony_cireject the livepatch module. Furthermore, we cannot apply relocations that
4362306a36Sopenharmony_ciaffect modules not yet loaded at patch module load time (e.g. a patch to a
4462306a36Sopenharmony_cidriver that is not loaded). Formerly, livepatch solved this problem by
4562306a36Sopenharmony_ciembedding special "dynrela" (dynamic rela) sections in the resulting patch
4662306a36Sopenharmony_cimodule ELF output. Using these dynrela sections, livepatch could resolve
4762306a36Sopenharmony_cisymbols while taking into account its scope and what module the symbol
4862306a36Sopenharmony_cibelongs to, and then manually apply the dynamic relocations. However this
4962306a36Sopenharmony_ciapproach required livepatch to supply arch-specific code in order to write
5062306a36Sopenharmony_cithese relocations. In the new format, livepatch manages its own SHT_RELA
5162306a36Sopenharmony_cirelocation sections in place of dynrela sections, and the symbols that the
5262306a36Sopenharmony_cirelas reference are special livepatch symbols (see section 2 and 3). The
5362306a36Sopenharmony_ciarch-specific livepatch relocation code is replaced by a call to
5462306a36Sopenharmony_ciapply_relocate_add().
5562306a36Sopenharmony_ci
5662306a36Sopenharmony_ci2. Livepatch modinfo field
5762306a36Sopenharmony_ci==========================
5862306a36Sopenharmony_ci
5962306a36Sopenharmony_ciLivepatch modules are required to have the "livepatch" modinfo attribute.
6062306a36Sopenharmony_ciSee the sample livepatch module in samples/livepatch/ for how this is done.
6162306a36Sopenharmony_ci
6262306a36Sopenharmony_ciLivepatch modules can be identified by users by using the 'modinfo' command
6362306a36Sopenharmony_ciand looking for the presence of the "livepatch" field. This field is also
6462306a36Sopenharmony_ciused by the kernel module loader to identify livepatch modules.
6562306a36Sopenharmony_ci
6662306a36Sopenharmony_ciExample:
6762306a36Sopenharmony_ci--------
6862306a36Sopenharmony_ci
6962306a36Sopenharmony_ci**Modinfo output:**
7062306a36Sopenharmony_ci
7162306a36Sopenharmony_ci::
7262306a36Sopenharmony_ci
7362306a36Sopenharmony_ci	% modinfo livepatch-meminfo.ko
7462306a36Sopenharmony_ci	filename:		livepatch-meminfo.ko
7562306a36Sopenharmony_ci	livepatch:		Y
7662306a36Sopenharmony_ci	license:		GPL
7762306a36Sopenharmony_ci	depends:
7862306a36Sopenharmony_ci	vermagic:		4.3.0+ SMP mod_unload
7962306a36Sopenharmony_ci
8062306a36Sopenharmony_ci3. Livepatch relocation sections
8162306a36Sopenharmony_ci================================
8262306a36Sopenharmony_ci
8362306a36Sopenharmony_ciA livepatch module manages its own ELF relocation sections to apply
8462306a36Sopenharmony_cirelocations to modules as well as to the kernel (vmlinux) at the
8562306a36Sopenharmony_ciappropriate time. For example, if a patch module patches a driver that is
8662306a36Sopenharmony_cinot currently loaded, livepatch will apply the corresponding livepatch
8762306a36Sopenharmony_cirelocation section(s) to the driver once it loads.
8862306a36Sopenharmony_ci
8962306a36Sopenharmony_ciEach "object" (e.g. vmlinux, or a module) within a patch module may have
9062306a36Sopenharmony_cimultiple livepatch relocation sections associated with it (e.g. patches to
9162306a36Sopenharmony_cimultiple functions within the same object). There is a 1-1 correspondence
9262306a36Sopenharmony_cibetween a livepatch relocation section and the target section (usually the
9362306a36Sopenharmony_citext section of a function) to which the relocation(s) apply. It is
9462306a36Sopenharmony_cialso possible for a livepatch module to have no livepatch relocation
9562306a36Sopenharmony_cisections, as in the case of the sample livepatch module (see
9662306a36Sopenharmony_cisamples/livepatch).
9762306a36Sopenharmony_ci
9862306a36Sopenharmony_ciSince ELF information is preserved for livepatch modules (see Section 5), a
9962306a36Sopenharmony_cilivepatch relocation section can be applied simply by passing in the
10062306a36Sopenharmony_ciappropriate section index to apply_relocate_add(), which then uses it to
10162306a36Sopenharmony_ciaccess the relocation section and apply the relocations.
10262306a36Sopenharmony_ci
10362306a36Sopenharmony_ciEvery symbol referenced by a rela in a livepatch relocation section is a
10462306a36Sopenharmony_cilivepatch symbol. These must be resolved before livepatch can call
10562306a36Sopenharmony_ciapply_relocate_add(). See Section 3 for more information.
10662306a36Sopenharmony_ci
10762306a36Sopenharmony_ci3.1 Livepatch relocation section format
10862306a36Sopenharmony_ci=======================================
10962306a36Sopenharmony_ci
11062306a36Sopenharmony_ciLivepatch relocation sections must be marked with the SHF_RELA_LIVEPATCH
11162306a36Sopenharmony_cisection flag. See include/uapi/linux/elf.h for the definition. The module
11262306a36Sopenharmony_ciloader recognizes this flag and will avoid applying those relocation sections
11362306a36Sopenharmony_ciat patch module load time. These sections must also be marked with SHF_ALLOC,
11462306a36Sopenharmony_ciso that the module loader doesn't discard them on module load (i.e. they will
11562306a36Sopenharmony_cibe copied into memory along with the other SHF_ALLOC sections).
11662306a36Sopenharmony_ci
11762306a36Sopenharmony_ciThe name of a livepatch relocation section must conform to the following
11862306a36Sopenharmony_ciformat::
11962306a36Sopenharmony_ci
12062306a36Sopenharmony_ci  .klp.rela.objname.section_name
12162306a36Sopenharmony_ci  ^        ^^     ^ ^          ^
12262306a36Sopenharmony_ci  |________||_____| |__________|
12362306a36Sopenharmony_ci     [A]      [B]        [C]
12462306a36Sopenharmony_ci
12562306a36Sopenharmony_ci[A]
12662306a36Sopenharmony_ci  The relocation section name is prefixed with the string ".klp.rela."
12762306a36Sopenharmony_ci
12862306a36Sopenharmony_ci[B]
12962306a36Sopenharmony_ci  The name of the object (i.e. "vmlinux" or name of module) to
13062306a36Sopenharmony_ci  which the relocation section belongs follows immediately after the prefix.
13162306a36Sopenharmony_ci
13262306a36Sopenharmony_ci[C]
13362306a36Sopenharmony_ci  The actual name of the section to which this relocation section applies.
13462306a36Sopenharmony_ci
13562306a36Sopenharmony_ciExamples:
13662306a36Sopenharmony_ci---------
13762306a36Sopenharmony_ci
13862306a36Sopenharmony_ci**Livepatch relocation section names:**
13962306a36Sopenharmony_ci
14062306a36Sopenharmony_ci::
14162306a36Sopenharmony_ci
14262306a36Sopenharmony_ci  .klp.rela.ext4.text.ext4_attr_store
14362306a36Sopenharmony_ci  .klp.rela.vmlinux.text.cmdline_proc_show
14462306a36Sopenharmony_ci
14562306a36Sopenharmony_ci**`readelf --sections` output for a patch
14662306a36Sopenharmony_cimodule that patches vmlinux and modules 9p, btrfs, ext4:**
14762306a36Sopenharmony_ci
14862306a36Sopenharmony_ci::
14962306a36Sopenharmony_ci
15062306a36Sopenharmony_ci  Section Headers:
15162306a36Sopenharmony_ci  [Nr] Name                          Type                    Address          Off    Size   ES Flg Lk Inf Al
15262306a36Sopenharmony_ci  [ snip ]
15362306a36Sopenharmony_ci  [29] .klp.rela.9p.text.caches.show RELA                    0000000000000000 002d58 0000c0 18 AIo 64   9  8
15462306a36Sopenharmony_ci  [30] .klp.rela.btrfs.text.btrfs.feature.attr.show RELA     0000000000000000 002e18 000060 18 AIo 64  11  8
15562306a36Sopenharmony_ci  [ snip ]
15662306a36Sopenharmony_ci  [34] .klp.rela.ext4.text.ext4.attr.store RELA              0000000000000000 002fd8 0000d8 18 AIo 64  13  8
15762306a36Sopenharmony_ci  [35] .klp.rela.ext4.text.ext4.attr.show RELA               0000000000000000 0030b0 000150 18 AIo 64  15  8
15862306a36Sopenharmony_ci  [36] .klp.rela.vmlinux.text.cmdline.proc.show RELA         0000000000000000 003200 000018 18 AIo 64  17  8
15962306a36Sopenharmony_ci  [37] .klp.rela.vmlinux.text.meminfo.proc.show RELA         0000000000000000 003218 0000f0 18 AIo 64  19  8
16062306a36Sopenharmony_ci  [ snip ]                                       ^                                             ^
16162306a36Sopenharmony_ci                                                 |                                             |
16262306a36Sopenharmony_ci                                                [*]                                           [*]
16362306a36Sopenharmony_ci
16462306a36Sopenharmony_ci[*]
16562306a36Sopenharmony_ci  Livepatch relocation sections are SHT_RELA sections but with a few special
16662306a36Sopenharmony_ci  characteristics. Notice that they are marked SHF_ALLOC ("A") so that they will
16762306a36Sopenharmony_ci  not be discarded when the module is loaded into memory, as well as with the
16862306a36Sopenharmony_ci  SHF_RELA_LIVEPATCH flag ("o" - for OS-specific).
16962306a36Sopenharmony_ci
17062306a36Sopenharmony_ci**`readelf --relocs` output for a patch module:**
17162306a36Sopenharmony_ci
17262306a36Sopenharmony_ci::
17362306a36Sopenharmony_ci
17462306a36Sopenharmony_ci  Relocation section '.klp.rela.btrfs.text.btrfs_feature_attr_show' at offset 0x2ba0 contains 4 entries:
17562306a36Sopenharmony_ci      Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
17662306a36Sopenharmony_ci  000000000000001f  0000005e00000002 R_X86_64_PC32          0000000000000000 .klp.sym.vmlinux.printk,0 - 4
17762306a36Sopenharmony_ci  0000000000000028  0000003d0000000b R_X86_64_32S           0000000000000000 .klp.sym.btrfs.btrfs_ktype,0 + 0
17862306a36Sopenharmony_ci  0000000000000036  0000003b00000002 R_X86_64_PC32          0000000000000000 .klp.sym.btrfs.can_modify_feature.isra.3,0 - 4
17962306a36Sopenharmony_ci  000000000000004c  0000004900000002 R_X86_64_PC32          0000000000000000 .klp.sym.vmlinux.snprintf,0 - 4
18062306a36Sopenharmony_ci  [ snip ]                                                                   ^
18162306a36Sopenharmony_ci                                                                             |
18262306a36Sopenharmony_ci                                                                            [*]
18362306a36Sopenharmony_ci
18462306a36Sopenharmony_ci[*]
18562306a36Sopenharmony_ci  Every symbol referenced by a relocation is a livepatch symbol.
18662306a36Sopenharmony_ci
18762306a36Sopenharmony_ci4. Livepatch symbols
18862306a36Sopenharmony_ci====================
18962306a36Sopenharmony_ci
19062306a36Sopenharmony_ciLivepatch symbols are symbols referred to by livepatch relocation sections.
19162306a36Sopenharmony_ciThese are symbols accessed from new versions of functions for patched
19262306a36Sopenharmony_ciobjects, whose addresses cannot be resolved by the module loader (because
19362306a36Sopenharmony_cithey are local or unexported global syms). Since the module loader only
19462306a36Sopenharmony_ciresolves exported syms, and not every symbol referenced by the new patched
19562306a36Sopenharmony_cifunctions is exported, livepatch symbols were introduced. They are used
19662306a36Sopenharmony_cialso in cases where we cannot immediately know the address of a symbol when
19762306a36Sopenharmony_cia patch module loads. For example, this is the case when livepatch patches
19862306a36Sopenharmony_cia module that is not loaded yet. In this case, the relevant livepatch
19962306a36Sopenharmony_cisymbols are resolved simply when the target module loads. In any case, for
20062306a36Sopenharmony_ciany livepatch relocation section, all livepatch symbols referenced by that
20162306a36Sopenharmony_cisection must be resolved before livepatch can call apply_relocate_add() for
20262306a36Sopenharmony_cithat reloc section.
20362306a36Sopenharmony_ci
20462306a36Sopenharmony_ciLivepatch symbols must be marked with SHN_LIVEPATCH so that the module
20562306a36Sopenharmony_ciloader can identify and ignore them. Livepatch modules keep these symbols
20662306a36Sopenharmony_ciin their symbol tables, and the symbol table is made accessible through
20762306a36Sopenharmony_cimodule->symtab.
20862306a36Sopenharmony_ci
20962306a36Sopenharmony_ci4.1 A livepatch module's symbol table
21062306a36Sopenharmony_ci=====================================
21162306a36Sopenharmony_ciNormally, a stripped down copy of a module's symbol table (containing only
21262306a36Sopenharmony_ci"core" symbols) is made available through module->symtab (See layout_symtab()
21362306a36Sopenharmony_ciin kernel/module/kallsyms.c). For livepatch modules, the symbol table copied
21462306a36Sopenharmony_ciinto memory on module load must be exactly the same as the symbol table produced
21562306a36Sopenharmony_ciwhen the patch module was compiled. This is because the relocations in each
21662306a36Sopenharmony_cilivepatch relocation section refer to their respective symbols with their symbol
21762306a36Sopenharmony_ciindices, and the original symbol indices (and thus the symtab ordering) must be
21862306a36Sopenharmony_cipreserved in order for apply_relocate_add() to find the right symbol.
21962306a36Sopenharmony_ci
22062306a36Sopenharmony_ciFor example, take this particular rela from a livepatch module:::
22162306a36Sopenharmony_ci
22262306a36Sopenharmony_ci  Relocation section '.klp.rela.btrfs.text.btrfs_feature_attr_show' at offset 0x2ba0 contains 4 entries:
22362306a36Sopenharmony_ci      Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
22462306a36Sopenharmony_ci  000000000000001f  0000005e00000002 R_X86_64_PC32          0000000000000000 .klp.sym.vmlinux.printk,0 - 4
22562306a36Sopenharmony_ci
22662306a36Sopenharmony_ci  This rela refers to the symbol '.klp.sym.vmlinux.printk,0', and the symbol index is encoded
22762306a36Sopenharmony_ci  in 'Info'. Here its symbol index is 0x5e, which is 94 in decimal, which refers to the
22862306a36Sopenharmony_ci  symbol index 94.
22962306a36Sopenharmony_ci  And in this patch module's corresponding symbol table, symbol index 94 refers to that very symbol:
23062306a36Sopenharmony_ci  [ snip ]
23162306a36Sopenharmony_ci  94: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.printk,0
23262306a36Sopenharmony_ci  [ snip ]
23362306a36Sopenharmony_ci
23462306a36Sopenharmony_ci4.2 Livepatch symbol format
23562306a36Sopenharmony_ci===========================
23662306a36Sopenharmony_ci
23762306a36Sopenharmony_ciLivepatch symbols must have their section index marked as SHN_LIVEPATCH, so
23862306a36Sopenharmony_cithat the module loader can identify them and not attempt to resolve them.
23962306a36Sopenharmony_ciSee include/uapi/linux/elf.h for the actual definitions.
24062306a36Sopenharmony_ci
24162306a36Sopenharmony_ciLivepatch symbol names must conform to the following format::
24262306a36Sopenharmony_ci
24362306a36Sopenharmony_ci  .klp.sym.objname.symbol_name,sympos
24462306a36Sopenharmony_ci  ^       ^^     ^ ^         ^ ^
24562306a36Sopenharmony_ci  |_______||_____| |_________| |
24662306a36Sopenharmony_ci     [A]     [B]       [C]    [D]
24762306a36Sopenharmony_ci
24862306a36Sopenharmony_ci[A]
24962306a36Sopenharmony_ci  The symbol name is prefixed with the string ".klp.sym."
25062306a36Sopenharmony_ci
25162306a36Sopenharmony_ci[B]
25262306a36Sopenharmony_ci  The name of the object (i.e. "vmlinux" or name of module) to
25362306a36Sopenharmony_ci  which the symbol belongs follows immediately after the prefix.
25462306a36Sopenharmony_ci
25562306a36Sopenharmony_ci[C]
25662306a36Sopenharmony_ci  The actual name of the symbol.
25762306a36Sopenharmony_ci
25862306a36Sopenharmony_ci[D]
25962306a36Sopenharmony_ci  The position of the symbol in the object (as according to kallsyms)
26062306a36Sopenharmony_ci  This is used to differentiate duplicate symbols within the same
26162306a36Sopenharmony_ci  object. The symbol position is expressed numerically (0, 1, 2...).
26262306a36Sopenharmony_ci  The symbol position of a unique symbol is 0.
26362306a36Sopenharmony_ci
26462306a36Sopenharmony_ciExamples:
26562306a36Sopenharmony_ci---------
26662306a36Sopenharmony_ci
26762306a36Sopenharmony_ci**Livepatch symbol names:**
26862306a36Sopenharmony_ci
26962306a36Sopenharmony_ci::
27062306a36Sopenharmony_ci
27162306a36Sopenharmony_ci	.klp.sym.vmlinux.snprintf,0
27262306a36Sopenharmony_ci	.klp.sym.vmlinux.printk,0
27362306a36Sopenharmony_ci	.klp.sym.btrfs.btrfs_ktype,0
27462306a36Sopenharmony_ci
27562306a36Sopenharmony_ci**`readelf --symbols` output for a patch module:**
27662306a36Sopenharmony_ci
27762306a36Sopenharmony_ci::
27862306a36Sopenharmony_ci
27962306a36Sopenharmony_ci  Symbol table '.symtab' contains 127 entries:
28062306a36Sopenharmony_ci     Num:    Value          Size Type    Bind   Vis     Ndx         Name
28162306a36Sopenharmony_ci     [ snip ]
28262306a36Sopenharmony_ci      73: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.snprintf,0
28362306a36Sopenharmony_ci      74: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.capable,0
28462306a36Sopenharmony_ci      75: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.find_next_bit,0
28562306a36Sopenharmony_ci      76: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT OS [0xff20] .klp.sym.vmlinux.si_swapinfo,0
28662306a36Sopenharmony_ci    [ snip ]                                               ^
28762306a36Sopenharmony_ci                                                           |
28862306a36Sopenharmony_ci                                                          [*]
28962306a36Sopenharmony_ci
29062306a36Sopenharmony_ci[*]
29162306a36Sopenharmony_ci  Note that the 'Ndx' (Section index) for these symbols is SHN_LIVEPATCH (0xff20).
29262306a36Sopenharmony_ci  "OS" means OS-specific.
29362306a36Sopenharmony_ci
29462306a36Sopenharmony_ci5. Symbol table and ELF section access
29562306a36Sopenharmony_ci======================================
29662306a36Sopenharmony_ciA livepatch module's symbol table is accessible through module->symtab.
29762306a36Sopenharmony_ci
29862306a36Sopenharmony_ciSince apply_relocate_add() requires access to a module's section headers,
29962306a36Sopenharmony_cisymbol table, and relocation section indices, ELF information is preserved for
30062306a36Sopenharmony_cilivepatch modules and is made accessible by the module loader through
30162306a36Sopenharmony_cimodule->klp_info, which is a :c:type:`klp_modinfo` struct. When a livepatch module
30262306a36Sopenharmony_ciloads, this struct is filled in by the module loader.
303