Kevin Koo

Domain Issue

My domain has not been renewed properly for the last two weeks. Now it has been working. Thanks for visiting. 🙂

Dynamic Linking in ELF

ELF is the binary format that allows for being both executable and linkable. It is de-facto standard in Linux.

A. Linking Overview

As the size of the program functionality grows, modulization helps programmers to maintain their code with efficiency. During compilation, an object file is generated per module. Afterwards, a linker (i.e., ld or ld.gold) takes one or more object files and combines them into a single executable file, library file, or another object file. A linker plays an pivotal role to resolve the locations like re-targeting absolute jumps.

An object file contains a symbol – a primitive datatype to be identified – that references another object file in nature. There are two symbol kinds: local symbols and external symbols. A local symbol resides in the same object file (often for the relocation purpose at linking time). For the latter, if an external symbol is defined inside the object file it can be called from other modules. If undefined, it requires to find the symbol to which references. The referenced symbol can be located in either another object file or a library file, a collection of object files that can be shared by other executables.

A library can be static or dynamic. If an application employs a symbol in a static library (.a extension), the compiler directly merges the object file that contains the symbol with a final executable. When the object file contains another symbol in another object file, it should be resolved and combined at compilation time as well although the object file is not required by the original program. In case of a dynamic library (shared object or .so extension), the final executable does not embed the object file. Instead, it delays the resolution of undefined symbols until runtime and lets a dynamic linker do its job. Hence, a statically linked executable can run itself without a library at runtime whereas a dynamically linked one cannot.

Here we demystify how dynamic linking works with a simple example for the shared object or position independent code (-fPIC option in a gcc and clang) in x86_64 (AKA AMD64).

B. Sample Program

Here is a tiny source code that has two modules (test.c and func.c) and one header file (func.h).

// test.c
#include <stdio.h>
#include "func.h"

void calc(int x, int y) {
        printf("x + y = %d\n", add(x, y));
        printf("x - y = %d\n", sub(x, y));
}

int main(void) {
        int a = 10;
        int b = 7;
        calc(10, 7);
        return 0;
}

// func.c
#include "func.h"

int add(int x, int y) {
        int a = x;
        int b = y;
        return a + b;
}


int sub(int x, int y) {
        int a = x;
        int b = y;
        return a - b;
}

// func.h 
extern int add(int, int);
extern int sub(int, int);
extern void calc(int, int);

// Compile the files above with clang
// $ clang -o main func.c calc.c

// test.c

#include <stdio.h>

#include "func.h"

void calc(int x, int y) {

printf("x + y = %d\n", add(x, y));

printf("x - y = %d\n", sub(x, y));

}

int main(void) {

int a = 10;

int b = 7;

calc(10, 7);

return 0;

}

// func.c

#include "func.h"

int add(int x, int y) {

int a = x;

int b = y;

return a + b;

}

int sub(int x, int y) {

int a = x;

int b = y;

return a - b;

}

// func.h

extern int add(int, int);

extern int sub(int, int);

extern void calc(int, int);

// Compile the files above with clang

// $ clang -o main func.c calc.c

C. ELF Header and Tables for Program Header and Section Header

ELF consists of three parts: ELF Header, Program Header Table and Section Header Table.

First off, ELF header is quite self-explantionary with the defined struct itself in Elf64_Ehdr. See the comments below.

# define ELF_NIDENT      16
 
typedef struct {
        uint8_t         e_ident[ELF_NIDENT]; // ELF Identification: See [a]
        Elf32_Half      e_type;      // Object File Type: See [b]
        Elf32_Half      e_machine;   // Required Architecture
        Elf32_Word      e_version;   // Object Version
        Elf32_Addr      e_entry;     // Entry Point VA*
        Elf32_Off       e_phoff;     // Program Header Offset
        Elf32_Off       e_shoff;     // Section Header Offset
        Elf32_Word      e_flags;     // Processor-Specific Flags
        Elf32_Half      e_ehsize;    // ELF Header Size in Bytes
        Elf32_Half      e_phentsize; // Size of each Program Header Entry
        Elf32_Half      e_phnum;     // Number of Program Headers
        Elf32_Half      e_shentsize; // Size of each Section Header Entry
        Elf32_Half      e_shnum;     // Number of Section Headers
        Elf32_Half      e_shstrndx;  // Section Header String Table Index 
}
Elf64_Ehdr;

// [a] Elf_Ident Enum
enum Elf_Ident {
        EI_MAG0         = 0, // 0x7F
        EI_MAG1         = 1, // 'E'
        EI_MAG2         = 2, // 'L'
        EI_MAG3         = 3, // 'F'
        EI_CLASS        = 4, // Architecture (32/64)
        EI_DATA         = 5, // Byte Order: ELFDATA2LSB=0x1, ELFDATA2MSB=0x2
        EI_VERSION      = 6, // File Version
        EI_OSABI        = 7, // OS Specific
        EI_ABIVERSION   = 8, // OS Specific
        EI_PAD          = 9  // Padding (Unused)
};
 
# define ELFMAG0      0x7F // e_ident[EI_MAG0]
# define ELFMAG1      'E'  // e_ident[EI_MAG1]
# define ELFMAG2      'L'  // e_ident[EI_MAG2]
# define ELFMAG3      'F'  // e_ident[EI_MAG3]
# define ELFDATA2LSB  (1)  // Little Endian
# define ELFCLASS64   (1)  // 64-bit Architecture

// [b] Elf_type structure
enum Elf_Type {
        ET_NONE         = 0, // Unkown Type
        ET_REL          = 1, // Relocatable File
        ET_EXEC         = 2  // Executable File
        ET_DYN          = 3  // Shared Object File
        ET_CORE         = 4  // Core File
        ET_LOPROC       = 0xff00  // Processor-Specific
        ET_HIPROC       = 0xffff  // Processor-Specific 
};

# define ELF_NIDENT 16

typedef struct {

uint8_t e_ident[ELF_NIDENT]; // ELF Identification: See [a]

Elf32_Half e_type; // Object File Type: See [b]

Elf32_Half e_machine; // Required Architecture

Elf32_Word e_version; // Object Version

Elf32_Addr e_entry; // Entry Point VA*

Elf32_Off e_phoff; // Program Header Offset

Elf32_Off e_shoff; // Section Header Offset

Elf32_Word e_flags; // Processor-Specific Flags

Elf32_Half e_ehsize; // ELF Header Size in Bytes

Elf32_Half e_phentsize; // Size of each Program Header Entry

Elf32_Half e_phnum; // Number of Program Headers

Elf32_Half e_shentsize; // Size of each Section Header Entry

Elf32_Half e_shnum; // Number of Section Headers

Elf32_Half e_shstrndx; // Section Header String Table Index

}

Elf64_Ehdr;

// [a] Elf_Ident Enum

enum Elf_Ident {

EI_MAG0 = 0, // 0x7F

EI_MAG1 = 1, // 'E'

EI_MAG2 = 2, // 'L'

EI_MAG3 = 3, // 'F'

EI_CLASS = 4, // Architecture (32/64)

EI_DATA = 5, // Byte Order: ELFDATA2LSB=0x1, ELFDATA2MSB=0x2

EI_VERSION = 6, // File Version

EI_OSABI = 7, // OS Specific

EI_ABIVERSION = 8, // OS Specific

EI_PAD = 9 // Padding (Unused)

};

# define ELFMAG0 0x7F // e_ident[EI_MAG0]

# define ELFMAG1 'E' // e_ident[EI_MAG1]

# define ELFMAG2 'L' // e_ident[EI_MAG2]

# define ELFMAG3 'F' // e_ident[EI_MAG3]

# define ELFDATA2LSB (1) // Little Endian

# define ELFCLASS64 (1) // 64-bit Architecture

// [b] Elf_type structure

enum Elf_Type {

ET_NONE = 0, // Unkown Type

ET_REL = 1, // Relocatable File

ET_EXEC = 2 // Executable File

ET_DYN = 3 // Shared Object File

ET_CORE = 4 // Core File

ET_LOPROC = 0xff00 // Processor-Specific

ET_HIPROC = 0xffff // Processor-Specific

};

The program header table (PHT) describes how a loader maps the binary into virtual address space (or VA) when loading, whereas the section header table (SHT) has the entries of each defined section header when linking. Each mapped region in VA by a PHT entry is often called a segment from a loader view. As is a section by a SHT entry from a linker view.

The final executable file main is shown as following (Figure 1). The linker view on the left shows how each section is stored as a file at offline. The loader view on the right shows how each segment is loaded as a process at runtime. For instance, [S1] is the first section whose size is 0x1c with (A)llocatable by a loader. R, X and W denote readable, executable and writable respectively. On a loader view, there are four major chucks of memory: 0x400000-0x401000 (RX), 0x401000-0x402000 (RW), regions for shared objects and the space for stack and heap.

<Figure 1>
                <Linker View>                               <Loader View>
                
FileOffset                                  VirtualAddr  LOAD (SZ=0x1000)
          .-----------------------------. ------------> .------------------.
   0x0000 |  ELF Header                 |      0x400000 |  ELF_HDR (R)     |
          .-----------------------------.               |------------------|
   0x0040 |  Program Header Table       |      0x400040 |  PHDR (R)        |
          |                             |               |                  |
          |                             |               |                  |
          |                             |               |                  |
          |                             |               |                  |
          .-----------------------------.               |------------------|
   0x0200 | [S1] .interp (0x1c) A       |      0x400200 |  INTERP (R)      |
          .-----------------------------.               |------------------|
   0x021c | [S2] .note.ABI-tag (0x20) A |               |  (RX)            |
          .-----------------------------.               |                  |
   0x0240 | [S3] .dynsym (0xa8) A       |               |                  |
          .-----------------------------.               |                  |
   0x02e8 | [S4] .dynstr (0x89) A       |               |                  |
          .-----------------------------.               |                  |
   0x0378 | [S5] .hash (0x30) A         |               |                  |
          .-----------------------------.               |                  |
   0x03a8 | [S6] .gnu.version (0xe) A   |               |                  |
          .-----------------------------.               |                  |
   0x03b8 | [S7] .gnu.version_r (0x20) A|               |                  |
          .-----------------------------.               |                  |
   0x03d8 | [S8] .rela.dyn (0x18) A     |               |                  |
          .-----------------------------.               |                  |
   0x03f0 | [S9] .rela.plt (0x48) AI    |               |                  |
          .-----------------------------. ------------> |------------------|
   0x0438 | [S10] .init (0x1a) AX       |               |                  |
          .-----------------------------.               |                  |
   0x0460 | [S11] .plt (0x40) AX        |               |                  |
          .-----------------------------.               |                  |
   0x04a0 | [S12] .text (0xa8) AX       |               |                  |
          |                             |               |                  |
          .-----------------------------.               |                  |
   0x06f4 | [S13] .fini (0x9) AX        |               |                  |
          .-----------------------------. ------------> |------------------|
   0x0700 | [S14] .rodata (0x1c) AMS    |               |                  |
          .-----------------------------.               |                  |
   0x0720 | [S15] .eh_frame (0xd4) A    |               |                  |
          .-----------------------------.               |                  |
   0x07f4 | [S16] .eh_frame_hdr (0x2c) A|               |                  |
          .-----------------------------. ------------> .------------------.
   0x0820 | [S17] .dynamic (0x1d) WA    |    |
          |                             |    |          LOAD (SZ=0x1000)
          .-----------------------------.    |          .------------------.
   0x09f0 | [S18] .got (0x8) WA         |    | 0x401000 |                  |
          .-----------------------------.    \--------> .------------------.
   0x09f8 | [S19] .got.plt (0x30) WA    |      0x401820 |  (RW)            |
          .-----------------------------.               |                  |
   0x0a28 | [S20] .data (0x10) WA       |               |                  |
          .-----------------------------.               |                  |
   0x0a38 | [S21] .jcr (0x8) WA         |               |                  |
          .-----------------------------.               |                  |
   0x0a40 | [S22] .tm_clone_tbl (0x0)WA |               |                  |
          .-----------------------------.               |                  |
   0x0a40 | [S23] .fini_array (0x8) WA  |               |                  |
          .-----------------------------.               |                  |
   0x0a48 | [S24] .init_array (0x8) WA  |               |                  |
          .-----------------------------.               |                  |
   0x0a50 | [S25] .bss (0x4) WA         |      0x401a50 |                  |
          .-----------------------------. ---> 0x402000 .------------------.
   0x0a50 | [S26] .comment (0x62) MS    | \
          .-----------------------------. |             .------------------.
   0x0ab4 | [S27] .note.gnu.gold (0x1c) | | 7ffff7a0e000|  Shared Objects  |
          .-----------------------------. |             |  libc, ld, ...   |
   0x0ad0 | [S28] .symtab (0x408)       | |             |                  |
          .-----------------------------. |             .------------------.
   0x0ed8 | [S29] .strtab (0x23f)       | | 
          .-----------------------------. | No mapping in a virtual address
   0x1117 | [S30] .shstrtab (0x118)     | |
          .-----------------------------. |             .------------------.
   0x1230 |  Section Header Table       | |             |  Heap            |
          |   Entry [S0] NULL           | |             |                  |
          |   Entry [S1] .interp        | |             |                  |
          |         .....               | | 7ffffffde000|  Stack           |
          |   Entry [S30] .shstrtab     | |             .------------------.
   0x19f0 .-----------------------------./

FileOffset VirtualAddr LOAD (SZ=0x1000)

.-----------------------------. ------------> .------------------.

0x0000 | ELF Header | 0x400000 | ELF_HDR (R) |

.-----------------------------. |------------------|

0x0040 | Program Header Table | 0x400040 | PHDR (R) |

| | | |

.-----------------------------. |------------------|

0x0200 | [S1] .interp (0x1c) A | 0x400200 | INTERP (R) |

.-----------------------------. |------------------|

0x021c | [S2] .note.ABI-tag (0x20) A | | (RX) |

.-----------------------------. | |

0x0240 | [S3] .dynsym (0xa8) A | | |

.-----------------------------. | |

0x02e8 | [S4] .dynstr (0x89) A | | |

.-----------------------------. | |

0x0378 | [S5] .hash (0x30) A | | |

.-----------------------------. | |

0x03a8 | [S6] .gnu.version (0xe) A | | |

.-----------------------------. | |

0x03b8 | [S7] .gnu.version_r (0x20) A| | |

.-----------------------------. | |

0x03d8 | [S8] .rela.dyn (0x18) A | | |

.-----------------------------. | |

0x03f0 | [S9] .rela.plt (0x48) AI | | |

.-----------------------------. ------------> |------------------|

0x0438 | [S10] .init (0x1a) AX | | |

.-----------------------------. | |

0x0460 | [S11] .plt (0x40) AX | | |

.-----------------------------. | |

0x04a0 | [S12] .text (0xa8) AX | | |

| | | |

.-----------------------------. | |

0x06f4 | [S13] .fini (0x9) AX | | |

.-----------------------------. ------------> |------------------|

0x0700 | [S14] .rodata (0x1c) AMS | | |

.-----------------------------. | |

0x0720 | [S15] .eh_frame (0xd4) A | | |

.-----------------------------. | |

0x07f4 | [S16] .eh_frame_hdr (0x2c) A| | |

.-----------------------------. ------------> .------------------.

0x0820 | [S17] .dynamic (0x1d) WA | |

| | | LOAD (SZ=0x1000)

.-----------------------------. | .------------------.

0x09f0 | [S18] .got (0x8) WA | | 0x401000 | |

.-----------------------------. \--------> .------------------.

0x09f8 | [S19] .got.plt (0x30) WA | 0x401820 | (RW) |

.-----------------------------. | |

0x0a28 | [S20] .data (0x10) WA | | |

.-----------------------------. | |

0x0a38 | [S21] .jcr (0x8) WA | | |

.-----------------------------. | |

0x0a40 | [S22] .tm_clone_tbl (0x0)WA | | |

.-----------------------------. | |

0x0a40 | [S23] .fini_array (0x8) WA | | |

.-----------------------------. | |

0x0a48 | [S24] .init_array (0x8) WA | | |

.-----------------------------. | |

0x0a50 | [S25] .bss (0x4) WA | 0x401a50 | |

.-----------------------------. ---> 0x402000 .------------------.

0x0a50 | [S26] .comment (0x62) MS | \

.-----------------------------. | .------------------.

.-----------------------------. | | libc, ld, ... |

0x0ad0 | [S28] .symtab (0x408) | | | |

.-----------------------------. | .------------------.

0x0ed8 | [S29] .strtab (0x23f) | |

.-----------------------------. | No mapping in a virtual address

0x1117 | [S30] .shstrtab (0x118) | |

.-----------------------------. | .------------------.

| Entry [S30] .shstrtab | | .------------------.

0x19f0 .-----------------------------./

A friendly ‘readelf‘ command illustrates what each segment and section look like by reading the structure. Note that there are a lot of sections appended during compilation.

// Section to Segment mapping:
Segment Sections...
00
01     .interp
02     .interp .note.ABI-tag .dynsym .dynstr .hash .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame .eh_frame_hdr
03     .dynamic .got .got.plt .data .jcr .tm_clone_table .fini_array .init_array .bss
04     .dynamic
05     .note.ABI-tag
06     .eh_frame_hdr
07

// Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .interp           PROGBITS         0000000000400200  00000200
       000000000000001c  0000000000000000   A       0     0     1
  [ 2] .note.ABI-tag     NOTE             000000000040021c  0000021c
       0000000000000020  0000000000000000   A       0     0     4
  [ 3] .dynsym           DYNSYM           0000000000400240  00000240
       00000000000000a8  0000000000000018   A       4     1     8
  [ 4] .dynstr           STRTAB           00000000004002e8  000002e8
       0000000000000089  0000000000000000   A       0     0     1
  [ 5] .hash             HASH             0000000000400378  00000378
       0000000000000030  0000000000000004   A       3     0     8
  [ 6] .gnu.version      VERSYM           00000000004003a8  000003a8
       000000000000000e  0000000000000002   A       3     0     2
  [ 7] .gnu.version_r    VERNEED          00000000004003b8  000003b8
       0000000000000020  0000000000000000   A       4     1     4
  [ 8] .rela.dyn         RELA             00000000004003d8  000003d8
       0000000000000018  0000000000000018   A       3     0     8
  [ 9] .rela.plt         RELA             00000000004003f0  000003f0
       0000000000000048  0000000000000018  AI       3    11     8
  [10] .init             PROGBITS         0000000000400438  00000438
       000000000000001a  0000000000000000  AX       0     0     4
  [11] .plt              PROGBITS         0000000000400460  00000460
       0000000000000040  0000000000000010  AX       0     0     16
  [12] .text             PROGBITS         00000000004004a0  000004a0
       0000000000000252  0000000000000000  AX       0     0     16
  [13] .fini             PROGBITS         00000000004006f4  000006f4
       0000000000000009  0000000000000000  AX       0     0     4
  [14] .rodata           PROGBITS         0000000000400700  00000700
       000000000000001c  0000000000000000 AMS       0     0     4
  [15] .eh_frame         PROGBITS         0000000000400720  00000720
       00000000000000d4  0000000000000000   A       0     0     8
  [16] .eh_frame_hdr     PROGBITS         00000000004007f4  000007f4
       000000000000002c  0000000000000000   A       0     0     4
  [17] .dynamic          DYNAMIC          0000000000401820  00000820
       00000000000001d0  0000000000000010  WA       4     0     8
  [18] .got              PROGBITS         00000000004019f0  000009f0
       0000000000000008  0000000000000000  WA       0     0     8
  [19] .got.plt          PROGBITS         00000000004019f8  000009f8
       0000000000000030  0000000000000000  WA       0     0     8
  [20] .data             PROGBITS         0000000000401a28  00000a28
       0000000000000010  0000000000000000  WA       0     0     8
  [21] .jcr              PROGBITS         0000000000401a38  00000a38
       0000000000000008  0000000000000000  WA       0     0     8
  [22] .tm_clone_table   PROGBITS         0000000000401a40  00000a40
       0000000000000000  0000000000000000  WA       0     0     8
  [23] .fini_array       FINI_ARRAY       0000000000401a40  00000a40
       0000000000000008  0000000000000000  WA       0     0     8
  [24] .init_array       INIT_ARRAY       0000000000401a48  00000a48
       0000000000000008  0000000000000000  WA       0     0     8
  [25] .bss              NOBITS           0000000000401a50  00000a50
       0000000000000004  0000000000000000  WA       0     0     4
  [26] .comment          PROGBITS         0000000000000000  00000a50
       0000000000000062  0000000000000001  MS       0     0     1
  [27] .note.gnu.gold-ve NOTE             0000000000000000  00000ab4
       000000000000001c  0000000000000000           0     0     4
  [28] .symtab           SYMTAB           0000000000000000  00000ad0
       0000000000000408  0000000000000018          29    22     8
  [29] .strtab           STRTAB           0000000000000000  00000ed8
       000000000000023f  0000000000000000           0     0     1
  [30] .shstrtab         STRTAB           0000000000000000  00001117
       0000000000000118  0000000000000000           0     0     1

Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
  I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

// Section to Segment mapping:

Segment Sections...

01 .interp

02 .interp .note.ABI-tag .dynsym .dynstr .hash .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame .eh_frame_hdr

03 .dynamic .got .got.plt .data .jcr .tm_clone_table .fini_array .init_array .bss

04 .dynamic

05 .note.ABI-tag

06 .eh_frame_hdr

// Section Headers:

[Nr] Name Type Address Offset

Size EntSize Flags Link Info Align

[ 0] NULL 0000000000000000 00000000

0000000000000000 0000000000000000 0 0 0

[ 1] .interp PROGBITS 0000000000400200 00000200

000000000000001c 0000000000000000 A 0 0 1

[ 2] .note.ABI-tag NOTE 000000000040021c 0000021c

0000000000000020 0000000000000000 A 0 0 4

[ 3] .dynsym DYNSYM 0000000000400240 00000240

00000000000000a8 0000000000000018 A 4 1 8

[ 4] .dynstr STRTAB 00000000004002e8 000002e8

0000000000000089 0000000000000000 A 0 0 1

[ 5] .hash HASH 0000000000400378 00000378

0000000000000030 0000000000000004 A 3 0 8

[ 6] .gnu.version VERSYM 00000000004003a8 000003a8

000000000000000e 0000000000000002 A 3 0 2

[ 7] .gnu.version_r VERNEED 00000000004003b8 000003b8

0000000000000020 0000000000000000 A 4 1 4

[ 8] .rela.dyn RELA 00000000004003d8 000003d8

0000000000000018 0000000000000018 A 3 0 8

[ 9] .rela.plt RELA 00000000004003f0 000003f0

0000000000000048 0000000000000018 AI 3 11 8

[10] .init PROGBITS 0000000000400438 00000438

000000000000001a 0000000000000000 AX 0 0 4

[11] .plt PROGBITS 0000000000400460 00000460

0000000000000040 0000000000000010 AX 0 0 16

[12] .text PROGBITS 00000000004004a0 000004a0

0000000000000252 0000000000000000 AX 0 0 16

[13] .fini PROGBITS 00000000004006f4 000006f4

0000000000000009 0000000000000000 AX 0 0 4

[14] .rodata PROGBITS 0000000000400700 00000700

000000000000001c 0000000000000000 AMS 0 0 4

[15] .eh_frame PROGBITS 0000000000400720 00000720

00000000000000d4 0000000000000000 A 0 0 8

[16] .eh_frame_hdr PROGBITS 00000000004007f4 000007f4

000000000000002c 0000000000000000 A 0 0 4

[17] .dynamic DYNAMIC 0000000000401820 00000820

00000000000001d0 0000000000000010 WA 4 0 8

[18] .got PROGBITS 00000000004019f0 000009f0

0000000000000008 0000000000000000 WA 0 0 8

[19] .got.plt PROGBITS 00000000004019f8 000009f8

0000000000000030 0000000000000000 WA 0 0 8

[20] .data PROGBITS 0000000000401a28 00000a28

0000000000000010 0000000000000000 WA 0 0 8

[21] .jcr PROGBITS 0000000000401a38 00000a38

0000000000000008 0000000000000000 WA 0 0 8

[22] .tm_clone_table PROGBITS 0000000000401a40 00000a40

0000000000000000 0000000000000000 WA 0 0 8

[23] .fini_array FINI_ARRAY 0000000000401a40 00000a40

0000000000000008 0000000000000000 WA 0 0 8

[24] .init_array INIT_ARRAY 0000000000401a48 00000a48

0000000000000008 0000000000000000 WA 0 0 8

[25] .bss NOBITS 0000000000401a50 00000a50

0000000000000004 0000000000000000 WA 0 0 4

[26] .comment PROGBITS 0000000000000000 00000a50

0000000000000062 0000000000000001 MS 0 0 1

[27] .note.gnu.gold-ve NOTE 0000000000000000 00000ab4

000000000000001c 0000000000000000 0 0 4

[28] .symtab SYMTAB 0000000000000000 00000ad0

0000000000000408 0000000000000018 29 22 8

[29] .strtab STRTAB 0000000000000000 00000ed8

000000000000023f 0000000000000000 0 0 1

[30] .shstrtab STRTAB 0000000000000000 00001117

0000000000000118 0000000000000000 0 0 1

Key to Flags:

W (write), A (alloc), X (execute), M (merge), S (strings), l (large)

I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)

O (extra OS processing required) o (OS specific), p (processor specific)

D. Linking process

Before moving on, here is what the object looks like (in crt1.o). The relocation record shows that there are four locations that cannot be resolved while compilation process. That is why the 4-byte address is empty filled with 0x0s. The first relocation entry at offset 12 has the reference of __libc_csu_fini, defined in another object. We can see that _start function actually calls our main function at offset 0x20, the entry point of the final executable.

0000000000000000 <_start>:
   0:   31 ed                   xor    ebp,ebp
   2:   49 89 d1                mov    r9,rdx
   5:   5e                      pop    rsi
   6:   48 89 e2                mov    rdx,rsp
   9:   48 83 e4 f0             and    rsp,0xfffffffffffffff0
   d:   50                      push   rax
   e:   54                      push   rsp
   f:   49 c7 c0 00 00 00 00    mov    r8,0x0
  16:   48 c7 c1 00 00 00 00    mov    rcx,0x0
  1d:   48 c7 c7 00 00 00 00    mov    rdi,0x0
  24:   e8 00 00 00 00          call   29 <_start+0x29>
  29:   f4                      hlt

0000000000000000 <_start>:

0: 31 ed xor ebp,ebp

2: 49 89 d1 mov r9,rdx

5: 5e pop rsi

6: 48 89 e2 mov rdx,rsp

9: 48 83 e4 f0 and rsp,0xfffffffffffffff0

d: 50 push rax

e: 54 push rsp

f: 49 c7 c0 00 00 00 00 mov r8,0x0

16: 48 c7 c1 00 00 00 00 mov rcx,0x0

1d: 48 c7 c7 00 00 00 00 mov rdi,0x0

24: e8 00 00 00 00 call 29 <_start+0x29>

29: f4 hlt

RELOCATION RECORDS FOR [.text]:
OFFSET           TYPE              VALUE
0000000000000012 R_X86_64_32S      __libc_csu_fini
0000000000000019 R_X86_64_32S      __libc_csu_init
0000000000000020 R_X86_64_32S      main
0000000000000025 R_X86_64_PC32     __libc_start_main-0x0000000000000004

RELOCATION RECORDS FOR [.text]:

OFFSET TYPE VALUE

0000000000000012 R_X86_64_32S __libc_csu_fini

0000000000000019 R_X86_64_32S __libc_csu_init

0000000000000020 R_X86_64_32S main

0000000000000025 R_X86_64_PC32 __libc_start_main-0x0000000000000004

Now, when the given program is compiled by default, gcc or clang driver combines necessary files (i.e., CRT) to allow a loader to handle it properly. The Figure 2 illustrates them. (*) means the function is defined in the object file, whereas others are declared outside of the file. (i.e., using extern keyword in C) For example, crt1.o defines _start in the file but the function __libc_start_main has to be resolved in a linking time (or maybe later).

<Figure 2>
.-------------------.-----------------------.---------------------------.----------.
| crt1.o            | cri.o                 | crtbegin.o                | crtn.o   |
.-------------------|-----------------------|---------------------------.----------.
| _start*           | __init*               | deregister_tm_clones *    | .init    |
| __libc_csu_fini   | __fini*               | register_tm_clones *      | .fini    |
| __libc_csu_init   | __gmon_start          | __do_global_dtors_aux *   |          |
| __libc_start_main | _GLOBAL_OFFSET_TABLE_ | frame_dummy *             |          |
|                   |                       | __TMC_END__               |          |
|                   |                       | _ITM_deregisterTMCloneTab |          |
|                   |                       | _ITM_registerTMCloneTable |          |
|                   |                       | _Jv_RegisterClasses       |          |
.-------------------------------------------.---------------------------.----------.
| libc.a                         | crtend.o | func.o                    | test.o   |
.--------------------------------.----------.---------------------------.----------.
| __libc_csu_fini (elf-init.o) * | NONE     | add *                     | calc *   |
| __libc_csu_init (elf-init.o) * |          | sub *                     | main *   |
| printf (printf.o) *            |          |                           | add      |
|                                |          |                           | printf   |
|                                |          |                           | sub      |
.--------------------------------.----------.---------------------------.----------.

.-------------------.-----------------------.---------------------------.----------.

.-------------------|-----------------------|---------------------------.----------.

.-------------------------------------------.---------------------------.----------.

.--------------------------------.----------.---------------------------.----------.

| printf (printf.o) * | | | add |

| | | | sub |

.--------------------------------.----------.---------------------------.----------.

Let’s see the layout of all functions from each object file. Figure 3 is part of sections from 10 to 13 in Figure 1. Interesting enough, the layout of all functions in a single object file is not inter-mixed.

<Figure 3>
         |          ...           |
         |========================|
   .init | _init_proc             |  __init (cri.o) + .init (crtn.o)
         |========================|
   .plt  | __libc_start_main      | \ The addresses will be resolved 
         | __gmon_start__         | | at runtime by dynamic linker,
         | _printf                | | the .plt entry is fixed up 
         |========================| / when each function is invoked.
   .text | _start                 |   crt1.o
         |------------------------| \
         | deregister_tm_clones   | |
         | register_tm_clones     | | crtbegin.o
         | __do_global_dtors_aux  | |
         | frame_dummy            | |
         |------------------------| /
         | calc                   | \
         | main                   | | test.o
         |------------------------| /
         | add                    | \
         | sub                    | | func.o
         |------------------------| /
         | __libc_csu_init        | \
         | __libc_csu_fini        | | libc.a (included in a binary)
         |========================| /
    .fini| _term_proc             | __fini (cri.o) + .fini (crtn.o)
         |========================|
         |          ...           |

| ... |

|========================|

.init | _init_proc | __init (cri.o) + .init (crtn.o)

|========================|

.plt | __libc_start_main | \ The addresses will be resolved

| __gmon_start__ | | at runtime by dynamic linker,

| _printf | | the .plt entry is fixed up

|========================| / when each function is invoked.

.text | _start | crt1.o

|------------------------| \

| deregister_tm_clones | |

| register_tm_clones | | crtbegin.o

| __do_global_dtors_aux | |

| frame_dummy | |

|------------------------| /

| calc | \

| main | | test.o

|------------------------| /

| add | \

| sub | | func.o

|------------------------| /

| __libc_csu_init | \

| __libc_csu_fini | | libc.a (included in a binary)

|========================| /

.fini| _term_proc | __fini (cri.o) + .fini (crtn.o)

|========================|

| ... |

E. Dynamic Linking

When dynamic linking is required (modern compiler set it by default), a compiler generates .dynamic section (section index 17 above). Note that executable files and shared object files have a separate procedure linkage table (PLT).

With the sample we have, the dynamic section contains 24 entries as following. Pay attention to the highlighted, which are required by dynamic linker. The section .plt.got (PLTGOT) is the very place that the final fix-ups are stored by dynamic linker. The .rela.plt (JMPREL) and .rela.dyn (RELA) are the relocation section tables that describe relocation entries.

Dynamic section at offset 0x820 contains 24 entries:
  Tag        Type                    Name/Value
 0x0000000000000003 (PLTGOT)        0x4019f8      // address of .plt.got section
 0x0000000000000002 (PLTRELSZ)      72 (bytes)    // size of .plt.got section
 0x0000000000000017 (JMPREL)        0x4003f0      // address of .rela.plt section
 0x0000000000000014 (PLTREL)        RELA          // relocation type (REL / RELA)
 0x0000000000000007 (RELA)          0x4003d8      // address of .rela.dyn section
 0x0000000000000008 (RELASZ)        24 (bytes)    // total size of .rela.dyn section
 0x0000000000000009 (RELAENT)       24 (bytes)    // size of an entry in .rela.dyn section
 0x0000000000000015 (DEBUG)         0x0
 0x0000000000000006 (SYMTAB)        0x400240      // address of .symtab section
 0x000000000000000b (SYMENT)        24 (bytes)    // size of an entry in .symtab section
 0x0000000000000005 (STRTAB)        0x4002e8      // address of .strtab section
 0x000000000000000a (STRSZ)         137 (bytes)   // size of .strtab section
 0x0000000000000004 (HASH)          0x400378      // address of hash section
 0x0000000000000001 (NEEDED)        Shared library: [libc.so.6] 
 0x000000000000000c (INIT)          0x400438      // address of .init section
 0x000000000000000d (FINI)          0x4006f4      // address of .fini section
 0x000000000000001a (FINI_ARRAY)    0x401a40      // address of .fini_array section
 0x000000000000001c (FINI_ARRAYSZ)  8 (bytes)     // size of ._fini_array
 0x0000000000000019 (INIT_ARRAY)    0x401a48      // address of .init_array section
 0x000000000000001b (INIT_ARRAYSZ)  8 (bytes)     // size of ._fini_array
 0x000000006ffffff0 (VERSYM)        0x4003a8      // address of .gnu.version section
 0x000000006ffffffe (VERNEED)       0x4003b8      // address of .gnu.version_r section
 0x000000006fffffff (VERNEEDNUM)    1             // number of needed versions
 0x0000000000000000 (NULL)          0x0           // end of .dynamic section

Dynamic section at offset 0x820 contains 24 entries:

Tag Type Name/Value

0x0000000000000003 (PLTGOT) 0x4019f8 // address of .plt.got section

0x0000000000000002 (PLTRELSZ) 72 (bytes) // size of .plt.got section

0x0000000000000017 (JMPREL) 0x4003f0 // address of .rela.plt section

0x0000000000000014 (PLTREL) RELA // relocation type (REL / RELA)

0x0000000000000007 (RELA) 0x4003d8 // address of .rela.dyn section

0x0000000000000008 (RELASZ) 24 (bytes) // total size of .rela.dyn section

0x0000000000000009 (RELAENT) 24 (bytes) // size of an entry in .rela.dyn section

0x0000000000000015 (DEBUG) 0x0

0x0000000000000006 (SYMTAB) 0x400240 // address of .symtab section

0x000000000000000b (SYMENT) 24 (bytes) // size of an entry in .symtab section

0x0000000000000005 (STRTAB) 0x4002e8 // address of .strtab section

0x000000000000000a (STRSZ) 137 (bytes) // size of .strtab section

0x0000000000000004 (HASH) 0x400378 // address of hash section

0x0000000000000001 (NEEDED) Shared library: [libc.so.6]

0x000000000000000c (INIT) 0x400438 // address of .init section

0x000000000000000d (FINI) 0x4006f4 // address of .fini section

0x000000000000001a (FINI_ARRAY) 0x401a40 // address of .fini_array section

0x000000000000001c (FINI_ARRAYSZ) 8 (bytes) // size of ._fini_array

0x0000000000000019 (INIT_ARRAY) 0x401a48 // address of .init_array section

0x000000000000001b (INIT_ARRAYSZ) 8 (bytes) // size of ._fini_array

0x000000006ffffff0 (VERSYM) 0x4003a8 // address of .gnu.version section

0x000000006ffffffe (VERNEED) 0x4003b8 // address of .gnu.version_r section

0x000000006fffffff (VERNEEDNUM) 1 // number of needed versions

0x0000000000000000 (NULL) 0x0 // end of .dynamic section

Here are the symbols in relocation sections. The type “R_X86_64_JUMP_SLOT” means these symbols need to be resolved at runtime by dynamic linker. The offset is the location that resolved reference has to be stored.

Relocation section '.rela.dyn' at offset 0x3d8 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000004019f0  000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x3f0 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000401a10  000200000007 R_X86_64_JUMP_SLO 0000000000000000 __libc_start_main@GLIBC_2.2.5 + 0
000000401a18  000300000007 R_X86_64_JUMP_SLO 0000000000000000 __gmon_start__ + 0
000000401a20  000100000007 R_X86_64_JUMP_SLO 0000000000000000 printf@GLIBC_2.2.5 + 0

Relocation section '.rela.dyn' at offset 0x3d8 contains 1 entries:

Offset Info Type Sym. Value Sym. Name + Addend

0000004019f0 000300000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0

Relocation section '.rela.plt' at offset 0x3f0 contains 3 entries:

Offset Info Type Sym. Value Sym. Name + Addend

000000401a10 000200000007 R_X86_64_JUMP_SLO 0000000000000000 __libc_start_main@GLIBC_2.2.5 + 0

000000401a18 000300000007 R_X86_64_JUMP_SLO 0000000000000000 __gmon_start__ + 0

000000401a20 000100000007 R_X86_64_JUMP_SLO 0000000000000000 printf@GLIBC_2.2.5 + 0

The Figure 4 (before resolution) and 5 (after resolution) illustrate how dynamic linker resolves the references on the fly. With disassembly, three symbols are called at 0x400448, 0x4004c4 and 0x4005c7. At first, they are supposed to jump to somewhere in PLT. Again, another jump instruction in PLT corresponds to somewhere in a .got.plt. The value in .got.plt has the address of next instruction in .plt that has pointed to itself (+6).

For example, the address of printf@plt is 0x400490, and it jumps to 0x400496 after dereferencing (rip+0x158a is 0x401a20, and 0x400496 is stored in there). Then it pushes 0x2 and jumps to .plt table.

<Figure 4>

      .text                           .plt <--------------------------------------\
      .-------------------.           .---------------------------.               |
4004a0|                   |     400460|   pushq [rip+0x159a]      | <_GOT+8>      |
      |                   |           |---------------------------|               |
      |                   |           |   jmpq  [rip+0x159c]      | <_GOT+16>     |
      |-------------------|           |---------------------------|               |
400448| call <__gmon_     |     400470|   jmpq  [rip+0x159a]      | <__libc_..@ --|--\
      |  start__@plt>     | /-------> |   push  0x0               |  got.plt>     |  |
      |-------------------| |         |   jmp   0x400460          | --------------|  |
      |-------------------| |         |---------------------------|               |  |
4004c4| call <__libc_     | |   400480|   jmpq  [rip+0x1592]      | <__gmon_..@ --|--|--\
      |  start_main@plt>  | | /-----> |   push  0x1               |  got.plt      |  |  |
      |-------------------| | |       |   jmp   0x400460          |---------------|  |  |
      |-------------------| | |       |---------------------------|               |  |  |
4005c7| call <printf@plt> | | | 400490|   jmpq  [rip+0x158a]      | <printf@ -----|--------\
      |-------------------| | | /---> |   push  0x2               |  got.plt>     |  |  |  |
      |                   | | | |     |   jmp   0x400460          | -------------/   |  |  |
      |                   | | | |     .---------------------------.                  |  |  |
      |                   | | | |       <Procedure Linkage Table>                    |  |  |
      |                   | | | |                                                    |  |  |
      |                   | | | |     .got                                           |  |  |
      .-------------------. | | |     .---------------------------.                  |  |  |
                            | | |     |            0x0            |4019f0            |  |  |
      .got.plt sec_hdr      | | |     |---------------------------|                  |  |  |
      .-------------------. | | |     |   (.dynamic) 0x0401820    |4019f8            |  |  |
      | PLTGOT            | | | |     .---------------------------.                  |  |  |
      |                   | | | |          <Global Offset Table>                     |  |  |
      |                   | | | |                                                    |  |  |
      |                   | | | |                                                    |  |  |
      |                   | | | |     .got.plt                                       |  |  |
      |                   | | | |     .---------------------------.                  |  |  |
      .-------------------. | | |     |           0x0             |401a00            |  |  |
                            | | |     |           0x0             |401a08            |  |  |
      .rela.plt sec_hdr     \-|-|---- |         0x400476          |401a10 <----------/  |  |
      .-------------------.   \-|---- |         0x400486          |401a18 <-------------/  |
      | JMPREL            |     \---- |         0x400496          |401a20 <----------------/
      |   r_offset        |           .---------------------------.
      |   r_info          | <-- R_X86_64_JUMP_SLO type
      |   r_addend        |           
      .-------------------.

.text .plt <--------------------------------------\

.-------------------. .---------------------------. |

4004a0| | 400460| pushq [rip+0x159a] | <_GOT+8> |

| | |---------------------------| |

| | | jmpq [rip+0x159c] | <_GOT+16> |

|-------------------| |---------------------------| |

400448| call <__gmon_ | 400470| jmpq [rip+0x159a] | <__libc_..@ --|--\

|-------------------| | | jmp 0x400460 | --------------| |

|-------------------| | |---------------------------| | |

4004c4| call <__libc_ | | 400480| jmpq [rip+0x1592] | <__gmon_..@ --|--|--\

|-------------------| | | | jmp 0x400460 |---------------| | |

|-------------------| | | |---------------------------| | | |

4005c7| call <printf@plt> | | | 400490| jmpq [rip+0x158a] | <printf@ -----|--------\

|-------------------| | | /---> | push 0x2 | got.plt> | | | |

| | | | | | jmp 0x400460 | -------------/ | | |

| | | | | .---------------------------. | | |

| | | | | | | |

| | | | | .got | | |

.-------------------. | | | .---------------------------. | | |

| | | | 0x0 |4019f0 | | |

.got.plt sec_hdr | | | |---------------------------| | | |

.-------------------. | | | | (.dynamic) 0x0401820 |4019f8 | | |

| PLTGOT | | | | .---------------------------. | | |

| | | | | | | |

| | | | | .---------------------------. | | |

.-------------------. | | | | 0x0 |401a00 | | |

| | | | 0x0 |401a08 | | |

.rela.plt sec_hdr \-|-|---- | 0x400476 |401a10 <----------/ | |

.-------------------. \-|---- | 0x400486 |401a18 <-------------/ |

| JMPREL | \---- | 0x400496 |401a20 <----------------/

| r_offset | .---------------------------.

| r_info | <-- R_X86_64_JUMP_SLO type

| r_addend |

.-------------------.

Here is the snapshot after the resolution of __libc_start_main and printf at glibc. The code for __gmon_start__ is already in the final executable. (thus 0x400486). At this point, all references are successfully resolved by dynamic linker. Note that the reference is resolved only once when it is called for the first time.

The address of the routine for the resolution is stored at 0x401a08, which is dl_runtime_resolve_avx in this example.

<Figure 5>
      .text                               .plt <--------------------------------------\
      .-------------------.               .---------------------------.               |
4004a0|                   |         400460|   pushq [rip+0x159a]      | <_GOT+8>      |
      |                   |               |---------------------------|               |
      |                   |               |   jmpq  [rip+0x159c]      | <_GOT+16>     |
      |-------------------|               |---------------------------|               |
400448| call <__gmon_     |  /----> 400470|   jmpq  [rip+0x159a]      | <__libc_..@ --|--\
      |  start__@plt>     |--|\           |   push  0x0               |  got.plt>     |  |
      |-------------------|  ||           |   jmp   0x400460          | --------------|  |
      |-------------------|  ||           |---------------------------|               |  |
4004c4| call <__libc_     | -/\---> 400480|   jmpq  [rip+0x1592]      | <__gmon_..@ --|--|--\
      |  start_main@plt>  |               |   push  0x1               |  got.plt      |  |  |
      |-------------------|               |   jmp   0x400460          |---------------|  |  |
      |-------------------|               |---------------------------|               |  |  |
4005c7| call <printf@plt> | ------> 400490|   jmpq  [rip+0x158a]      | <printf@ -----|--------\
      |-------------------|               |   push  0x2               |  got.plt>     |  |  |  |
      |                   |               |   jmp   0x400460          | -------------/   |  |  |
      |                   |               .---------------------------.                  |  |  |
      |                   |                 <Procedure Linkage Table>                    |  |  |
      |                   |                                                              |  |  |
      |                   |               .got                                           |  |  |
      .-------------------.               .---------------------------.                  |  |  |
                                          |            0x0            |4019f0            |  |  |
      .got.plt sec_hdr                    |---------------------------|                  |  |  |
      .-------------------.               |    (.dynamic) 0x0401820   |4019f8            |  |  |
      | PLTGOT            |               .---------------------------.                  |  |  |
      |                   |                    <Global Offset Table>                     |  |  |
      |                   |                                                              |  |  |
      |                   |           /-- [dl_runtime_resolve_avx] <------------\        |  |  |
      |                   |           |   .got.plt                              |        |  |  |
      |                   |           \-> .---------------------------.         |        |  |  |
      .-------------------.               |    0x00007ffff7ffe168     |401a00   |        |  |  |
                                          |    0x00007ffff7dee6a0     |401a08 --/        |  |  |
      .rela.plt sec_hdr                   |    0x00007ffff7a2e740     |401a10 <----------/  |  |
      .-------------------.               |    0x0000000000400486     |401a18 <-------------/  |
      | JMPREL            |               |    0x00007ffff7a637b0     |401a20 <----------------/
      |   r_offset        |               .---------------------------.
      |   r_info          |     <-- R_X86_64_JUMP_SLO type
      |   r_addend        |               
      .-------------------.

.text .plt <--------------------------------------\

.-------------------. .---------------------------. |

4004a0| | 400460| pushq [rip+0x159a] | <_GOT+8> |

| | |---------------------------| |

| | | jmpq [rip+0x159c] | <_GOT+16> |

|-------------------| |---------------------------| |

400448| call <__gmon_ | /----> 400470| jmpq [rip+0x159a] | <__libc_..@ --|--\

|-------------------| || | jmp 0x400460 | --------------| |

|-------------------| || |---------------------------| | |

4004c4| call <__libc_ | -/\---> 400480| jmpq [rip+0x1592] | <__gmon_..@ --|--|--\

|-------------------| | jmp 0x400460 |---------------| | |

|-------------------| |---------------------------| | | |

4005c7| call <printf@plt> | ------> 400490| jmpq [rip+0x158a] | <printf@ -----|--------\

|-------------------| | push 0x2 | got.plt> | | | |

| | | jmp 0x400460 | -------------/ | | |

| | .---------------------------. | | |

| | | | |

| | .got | | |

.-------------------. .---------------------------. | | |

| 0x0 |4019f0 | | |

.got.plt sec_hdr |---------------------------| | | |

.-------------------. | (.dynamic) 0x0401820 |4019f8 | | |

| PLTGOT | .---------------------------. | | |

| | | | |

| | /-- [dl_runtime_resolve_avx] <------------\ | | |

| | \-> .---------------------------. | | | |

.-------------------. | 0x00007ffff7ffe168 |401a00 | | | |

| 0x00007ffff7dee6a0 |401a08 --/ | | |

.rela.plt sec_hdr | 0x00007ffff7a2e740 |401a10 <----------/ | |

.-------------------. | 0x0000000000400486 |401a18 <-------------/ |

| JMPREL | | 0x00007ffff7a637b0 |401a20 <----------------/

| r_offset | .---------------------------.

| r_info | <-- R_X86_64_JUMP_SLO type

| r_addend |

.-------------------.

For more curious readers, here are the source files from glibc that defines __dl_runtime_resolve and _dl_fixup internally. With several breakpoints in debugging, the routine stores the link_map at %rdi register and the reloc_index at %rsi register. This index is the very one pushed in .plt section.

#define __dl_runtime_resolve at sysdeps/i386/dl-trampoline.S in the glibc-2.24
In sysdeps/x86_64/dl-trampoline.h, line 64
        # Copy args pushed by PLT in register.
        # %rdi: link_map, %rsi: reloc_index
        mov (LOCAL_STORAGE_AREA + 8)(%BASE), %RSI_LP (a)
        mov LOCAL_STORAGE_AREA(%BASE), %RDI_LP (b)
        call _dl_fixup          # Call resolver. (c)
        mov %RAX_LP, %R11_LP    # Save return value (d)
        
In sysdeps/elf/dl-runtime.c, line 66
    DL_FIXUP_VALUE_TYPE _dl_fixup (
           # ifdef ELF_MACHINE_RUNTIME_FIXUP_ARGS 
           ELF_MACHINE_RUNTIME_FIXUP_ARGS, # endif
           struct link_map *l, ElfW(Word) reloc_arg)

#define __dl_runtime_resolve at sysdeps/i386/dl-trampoline.S in the glibc-2.24

In sysdeps/x86_64/dl-trampoline.h, line 64

# Copy args pushed by PLT in register.

# %rdi: link_map, %rsi: reloc_index

mov (LOCAL_STORAGE_AREA + 8)(%BASE), %RSI_LP (a)

mov LOCAL_STORAGE_AREA(%BASE), %RDI_LP (b)

call _dl_fixup # Call resolver. (c)

mov %RAX_LP, %R11_LP # Save return value (d)

In sysdeps/elf/dl-runtime.c, line 66

DL_FIXUP_VALUE_TYPE _dl_fixup (

# ifdef ELF_MACHINE_RUNTIME_FIXUP_ARGS

ELF_MACHINE_RUNTIME_FIXUP_ARGS, # endif

struct link_map *l, ElfW(Word) reloc_arg)

gdb-peda$ x/30i 0x00007ffff7dee6a0
   0x7ffff7dee6a0 <_dl_runtime_resolve_avx>:    push   rbx
   0x7ffff7dee6a1 <_dl_runtime_resolve_avx+1>:  mov    rbx,rsp
   0x7ffff7dee6a4 <_dl_runtime_resolve_avx+4>:  and    rsp,0xffffffffffffffe0
   0x7ffff7dee6a8 <_dl_runtime_resolve_avx+8>:  sub    rsp,0x180
   0x7ffff7dee6af <_dl_runtime_resolve_avx+15>: mov    QWORD PTR [rsp+0x140],rax
   0x7ffff7dee6b7 <_dl_runtime_resolve_avx+23>: mov    QWORD PTR [rsp+0x148],rcx
   0x7ffff7dee6bf <_dl_runtime_resolve_avx+31>: mov    QWORD PTR [rsp+0x150],rdx
   0x7ffff7dee6c7 <_dl_runtime_resolve_avx+39>: mov    QWORD PTR [rsp+0x158],rsi
   0x7ffff7dee6cf <_dl_runtime_resolve_avx+47>: mov    QWORD PTR [rsp+0x160],rdi
   0x7ffff7dee6d7 <_dl_runtime_resolve_avx+55>: mov    QWORD PTR [rsp+0x168],r8
   0x7ffff7dee6df <_dl_runtime_resolve_avx+63>: mov    QWORD PTR [rsp+0x170],r9
   0x7ffff7dee6e7 <_dl_runtime_resolve_avx+71>: vmovdqa YMMWORD PTR [rsp],ymm0
   0x7ffff7dee6ec <_dl_runtime_resolve_avx+76>: vmovdqa YMMWORD PTR [rsp+0x20],ymm1
   0x7ffff7dee6f2 <_dl_runtime_resolve_avx+82>: vmovdqa YMMWORD PTR [rsp+0x40],ymm2
   0x7ffff7dee6f8 <_dl_runtime_resolve_avx+88>: vmovdqa YMMWORD PTR [rsp+0x60],ymm3
   0x7ffff7dee6fe <_dl_runtime_resolve_avx+94>: vmovdqa YMMWORD PTR [rsp+0x80],ymm4
   0x7ffff7dee707 <_dl_runtime_resolve_avx+103>:        vmovdqa YMMWORD PTR [rsp+0xa0],ymm5
   0x7ffff7dee710 <_dl_runtime_resolve_avx+112>:        vmovdqa YMMWORD PTR [rsp+0xc0],ymm6
   0x7ffff7dee719 <_dl_runtime_resolve_avx+121>:        vmovdqa YMMWORD PTR [rsp+0xe0],ymm7
   0x7ffff7dee722 <_dl_runtime_resolve_avx+130>:        bndmov [rsp+0x100],bnd0
   0x7ffff7dee72b <_dl_runtime_resolve_avx+139>:        bndmov [rsp+0x110],bnd1
   0x7ffff7dee734 <_dl_runtime_resolve_avx+148>:        bndmov [rsp+0x120],bnd2
   0x7ffff7dee73d <_dl_runtime_resolve_avx+157>:        bndmov [rsp+0x130],bnd3
   0x7ffff7dee746 <_dl_runtime_resolve_avx+166>:        mov    rsi,QWORD PTR [rbx+0x10]
   0x7ffff7dee74a <_dl_runtime_resolve_avx+170>:        mov    rdi,QWORD PTR [rbx+0x8]
   0x7ffff7dee74e <_dl_runtime_resolve_avx+174>:        call   0x7ffff7de6820 <_dl_fixup>
   0x7ffff7dee753 <_dl_runtime_resolve_avx+179>:        mov    r11,rax
   0x7ffff7dee756 <_dl_runtime_resolve_avx+182>:        bndmov bnd3,[rsp+0x130]
   0x7ffff7dee75f <_dl_runtime_resolve_avx+191>:        bndmov bnd2,[rsp+0x120]
   0x7ffff7dee768 <_dl_runtime_resolve_avx+200>:        bndmov bnd1,[rsp+0x110]

gdb-peda$ x/30i 0x7ffff7de6820
   0x7ffff7de6820 <_dl_fixup>:  push   rbx
   0x7ffff7de6821 <_dl_fixup+1>:        mov    r10,rdi
   0x7ffff7de6824 <_dl_fixup+4>:        mov    esi,esi
   0x7ffff7de6826 <_dl_fixup+6>:        lea    rdx,[rsi+rsi*2]
   0x7ffff7de682a <_dl_fixup+10>:       sub    rsp,0x10
   0x7ffff7de682e <_dl_fixup+14>:       mov    rax,QWORD PTR [rdi+0x68] // 69 l_info[DT_STRTAB]
   0x7ffff7de6832 <_dl_fixup+18>:       mov    rdi,QWORD PTR [rax+0x8]  // 69 [rdi = strtab] const char *strtab = (const void *) D_PTR (l, l_info[DT_STRTAB]);
   0x7ffff7de6836 <_dl_fixup+22>:       mov    rax,QWORD PTR [r10+0xf8] // 72 (D_PTR (l, l_info[DT_JMPREL])
   0x7ffff7de683d <_dl_fixup+29>:       mov    rax,QWORD PTR [rax+0x8]  // 72 const PLTREL *const reloc = (const void *) (D_PTR (l, l_info[DT_JMPREL]) + reloc_offset)
   0x7ffff7de6841 <_dl_fixup+33>:       lea    r8,[rax+rdx*8] 
   0x7ffff7de6845 <_dl_fixup+37>:       mov    rax,QWORD PTR [r10+0x70] // 68 (const void *) D_PTR (l, l_info[DT_SYMTAB])
   0x7ffff7de6849 <_dl_fixup+41>:       mov    rcx,QWORD PTR [r8+0x8]   // 73 reloc->r_info
   0x7ffff7de684d <_dl_fixup+45>:       mov    rax,QWORD PTR [rax+0x8]  // 73 const &symtab[ELFW(R_SYM) (reloc->r_info)]
   0x7ffff7de6851 <_dl_fixup+49>:       mov    rdx,rcx  // 73 const ElfW(Sym) *sym = &symtab[ELFW(R_SYM) (reloc->r_info)]
   0x7ffff7de6854 <_dl_fixup+52>:       shr    rdx,0x20 // 73
   0x7ffff7de6858 <_dl_fixup+56>:       lea    rsi,[rdx+rdx*2] // 73
   0x7ffff7de685c <_dl_fixup+60>:       lea    rsi,[rax+rsi*8] // 73 [rsi = sym, 0x400270]
   0x7ffff7de6860 <_dl_fixup+64>:       mov    rax,QWORD PTR [r10] // 74 
   0x7ffff7de6863 <_dl_fixup+67>:       mov    QWORD PTR [rsp+0x8],rsi // 73
   0x7ffff7de6868 <_dl_fixup+72>:       mov    rbx,rax // 74
   0x7ffff7de686b <_dl_fixup+75>:       add    rbx,QWORD PTR [r8] // 74 void *const rel_addr = (void *)(l->l_addr + reloc->r_offset);
   0x7ffff7de686e <_dl_fixup+78>:       cmp    ecx,0x7 // 79
   0x7ffff7de6871 <_dl_fixup+81>:       jne    0x7ffff7de69c7 <_dl_fixup+423> // 79
   0x7ffff7de6877 <_dl_fixup+87>:       test   BYTE PTR [rsi+0x5],0x3
   0x7ffff7de687b <_dl_fixup+91>:       jne    0x7ffff7de6919 <_dl_fixup+249>
   0x7ffff7de6881 <_dl_fixup+97>:       mov    rax,QWORD PTR [r10+0x1c8]

gdb-peda$ x/30i 0x00007ffff7dee6a0

0x7ffff7dee6a0 <_dl_runtime_resolve_avx>: push rbx

0x7ffff7dee6a1 <_dl_runtime_resolve_avx+1>: mov rbx,rsp

0x7ffff7dee6a4 <_dl_runtime_resolve_avx+4>: and rsp,0xffffffffffffffe0

0x7ffff7dee6a8 <_dl_runtime_resolve_avx+8>: sub rsp,0x180

0x7ffff7dee6af <_dl_runtime_resolve_avx+15>: mov QWORD PTR [rsp+0x140],rax

0x7ffff7dee6b7 <_dl_runtime_resolve_avx+23>: mov QWORD PTR [rsp+0x148],rcx

0x7ffff7dee6bf <_dl_runtime_resolve_avx+31>: mov QWORD PTR [rsp+0x150],rdx

0x7ffff7dee6c7 <_dl_runtime_resolve_avx+39>: mov QWORD PTR [rsp+0x158],rsi

0x7ffff7dee6cf <_dl_runtime_resolve_avx+47>: mov QWORD PTR [rsp+0x160],rdi

0x7ffff7dee6d7 <_dl_runtime_resolve_avx+55>: mov QWORD PTR [rsp+0x168],r8

0x7ffff7dee6df <_dl_runtime_resolve_avx+63>: mov QWORD PTR [rsp+0x170],r9

0x7ffff7dee6e7 <_dl_runtime_resolve_avx+71>: vmovdqa YMMWORD PTR [rsp],ymm0

0x7ffff7dee6ec <_dl_runtime_resolve_avx+76>: vmovdqa YMMWORD PTR [rsp+0x20],ymm1

0x7ffff7dee6f2 <_dl_runtime_resolve_avx+82>: vmovdqa YMMWORD PTR [rsp+0x40],ymm2

0x7ffff7dee6f8 <_dl_runtime_resolve_avx+88>: vmovdqa YMMWORD PTR [rsp+0x60],ymm3

0x7ffff7dee6fe <_dl_runtime_resolve_avx+94>: vmovdqa YMMWORD PTR [rsp+0x80],ymm4

0x7ffff7dee707 <_dl_runtime_resolve_avx+103>: vmovdqa YMMWORD PTR [rsp+0xa0],ymm5

0x7ffff7dee710 <_dl_runtime_resolve_avx+112>: vmovdqa YMMWORD PTR [rsp+0xc0],ymm6

0x7ffff7dee719 <_dl_runtime_resolve_avx+121>: vmovdqa YMMWORD PTR [rsp+0xe0],ymm7

0x7ffff7dee722 <_dl_runtime_resolve_avx+130>: bndmov [rsp+0x100],bnd0

0x7ffff7dee72b <_dl_runtime_resolve_avx+139>: bndmov [rsp+0x110],bnd1

0x7ffff7dee734 <_dl_runtime_resolve_avx+148>: bndmov [rsp+0x120],bnd2

0x7ffff7dee73d <_dl_runtime_resolve_avx+157>: bndmov [rsp+0x130],bnd3

0x7ffff7dee746 <_dl_runtime_resolve_avx+166>: mov rsi,QWORD PTR [rbx+0x10]

0x7ffff7dee74a <_dl_runtime_resolve_avx+170>: mov rdi,QWORD PTR [rbx+0x8]

0x7ffff7dee74e <_dl_runtime_resolve_avx+174>: call 0x7ffff7de6820 <_dl_fixup>

0x7ffff7dee753 <_dl_runtime_resolve_avx+179>: mov r11,rax

0x7ffff7dee756 <_dl_runtime_resolve_avx+182>: bndmov bnd3,[rsp+0x130]

0x7ffff7dee75f <_dl_runtime_resolve_avx+191>: bndmov bnd2,[rsp+0x120]

0x7ffff7dee768 <_dl_runtime_resolve_avx+200>: bndmov bnd1,[rsp+0x110]

gdb-peda$ x/30i 0x7ffff7de6820

0x7ffff7de6820 <_dl_fixup>: push rbx

0x7ffff7de6821 <_dl_fixup+1>: mov r10,rdi

0x7ffff7de6824 <_dl_fixup+4>: mov esi,esi

0x7ffff7de6826 <_dl_fixup+6>: lea rdx,[rsi+rsi*2]

0x7ffff7de682a <_dl_fixup+10>: sub rsp,0x10

0x7ffff7de682e <_dl_fixup+14>: mov rax,QWORD PTR [rdi+0x68] // 69 l_info[DT_STRTAB]

0x7ffff7de6832 <_dl_fixup+18>: mov rdi,QWORD PTR [rax+0x8] // 69 [rdi = strtab] const char *strtab = (const void *) D_PTR (l, l_info[DT_STRTAB]);

0x7ffff7de6836 <_dl_fixup+22>: mov rax,QWORD PTR [r10+0xf8] // 72 (D_PTR (l, l_info[DT_JMPREL])

0x7ffff7de683d <_dl_fixup+29>: mov rax,QWORD PTR [rax+0x8] // 72 const PLTREL *const reloc = (const void *) (D_PTR (l, l_info[DT_JMPREL]) + reloc_offset)

0x7ffff7de6841 <_dl_fixup+33>: lea r8,[rax+rdx*8]

0x7ffff7de6845 <_dl_fixup+37>: mov rax,QWORD PTR [r10+0x70] // 68 (const void *) D_PTR (l, l_info[DT_SYMTAB])

0x7ffff7de6849 <_dl_fixup+41>: mov rcx,QWORD PTR [r8+0x8] // 73 reloc->r_info

0x7ffff7de684d <_dl_fixup+45>: mov rax,QWORD PTR [rax+0x8] // 73 const &symtab[ELFW(R_SYM) (reloc->r_info)]

0x7ffff7de6851 <_dl_fixup+49>: mov rdx,rcx // 73 const ElfW(Sym) *sym = &symtab[ELFW(R_SYM) (reloc->r_info)]

0x7ffff7de6854 <_dl_fixup+52>: shr rdx,0x20 // 73

0x7ffff7de6858 <_dl_fixup+56>: lea rsi,[rdx+rdx*2] // 73

0x7ffff7de685c <_dl_fixup+60>: lea rsi,[rax+rsi*8] // 73 [rsi = sym, 0x400270]

0x7ffff7de6860 <_dl_fixup+64>: mov rax,QWORD PTR [r10] // 74

0x7ffff7de6863 <_dl_fixup+67>: mov QWORD PTR [rsp+0x8],rsi // 73

0x7ffff7de6868 <_dl_fixup+72>: mov rbx,rax // 74

0x7ffff7de686b <_dl_fixup+75>: add rbx,QWORD PTR [r8] // 74 void *const rel_addr = (void *)(l->l_addr + reloc->r_offset);

0x7ffff7de686e <_dl_fixup+78>: cmp ecx,0x7 // 79

0x7ffff7de6871 <_dl_fixup+81>: jne 0x7ffff7de69c7 <_dl_fixup+423> // 79

0x7ffff7de6877 <_dl_fixup+87>: test BYTE PTR [rsi+0x5],0x3

0x7ffff7de687b <_dl_fixup+91>: jne 0x7ffff7de6919 <_dl_fixup+249>

0x7ffff7de6881 <_dl_fixup+97>: mov rax,QWORD PTR [r10+0x1c8]

[References]

http://www.skyfree.org/linux/references/ELF_Format.pdf
https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf
https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/elf.h
https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=elf/dl-reloc.c
http://www.cs.stevens.edu/~jschauma/810/elf.html

Cyber Grand Challenge by DARPA

1. Overview

2014년 여름, 미 국방성 연구부문을 담당하고 있는 DARPA (Defense Advanced Research Projects Agency)에서 매우 흥미로운 competition event를 진행한다. 이름하여 Cyber Grand Challenge – 자동화된 공격방어 시스템 (automated cyber reasoning system) 하에 상호 간의 공격과 방어를 모두 기계가 실시간으로 담당하게 해 경쟁하는 대회다.

많은 CTF Challenge 대회가 있지만, 사람의 개입없이 순수히 기계만을 이용해 취약점을 찾고, 공격과 방어를 하는 경우는 세계적으로 처음이었다. 게다가 상당히 큰 금액의 상금은 보안인의 관심을 끌기에 충분했다.

대표링크는 여기 https://cgc.darpa.mil 로 각종 문서와 대회결과를 확인할 수 있다. 대회는 3년간 예선(2014년)과 최종후보 결선전(2015년)을 거쳐 2016년 8월 4일 DEF CON에서 Final 결승전을 치룬다. Final은 7개의 팀이 겨룰 예정이며, 우승자는 무려 20만불의 상금을 거머쥘 수 있다.

작년 USENIX technical session에서 Invite Talk으로 DARPA의 Mike가 두번의 대회를 치르면서 있었던 과정과 결과를 발표했다. 발표자료는 여기를 참고하기 바란다.

UPDATED (As of Aug. 8, 2016)
a. Mayhem declared the final winner of historic Cyber Grand Challenge.
b. Mayhem has been developed as a cyber reasoning system since 2012 by Sang Kil Cha et al., see the paper, Unleashing MAYHEM on Binary Code, for more details)
c. CFE File Archive is available now.

2. DECREE

무엇보다도 DARPA는 취약점 자체에 초점을 맞출 수 있도록 기존 리눅스 환경을 변형한 DECREE라는 운영제체를 개발했다. DECREE는 수백 개의 system call이 존재하는 리눅스와는 다르게 실행파일이 다음과 같이 단 7개의 system call만을 이용하도록 설계했다. (Header 파일 참조)

int transmit(int fd, const void *buf, size_t count, size_t *tx_bytes);
int receive(int fd, void *buf, size_t count, size_t *rx_bytes);
int fdwait(int nfds, fd_set *readfds, fd_set *writefds);
int allocate(size_t length, int is_X, void **addr);
int deallocate(void *addr, size_t length);
int random(void *buf, size_t count, size_t *rnd_bytes);

새로운 운영체제에서 작동할 수 있는 실행파일을 컴파일할 수 있도록 gcc를 수정하고, 실행파일포맷 또한 기존의 ELF를 기반으로 한 cgc 포맷을 새로 정의했다. 이는 기존의 어떤 운영체제에서도 파일의 실행은 물론 대회에서 발견한 취약점과 exploit 모두 무미의함을 의미한다. 그리고 IPC (Inter-Process Communication) 도 무척 제한적이다. 공유 메모리를 지원하지 않으며, 단순한 양방향 통신만을 지원한다.

CyberGrandChallenge Github에 있는 walk-through 문서를 따라 VirtualBox와 vagrant를 설치하고 vbox를 추가한 후 다음과 같이 5개의 VM을 차례로 부팅할 수 있다.

~/Repos/cgc $ vagrant up
Bringing machine 'cb' up with 'virtualbox' provider...
Bringing machine 'ids' up with 'virtualbox' provider...
Bringing machine 'pov' up with 'virtualbox' provider...
Bringing machine 'crs' up with 'virtualbox' provider...
Bringing machine 'ti' up with 'virtualbox' provider...
[cb] Clearing any previously set forwarded ports...
[cb] Clearing any previously set network interfaces...
[cb] Preparing network interfaces based on configuration...
[cb] Forwarding ports...
[cb] -- 22 => 2222 (adapter 1)
[cb] Running 'pre-boot' VM customizations...
[cb] Booting VM...
[cb] Waiting for machine to boot. This may take a few minutes...
[cb] Machine booted and ready!
[cb] Setting hostname...
[cb] Configuring and enabling network interfaces...
[cb] Mounting shared folders...
[cb] -- /vagrant
[cb] VM already provisioned. Run `vagrant provision` or use `--provision` to force it
[ids] Clearing any previously set forwarded ports...
[ids] Fixed port collision for 22 => 2222. Now on port 2200.
[ids] Clearing any previously set network interfaces...
[ids] Preparing network interfaces based on configuration...
[ids] Forwarding ports...
[ids] -- 22 => 2200 (adapter 1)
[ids] Running 'pre-boot' VM customizations...
[ids] Booting VM...
[ids] Waiting for machine to boot. This may take a few minutes...
[ids] Machine booted and ready!
[ids] Setting hostname...
[ids] Configuring and enabling network interfaces...
[ids] Mounting shared folders...
[ids] -- /vagrant
[ids] VM already provisioned. Run `vagrant provision` or use `--provision` to force it
[pov] Clearing any previously set forwarded ports...
[pov] Fixed port collision for 22 => 2200. Now on port 2201.
[pov] Clearing any previously set network interfaces...
[pov] Preparing network interfaces based on configuration...
[pov] Forwarding ports...
[pov] -- 22 => 2201 (adapter 1)
[pov] Running 'pre-boot' VM customizations...
[pov] Booting VM...
[pov] Waiting for machine to boot. This may take a few minutes...
[pov] Machine booted and ready!
[pov] Setting hostname...
[pov] Configuring and enabling network interfaces...
[pov] Mounting shared folders...
[pov] -- /vagrant
[pov] VM already provisioned. Run `vagrant provision` or use `--provision` to force it
[crs] Clearing any previously set forwarded ports...
[crs] Fixed port collision for 22 => 2201. Now on port 2202.
[crs] Clearing any previously set network interfaces...
[crs] Preparing network interfaces based on configuration...
[crs] Forwarding ports...
[crs] -- 22 => 2202 (adapter 1)
[crs] Running 'pre-boot' VM customizations...
[crs] Booting VM...
[crs] Waiting for machine to boot. This may take a few minutes...
[crs] Machine booted and ready!
[crs] Setting hostname...
[crs] Configuring and enabling network interfaces...
[crs] Mounting shared folders...
[crs] -- /vagrant
[crs] VM already provisioned. Run `vagrant provision` or use `--provision` to force it
[ti] Clearing any previously set forwarded ports...
[ti] Fixed port collision for 22 => 2202. Now on port 2203.
[ti] Clearing any previously set network interfaces...
[ti] Preparing network interfaces based on configuration...
[ti] Forwarding ports...
[ti] -- 22 => 2203 (adapter 1)
[ti] Running 'pre-boot' VM customizations...
[ti] Booting VM...
[ti] Waiting for machine to boot. This may take a few minutes...
[ti] Machine booted and ready!
[ti] Setting hostname...
[ti] Configuring and enabling network interfaces...
[ti] Mounting shared folders...
[ti] -- /vagrant
[ti] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

~/Repos/cgc $ vagrant up

Bringing machine 'cb' up with 'virtualbox' provider...

Bringing machine 'ids' up with 'virtualbox' provider...

Bringing machine 'pov' up with 'virtualbox' provider...

Bringing machine 'crs' up with 'virtualbox' provider...

Bringing machine 'ti' up with 'virtualbox' provider...

[cb] Clearing any previously set forwarded ports...

[cb] Clearing any previously set network interfaces...

[cb] Preparing network interfaces based on configuration...

[cb] Forwarding ports...

[cb] -- 22 => 2222 (adapter 1)

[cb] Running 'pre-boot' VM customizations...

[cb] Booting VM...

[cb] Waiting for machine to boot. This may take a few minutes...

[cb] Machine booted and ready!

[cb] Setting hostname...

[cb] Configuring and enabling network interfaces...

[cb] Mounting shared folders...

[cb] -- /vagrant

[cb] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

[ids] Clearing any previously set forwarded ports...

[ids] Fixed port collision for 22 => 2222. Now on port 2200.

[ids] Clearing any previously set network interfaces...

[ids] Preparing network interfaces based on configuration...

[ids] Forwarding ports...

[ids] -- 22 => 2200 (adapter 1)

[ids] Running 'pre-boot' VM customizations...

[ids] Booting VM...

[ids] Waiting for machine to boot. This may take a few minutes...

[ids] Machine booted and ready!

[ids] Setting hostname...

[ids] Configuring and enabling network interfaces...

[ids] Mounting shared folders...

[ids] -- /vagrant

[ids] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

[pov] Clearing any previously set forwarded ports...

[pov] Fixed port collision for 22 => 2200. Now on port 2201.

[pov] Clearing any previously set network interfaces...

[pov] Preparing network interfaces based on configuration...

[pov] Forwarding ports...

[pov] -- 22 => 2201 (adapter 1)

[pov] Running 'pre-boot' VM customizations...

[pov] Booting VM...

[pov] Waiting for machine to boot. This may take a few minutes...

[pov] Machine booted and ready!

[pov] Setting hostname...

[pov] Configuring and enabling network interfaces...

[pov] Mounting shared folders...

[pov] -- /vagrant

[pov] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

[crs] Clearing any previously set forwarded ports...

[crs] Fixed port collision for 22 => 2201. Now on port 2202.

[crs] Clearing any previously set network interfaces...

[crs] Preparing network interfaces based on configuration...

[crs] Forwarding ports...

[crs] -- 22 => 2202 (adapter 1)

[crs] Running 'pre-boot' VM customizations...

[crs] Booting VM...

[crs] Waiting for machine to boot. This may take a few minutes...

[crs] Machine booted and ready!

[crs] Setting hostname...

[crs] Configuring and enabling network interfaces...

[crs] Mounting shared folders...

[crs] -- /vagrant

[crs] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

[ti] Clearing any previously set forwarded ports...

[ti] Fixed port collision for 22 => 2202. Now on port 2203.

[ti] Clearing any previously set network interfaces...

[ti] Preparing network interfaces based on configuration...

[ti] Forwarding ports...

[ti] -- 22 => 2203 (adapter 1)

[ti] Running 'pre-boot' VM customizations...

[ti] Booting VM...

[ti] Waiting for machine to boot. This may take a few minutes...

[ti] Machine booted and ready!

[ti] Setting hostname...

[ti] Configuring and enabling network interfaces...

[ti] Mounting shared folders...

[ti] -- /vagrant

[ti] VM already provisioned. Run `vagrant provision` or use `--provision` to force it

다음은 default로 설정되어 있는 CRS서버에 ssh로 연결한 모습이다. 게스트 운영체에제서 /vagrant로 이동하면 자동으로 Vagrant 스크립트에 의해 호스트 운영체제의 현재 디렉토리를 공유하고 있음을 알 수 있다.

~/Repos/cgc $ vagrant ssh
Linux crs 3.13.11-ckt32-cgc #1 SMP Tue Jul 12 14:51:24 UTC 2016 i686

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Jul 29 05:31:37 2016 from 10.0.2.2
vagrant@crs:~$ cd /vagrant/
vagrant@crs:/vagrant$ ls
Vagrantfile  peda  samples  vm.json

~/Repos/cgc $ vagrant ssh

Linux crs 3.13.11-ckt32-cgc #1 SMP Tue Jul 12 14:51:24 UTC 2016 i686

The programs included with the Debian GNU/Linux system are free software;

the exact distribution terms for each program are described in the

individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent

permitted by applicable law.

Last login: Fri Jul 29 05:31:37 2016 from 10.0.2.2

vagrant@crs:~$ cd /vagrant/

vagrant@crs:/vagrant$ ls

Vagrantfile peda samples vm.json

3. Challenge Binaries and Utilities

2015년 Grand Challenge에서 DARPA는 대회용으로 131개의 Compile된 Binary만을 공개했다. 하지만, 현재 Github Repository에서 모든 바이너리의 소스코드와 취약점, PoV (Proof of Vulnerability), 그리고 상세한 설명 (Service Description)을 제공하고 있다.

131개의 바이너리는 단순하게 만든 운영체제에서 실제 서비스와 유사할 만큼의 복잡도를 가진 서비스를 제공할 수 있다. 72개의 CC 파일 (7000 LOC), 1236개의 헤더 (19만 LOC), 1996개의 C 파일 (20만 LOC), 6000여 개의 함수와 590개의 PoV가 이를 대변한다.

(1) CGC Executable Format (CGCEF)

CGC 실행 포맷은 처음 15바이트의 헤더가 ELF와 다르다. \x7fCGC로 시작함을 알 수 있다.

#define CI_NIDENT  16
typedef struct{
    unsigned char  e_ident[EI_NIDENT];
#define C_IDENT "\x7fCGC\x01\x01\x01\x43\x01\x00\x00\x00\x00\x00\x00"
    /* ELF vs CGC identification
    * ELF          CGC
    *  0x7f        0x7f
    *  'E'        'C'
    *  'L'        'G'
    *  'F'        'C'
    *  class      1      : '1' translates to 32bit ELF
    *  data        1      : '1' translates to little endian ELF
    *  version    1      : '1' translates to little endian ELF
    *  osabi      \x43    : '1' CGC
    *  abiversion  1      : '1' translates to version 1
    *  padding    random values
    */
    uint16_t    e_type;        /* Must be 2 for executable */
    uint16_t    e_machine;      /* Must be 3 for i386 */
    uint32_t    e_version;      /* Must be 1 */
    uint32_t    e_entry;        /* Virtual address entry point */
    uint32_t    e_phoff;        /* Program Header offset */
    uint32_t    e_shoff;        /* Section Header offset */
    uint32_t    e_flags;        /* Must be 0 */
    uint16_t    e_ehsize;      /* CGC header's size */
    uint16_t    e_phentsize;    /* Program header entry size */
    uint16_t    e_phnum;        /* # program header entries */
    uint16_t    e_shentsize;    /* Section header entry size */
    uint16_t    e_shnum;        /* # section header entries */
    uint16_t    e_shstrndx;    /* sect header # of str table */
} CGC32_hdr;

#define CI_NIDENT 16

typedef struct{

unsigned char e_ident[EI_NIDENT];

#define C_IDENT "\x7fCGC\x01\x01\x01\x43\x01\x00\x00\x00\x00\x00\x00"

/* ELF vs CGC identification

* ELF CGC

* 0x7f 0x7f

* 'E' 'C'

* 'L' 'G'

* 'F' 'C'

* class 1 : '1' translates to 32bit ELF

* data 1 : '1' translates to little endian ELF

* version 1 : '1' translates to little endian ELF

* osabi \x43 : '1' CGC

* abiversion 1 : '1' translates to version 1

* padding random values

uint16_t e_type; /* Must be 2 for executable */

uint16_t e_machine; /* Must be 3 for i386 */

uint32_t e_version; /* Must be 1 */

uint32_t e_entry; /* Virtual address entry point */

uint32_t e_phoff; /* Program Header offset */

uint32_t e_shoff; /* Section Header offset */

uint32_t e_flags; /* Must be 0 */

uint16_t e_ehsize; /* CGC header's size */

uint16_t e_phentsize; /* Program header entry size */

uint16_t e_phnum; /* # program header entries */

uint16_t e_shentsize; /* Section header entry size */

uint16_t e_shnum; /* # section header entries */

uint16_t e_shstrndx; /* sect header # of str table */

} CGC32_hdr;

CGC 실행포맷은 cgcef-verify라는 도구로 정상여부를 확인할 수 있으며, readelf와 같이 readcgcef로 읽을 수 있다. PATH 환경변수를 살짝 조정해서 기존의 bintools을 편하게 대체할 수도 있다.

vagrant@crs:/vagrant$ cgcef_verify ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001
vagrant@crs:/vagrant$ readcgcef -e ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001
CGCEF Header:
  Magic:   7f 43 47 43 01 01 01 43 01 4d 65 72 69 6e 6f 00 
  Class:                             CGCEF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            CGC
  ABI Version:                       1
  Type:                              EXEC (Executable file)
  Machine:                           Intel i386
  Version:                           0x1
  Entry point address:               0x804a789
  Start of program headers:          52 (bytes into file)
  Start of section headers:          94208 (bytes into file)
  Flags:                             0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         3
  Size of section headers:           40 (bytes)
  Number of section headers:         6
  Section header string table index: 5

CGCEf file type is EXEC (Executable file)
Entry point 0x804a789
There are 3 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  PHDR           0x000034 0x08048034 0x08048034 0x00060 0x00060 R   0x4
  LOAD           0x000000 0x08048000 0x08048000 0x02da0 0x02da0 R E 0x1000
  LOAD           0x002da0 0x0804bda0 0x0804bda0 0x14217 0x14217 RW  0x1000

 Section to Segment mapping:
  Segment Sections...
   00     
   01     .text .rodata 
   02     .data 
There are 6 section headers, starting at offset 0x17000:

Section Headers:
  [Nr] Name           Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 0] (null)         NULL            00000000 000000 000000 00      0   0  0
  [ 1] .text          PROGBITS        080480a0 0000a0 002a03 00  AX  0   0 16
  [ 2] .rodata        PROGBITS        0804aab0 002ab0 0002f0 00   A  0   0 16
  [ 3] .data          PROGBITS        0804bda0 002da0 014217 00  WA  0   0  4
  [ 4] .comment       PROGBITS        00000000 016fb7 00001e 01  MS  0   0  1
  [ 5] .shstrtab      STRTAB          00000000 016fd5 000028 00      0   0  1
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings)
  I (info), L (link order), G (group), x (unknown)
  O (extra OS processing required) o (OS specific), p (processor specific)

vagrant@crs:/vagrant$ ls /usr/i386-linux-cgc/bin
ar            clang-check   lli               llvm-cov        llvm-mc        llvm-rtdyld      nm
as            clang-format  lli-child-target  llvm-diff       llvm-mcmarkup  llvm-size        objcopy
bugpoint      clang-tblgen  llvm-ar           llvm-dis        llvm-nm        llvm-stress      objdump
c-index-test  ld            llvm-as           llvm-dwarfdump  llvm-objdump   llvm-symbolizer  opt
clang         ld.bfd        llvm-bcanalyzer   llvm-extract    llvm-ranlib    llvm-tblgen      ranlib
clang++       llc           llvm-config       llvm-link       llvm-readobj   macho-dump       strip

vagrant@crs:/vagrant$ export PATH=/usr/i386-linux-cgc/bin:$PATH
vagrant@crs:/vagrant$ which objdump
/usr/i386-linux-cgc/bin/objdump

./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001: file format cgc32-i386
architecture: i386, flags 0x00000102:
EXEC_P, D_PAGED
start address 0x0804a789

vagrant@crs:/vagrant$ cgcef_verify ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001

vagrant@crs:/vagrant$ readcgcef -e ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001

CGCEF Header:

Magic: 7f 43 47 43 01 01 01 43 01 4d 65 72 69 6e 6f 00

Class: CGCEF32

Data: 2's complement, little endian

Version: 1 (current)

OS/ABI: CGC

ABI Version: 1

Type: EXEC (Executable file)

Machine: Intel i386

Version: 0x1

Entry point address: 0x804a789

Start of program headers: 52 (bytes into file)

Start of section headers: 94208 (bytes into file)

Flags: 0

Size of this header: 52 (bytes)

Size of program headers: 32 (bytes)

Number of program headers: 3

Size of section headers: 40 (bytes)

Number of section headers: 6

Section header string table index: 5

CGCEf file type is EXEC (Executable file)

Entry point 0x804a789

There are 3 program headers, starting at offset 52

Program Headers:

Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align

PHDR 0x000034 0x08048034 0x08048034 0x00060 0x00060 R 0x4

LOAD 0x000000 0x08048000 0x08048000 0x02da0 0x02da0 R E 0x1000

LOAD 0x002da0 0x0804bda0 0x0804bda0 0x14217 0x14217 RW 0x1000

Section to Segment mapping:

Segment Sections...

01 .text .rodata

02 .data

There are 6 section headers, starting at offset 0x17000:

Section Headers:

[Nr] Name Type Addr Off Size ES Flg Lk Inf Al

[ 0] (null) NULL 00000000 000000 000000 00 0 0 0

[ 1] .text PROGBITS 080480a0 0000a0 002a03 00 AX 0 0 16

[ 2] .rodata PROGBITS 0804aab0 002ab0 0002f0 00 A 0 0 16

[ 3] .data PROGBITS 0804bda0 002da0 014217 00 WA 0 0 4

[ 4] .comment PROGBITS 00000000 016fb7 00001e 01 MS 0 0 1

[ 5] .shstrtab STRTAB 00000000 016fd5 000028 00 0 0 1

Key to Flags:

W (write), A (alloc), X (execute), M (merge), S (strings)

I (info), L (link order), G (group), x (unknown)

O (extra OS processing required) o (OS specific), p (processor specific)

vagrant@crs:/vagrant$ ls /usr/i386-linux-cgc/bin

ar clang-check lli llvm-cov llvm-mc llvm-rtdyld nm

as clang-format lli-child-target llvm-diff llvm-mcmarkup llvm-size objcopy

bugpoint clang-tblgen llvm-ar llvm-dis llvm-nm llvm-stress objdump

c-index-test ld llvm-as llvm-dwarfdump llvm-objdump llvm-symbolizer opt

clang ld.bfd llvm-bcanalyzer llvm-extract llvm-ranlib llvm-tblgen ranlib

clang++ llc llvm-config llvm-link llvm-readobj macho-dump strip

vagrant@crs:/vagrant$ export PATH=/usr/i386-linux-cgc/bin:$PATH

vagrant@crs:/vagrant$ which objdump

/usr/i386-linux-cgc/bin/objdump

./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001: file format cgc32-i386

architecture: i386, flags 0x00000102:

EXEC_P, D_PAGED

start address 0x0804a789

(2) CGCEF Program Headers

CGCEF는 섹션유형이(section type) 4개밖에 없다. 모든 바이너리는 정적으로 연결(Statically Linked)되어 있으며, 동적링킹 (Dynamic Linked)과 thread를 사용하지 않는다. 따라서 TLS (Thread Local Storage) 섹션도 존재하지 않는다.

typedef struct{
    uint32_t        p_type;        /* Section type */
#define PT_NULL    0              /* Unused header */
#define PT_LOAD    1              /* Segment loaded into mem */
#define PT_PHDR    6              /* Program hdr tbl itself */
#define PT_CGCPOV2  0x6ccccccc      /* CFE Type 2 PoV flag sect */
    uint32_t        p_offset;      /* Offset into the file */
    uint32_t        p_vaddr;        /* Virtual program address */
    uint32_t        p_paddr;        /* Set to zero */
    uint32_t        p_filesz;      /* Section bytes in file */
    uint32_t        p_memsz;        /* Section bytes in memory */
    uint32_t        p_flags;        /* section flags */
#define PF_X        (1<<0)          /* Mapped executable */
#define PF_W        (1<<1)          /* Mapped writable */
#define PF_R        (1<<2)          /* Mapped readable */
    /* Acceptable flag combinations are:
    *        PF_R
    *        PF_R|PF_W
    *        PF_R|PF_X
    *        PF_R|PF_W|PF_X
    */
    uint32_t        p_align;        /* Only used by core dumps */
} CGC32_Phdr;

typedef struct{

uint32_t p_type; /* Section type */

#define PT_NULL 0 /* Unused header */

#define PT_LOAD 1 /* Segment loaded into mem */

#define PT_PHDR 6 /* Program hdr tbl itself */

#define PT_CGCPOV2 0x6ccccccc /* CFE Type 2 PoV flag sect */

uint32_t p_offset; /* Offset into the file */

uint32_t p_vaddr; /* Virtual program address */

uint32_t p_paddr; /* Set to zero */

uint32_t p_filesz; /* Section bytes in file */

uint32_t p_memsz; /* Section bytes in memory */

uint32_t p_flags; /* section flags */

#define PF_X (1<<0) /* Mapped executable */

#define PF_W (1<<1) /* Mapped writable */

#define PF_R (1<<2) /* Mapped readable */

/* Acceptable flag combinations are:

* PF_R

* PF_R|PF_W

* PF_R|PF_X

* PF_R|PF_W|PF_X

uint32_t p_align; /* Only used by core dumps */

} CGC32_Phdr;

vagrant@crs:/vagrant$ objdump -h ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001

./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001: file format cgc32-i386

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00002a03  080480a0  080480a0  000000a0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .rodata       000002f0  0804aab0  0804aab0  00002ab0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .data         00014217  0804bda0  0804bda0  00002da0  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  3 .comment      0000001e  00000000  00000000  00016fb7  2**0
                  CONTENTS, READONLY

vagrant@crs:/vagrant$ objdump -h ./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001

./samples/cqe-challenges/CROMU_00001/bin/CROMU_00001: file format cgc32-i386

Sections:

Idx Name Size VMA LMA File off Algn

0 .text 00002a03 080480a0 080480a0 000000a0 2**4

CONTENTS, ALLOC, LOAD, READONLY, CODE

1 .rodata 000002f0 0804aab0 0804aab0 00002ab0 2**4

CONTENTS, ALLOC, LOAD, READONLY, DATA

2 .data 00014217 0804bda0 0804bda0 00002da0 2**2

CONTENTS, ALLOC, LOAD, DATA

3 .comment 0000001e 00000000 00000000 00016fb7 2**0

CONTENTS, READONLY

(3) CGCEF Section Headers

섹션헤더는 디버깅할 때 정보 제공용으로만 사용하며, Loader는 이 섹션을 무시하고 로딩한다. 배포용(release)으로 제공하는 바이너리는 보통 이 섹션이 제거되고(stripped) 없다고 보면 된다.

typedef struct {
    uint32_t        sh_name;        /* Name (index into strtab) */
    uint32_t        sh_type;        /* Section type */
#define SHT_SYMTAB  2              /* Symbol table */
#define SHT_STRTAB  3              /* String Table */
    uint32_t        sh_flags;
#define SHT_WRITE  (1<<0)          /* Section is writable */
#define SHT_ALLOC  (1<<1)          /* Section is in memory */
#define SHT_EXECINSTR (1<<2)        /* Section contains code */
    uint32_t        sh_addr;        /* Address of section */
    uint32_t        sh_offset;      /* Offset into file */
    uint32_t        sh_size;        /* Section size in file */
    uint32_t        sh_link;
            /* When sh_type is SHT_SYMTAB, sh_link is the index of
            * the associated SHT_STRTAB section
            */
    uint32_t        sh_info;
            /* When sh_type is SHT_SYMTAB, info is one greater
            * than the symbol table index of the last local
            * symbol.
            */
    uint32_t        sh_addralign;  /* Alignment constraints */
    uint32_t        sh_entsize;
            /* Size in bytes of each entry pointed to by this
            * section table */
} CGC32_Shdr;

typedef struct {

uint32_t sh_name; /* Name (index into strtab) */

uint32_t sh_type; /* Section type */

#define SHT_SYMTAB 2 /* Symbol table */

#define SHT_STRTAB 3 /* String Table */

uint32_t sh_flags;

#define SHT_WRITE (1<<0) /* Section is writable */

#define SHT_ALLOC (1<<1) /* Section is in memory */

#define SHT_EXECINSTR (1<<2) /* Section contains code */

uint32_t sh_addr; /* Address of section */

uint32_t sh_offset; /* Offset into file */

uint32_t sh_size; /* Section size in file */

uint32_t sh_link;

/* When sh_type is SHT_SYMTAB, sh_link is the index of

* the associated SHT_STRTAB section

uint32_t sh_info;

/* When sh_type is SHT_SYMTAB, info is one greater

* than the symbol table index of the last local

* symbol.

uint32_t sh_addralign; /* Alignment constraints */

uint32_t sh_entsize;

/* Size in bytes of each entry pointed to by this

* section table */

} CGC32_Shdr;

(4) IDA Pro modules for CGC

Reversing을 할 때 표준도구처럼 많은 사람들이 애용하는 IDA Pro에서도 Loading할 수 있도 Module을 제공한다. 여기서 (http://idabook.com/cgc/) 버전에 맞는 모듈을 다운받아 설치하면 CGC 바이너리를 로딩할 때 자동으로 Loader를 선택해 준다.

(5) How to run CGC binary as a service

일일히 실행하기 번거로우면, 다음과 같이 임의로 서버와 클라이언트를 실행해 확인해 볼 수 있다. 다음과 같이 cqe 바이너리의 poller directory 내에 for-testing과 for-release 두 가지 형태의 많은 sample data로 테스트해 볼 수 있다.

# SEVER SIDE
vagrant@crs:/vagrant/cqe-challenges/KPRCA_00022/build/release/bin$ cb-server --insecure -p 10000 -d /vagrant/cqe-challenges/CROMU_00015/bin /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015
connection from: 127.0.0.1:4499
negotiation flag: 0
getting random seed
seed: 22733022E8A0D77BB3F089EA9A2DD56593E0373B739737F976F4EFC937BEB1A4FFEAF08F62D708E22C928970F728052F
stat: /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015 filesize 159244
CB exited (pid: 7672, exit code: 0)
total children: 1
total maxrss 96
total minflt 68
total utime 0.000000
total sw-cpu-clock 13506359
total sw-task-clock 13504409
CB exited (pid: 7671, exit code: 0)

# CLIENT SIDE
vagrant@crs:/vagrant/cqe-challenges/CROMU_00015$ cb-replay --host 127.0.0.1 --port 10000 ./poller/for-testing/POLL_00800.xml 
# CROMU_00015 - ./poller/for-testing/POLL_00800.xml
# connected to ('127.0.0.1', 10000)
ok 1 - match: string
ok 2 - write: sent 2 bytes
ok 3 - match: string
ok 4 - write: sent 11 bytes
ok 5 - match: string
ok 6 - write: sent 2 bytes
ok 7 - match: string
ok 8 - match: string
ok 9 - write: sent 2 bytes
ok 10 - match: string
ok 11 - match: string
ok 12 - write: sent 2 bytes
ok 13 - match: string
ok 14 - write: sent 2 bytes
ok 15 - match: string
ok 16 - write: sent 3 bytes
ok 17 - match: string
ok 18 - write: sent 5 bytes
ok 19 - match: string
ok 20 - write: sent 2 bytes
ok 21 - match: string
ok 22 - write: sent 8 bytes
ok 23 - match: string
ok 24 - write: sent 2 bytes
ok 25 - match: string
ok 26 - write: sent 8 bytes
ok 27 - match: string
ok 28 - write: sent 2 bytes
ok 29 - match: string
ok 30 - write: sent 9 bytes
ok 31 - match: string
ok 32 - write: sent 3 bytes
ok 33 - match: string
ok 34 - write: sent 6 bytes
ok 35 - match: string
ok 36 - write: sent 2 bytes
ok 37 - match: string
ok 38 - write: sent 8 bytes
ok 39 - match: string
ok 40 - write: sent 3 bytes
ok 41 - match: string
ok 42 - write: sent 9 bytes
ok 43 - match: string
ok 44 - write: sent 2 bytes
ok 45 - match: string
ok 46 - write: sent 8 bytes
ok 47 - match: string
ok 48 - write: sent 2 bytes
ok 49 - match: string
ok 50 - write: sent 8 bytes
ok 51 - match: string
ok 52 - write: sent 2 bytes
ok 53 - match: string
ok 54 - match: string
ok 55 - write: sent 2 bytes
ok 56 - match: string
ok 57 - write: sent 9 bytes
ok 58 - match: string
ok 59 - write: sent 2 bytes
ok 60 - match: string
ok 61 - write: sent 9 bytes
ok 62 - match: string
ok 63 - write: sent 2 bytes
ok 64 - match: string
ok 65 - write: sent 8 bytes
ok 66 - match: string
ok 67 - write: sent 2 bytes
ok 68 - match: string
ok 69 - write: sent 8 bytes
ok 70 - match: string
ok 71 - write: sent 3 bytes
ok 72 - match: string
ok 73 - write: sent 2 bytes
ok 74 - match: string
ok 75 - write: sent 452 bytes
ok 76 - match: string
ok 77 - write: sent 2 bytes
ok 78 - match: string
ok 79 - match: string
ok 80 - write: sent 2 bytes
ok 81 - match: string
ok 82 - write: sent 2 bytes
ok 83 - match: string
ok 84 - write: sent 2 bytes
ok 85 - match: string
ok 86 - write: sent 8 bytes
ok 87 - match: string
ok 88 - write: sent 2 bytes
ok 89 - match: string
ok 90 - write: sent 8 bytes
ok 91 - match: string
ok 92 - write: sent 3 bytes
ok 93 - match: string
ok 94 - write: sent 8 bytes
ok 95 - match: string
ok 96 - write: sent 2 bytes
ok 97 - match: string
ok 98 - write: sent 8 bytes
ok 99 - match: string
ok 100 - write: sent 2 bytes
ok 101 - match: string
ok 102 - write: sent 8 bytes
ok 103 - match: string
ok 104 - write: sent 3 bytes
ok 105 - match: string
ok 106 - write: sent 5 bytes
ok 107 - match: string
ok 108 - write: sent 2 bytes
ok 109 - match: string
ok 110 - write: sent 8 bytes
ok 111 - match: string
ok 112 - write: sent 2 bytes
ok 113 - match: string
ok 114 - write: sent 8 bytes
ok 115 - match: string
ok 116 - write: sent 2 bytes
ok 117 - match: string
ok 118 - write: sent 8 bytes
ok 119 - match: string
ok 120 - write: sent 2 bytes
ok 121 - match: string
ok 122 - write: sent 8 bytes
ok 123 - match: string
ok 124 - write: sent 2 bytes
ok 125 - match: string
ok 126 - write: sent 8 bytes
ok 127 - match: string
ok 128 - write: sent 3 bytes
ok 129 - match: string
ok 130 - write: sent 12 bytes
ok 131 - match: string
ok 132 - write: sent 3 bytes
ok 133 - match: string
ok 134 - write: sent 2 bytes
ok 135 - match: string
ok 136 - write: sent 13 bytes
ok 137 - match: string
ok 138 - write: sent 2 bytes
ok 139 - match: string
# tests passed: 139
# tests failed: 0
# total tests passed: 139
# total tests failed: 0
# polls passed: 1
# polls failed: 0

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

# SEVER SIDE

vagrant@crs:/vagrant/cqe-challenges/KPRCA_00022/build/release/bin$ cb-server --insecure -p 10000 -d /vagrant/cqe-challenges/CROMU_00015/bin /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015

connection from: 127.0.0.1:4499

negotiation flag: 0

getting random seed

seed: 22733022E8A0D77BB3F089EA9A2DD56593E0373B739737F976F4EFC937BEB1A4FFEAF08F62D708E22C928970F728052F

stat: /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015 filesize 159244

CB exited (pid: 7672, exit code: 0)

total children: 1

total maxrss 96

total minflt 68

total utime 0.000000

total sw-cpu-clock 13506359

total sw-task-clock 13504409

CB exited (pid: 7671, exit code: 0)

# CLIENT SIDE

vagrant@crs:/vagrant/cqe-challenges/CROMU_00015$ cb-replay --host 127.0.0.1 --port 10000 ./poller/for-testing/POLL_00800.xml

# CROMU_00015 - ./poller/for-testing/POLL_00800.xml

# connected to ('127.0.0.1', 10000)

ok 1 - match: string

ok 2 - write: sent 2 bytes

ok 3 - match: string

ok 4 - write: sent 11 bytes

ok 5 - match: string

ok 6 - write: sent 2 bytes

ok 7 - match: string

ok 8 - match: string

ok 9 - write: sent 2 bytes

ok 10 - match: string

ok 11 - match: string

ok 12 - write: sent 2 bytes

ok 13 - match: string

ok 14 - write: sent 2 bytes

ok 15 - match: string

ok 16 - write: sent 3 bytes

ok 17 - match: string

ok 18 - write: sent 5 bytes

ok 19 - match: string

ok 20 - write: sent 2 bytes

ok 21 - match: string

ok 22 - write: sent 8 bytes

ok 23 - match: string

ok 24 - write: sent 2 bytes

ok 25 - match: string

ok 26 - write: sent 8 bytes

ok 27 - match: string

ok 28 - write: sent 2 bytes

ok 29 - match: string

ok 30 - write: sent 9 bytes

ok 31 - match: string

ok 32 - write: sent 3 bytes

ok 33 - match: string

ok 34 - write: sent 6 bytes

ok 35 - match: string

ok 36 - write: sent 2 bytes

ok 37 - match: string

ok 38 - write: sent 8 bytes

ok 39 - match: string

ok 40 - write: sent 3 bytes

ok 41 - match: string

ok 42 - write: sent 9 bytes

ok 43 - match: string

ok 44 - write: sent 2 bytes

ok 45 - match: string

ok 46 - write: sent 8 bytes

ok 47 - match: string

ok 48 - write: sent 2 bytes

ok 49 - match: string

ok 50 - write: sent 8 bytes

ok 51 - match: string

ok 52 - write: sent 2 bytes

ok 53 - match: string

ok 54 - match: string

ok 55 - write: sent 2 bytes

ok 56 - match: string

ok 57 - write: sent 9 bytes

ok 58 - match: string

ok 59 - write: sent 2 bytes

ok 60 - match: string

ok 61 - write: sent 9 bytes

ok 62 - match: string

ok 63 - write: sent 2 bytes

ok 64 - match: string

ok 65 - write: sent 8 bytes

ok 66 - match: string

ok 67 - write: sent 2 bytes

ok 68 - match: string

ok 69 - write: sent 8 bytes

ok 70 - match: string

ok 71 - write: sent 3 bytes

ok 72 - match: string

ok 73 - write: sent 2 bytes

ok 74 - match: string

ok 75 - write: sent 452 bytes

ok 76 - match: string

ok 77 - write: sent 2 bytes

ok 78 - match: string

ok 79 - match: string

ok 80 - write: sent 2 bytes

ok 81 - match: string

ok 82 - write: sent 2 bytes

ok 83 - match: string

ok 84 - write: sent 2 bytes

ok 85 - match: string

ok 86 - write: sent 8 bytes

ok 87 - match: string

ok 88 - write: sent 2 bytes

ok 89 - match: string

ok 90 - write: sent 8 bytes

ok 91 - match: string

ok 92 - write: sent 3 bytes

ok 93 - match: string

ok 94 - write: sent 8 bytes

ok 95 - match: string

ok 96 - write: sent 2 bytes

ok 97 - match: string

ok 98 - write: sent 8 bytes

ok 99 - match: string

ok 100 - write: sent 2 bytes

ok 101 - match: string

ok 102 - write: sent 8 bytes

ok 103 - match: string

ok 104 - write: sent 3 bytes

ok 105 - match: string

ok 106 - write: sent 5 bytes

ok 107 - match: string

ok 108 - write: sent 2 bytes

ok 109 - match: string

ok 110 - write: sent 8 bytes

ok 111 - match: string

ok 112 - write: sent 2 bytes

ok 113 - match: string

ok 114 - write: sent 8 bytes

ok 115 - match: string

ok 116 - write: sent 2 bytes

ok 117 - match: string

ok 118 - write: sent 8 bytes

ok 119 - match: string

ok 120 - write: sent 2 bytes

ok 121 - match: string

ok 122 - write: sent 8 bytes

ok 123 - match: string

ok 124 - write: sent 2 bytes

ok 125 - match: string

ok 126 - write: sent 8 bytes

ok 127 - match: string

ok 128 - write: sent 3 bytes

ok 129 - match: string

ok 130 - write: sent 12 bytes

ok 131 - match: string

ok 132 - write: sent 3 bytes

ok 133 - match: string

ok 134 - write: sent 2 bytes

ok 135 - match: string

ok 136 - write: sent 13 bytes

ok 137 - match: string

ok 138 - write: sent 2 bytes

ok 139 - match: string

# tests passed: 139

# tests failed: 0

# total tests passed: 139

# total tests failed: 0

# polls passed: 1

# polls failed: 0

또한 pov 디렉토리에는 실제 취약점을 trigger하는 입력값(input)을 가지고 있어 실제 취약한 코드가 어디인지 살펴보기에 안성맞춤이다. 아래 예제는 client와 통신한 후 exit code가 0이 아닌 상태로 종결되었기에 서버 측에서 실제 register status를 보여주고 있다.

# SERVER SIDE
vagrant@crs:/vagrant/cqe-challenges/KPRCA_00022/build/release/bin$ cb-server --insecure -p 10000 -d /vagrant/cqe-challenges/CROMU_00015/bin /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015
connection from: 127.0.0.1:38142
negotiation flag: 0
getting random seed
seed: 9D7F1CABD61E527C629EEB20FD4A6E75D01A326A24CA2572DB294255EF554755FC498388FCAB7BCDFBAD3AB1B07BB0B6
stat: /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015 filesize 159244
register states - eax: 0805a000 ecx: 00000000 edx: b7ffe000 ebx: 00000000 esp: baaaad18 ebp: baaaad24 esi: baaaaef0 edi: 00000000 eip: 08058e27
CB generated signal (pid: 7686, signal: 11)
total children: 1
total maxrss 68
total minflt 20
total utime 0.000000
total sw-cpu-clock 1076908
total sw-task-clock 1208191

# CLIENT SIDE
vagrant@crs:/vagrant/cqe-challenges/CROMU_00015$ cb-replay --host 127.0.0.1 --port 10000 ./pov/POV_00006.xml  
# service - ./pov/POV_00006.xml
# connected to ('127.0.0.1', 10000)
ok 1 - read length
ok 2 - write: sent 2 bytes
ok 3 - read length
ok 4 - write: sent 29 bytes
ok 5 - slept 1.000000
# tests passed: 5
# tests failed: 0
# total tests passed: 5
# total tests failed: 0
# polls passed: 1
# polls failed: 0

# SERVER SIDE

vagrant@crs:/vagrant/cqe-challenges/KPRCA_00022/build/release/bin$ cb-server --insecure -p 10000 -d /vagrant/cqe-challenges/CROMU_00015/bin /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015

connection from: 127.0.0.1:38142

negotiation flag: 0

getting random seed

seed: 9D7F1CABD61E527C629EEB20FD4A6E75D01A326A24CA2572DB294255EF554755FC498388FCAB7BCDFBAD3AB1B07BB0B6

stat: /vagrant/cqe-challenges/CROMU_00015/bin/CROMU_00015 filesize 159244

register states - eax: 0805a000 ecx: 00000000 edx: b7ffe000 ebx: 00000000 esp: baaaad18 ebp: baaaad24 esi: baaaaef0 edi: 00000000 eip: 08058e27

CB generated signal (pid: 7686, signal: 11)

total children: 1

total maxrss 68

total minflt 20

total utime 0.000000

total sw-cpu-clock 1076908

total sw-task-clock 1208191

# CLIENT SIDE

vagrant@crs:/vagrant/cqe-challenges/CROMU_00015$ cb-replay --host 127.0.0.1 --port 10000 ./pov/POV_00006.xml

# service - ./pov/POV_00006.xml

# connected to ('127.0.0.1', 10000)

ok 1 - read length

ok 2 - write: sent 2 bytes

ok 3 - read length

ok 4 - write: sent 29 bytes

ok 5 - slept 1.000000

# tests passed: 5

# tests failed: 0

# total tests passed: 5

# total tests failed: 0

# polls passed: 1

# polls failed: 0

4. CWE Distribution for CGC Binaries

CWE는 MITRE (https://cwe.mitre.org/)에서 제공하는 일종의 취약점 분류(Common Weakness Enumeration)라고 볼 수 있다. Semi-Final에서 제공한 131개의 바이너리 Description을 기반으로 살펴본 결과 다음 표와 같이 대략 60여개의 분류와 300개 이상의 취약점을 가지고 있음을 알 수 있다.

CWE No	CWE Description	Total
CWE-20	Improper Input Validation	9
CWE-22	Improper limitation of a pathname to a restricted directory	1
CWE-59	Improper link resolution before file access	1
CWE-61	UNIX symbolic link following	1
CWE-119	Improper Restriction of Operations within the Bounds of a Memory Buffer	6
CWE-120	Buffer Copy without Checking Size of Input ('Classic Buffer Overflow')	22
CWE-121	Stack-based buffer overflow	24
CWE-122	Heap-Based Buffer Overflow	29
CWE-123	Write-what-where Condition	1
CWE-125	Out-of-bounds read	13
CWE-127	Buffer Under-read	1
CWE-128	Wrap-around Error	1
CWE-129	Improper Validation of Array Index	14
CWE-131	Improper Calculation of Buffer Size	11
CWE-134	Uncontrolled Format Sting	7
CWE-170	Improper Null Termination	1
CWE-176	Improper handling of unicode encoding	1
CWE-190	Integer overflow or wraparound	16
CWE-191	Integer underflow (Wrap or Wraparound)	3
CWE-193	Off-by-one error	11
CWE-195	Signed to Unsigned Conversion Error	5
CWE-196	Unsigned to Signed Conversion Error	1
CWE-200	Information Exposure	1
CWE-201	Information Exposure Through Sent Data	1
CWE-252	Unchecked Return Value	1
CWE-275	Permission issues	1
CWE-326	Inadequate Encryption Strength	1
CWE-327	Use of a Broken or Risky Cryptographic Function	1
CWE-328	Reversible One-Way Hash	1
CWE-367	TOCTOU Error	2
CWE-369	Divide by zero	1
CWE-400	Uncontrolled Resource Consumption	1
CWE-415	Double Free	1
CWE-416	Use After Free	8
CWE-434	Unrestricted upload of file with dangerous type	1
CWE-457	Use of uninitialized variable	5
CWE-467	Use of sizeof() on a Pointer Type	2
CWE-469	Use of Pointer Subtraction to Determine Size	1
CWE-471	Modification if Assumed-Immutable Data	1
CWE-476	Null Pointer Dereference	24
CWE-665	Improper Initialization	1
CWE-674	Uncontrolled recursion	5
CWE-680	Integer Overflow to Buffer Overflow	2
CWE-682	Incorrect calculation	1
CWE-690	Unchecked Return Value	1
CWE-704	Incorrect type conversion or cast	3
CWE-755	Improper Handling of Exceptional Conditions	1
CWE-763	Release of Invalid Pointer or Reference	1
CWE-783	Operator Precedence Logic Error	1
CWE-785	Use of Path Manipulation Function without Maximum-sized Buffer	1
CWE-787	Out-of-bounds Write	22
CWE-788	Access of Memory Location After End of Buffer	6
CWE-798	Use of Hard-coded Credentials	1
CWE-805	Buffer Access with Incorrect Length Value	2
CWE-822	Untrusted Pointer Dereference	4
CWE-823	Use of Out-of-range Pointer Offset	2
CWE-824	Access of Uninitialized Pointer	5
CWE-825	Expired Pointer Dereference	1
CWE-839	Numeric Range Comparison Without Minimum Check	4
CWE-843	Access of Resource Using Incompatible Type 'Type Confusion'	5
CWE-908	Use of Uninitialized Resource	3

TOP 5는 짐작할 수 있듯이 Classic Buffer Overflow, Stack-based Overflow, Heap-based Overflow, Out-of-Bounds Write, Null Pointer Dereferencing이다.