Early interrupts handler
Early interrupt and exception handling
In the previous part we stopped before setting of early interrupt handlers. At this moment we are in the decompressed Linux kernel, we have basic paging structure for early boot and our current goal is to finish early preparation before the main kernel code will start to work.
We already started to do this preparation in the previous first part of this chapter. We continue in this part and will know more about interrupt and exception handling.
Remember that we stopped before following function:
from the arch/x86/kernel/head64.c source code file. But before we start to sort out this function, we need to know about interrupts and handlers.
Some theory
An interrupt is an event caused by software or hardware to the CPU. For example a user have pressed a key on keyboard. On interrupt, CPU stops the current task and transfer control to the special routine which is called - interrupt handler. An interrupt handler handles and interrupt and transfer control back to the previously stopped task. We can split interrupts on three types:
Software interrupts - when a software signals CPU that it needs kernel attention. These interrupts are generally used for system calls;
Hardware interrupts - when a hardware event happens, for example button is pressed on a keyboard;
Exceptions - interrupts generated by CPU, when the CPU detects error, for example division by zero or accessing a memory page which is not in RAM.
Every interrupt and exception is assigned a unique number which is called - vector number. Vector number can be any number from 0 to 255. There is common practice to use first 32 vector numbers for exceptions, and vector numbers from 32 to 255 are used for user-defined interrupts.
CPU uses vector number as an index in the Interrupt Descriptor Table (we will see description of it soon). CPU catches interrupts from the APIC or through its pins. Following table shows 0-31 exceptions:
To react on interrupt CPU uses special structure - Interrupt Descriptor Table or IDT. IDT is an array of 8-byte descriptors like Global Descriptor Table, but IDT entries are called gates. CPU multiplies vector number by 8 to find the IDT entry. But in 64-bit mode IDT is an array of 16-byte descriptors and CPU multiplies vector number by 16 to find the entry in the IDT. We remember from the previous part that CPU uses special GDTR register to locate Global Descriptor Table, so CPU uses special register IDTR for Interrupt Descriptor Table and lidt instruction for loading base address of the table into this register.
64-bit mode IDT entry has following structure:
Where:
Offset- is offset to entry point of an interrupt handler;DPL- Descriptor Privilege Level;P- Segment Present flag;Segment selector- a code segment selector in GDT or LDT (actually in linux, it must point to a valid descriptor in your GDT.)
IST- provides ability to switch to a new stack for interrupts handling.
And the last Type field describes type of the IDT entry. There are three different kinds of gates for interrupts:
Task gate
Interrupt gate
Trap gate
Interrupt and trap gates contain a far pointer to the entry point of the interrupt handler. Only one difference between these types is how CPU handles IF flag. If interrupt handler was accessed through interrupt gate, CPU clear the IF flag to prevent other interrupts while current interrupt handler executes. After that current interrupt handler executes, CPU sets the IF flag again with iret instruction.
Other bits in the interrupt descriptor is reserved and must be 0. Now let's look how CPU handles interrupts:
CPU save flags register,
CS, and instruction pointer on the stack.If interrupt causes an error code (like
#PFfor example), CPU saves an error on the stack after instruction pointer;After interrupt handler executes,
iretinstruction will be used to return from it.
Now let's back to code.
Fill and load IDT
We stopped at the following function:
idt_setup_early_handler is defined in the arch/x86/kernel/idt.c like the following:
where NUM_EXCEPTION_VECTORS expands to 32. As we can see, We're filling only first 32 IDT entries in the loop, because all of the early setup runs with interrupts disabled, so there is no need to set up interrupt handlers for vectors greater than 32. Here we call set_intr_gate in the loop, which takes two parameters:
Number of an interrupt or
vector number;Address of the idt handler.
and inserts an interrupt gate to the IDT table which is represented by the &idt_descr array.
The early_idt_handler_array array is declared in the arch/x86/include/asm/segment.h header file and contains addresses of the first 32 exception handlers:
The early_idt_handler_array is 288 bytes array which contains address of exception entry points every nine bytes. Every nine bytes of this array consist of two bytes optional instruction for pushing dummy error code if an exception does not provide it, two bytes instruction for pushing vector number to the stack and five bytes of jump to the common exception handler code. You will see more detail in the next paragraph.
The set_intr_gate function is defined in the arch/x86/kernel/idt.c source file and looks:
First of all it checks that passed vector number is not greater than 255 with BUG_ON macro. We need to do this because we are limited to have up to 256 interrupts. After this, we fill the idt data with the given arguments and others, which will be passed to idt_setup_from_table. The idt_setup_from_table function is defined in the same file as the set_intr_gate function like the following:
which fill temporary idt descriptor with the given arguments and others. And then we just copy it to the certain element of the idt_table array. idt_table is an array of idt entries:
Now we are moving back to main loop code. After main loop finishes, we can load Interrupt Descriptor table with the call of the:
where idt_descr is:
and load_idt just executes lidt instruction:
Okay, now we have filled and loaded Interrupt Descriptor Table, we know how the CPU acts during an interrupt. So now time to deal with interrupts handlers.
Early interrupts handlers
As you can read above, we filled IDT with the address of the early_idt_handler_array. In this section, we are going to look into it in detail. We can find it in the arch/x86/kernel/head_64.S assembly file:
We can see here, interrupt handlers generation for the first 32 exceptions. We check here, if exception has an error code then we do nothing, if exception does not return error code, we push zero to the stack. We do it for that stack was uniform. After that we push vector number on the stack and jump on the early_idt_handler_common which is generic interrupt handler for now. After all, every nine bytes of the early_idt_handler_array array consists of optional push of an error code, push of vector number and jump instruction to early_idt_handler_common. We can see it in the output of the objdump util:
As we may know, CPU pushes flag register, CS and RIP on the stack before calling interrupt handler. So before early_idt_handler_common will be executed, stack will contain following data:
Now let's look on the early_idt_handler_common implementation. It locates in the same arch/x86/kernel/head_64.S assembly file. First of all we increment early_recursion_flag to prevent recursion in the early_idt_handler_common:
The early_recursion_flag is defined in the same assembly file as the early_idt_handler_common symbol as follows:
Next we save general registers on the stack:
Okay, now the stack contains following data:
We need to do it to prevent wrong values of registers when we return from the interrupt handler. After this we check the vector number, and if it is #PF or Page Fault, we put value from the cr2 to the rdi register and call early_make_pgtable (we'll see it soon):
otherwise we call early_fixup_exception function by passing kernel stack pointer:
We'll see the implementation of the early_fixup_exception function later.
After we decrement the early_recursion_flag, we restore registers which we saved before from the stack and return from the handler with iretq.
It is the end of the interrupt handler. We will examine the page fault handling and the other exception handling in order.
Page fault handling
In the previous paragraph we saw the early interrupt handler which checks if the vector number is page fault and calls early_make_pgtable for building new page tables if it is. We need to have #PF handler in this step because there are plans to add ability to load kernel above 4G and make access to boot_params structure above the 4G.
You can find the implementation of early_make_pgtable in arch/x86/kernel/head64.c and takes one parameter - the value of cr2 register, which contains the address caused page fault. Let's look on it:
__PAGE_OFFSET is defined in the arch/x86/include/asm/page_64_types.h header file, and the suffix UL forces the page offset to be a unsigned long data type.
And the _AC macro is defined in the include/uapi/linux/const.h header file:
Where __PAGE_OFFSET expands to 0xffff888000000000. But, why is it possible to translate a virtual address to a physical address by subtracting __PAGE_OFFSET? The answer is in the Documentation/x86/x86_64/mm.rst documentation:
As explained above, the virtual address space ffff888000000000-ffffc87fffffffff is direct mapping of all physical memory. When the kernel wants to access all physical memory, it uses direct mapping.
Okay, let's get back to discussing early_make_pgtable. We initialize pmd and pass it to the __early_make_pgtable function along with address. The __early_make_pgtable function is defined in the same file as the early_make_pgtable function as follows:
It starts from the definition of some variables which have *val_t types. All of these types are declared as alias of unsigned long using typedef.
After we made the check that we have no invalid address, we're getting the address of the Page Global Directory entry which contains base address of Page Upper Directory and put its value to the pgd variable:
And we check if pgd is presented. If it is, we assign the base address of the page upper directory table to pud_p:
where PTE_PFN_MASK is a macro which mask lower 12 bits of (pte|pmd|pud|pgd)val_t.
If pgd is not presented, we check if next_early_pgt is not greater than EARLY_DYNAMIC_PAGE_TABLES which is 64 and present a fixed number of buffers to set up new page tables on demand. If next_early_pgt is greater than EARLY_DYNAMIC_PAGE_TABLES we reset page tables and start again from again label. If next_early_pgt is less than EARLY_DYNAMIC_PAGE_TABLES, we assign the next entry of early_dynamic_pgts to pud_p and fill whole entry of the page upper directory with 0, then fill the page global directory entry with the base address and some access rights:
And we fix pud_p to point to correct entry and assign its value to pud with the following:
And then we do the same routine as above, but to the page middle directory.
In the end we assign the given pmd which is passed by the early_make_pgtable function to the certain entry of page middle directory which maps kernel text+data virtual addresses:
After page fault handler finished its work, as a result, early_top_pgt contains entries which point to the valid addresses.
Other exception handling
In early interrupt phase, exceptions other than page fault are handled by early_fixup_exception function which is defined in arch/x86/mm/extable.c and takes two parameters - pointer to kernel stack which consists of saved registers and vector number:
First of all we need to make some checks as the following:
Here we just ignore NMI and make sure that we are not in recursive situation.
After that, we get into:
The fixup_exception function finds the actual handler and call it. It is defined in the same file as early_fixup_exception function as the following:
The ex_handler_t is a type of function pointer, which is defined like:
The search_exception_tables function looks up the given address in the exception table (i.e. the contents of the ELF section, __ex_table). After that, we get the actual address by ex_fixup_handler function. At last we call actual handler. For more information about exception table, you can refer to Documentation/x86/exception-tables.txt.
Let's get back to the early_fixup_exception function, the next step is:
The fixup_bug function is defined in arch/x86/kernel/traps.c. Let's have a look on the function implementation:
All what this function does is just returns 1 if the exception is generated because #UD (or Invalid Opcode) occurred and the report_bug function returns BUG_TRAP_TYPE_WARN, otherwise returns 0.
Conclusion
This is the end of the second part about Linux kernel insides. If you have questions or suggestions, ping me in twitter 0xAX, drop me email or just create issue. In the next part we will see all steps before kernel entry point - start_kernel function.
Please note that English is not my first language and I am really sorry for any inconvenience. If you found any mistakes please send me PR to linux-insides.
Links
Last updated