Arm MMU in linux: page table Initialization

This post covers details of MMU initialization code of linux 2.6.11 kernel for ARM and few internal tweaks used for mapping arm page tables to x86 style page tables which linux memory management code expects. We'll cover only architecutre specific details, generic MM code of kernel will not be covered here. For all the architecutre specific details we take ARM 11 (Architecture V6) as our reference. 

Following conventions are used in this writeup:

• All details are for uniprocessor system, nothing related to SMP is covered here.

Kernel's execution starts at stext in (arch/arm/kernel/head.S) , at this point kernel expects that MMU is off, I-cache is off, D-cache is off. After looking up of processor type and machine number, we call__create_page_tables which sets up the initial MMU tables for which are used only during initialization of MMU. Here we create MMU mapping tables for 4MB of memory starting from kernel's text (More preciesly it starts at a 1MB section in which kernel's text starts). Following is the code snippet which creates these mappings: 

-----------------------------

_create_page_tables: 

1. ldr r5, [r8, #MACHINFO_PHYSRAM] @ physram
2. pgtbl r4, r5 @ page table address
3. mov r0, r4
4. mov r3, #0
5. add r6, r0, #0x4000
6. 1: str r3, [r0], #4
7. str r3, [r0], #4
8. str r3, [r0], #4
9. str r3, [r0], #4
10. teq r0, r6
11. bne 1b
12. ldr r7, [r10, #PROCINFO_MMUFLAGS] @ mmuflags
13. mov r6, pc, lsr #20 @ start of kernel section
14. orr r3, r7, r6, lsl #20 @ flags + kernel base
15. str r3, [r4, r6, lsl #2] @ identity mapping
16. add r0, r4, #(TEXTADDR & 0xff000000) >> 18 @ start of kernel
17. str r3, [r0, #(TEXTADDR & 0x00f00000) >> 18]!
18. add r3, r3, #1 <<>
19. str r3, [r0, #4]! @ KERNEL + 1MB
20. add r3, r3, #1 <<>
21. str r3, [r0, #4]! @ KERNEL + 2MB
22. add r3, r3, #1 <<>
23. str r3, [r0, #4] @ KERNEL + 3MB

-----------------------------

In this function we first of all find out the physical address where our RAM starts, and locate the address where we'll store our inital MMU tables (line #1 and #2). We create these inital MMU tables 16kb below kernel entry point. Line #12- #15 create an identity mapping of 1MB starting from kernel entry point (physical address of kernel entry, i.estext). Here we do'nt create a second level page table for these mapping, instead of that we specify in first level descriptor that these mappings are for section (each section mapping is of size 1MB). Simiarly we create mapping for 4 more sections (these 4 are non identity mappings, size of each section is 1MB here also) starting at TEXTADDR (virtual address of kernel entry point)


so the initial memory map looks something line this:
After the initial page tables are setup, next step is to enable MMU. This code is tricky, does lot of deep magic. 

----------------------------

1. ldr r13, __switch_data @ address to jump to after mmu has been enabled
2. adr lr, __enable_mmu @ return (PIC) address
3. add pc, r10, #PROCINFO_INITFUNC
4. 
   .type __switch_data, %object


5. __switch_data: .long __mmap_switched
6. .long __data_loc @ r4
7. .long __data_start @ r5
8. .long __bss_start @ r6
9. .long _end @ r7
10. .long processor_id @ r4
11. .long __machine_arch_type @ r5
12. .long cr_alignment @ r6
13. .long init_thread_union+8192 @ sp

-----------------------------

line #1 puts virtual address of __mmap_switched in r13, after enabling MMU kernel will jump to address in r13. The virtual address that is used here is not from identity mapping, but it is PAGE_OFFSET + physical address of __mmap_switched. And now since in__mmap_switched we start refering to virtual addresses of variables and functions, so starting from __mmap_switched is no longer position independent.

At line #2-#3, we put position independent address of__enable_mmu ('adr' Psuedo instruction translates to PC relative addressing, that's why it is position independent) and jumps to at a offset of PROCINFO_INITFUNC (12 bytes) in __v6_proc_info structure (arch/arm/mm/proc-v6.S). At PROCINFO_INITFUNC in__v6_proc_info we have a branch to __v6_setup, which does following setup for enabling MMU:

• Clean and Invalidate D-cache and Icache, and invalidate TLB.
• Prepare the control register1 (C1) value that needs to be written when enabling mmu, and return the value which needs to be written in C1.

As __v6_setup returns we enter __enable_mmu, which just sets Cache enable bits, branch-prediction enable bits in 'r' which is the value to be written in C1 (value to be written in C1 was returned in 'r0' by __v6_setup) and then calls __turn_mmu_on.

--------------------------------------

__turn_mmu_on:

1. mov r0, r0
2. mcr p15, 0, r0, c1, c0, 0 @ write control reg
3. mrc p15, 0, r3, c0, c0, 0 @ read id reg
4. mov r3, r3
5. mov r3, r3
6. mov pc, r13

--------------------------------------

__turn_mmu_on just writes 'r0' to C1 to enable MMU. Line #4 and #5 are the nops to make sure that pipeline does not contain and invalid address access when C1 is written. Line #6 'r13' is moved in 'pc' and we enter __mmap_switched. Now MMU is ON, every address is virtual no physical addresses anymore. But the final kernel page tables are still not setup (final page tables will be setup bypaging_init and mappings created by __create_page_tables will be discarded), we are still running with 4Mb mapping that__create_page_tables had set it for us. __mmap_switched copies the data segment if required, clears the BSS and calls start_kernel.

The mappings page table mappings that we discussed above makes sure that the position dependent code of kernel startup runs peacefully, and these mappings are overwritten at later stages by a function called paging_init( ) ( start_kernel( ) -> setup_arch( )-> paging_init( )).

paging_init() populates the master L1 page table (init_mm) with linear mappings of complete SDRAM and the SOC specific address space (SOC specific memory mappings are created by mdesc->mapio()function, this function pointer is initialized by SOS specific code, arch/arm/mach-*). So in master L1 page table (init_mm) we have mappings which map virtual addresses in range PAGE_OFFSET - (PAGE_OFFSET + sdram size) to physical address of SDRAM start - (physical address of SDRAM start + sdram size). Also we have SOC specific mappings created by mdesc->mapio() function.

One more point worth noting here is that whenever a new process is created, a new L1 page table is allocated for it, and the kernel mappings (sdram mapping, SOC specific mappings) are copied to it from the master L1 page table (init_mm). Every process has its own user space mappings so no need to copy anything from anywhere.

Handling of mapings for VMALLOC REGION are is bit tricky, because VMALLOC virtual addressed are allocated when a process callsvmalloc( ). So if we have some processes' which were created before the process which called vmalloc then their L1 page tables will have no maping for new vmalloc'ed region. So how this is taken care of is very simple, the mappings for vmalloc'ed region are updated in the master L1 page table (init_mm) and when a process whose page tables do not have newly created mapping accesses the newly vmalloc'ed region, a page fault is generated. And kernel handles page faults in VMALLOC region specially by copying the mappings for newly vmalloc'ed area to page tables of the process which generated the fault.

Tweaks for integerating ARM 2 level page tables with linux implementation of 3 level ix86 style page tables

1. Linux assumes that it is dealing with 3 level page tables and ARM has 2 level page tables. For handling this, for ARM in ARCH include files __pmd is defined as a identity macro in (include/asm-arm/page.h):

-------------------------------------------------

#define __pmd(x) ((pmd_t) { (x) } )

--------------------------------------------------

So effectively for ARM, linux is told that pmd has just one entry, effectively bypassing pmd.

2. Memory management code of in Linux expects ix86 type page table entries, for example it uses 'P' (present ), 'D' (dirty) bits but ARM page table entries (PTEs) don't have these bits. As a workaround to provide ix86 PTE flags, what ARM page table implementation does is that, it tells linux that PGD has 2048 entries of 8 bytes each (whereas ARM hardware level 1 pagetable has 4096 entries of 4 bytes each).

Also it tells Linux that each PTE table has 512 entries (whereas ARM hardware PTE table of 256 entries).This means that the PTE table that is exposed to Linux is actually 2 ARM PTE tables, arranged contigously in memory. NowAfter these 2 ARM PTE tables (lets say h/w PTE table 1 and h/w PTE table 2), 2 more PTE tables (256 entries each, say linux PTE table 1 and 2) are allocated in memory contiguous to Arm hardware page table 2. Linux PTE table 1 and 2 contains PTE flags of ix86 style corresponding to entries in ARM PTE table 1 and 2. So whenever Linux needs ix86 style PTE flags it uses entries in Linux PTE table 1 and 2. ARM never looks into Linux PTE table 1 and 2 during hardware page table walk, it uses only h/w PTE tables as mentioned above. Refer to include/asm-arm/pgtable.h(line: 20-76) for details of how ARM h/w PTE table 1 and 2, and linux PTE table 1 and 2 are organized in memory.

So here we conclude the architecture specific page table related stuff for ARM.