Size Matters: An Exploration of Virtual Memory on iOS
Brian T. Kelley
9 min read
An out-of-memory crash while debugging an iOS app led to investigating the iOS virtual memory system and finding the virtual address space size varies across iOS devices. Fixing the debugging workflow required use of the Extended Virtual Addressing entitlement to enable the full 64-bit address space.
I ran into an odd out-of-memory problem the other day when attempting to debug an iOS app on device. The app consistently crashed shortly after launch, preventing me from investigating the bug. To unblock myself, I learned a lot about the iOS virtual memory implementation and journaled my findings (including a fix!) here.
Background
The term virtual memory describes an abstraction between a process (executable, app, etc.) and the computer’s physical memory (RAM). This is a feature provided by an operating system in conjunction with the computer’s CPU (specifically its memory management unit, or MMU).
In a system with virtual memory, each process has its own memory address space that defines its valid logical (née virtual) memory addresses. When a process reads from or writes to a logical memory address, the logical memory address is translated by the MMU into a physical address. This is why the term virtual is used—a process’s memory address is not intrinsically related to any physical address on the machine.
A corollary of this abstraction is that two processes may have a pointer with the same value (a logical memory address), but each pointer may map to a different physical address. Similarly, two processes may have pointers with different values, but each maps to the same physical address (e.g. a shared library).
Virtual memory is divided into units called a page. A page is a contiguous address range, and each page has the same, fixed size. The size of a page varies by operating system and hardware. 4 KiB is common on 32-bit hardware and common sizes on 64-bit hardware include 4 KiB, 16 KiB, and 64 KiB.
Not all addresses in a virtual memory address space need map to a physical address. If a process accesses an address in an unmapped page, the MMU raises a page fault exception.
The page fault exception may be used by the operating system to implement paging, or the movement of data from disk into main memory (typically called a page in, or reading a page from disk into RAM). This technique enables operating systems and applications to use more memory than is physically available on the machine by leveraging the disk to store pages that the operating system wants to evict from RAM (typically called page out, or moving a page from RAM out to disk). The process of transferring data between RAM and disk is also known as swapping.
The operating system may also handle a page fault by terminating the process (i.e. a crash). On 32-bit Apple platforms, the first page (addresses [0x0000, 0x0FFF]) is not accessible by the process, which is why a NULL pointer deference (an access of address 0) crashes. On 64-bit Apple platforms, the entire 4 GiB 32-bit address space (addresses [0x00000000, 0xFFFFFFFF]) is not accessible by the process, which catches both NULL pointer dereference bugs and 64-bit to 32-bit pointer truncation bugs.
Virtual Memory on iOS
In many respects, iOS virtual memory is similar to virtual memory in many other operating systems, but there are a few notable unique aspects of the iOS implementation.
No Page Outs
Unlike macOS (and most operating systems with virtual memory), iOS does not page out dirty memory to disk. (The term dirty refers to memory that has been written to by the process.) iOS only discards read-only pages of a memory-mapped file, which can be paged in again as necessary. The main executable, shared libraries, and some types of resources are typically memory-mapped into the process address space and are therefore candidates for eviction from RAM if the system is low on physical memory.
64-bit Virtual Address Space
A surprising aspect (to me at least!) of the iOS 64-bit virtual memory system is that the size of the virtual address space depends on the amount of physical memory installed on the device. Consider the following excerpts from xnu-7195.141.2 (the operating system kernel):
osfmk/mach/shared_region.h lines 86-87:#define SHARED_REGION_BASE_ARM64 0x180000000ULL
#define SHARED_REGION_SIZE_ARM64 0x100000000ULL#define ARM64_MIN_MAX_ADDRESS (SHARED_REGION_BASE_ARM64 + SHARED_REGION_SIZE_ARM64 + 0x20000000) // end of shared region + 512MB for various purposes
const vm_map_offset_t min_max_offset = ARM64_MIN_MAX_ADDRESS; // end of shared region + 512MB for various purposes
if (arm64_pmap_max_offset_default) {
max_offset_ret = arm64_pmap_max_offset_default;
} else if (max_mem > 0xC0000000) {
max_offset_ret = min_max_offset + 0x138000000; // Max offset is 13.375GB for devices with > 3GB of memory
} else if (max_mem > 0x40000000) {
max_offset_ret = min_max_offset + 0x38000000; // Max offset is 9.375GB for devices with > 1GB and <= 3GB of memory
} else {
max_offset_ret = min_max_offset;
}Using the above information[1], we can calculate the size of the virtual memory address space for various iOS devices running iOS 12 or later. (Subtract 3 GiB for iOS 11 and earlier.)
| RAM | Address Space | Devices |
|---|---|---|
> 3 GiB |
15.375 GiB |
|
> 1 GiB |
11.375 GiB |
|
<= 1 GiB |
10.5 GiB |
|
Available Address Space
8 GiB[2] of the virtual address space is not available for use by the process. As mentioned earlier:
-
The first 4 GiB of the 64-bit virtual address space (which is also the entire 32-bit address space!) cannot be read from, written to, or executed by the process. The Mach-O executable file format designates this area as
PAGE_ZERO, and the kernel requiresPAGE_ZEROfor arm64 processes. -
The size of the shared region, for use by the system, is fixed to 4 GiB.
With this information, we can update the table to include the size of the virtual address space that’s usable by a process.
| RAM | Address Space | Usable | Devices |
|---|---|---|---|
> 3 GiB |
15.375 GiB |
7.375 GiB |
|
> 1 GiB |
11.375 GiB |
3.375 GiB |
|
<= 1 GiB |
10.5 GiB |
2.5 GiB |
|
Considerations for Development
Generally speaking, when a Mach-O file is loaded into a process, the entire file is mapped into the process’s address space. This is particularly interesting for debug builds, which typically contain a considerable amount of debug information embedded into the executable binary.
And finally, we circle back to the origin of this post! 🤣 I was attempting to debug a large app on an iPhone X, but it would crash shortly after launch with this rather cryptic error message:
can't allocate region :*** mach_vm_map(size=1048576, flags: 100) failed (error code=3) MyApp(1234,0x2d580000) malloc: *** set a breakpoint in malloc_error_break to debug warning: could not execute support code to read Objective-C class data in the process. This may reduce the quality of type information available.
Using Instruments, I observed the application executable and its libraries totaled over 3 GiB in size, leaving less than 370 MiB of address space for the heap and thread stacks. I couldn’t debug the app on device because the debug information used so much of the address space there was literally no address space available for use by the allocator!
Extended Virtual Addressing
New in iOS 14 is the Extended Virtual Addressing entitlement, whose plist key is com.apple.developer.kernel.extended-virtual-addressing. If a process has this entitlement, the kernel enables jumbo mode, which provides the process with access to the full 64-bit address space.
osfmk/arm/pmap.c lines 13426-13432:} else if (option == ARM_PMAP_MAX_OFFSET_JUMBO) {
if (arm64_pmap_max_offset_default) {
// Allow the boot-arg to override jumbo size
max_offset_ret = arm64_pmap_max_offset_default;
} else {
max_offset_ret = MACH_VM_MAX_ADDRESS; // Max offset is 64GB for pmaps with special "jumbo" blessing
}Because this entitlement only increases the size of the virtual address space, it’s primarily useful for applications that map in a large amount of read-only data.
Mapping in a lot of read-only data is exactly what my large debug build is doing, so I added the entitlement to the app’s entitlements file for development builds. The entitlement resolved the out-of-memory crash after launch when debugging on device! 🚀 🎉 🎊 With that out of the way, I was able investigate the bug that brought me down this fascinating detour.
SHARED_REGION_SIZE_ARM64 from 1 GiB to 4 GiB but only added 1 GiB to the values in the comments.