FerrousPath

Level 4 · Expert· Lesson 1 of 9 · 35 min

Memory layout and representation

Size, alignment and padding, niches, fat pointers, dynamically sized and zero-sized types, uninitialised memory, and the repr attributes.

To write and audit unsafe code you need an exact model of how values are represented. This lesson is that model.

Size and alignment

Every type has a size (bytes it occupies) and an alignment (its address must be a multiple of this power of two). Size is always a multiple of alignment, so arrays need no gaps between elements. Struct fields are separated by padding where needed.

use std::mem::{align_of, size_of};

struct A { a: u8, b: u32, c: u16 }

#[repr(C)]
struct B { a: u8, b: u32, c: u16 }

#[repr(C, packed)]
struct C { a: u8, b: u32, c: u16 }

#[repr(align(64))]
struct CacheLine([u8; 8]);

fn main() {
    println!("default  A: size {:2} align {}", size_of::<A>(), align_of::<A>());
    println!("repr(C)  B: size {:2} align {}", size_of::<B>(), align_of::<B>());
    println!("packed   C: size {:2} align {}", size_of::<C>(), align_of::<C>());
    println!("align(64)  : size {:2} align {}", size_of::<CacheLine>(), align_of::<CacheLine>());
}
  • Default (repr(Rust)): field order is unspecified. The compiler reorders to reduce padding (8 bytes for A) and may change its strategy between versions. Never rely on it.
  • repr(C): declaration order, C padding rules (12 bytes). Required for FFI and for any code that reasons about field offsets.
  • repr(packed): no padding, alignment 1. Fields may be misaligned, and creating a reference to a misaligned field is UB — read them by value or with &raw const and read_unaligned.
  • repr(align(N)): raises alignment, for example to keep two atomics on separate cache lines and avoid false sharing.
  • repr(transparent): a single-field wrapper guaranteed to have the layout and ABI of its field — the foundation of safe newtype casts.
  • repr(u8), repr(i32)… on an enum fixes the discriminant type.

Niches and enum layout

A niche is a bit pattern that is not a valid value of a type. The compiler stores enum discriminants in the payload’s niches when it can.

use std::mem::size_of;
use std::num::NonZeroU32;
use std::ptr::NonNull;

enum Three { A, B, C }

fn main() {
    println!("&u8                 {}", size_of::<&u8>());
    println!("Option<&u8>         {}", size_of::<Option<&u8>>());
    println!("Option<Box<u8>>     {}", size_of::<Option<Box<u8>>>());
    println!("Option<NonNull<u8>> {}", size_of::<Option<NonNull<u8>>>());
    println!("u32 / Option<u32>   {} / {}", size_of::<u32>(), size_of::<Option<u32>>());
    println!("Option<NonZeroU32>  {}", size_of::<Option<NonZeroU32>>());
    println!("bool / Option<bool> {} / {}", size_of::<bool>(), size_of::<Option<bool>>());
    println!("Option<Option<bool>> {}", size_of::<Option<Option<bool>>>());
    println!("Three / Option<Three> {} / {}", size_of::<Three>(), size_of::<Option<Three>>());
    println!("Option<Vec<u8>>     {} (same as Vec<u8>: {})", size_of::<Option<Vec<u8>>>(), size_of::<Vec<u8>>());
}

References and Box are never null; bool uses 2 of 256 patterns; char leaves the surrogate range and everything above 0x10FFFF. The null-pointer optimisation for Option<&T>, Option<Box<T>>, Option<NonNull<T>>, Option<NonZero…> and Option<fn()> is guaranteed; other niche uses are an optimisation you may observe but not rely on.

This is also why invalid values are UB: transmute::<u8, bool>(2) creates a value that an enclosing Option<bool> might read as None.

Fat pointers and dynamically sized types

A dynamically sized type (DST) has a size known only at runtime: [T], str, and dyn Trait. DSTs can only be used behind a pointer, and that pointer is fat — it carries metadata.

use std::mem::{size_of, size_of_val};

trait Speak { fn speak(&self) -> String; }
impl Speak for u8 { fn speak(&self) -> String { format!("u8 {self}") } }
impl Speak for [u64; 4] { fn speak(&self) -> String { "array".into() } }

fn main() {
    println!("&u8 {}  &[u8] {}  &str {}  &dyn Speak {}",
        size_of::<&u8>(), size_of::<&[u8]>(), size_of::<&str>(), size_of::<&dyn Speak>());

    let slice: &[u16] = &[1, 2, 3];
    let a: &dyn Speak = &7u8;
    let b: &dyn Speak = &[0u64; 4];
    // size_of_val reads the metadata: length for slices, the vtable for trait objects.
    println!("{} {} {}", size_of_val(slice), size_of_val(a), size_of_val(b));
    println!("{} / {}", a.speak(), b.speak());
}

Every generic parameter has an implicit T: Sized bound. T: ?Sized removes it, which is how Box<T>, &T and Rc<T> accept str and dyn Trait. A struct may have a DST as its last field, making the struct itself a DST.

Zero-sized types

Types with size 0 — (), struct Marker;, [T; 0], PhantomData<T> — are real types with exactly one value and no runtime representation. The compiler removes all operations on them.

use std::collections::HashMap;
use std::mem::size_of;

struct Token;

fn main() {
    println!("{} {} {}", size_of::<()>(), size_of::<Token>(), size_of::<[u64; 0]>());

    // HashSet<K> is literally HashMap<K, ()>: the values occupy nothing.
    let mut set: HashMap<&str, ()> = HashMap::new();
    set.insert("a", ());

    // A Vec of ZSTs never allocates; it is just a counter.
    let mut v = Vec::new();
    for _ in 0..1_000_000 { v.push(Token); }
    println!("len {} capacity {}", v.len(), v.capacity());
}

Unsafe code must handle ZSTs deliberately: pointer arithmetic on them is a no-op, allocation of zero bytes is not permitted through alloc, and a “pointer to a ZST” is any non-null, aligned address — NonNull::dangling().

Uninitialised memory

Memory that has not been written is not “random bits”. It is a distinct state, and reading it at a type that requires initialised data — integers, references, bool — is undefined behaviour. MaybeUninit<T> is the type for storage that may not yet hold a valid T:

use std::mem::MaybeUninit;

fn squares<const N: usize>() -> [u64; N] {
    let mut out: [MaybeUninit<u64>; N] = [const { MaybeUninit::uninit() }; N];
    for (i, slot) in out.iter_mut().enumerate() {
        slot.write((i * i) as u64); // write does not read or drop the old contents
    }
    // SAFETY: every element was initialised by the loop above.
    out.map(|slot| unsafe { slot.assume_init() })
}

fn main() {
    println!("{:?}", squares::<6>());
}

Rules: write through MaybeUninit::write or a raw pointer ptr.write(v), never through *ptr = v (assignment drops the “old value”, which is garbage). Padding bytes are always uninitialised, which is why you cannot view an arbitrary struct as &[u8].

Validity versus safety invariants

Two different kinds of rule apply to a type:

  • The validity invariant must hold whenever a value of the type exists, even momentarily inside unsafe code. Violating it is immediate UB. Examples: a bool is 0 or 1; a reference is non-null, aligned and dereferenceable; a Box is non-null.
  • The safety invariant is what safe code may assume. Unsafe code may break it temporarily inside a module, provided safe code never observes the breakage. Examples: a String’s bytes are UTF-8; a Vec’s len is at most its capacity.

A Vec with len > capacity is a safety violation that becomes UB only when some later operation acts on it. A &T that is null is UB the moment it is created.

Transmutation and type punning

mem::transmute reinterprets the bits of one type as another of the same size. It is the most dangerous function in the language; nearly every use has a better alternative:

Goal Use
Integer ↔ float bits f32::to_bits, f32::from_bits
Integer ↔ bytes to_le_bytes, from_be_bytes…
Pointer ↔ integer ptr.addr(), ptr.expose_provenance(), with_addr
Slice of plain data ↔ bytes the bytemuck or zerocopy crates, which check the requirements
Wrapper ↔ inner type #[repr(transparent)] and a pointer cast

Exercises

  1. Without running it, compute the size and alignment of #[repr(C)] struct S { a: u8, b: u64, c: u8, d: u32, e: u16 }. Reorder the fields to minimise the size and verify with size_of.
  2. Design a #[repr(u8)] enum for a three-state value and an Option of it. What sizes do you get? Can you find the niche value with transmute to u8? (Print it; do not rely on it.)
  3. Write fn as_bytes<T: Copy>(x: &T) -> &[u8]. Explain why it is unsound for (u8, u32) and name the property T would need.
  4. Implement a fixed-capacity ArrayVec<T, const N: usize> backed by [MaybeUninit<T>; N] with push, pop, and a correct Drop.

Check your understanding

Pick an answer to see the explanation.

  1. 1.Why is size_of::<Option<&T>>() equal to size_of::<&T>()?

  2. 2.What does a &[T] consist of at runtime?

  3. 3.Is let x: u32 = unsafe { MaybeUninit::uninit().assume_init() }; undefined behaviour?

Key terms in this lesson

Finished the lesson and the exercises?