overview.rst

  1.. SPDX-License-Identifier: GPL-2.0
  2
  3========
  4Overview
  5========
  6
  7This section presents the configuration process of a CXL Type-3 memory device,
  8and how it is ultimately exposed to users as either a :code:`DAX` device or
  9normal memory pages via the kernel's page allocator.
 10
 11Portions marked with a bullet are points at which certain kernel objects
 12are generated.
 13
 141) Early Boot
 15
 16  a) BIOS, Build, and Boot Parameters
 17
 18    i) EFI_MEMORY_SP
 19    ii) CONFIG_EFI_SOFT_RESERVE
 20    iii) CONFIG_MHP_DEFAULT_ONLINE_TYPE
 21    iv) nosoftreserve
 22
 23  b) Memory Map Creation
 24
 25    i) EFI Memory Map / E820 Consulted for Soft-Reserved
 26
 27      * CXL Memory is set aside to be handled by the CXL driver
 28
 29      * Soft-Reserved IO Resource created for CFMWS entry
 30
 31  c) NUMA Node Creation
 32
 33    * Nodes created from ACPI CEDT CFMWS and SRAT Proximity domains (PXM)
 34
 35  d) Memory Tier Creation
 36
 37    * A default memory_tier is created with all nodes.
 38
 39  e) Contiguous Memory Allocation
 40
 41    * Any requested CMA is allocated from Online nodes
 42
 43  f) Init Finishes, Drivers start probing
 44
 452) ACPI and PCI Drivers
 46
 47  a) Detects PCI device is CXL, marking it for probe by CXL driver
 48
 493) CXL Driver Operation
 50
 51  a) Base device creation
 52
 53    * root, port, and memdev devices created
 54    * CEDT CFMWS IO Resource creation
 55
 56  b) Decoder creation
 57
 58    * root, switch, and endpoint decoders created
 59
 60  c) Logical device creation
 61
 62    * memory_region and endpoint devices created
 63
 64  d) Devices are associated with each other
 65
 66    * If auto-decoder (BIOS-programmed decoders), driver validates
 67      configurations, builds associations, and locks configs at probe time.
 68
 69    * If user-configured, validation and associations are built at
 70      decoder-commit time.
 71
 72  e) Regions surfaced as DAX region
 73
 74    * dax_region created
 75
 76    * DAX device created via DAX driver
 77
 784) DAX Driver Operation
 79
 80  a) DAX driver surfaces DAX region as one of two dax device modes
 81
 82    * kmem - dax device is converted to hotplug memory blocks
 83
 84      * DAX kmem IO Resource creation
 85
 86    * hmem - dax device is left as daxdev to be accessed as a file.
 87
 88      * If hmem, journey ends here.
 89
 90  b) DAX kmem surfaces memory region to Memory Hotplug to add to page
 91     allocator as "driver managed memory"
 92
 935) Memory Hotplug
 94
 95  a) mhp component surfaces a dax device memory region as multiple memory
 96     blocks to the page allocator
 97
 98    * blocks appear in :code:`/sys/bus/memory/devices` and linked to a NUMA node
 99
100  b) blocks are onlined into the requested zone (NORMAL or MOVABLE)
101
102    * Memory is marked "Driver Managed" to avoid kexec from using it as region
103      for kernel updates