FILE
xfs-online-fsck-design.rst
File preview with syntax highlighting.
linux-6.19.14/Documentation/filesystems/xfs/xfs-online-fsck-design.rst
1.. SPDX-License-Identifier: GPL-2.0 2.. _xfs_online_fsck_design: 3 4.. 5 Mapping of heading styles within this document: 6 Heading 1 uses "====" above and below 7 Heading 2 uses "====" 8 Heading 3 uses "----" 9 Heading 4 uses "````" 10 Heading 5 uses "^^^^" 11 Heading 6 uses "~~~~" 12 Heading 7 uses "...." 13 14 Sections are manually numbered because apparently that's what everyone 15 does in the kernel. 16 17====================== 18XFS Online Fsck Design 19====================== 20 21This document captures the design of the online filesystem check feature for 22XFS. 23The purpose of this document is threefold: 24 25- To help kernel distributors understand exactly what the XFS online fsck 26 feature is, and issues about which they should be aware. 27 28- To help people reading the code to familiarize themselves with the relevant 29 concepts and design points before they start digging into the code. 30 31- To help developers maintaining the system by capturing the reasons 32 supporting higher level decision making. 33 34As the online fsck code is merged, the links in this document to topic branches 35will be replaced with links to code. 36 37This document is licensed under the terms of the GNU Public License, v2. 38The primary author is Darrick J. Wong. 39 40This design document is split into seven parts. 41Part 1 defines what fsck tools are and the motivations for writing a new one. 42Parts 2 and 3 present a high level overview of how online fsck process works 43and how it is tested to ensure correct functionality. 44Part 4 discusses the user interface and the intended usage modes of the new 45program. 46Parts 5 and 6 show off the high level components and how they fit together, and 47then present case studies of how each repair function actually works. 48Part 7 sums up what has been discussed so far and speculates about what else 49might be built atop online fsck. 50 51.. contents:: Table of Contents 52 :local: 53 541. What is a Filesystem Check? 55============================== 56 57A Unix filesystem has four main responsibilities: 58 59- Provide a hierarchy of names through which application programs can associate 60 arbitrary blobs of data for any length of time, 61 62- Virtualize physical storage media across those names, and 63 64- Retrieve the named data blobs at any time. 65 66- Examine resource usage. 67 68Metadata directly supporting these functions (e.g. files, directories, space 69mappings) are sometimes called primary metadata. 70Secondary metadata (e.g. reverse mapping and directory parent pointers) support 71operations internal to the filesystem, such as internal consistency checking 72and reorganization. 73Summary metadata, as the name implies, condense information contained in 74primary metadata for performance reasons. 75 76The filesystem check (fsck) tool examines all the metadata in a filesystem 77to look for errors. 78In addition to looking for obvious metadata corruptions, fsck also 79cross-references different types of metadata records with each other to look 80for inconsistencies. 81People do not like losing data, so most fsck tools also contains some ability 82to correct any problems found. 83As a word of caution -- the primary goal of most Linux fsck tools is to restore 84the filesystem metadata to a consistent state, not to maximize the data 85recovered. 86That precedent will not be challenged here. 87 88Filesystems of the 20th century generally lacked any redundancy in the ondisk 89format, which means that fsck can only respond to errors by erasing files until 90errors are no longer detected. 91More recent filesystem designs contain enough redundancy in their metadata that 92it is now possible to regenerate data structures when non-catastrophic errors 93occur; this capability aids both strategies. 94 95+--------------------------------------------------------------------------+ 96| **Note**: | 97+--------------------------------------------------------------------------+ 98| System administrators avoid data loss by increasing the number of | 99| separate storage systems through the creation of backups; and they avoid | 100| downtime by increasing the redundancy of each storage system through the | 101| creation of RAID arrays. | 102| fsck tools address only the first problem. | 103+--------------------------------------------------------------------------+ 104 105TLDR; Show Me the Code! 106----------------------- 107 108Kernel and userspace code has been fully merged as of October 2025. 109 110Each kernel patchset adding an online repair function will use the same branch 111name across the kernel, xfsprogs, and fstests git repos. 112 113Existing Tools 114-------------- 115 116The online fsck tool described here will be the third tool in the history of 117XFS (on Linux) to check and repair filesystems. 118Two programs precede it: 119 120The first program, ``xfs_check``, was created as part of the XFS debugger 121(``xfs_db``) and can only be used with unmounted filesystems. 122It walks all metadata in the filesystem looking for inconsistencies in the 123metadata, though it lacks any ability to repair what it finds. 124Due to its high memory requirements and inability to repair things, this 125program is now deprecated and will not be discussed further. 126 127The second program, ``xfs_repair``, was created to be faster and more robust 128than the first program. 129Like its predecessor, it can only be used with unmounted filesystems. 130It uses extent-based in-memory data structures to reduce memory consumption, 131and tries to schedule readahead IO appropriately to reduce I/O waiting time 132while it scans the metadata of the entire filesystem. 133The most important feature of this tool is its ability to respond to 134inconsistencies in file metadata and directory tree by erasing things as needed 135to eliminate problems. 136Space usage metadata are rebuilt from the observed file metadata. 137 138Problem Statement 139----------------- 140 141The current XFS tools leave several problems unsolved: 142 1431. **User programs** suddenly **lose access** to the filesystem when unexpected 144 shutdowns occur as a result of silent corruptions in the metadata. 145 These occur **unpredictably** and often without warning. 146 1472. **Users** experience a **total loss of service** during the recovery period 148 after an **unexpected shutdown** occurs. 149 1503. **Users** experience a **total loss of service** if the filesystem is taken 151 offline to **look for problems** proactively. 152 1534. **Data owners** cannot **check the integrity** of their stored data without 154 reading all of it. 155 This may expose them to substantial billing costs when a linear media scan 156 performed by the storage system administrator might suffice. 157 1585. **System administrators** cannot **schedule** a maintenance window to deal 159 with corruptions if they **lack the means** to assess filesystem health 160 while the filesystem is online. 161 1626. **Fleet monitoring tools** cannot **automate periodic checks** of filesystem 163 health when doing so requires **manual intervention** and downtime. 164 1657. **Users** can be tricked into **doing things they do not desire** when 166 malicious actors **exploit quirks of Unicode** to place misleading names 167 in directories. 168 169Given this definition of the problems to be solved and the actors who would 170benefit, the proposed solution is a third fsck tool that acts on a running 171filesystem. 172 173This new third program has three components: an in-kernel facility to check 174metadata, an in-kernel facility to repair metadata, and a userspace driver 175program to drive fsck activity on a live filesystem. 176``xfs_scrub`` is the name of the driver program. 177The rest of this document presents the goals and use cases of the new fsck 178tool, describes its major design points in connection to those goals, and 179discusses the similarities and differences with existing tools. 180 181+--------------------------------------------------------------------------+ 182| **Note**: | 183+--------------------------------------------------------------------------+ 184| Throughout this document, the existing offline fsck tool can also be | 185| referred to by its current name "``xfs_repair``". | 186| The userspace driver program for the new online fsck tool can be | 187| referred to as "``xfs_scrub``". | 188| The kernel portion of online fsck that validates metadata is called | 189| "online scrub", and portion of the kernel that fixes metadata is called | 190| "online repair". | 191+--------------------------------------------------------------------------+ 192 193The naming hierarchy is broken up into objects known as directories and files 194and the physical space is split into pieces known as allocation groups. 195Sharding enables better performance on highly parallel systems and helps to 196contain the damage when corruptions occur. 197The division of the filesystem into principal objects (allocation groups and 198inodes) means that there are ample opportunities to perform targeted checks and 199repairs on a subset of the filesystem. 200 201While this is going on, other parts continue processing IO requests. 202Even if a piece of filesystem metadata can only be regenerated by scanning the 203entire system, the scan can still be done in the background while other file 204operations continue. 205 206In summary, online fsck takes advantage of resource sharding and redundant 207metadata to enable targeted checking and repair operations while the system 208is running. 209This capability will be coupled to automatic system management so that 210autonomous self-healing of XFS maximizes service availability. 211 2122. Theory of Operation 213====================== 214 215Because it is necessary for online fsck to lock and scan live metadata objects, 216online fsck consists of three separate code components. 217The first is the userspace driver program ``xfs_scrub``, which is responsible 218for identifying individual metadata items, scheduling work items for them, 219reacting to the outcomes appropriately, and reporting results to the system 220administrator. 221The second and third are in the kernel, which implements functions to check 222and repair each type of online fsck work item. 223 224+------------------------------------------------------------------+ 225| **Note**: | 226+------------------------------------------------------------------+ 227| For brevity, this document shortens the phrase "online fsck work | 228| item" to "scrub item". | 229+------------------------------------------------------------------+ 230 231Scrub item types are delineated in a manner consistent with the Unix design 232philosophy, which is to say that each item should handle one aspect of a 233metadata structure, and handle it well. 234 235Scope 236----- 237 238In principle, online fsck should be able to check and to repair everything that 239the offline fsck program can handle. 240However, online fsck cannot be running 100% of the time, which means that 241latent errors may creep in after a scrub completes. 242If these errors cause the next mount to fail, offline fsck is the only 243solution. 244This limitation means that maintenance of the offline fsck tool will continue. 245A second limitation of online fsck is that it must follow the same resource 246sharing and lock acquisition rules as the regular filesystem. 247This means that scrub cannot take *any* shortcuts to save time, because doing 248so could lead to concurrency problems. 249In other words, online fsck is not a complete replacement for offline fsck, and 250a complete run of online fsck may take longer than offline fsck. 251However, both of these limitations are acceptable tradeoffs to satisfy the 252different motivations of online fsck, which are to **minimize system downtime** 253and to **increase predictability of operation**. 254 255.. _scrubphases: 256 257Phases of Work 258-------------- 259 260The userspace driver program ``xfs_scrub`` splits the work of checking and 261repairing an entire filesystem into seven phases. 262Each phase concentrates on checking specific types of scrub items and depends 263on the success of all previous phases. 264The seven phases are as follows: 265 2661. Collect geometry information about the mounted filesystem and computer, 267 discover the online fsck capabilities of the kernel, and open the 268 underlying storage devices. 269 2702. Check allocation group metadata, all realtime volume metadata, and all quota 271 files. 272 Each metadata structure is scheduled as a separate scrub item. 273 If corruption is found in the inode header or inode btree and ``xfs_scrub`` 274 is permitted to perform repairs, then those scrub items are repaired to 275 prepare for phase 3. 276 Repairs are implemented by using the information in the scrub item to 277 resubmit the kernel scrub call with the repair flag enabled; this is 278 discussed in the next section. 279 Optimizations and all other repairs are deferred to phase 4. 280 2813. Check all metadata of every file in the filesystem. 282 Each metadata structure is also scheduled as a separate scrub item. 283 If repairs are needed and ``xfs_scrub`` is permitted to perform repairs, 284 and there were no problems detected during phase 2, then those scrub items 285 are repaired immediately. 286 Optimizations, deferred repairs, and unsuccessful repairs are deferred to 287 phase 4. 288 2894. All remaining repairs and scheduled optimizations are performed during this 290 phase, if the caller permits them. 291 Before starting repairs, the summary counters are checked and any necessary 292 repairs are performed so that subsequent repairs will not fail the resource 293 reservation step due to wildly incorrect summary counters. 294 Unsuccessful repairs are requeued as long as forward progress on repairs is 295 made somewhere in the filesystem. 296 Free space in the filesystem is trimmed at the end of phase 4 if the 297 filesystem is clean. 298 2995. By the start of this phase, all primary and secondary filesystem metadata 300 must be correct. 301 Summary counters such as the free space counts and quota resource counts 302 are checked and corrected. 303 Directory entry names and extended attribute names are checked for 304 suspicious entries such as control characters or confusing Unicode sequences 305 appearing in names. 306 3076. If the caller asks for a media scan, read all allocated and written data 308 file extents in the filesystem. 309 The ability to use hardware-assisted data file integrity checking is new 310 to online fsck; neither of the previous tools have this capability. 311 If media errors occur, they will be mapped to the owning files and reported. 312 3137. Re-check the summary counters and presents the caller with a summary of 314 space usage and file counts. 315 316This allocation of responsibilities will be :ref:`revisited <scrubcheck>` 317later in this document. 318 319Steps for Each Scrub Item 320------------------------- 321 322The kernel scrub code uses a three-step strategy for checking and repairing 323the one aspect of a metadata object represented by a scrub item: 324 3251. The scrub item of interest is checked for corruptions; opportunities for 326 optimization; and for values that are directly controlled by the system 327 administrator but look suspicious. 328 If the item is not corrupt or does not need optimization, resource are 329 released and the positive scan results are returned to userspace. 330 If the item is corrupt or could be optimized but the caller does not permit 331 this, resources are released and the negative scan results are returned to 332 userspace. 333 Otherwise, the kernel moves on to the second step. 334 3352. The repair function is called to rebuild the data structure. 336 Repair functions generally choose rebuild a structure from other metadata 337 rather than try to salvage the existing structure. 338 If the repair fails, the scan results from the first step are returned to 339 userspace. 340 Otherwise, the kernel moves on to the third step. 341 3423. In the third step, the kernel runs the same checks over the new metadata 343 item to assess the efficacy of the repairs. 344 The results of the reassessment are returned to userspace. 345 346Classification of Metadata 347-------------------------- 348 349Each type of metadata object (and therefore each type of scrub item) is 350classified as follows: 351 352Primary Metadata 353```````````````` 354 355Metadata structures in this category should be most familiar to filesystem 356users either because they are directly created by the user or they index 357objects created by the user 358Most filesystem objects fall into this class: 359 360- Free space and reference count information 361 362- Inode records and indexes 363 364- Storage mapping information for file data 365 366- Directories 367 368- Extended attributes 369 370- Symbolic links 371 372- Quota limits 373 374Scrub obeys the same rules as regular filesystem accesses for resource and lock 375acquisition. 376 377Primary metadata objects are the simplest for scrub to process. 378The principal filesystem object (either an allocation group or an inode) that 379owns the item being scrubbed is locked to guard against concurrent updates. 380The check function examines every record associated with the type for obvious 381errors and cross-references healthy records against other metadata to look for 382inconsistencies. 383Repairs for this class of scrub item are simple, since the repair function 384starts by holding all the resources acquired in the previous step. 385The repair function scans available metadata as needed to record all the 386observations needed to complete the structure. 387Next, it stages the observations in a new ondisk structure and commits it 388atomically to complete the repair. 389Finally, the storage from the old data structure are carefully reaped. 390 391Because ``xfs_scrub`` locks a primary object for the duration of the repair, 392this is effectively an offline repair operation performed on a subset of the 393filesystem. 394This minimizes the complexity of the repair code because it is not necessary to 395handle concurrent updates from other threads, nor is it necessary to access 396any other part of the filesystem. 397As a result, indexed structures can be rebuilt very quickly, and programs 398trying to access the damaged structure will be blocked until repairs complete. 399The only infrastructure needed by the repair code are the staging area for 400observations and a means to write new structures to disk. 401Despite these limitations, the advantage that online repair holds is clear: 402targeted work on individual shards of the filesystem avoids total loss of 403service. 404 405This mechanism is described in section 2.1 ("Off-Line Algorithm") of 406V. Srinivasan and M. J. Carey, `"Performance of On-Line Index Construction 407Algorithms" <https://minds.wisconsin.edu/bitstream/handle/1793/59524/TR1047.pdf>`_, 408*Extending Database Technology*, pp. 293-309, 1992. 409 410Most primary metadata repair functions stage their intermediate results in an 411in-memory array prior to formatting the new ondisk structure, which is very 412similar to the list-based algorithm discussed in section 2.3 ("List-Based 413Algorithms") of Srinivasan. 414However, any data structure builder that maintains a resource lock for the 415duration of the repair is *always* an offline algorithm. 416 417.. _secondary_metadata: 418 419Secondary Metadata 420`````````````````` 421 422Metadata structures in this category reflect records found in primary metadata, 423but are only needed for online fsck or for reorganization of the filesystem. 424 425Secondary metadata include: 426 427- Reverse mapping information 428 429- Directory parent pointers 430 431This class of metadata is difficult for scrub to process because scrub attaches 432to the secondary object but needs to check primary metadata, which runs counter 433to the usual order of resource acquisition. 434Frequently, this means that full filesystems scans are necessary to rebuild the 435metadata. 436Check functions can be limited in scope to reduce runtime. 437Repairs, however, require a full scan of primary metadata, which can take a 438long time to complete. 439Under these conditions, ``xfs_scrub`` cannot lock resources for the entire 440duration of the repair. 441 442Instead, repair functions set up an in-memory staging structure to store 443observations. 444Depending on the requirements of the specific repair function, the staging 445index will either have the same format as the ondisk structure or a design 446specific to that repair function. 447The next step is to release all locks and start the filesystem scan. 448When the repair scanner needs to record an observation, the staging data are 449locked long enough to apply the update. 450While the filesystem scan is in progress, the repair function hooks the 451filesystem so that it can apply pending filesystem updates to the staging 452information. 453Once the scan is done, the owning object is re-locked, the live data is used to 454write a new ondisk structure, and the repairs are committed atomically. 455The hooks are disabled and the staging area is freed. 456Finally, the storage from the old data structure are carefully reaped. 457 458Introducing concurrency helps online repair avoid various locking problems, but 459comes at a high cost to code complexity. 460Live filesystem code has to be hooked so that the repair function can observe 461updates in progress. 462The staging area has to become a fully functional parallel structure so that 463updates can be merged from the hooks. 464Finally, the hook, the filesystem scan, and the inode locking model must be 465sufficiently well integrated that a hook event can decide if a given update 466should be applied to the staging structure. 467 468In theory, the scrub implementation could apply these same techniques for 469primary metadata, but doing so would make it massively more complex and less 470performant. 471Programs attempting to access the damaged structures are not blocked from 472operation, which may cause application failure or an unplanned filesystem 473shutdown. 474 475Inspiration for the secondary metadata repair strategy was drawn from section 4762.4 of Srinivasan above, and sections 2 ("NSF: Index Build Without Side-File") 477and 3.1.1 ("Duplicate Key Insert Problem") in C. Mohan, `"Algorithms for 478Creating Indexes for Very Large Tables Without Quiescing Updates" 479<https://dl.acm.org/doi/10.1145/130283.130337>`_, 1992. 480 481The sidecar index mentioned above bears some resemblance to the side file 482method mentioned in Srinivasan and Mohan. 483Their method consists of an index builder that extracts relevant record data to 484build the new structure as quickly as possible; and an auxiliary structure that 485captures all updates that would be committed to the index by other threads were 486the new index already online. 487After the index building scan finishes, the updates recorded in the side file 488are applied to the new index. 489To avoid conflicts between the index builder and other writer threads, the 490builder maintains a publicly visible cursor that tracks the progress of the 491scan through the record space. 492To avoid duplication of work between the side file and the index builder, side 493file updates are elided when the record ID for the update is greater than the 494cursor position within the record ID space. 495 496To minimize changes to the rest of the codebase, XFS online repair keeps the 497replacement index hidden until it's completely ready to go. 498In other words, there is no attempt to expose the keyspace of the new index 499while repair is running. 500The complexity of such an approach would be very high and perhaps more 501appropriate to building *new* indices. 502 503**Future Work Question**: Can the full scan and live update code used to 504facilitate a repair also be used to implement a comprehensive check? 505 506*Answer*: In theory, yes. Check would be much stronger if each scrub function 507employed these live scans to build a shadow copy of the metadata and then 508compared the shadow records to the ondisk records. 509However, doing that is a fair amount more work than what the checking functions 510do now. 511The live scans and hooks were developed much later. 512That in turn increases the runtime of those scrub functions. 513 514Summary Information 515``````````````````` 516 517Metadata structures in this last category summarize the contents of primary 518metadata records. 519These are often used to speed up resource usage queries, and are many times 520smaller than the primary metadata which they represent. 521 522Examples of summary information include: 523 524- Summary counts of free space and inodes 525 526- File link counts from directories 527 528- Quota resource usage counts 529 530Check and repair require full filesystem scans, but resource and lock 531acquisition follow the same paths as regular filesystem accesses. 532 533The superblock summary counters have special requirements due to the underlying 534implementation of the incore counters, and will be treated separately. 535Check and repair of the other types of summary counters (quota resource counts 536and file link counts) employ the same filesystem scanning and hooking 537techniques as outlined above, but because the underlying data are sets of 538integer counters, the staging data need not be a fully functional mirror of the 539ondisk structure. 540 541Inspiration for quota and file link count repair strategies were drawn from 542sections 2.12 ("Online Index Operations") through 2.14 ("Incremental View 543Maintenance") of G. Graefe, `"Concurrent Queries and Updates in Summary Views 544and Their Indexes" 545<http://www.odbms.org/wp-content/uploads/2014/06/Increment-locks.pdf>`_, 2011. 546 547Since quotas are non-negative integer counts of resource usage, online 548quotacheck can use the incremental view deltas described in section 2.14 to 549track pending changes to the block and inode usage counts in each transaction, 550and commit those changes to a dquot side file when the transaction commits. 551Delta tracking is necessary for dquots because the index builder scans inodes, 552whereas the data structure being rebuilt is an index of dquots. 553Link count checking combines the view deltas and commit step into one because 554it sets attributes of the objects being scanned instead of writing them to a 555separate data structure. 556Each online fsck function will be discussed as case studies later in this 557document. 558 559Risk Management 560--------------- 561 562During the development of online fsck, several risk factors were identified 563that may make the feature unsuitable for certain distributors and users. 564Steps can be taken to mitigate or eliminate those risks, though at a cost to 565functionality. 566 567- **Decreased performance**: Adding metadata indices to the filesystem 568 increases the time cost of persisting changes to disk, and the reverse space 569 mapping and directory parent pointers are no exception. 570 System administrators who require the maximum performance can disable the 571 reverse mapping features at format time, though this choice dramatically 572 reduces the ability of online fsck to find inconsistencies and repair them. 573 574- **Incorrect repairs**: As with all software, there might be defects in the 575 software that result in incorrect repairs being written to the filesystem. 576 Systematic fuzz testing (detailed in the next section) is employed by the 577 authors to find bugs early, but it might not catch everything. 578 The kernel build system provides Kconfig options (``CONFIG_XFS_ONLINE_SCRUB`` 579 and ``CONFIG_XFS_ONLINE_REPAIR``) to enable distributors to choose not to 580 accept this risk. 581 The xfsprogs build system has a configure option (``--enable-scrub=no``) that 582 disables building of the ``xfs_scrub`` binary, though this is not a risk 583 mitigation if the kernel functionality remains enabled. 584 585- **Inability to repair**: Sometimes, a filesystem is too badly damaged to be 586 repairable. 587 If the keyspaces of several metadata indices overlap in some manner but a 588 coherent narrative cannot be formed from records collected, then the repair 589 fails. 590 To reduce the chance that a repair will fail with a dirty transaction and 591 render the filesystem unusable, the online repair functions have been 592 designed to stage and validate all new records before committing the new 593 structure. 594 595- **Misbehavior**: Online fsck requires many privileges -- raw IO to block 596 devices, opening files by handle, ignoring Unix discretionary access control, 597 and the ability to perform administrative changes. 598 Running this automatically in the background scares people, so the systemd 599 background service is configured to run with only the privileges required. 600 Obviously, this cannot address certain problems like the kernel crashing or 601 deadlocking, but it should be sufficient to prevent the scrub process from 602 escaping and reconfiguring the system. 603 The cron job does not have this protection. 604 605- **Fuzz Kiddiez**: There are many people now who seem to think that running 606 automated fuzz testing of ondisk artifacts to find mischievous behavior and 607 spraying exploit code onto the public mailing list for instant zero-day 608 disclosure is somehow of some social benefit. 609 In the view of this author, the benefit is realized only when the fuzz 610 operators help to **fix** the flaws, but this opinion apparently is not 611 widely shared among security "researchers". 612 The XFS maintainers' continuing ability to manage these events presents an 613 ongoing risk to the stability of the development process. 614 Automated testing should front-load some of the risk while the feature is 615 considered EXPERIMENTAL. 616 617Many of these risks are inherent to software programming. 618Despite this, it is hoped that this new functionality will prove useful in 619reducing unexpected downtime. 620 6213. Testing Plan 622=============== 623 624As stated before, fsck tools have three main goals: 625 6261. Detect inconsistencies in the metadata; 627 6282. Eliminate those inconsistencies; and 629 6303. Minimize further loss of data. 631 632Demonstrations of correct operation are necessary to build users' confidence 633that the software behaves within expectations. 634Unfortunately, it was not really feasible to perform regular exhaustive testing 635of every aspect of a fsck tool until the introduction of low-cost virtual 636machines with high-IOPS storage. 637With ample hardware availability in mind, the testing strategy for the online 638fsck project involves differential analysis against the existing fsck tools and 639systematic testing of every attribute of every type of metadata object. 640Testing can be split into four major categories, as discussed below. 641 642Integrated Testing with fstests 643------------------------------- 644 645The primary goal of any free software QA effort is to make testing as 646inexpensive and widespread as possible to maximize the scaling advantages of 647community. 648In other words, testing should maximize the breadth of filesystem configuration 649scenarios and hardware setups. 650This improves code quality by enabling the authors of online fsck to find and 651fix bugs early, and helps developers of new features to find integration 652issues earlier in their development effort. 653 654The Linux filesystem community shares a common QA testing suite, 655`fstests <https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/>`_, for 656functional and regression testing. 657Even before development work began on online fsck, fstests (when run on XFS) 658would run both the ``xfs_check`` and ``xfs_repair -n`` commands on the test and 659scratch filesystems between each test. 660This provides a level of assurance that the kernel and the fsck tools stay in 661alignment about what constitutes consistent metadata. 662During development of the online checking code, fstests was modified to run 663``xfs_scrub -n`` between each test to ensure that the new checking code 664produces the same results as the two existing fsck tools. 665 666To start development of online repair, fstests was modified to run 667``xfs_repair`` to rebuild the filesystem's metadata indices between tests. 668This ensures that offline repair does not crash, leave a corrupt filesystem 669after it exists, or trigger complaints from the online check. 670This also established a baseline for what can and cannot be repaired offline. 671To complete the first phase of development of online repair, fstests was 672modified to be able to run ``xfs_scrub`` in a "force rebuild" mode. 673This enables a comparison of the effectiveness of online repair as compared to 674the existing offline repair tools. 675 676General Fuzz Testing of Metadata Blocks 677--------------------------------------- 678 679XFS benefits greatly from having a very robust debugging tool, ``xfs_db``. 680 681Before development of online fsck even began, a set of fstests were created 682to test the rather common fault that entire metadata blocks get corrupted. 683This required the creation of fstests library code that can create a filesystem 684containing every possible type of metadata object. 685Next, individual test cases were created to create a test filesystem, identify 686a single block of a specific type of metadata object, trash it with the 687existing ``blocktrash`` command in ``xfs_db``, and test the reaction of a 688particular metadata validation strategy. 689 690This earlier test suite enabled XFS developers to test the ability of the 691in-kernel validation functions and the ability of the offline fsck tool to 692detect and eliminate the inconsistent metadata. 693This part of the test suite was extended to cover online fsck in exactly the 694same manner. 695 696In other words, for a given fstests filesystem configuration: 697 698* For each metadata object existing on the filesystem: 699 700 * Write garbage to it 701 702 * Test the reactions of: 703 704 1. The kernel verifiers to stop obviously bad metadata 705 2. Offline repair (``xfs_repair``) to detect and fix 706 3. Online repair (``xfs_scrub``) to detect and fix 707 708Targeted Fuzz Testing of Metadata Records 709----------------------------------------- 710 711The testing plan for online fsck includes extending the existing fs testing 712infrastructure to provide a much more powerful facility: targeted fuzz testing 713of every metadata field of every metadata object in the filesystem. 714``xfs_db`` can modify every field of every metadata structure in every 715block in the filesystem to simulate the effects of memory corruption and 716software bugs. 717Given that fstests already contains the ability to create a filesystem 718containing every metadata format known to the filesystem, ``xfs_db`` can be 719used to perform exhaustive fuzz testing! 720 721For a given fstests filesystem configuration: 722 723* For each metadata object existing on the filesystem... 724 725 * For each record inside that metadata object... 726 727 * For each field inside that record... 728 729 * For each conceivable type of transformation that can be applied to a bit field... 730 731 1. Clear all bits 732 2. Set all bits 733 3. Toggle the most significant bit 734 4. Toggle the middle bit 735 5. Toggle the least significant bit 736 6. Add a small quantity 737 7. Subtract a small quantity 738 8. Randomize the contents 739 740 * ...test the reactions of: 741 742 1. The kernel verifiers to stop obviously bad metadata 743 2. Offline checking (``xfs_repair -n``) 744 3. Offline repair (``xfs_repair``) 745 4. Online checking (``xfs_scrub -n``) 746 5. Online repair (``xfs_scrub``) 747 6. Both repair tools (``xfs_scrub`` and then ``xfs_repair`` if online repair doesn't succeed) 748 749This is quite the combinatoric explosion! 750 751Fortunately, having this much test coverage makes it easy for XFS developers to 752check the responses of XFS' fsck tools. 753Since the introduction of the fuzz testing framework, these tests have been 754used to discover incorrect repair code and missing functionality for entire 755classes of metadata objects in ``xfs_repair``. 756The enhanced testing was used to finalize the deprecation of ``xfs_check`` by 757confirming that ``xfs_repair`` could detect at least as many corruptions as 758the older tool. 759 760These tests have been very valuable for ``xfs_scrub`` in the same ways -- they 761allow the online fsck developers to compare online fsck against offline fsck, 762and they enable XFS developers to find deficiencies in the code base. 763 764Proposed patchsets include 765`fuzzing baselines 766<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fuzz-baseline>`_. 767 768Stress Testing 769-------------- 770 771A unique requirement to online fsck is the ability to operate on a filesystem 772concurrently with regular workloads. 773Although it is of course impossible to run ``xfs_scrub`` with *zero* observable 774impact on the running system, the online repair code should never introduce 775inconsistencies into the filesystem metadata, and regular workloads should 776never notice resource starvation. 777To verify that these conditions are being met, fstests has been enhanced in 778the following ways: 779 780* For each scrub item type, create a test to exercise checking that item type 781 while running ``fsstress``. 782* For each scrub item type, create a test to exercise repairing that item type 783 while running ``fsstress``. 784* Race ``fsstress`` and ``xfs_scrub -n`` to ensure that checking the whole 785 filesystem doesn't cause problems. 786* Race ``fsstress`` and ``xfs_scrub`` in force-rebuild mode to ensure that 787 force-repairing the whole filesystem doesn't cause problems. 788* Race ``xfs_scrub`` in check and force-repair mode against ``fsstress`` while 789 freezing and thawing the filesystem. 790* Race ``xfs_scrub`` in check and force-repair mode against ``fsstress`` while 791 remounting the filesystem read-only and read-write. 792* The same, but running ``fsx`` instead of ``fsstress``. (Not done yet?) 793 794Success is defined by the ability to run all of these tests without observing 795any unexpected filesystem shutdowns due to corrupted metadata, kernel hang 796check warnings, or any other sort of mischief. 797 7984. User Interface 799================= 800 801The primary user of online fsck is the system administrator, just like offline 802repair. 803Online fsck presents two modes of operation to administrators: 804A foreground CLI process for online fsck on demand, and a background service 805that performs autonomous checking and repair. 806 807Checking on Demand 808------------------ 809 810For administrators who want the absolute freshest information about the 811metadata in a filesystem, ``xfs_scrub`` can be run as a foreground process on 812a command line. 813The program checks every piece of metadata in the filesystem while the 814administrator waits for the results to be reported, just like the existing 815``xfs_repair`` tool. 816Both tools share a ``-n`` option to perform a read-only scan, and a ``-v`` 817option to increase the verbosity of the information reported. 818 819A new feature of ``xfs_scrub`` is the ``-x`` option, which employs the error 820correction capabilities of the hardware to check data file contents. 821The media scan is not enabled by default because it may dramatically increase 822program runtime and consume a lot of bandwidth on older storage hardware. 823 824The output of a foreground invocation is captured in the system log. 825 826The ``xfs_scrub_all`` program walks the list of mounted filesystems and 827initiates ``xfs_scrub`` for each of them in parallel. 828It serializes scans for any filesystems that resolve to the same top level 829kernel block device to prevent resource overconsumption. 830 831Background Service 832------------------ 833 834To reduce the workload of system administrators, the ``xfs_scrub`` package 835provides a suite of `systemd <https://systemd.io/>`_ timers and services that 836run online fsck automatically on weekends by default. 837The background service configures scrub to run with as little privilege as 838possible, the lowest CPU and IO priority, and in a CPU-constrained single 839threaded mode. 840This can be tuned by the systemd administrator at any time to suit the latency 841and throughput requirements of customer workloads. 842 843The output of the background service is also captured in the system log. 844If desired, reports of failures (either due to inconsistencies or mere runtime 845errors) can be emailed automatically by setting the ``EMAIL_ADDR`` environment 846variable in the following service files: 847 848* ``xfs_scrub_fail@.service`` 849* ``xfs_scrub_media_fail@.service`` 850* ``xfs_scrub_all_fail.service`` 851 852The decision to enable the background scan is left to the system administrator. 853This can be done by enabling either of the following services: 854 855* ``xfs_scrub_all.timer`` on systemd systems 856* ``xfs_scrub_all.cron`` on non-systemd systems 857 858This automatic weekly scan is configured out of the box to perform an 859additional media scan of all file data once per month. 860This is less foolproof than, say, storing file data block checksums, but much 861more performant if application software provides its own integrity checking, 862redundancy can be provided elsewhere above the filesystem, or the storage 863device's integrity guarantees are deemed sufficient. 864 865The systemd unit file definitions have been subjected to a security audit 866(as of systemd 249) to ensure that the xfs_scrub processes have as little 867access to the rest of the system as possible. 868This was performed via ``systemd-analyze security``, after which privileges 869were restricted to the minimum required, sandboxing was set up to the maximal 870extent possible with sandboxing and system call filtering; and access to the 871filesystem tree was restricted to the minimum needed to start the program and 872access the filesystem being scanned. 873The service definition files restrict CPU usage to 80% of one CPU core, and 874apply as nice of a priority to IO and CPU scheduling as possible. 875This measure was taken to minimize delays in the rest of the filesystem. 876No such hardening has been performed for the cron job. 877 878Health Reporting 879---------------- 880 881XFS caches a summary of each filesystem's health status in memory. 882The information is updated whenever ``xfs_scrub`` is run, or whenever 883inconsistencies are detected in the filesystem metadata during regular 884operations. 885System administrators should use the ``health`` command of ``xfs_spaceman`` to 886download this information into a human-readable format. 887If problems have been observed, the administrator can schedule a reduced 888service window to run the online repair tool to correct the problem. 889Failing that, the administrator can decide to schedule a maintenance window to 890run the traditional offline repair tool to correct the problem. 891 892**Future Work Question**: Should the health reporting integrate with the new 893inotify fs error notification system? 894Would it be helpful for sysadmins to have a daemon to listen for corruption 895notifications and initiate a repair? 896 897*Answer*: These questions remain unanswered, but should be a part of the 898conversation with early adopters and potential downstream users of XFS. 899 9005. Kernel Algorithms and Data Structures 901======================================== 902 903This section discusses the key algorithms and data structures of the kernel 904code that provide the ability to check and repair metadata while the system 905is running. 906The first chapters in this section reveal the pieces that provide the 907foundation for checking metadata. 908The remainder of this section presents the mechanisms through which XFS 909regenerates itself. 910 911Self Describing Metadata 912------------------------ 913 914Starting with XFS version 5 in 2012, XFS updated the format of nearly every 915ondisk block header to record a magic number, a checksum, a universally 916"unique" identifier (UUID), an owner code, the ondisk address of the block, 917and a log sequence number. 918When loading a block buffer from disk, the magic number, UUID, owner, and 919ondisk address confirm that the retrieved block matches the specific owner of 920the current filesystem, and that the information contained in the block is 921supposed to be found at the ondisk address. 922The first three components enable checking tools to disregard alleged metadata 923that doesn't belong to the filesystem, and the fourth component enables the 924filesystem to detect lost writes. 925 926Whenever a file system operation modifies a block, the change is submitted 927to the log as part of a transaction. 928The log then processes these transactions marking them done once they are 929safely persisted to storage. 930The logging code maintains the checksum and the log sequence number of the last 931transactional update. 932Checksums are useful for detecting torn writes and other discrepancies that can 933be introduced between the computer and its storage devices. 934Sequence number tracking enables log recovery to avoid applying out of date 935log updates to the filesystem. 936 937These two features improve overall runtime resiliency by providing a means for 938the filesystem to detect obvious corruption when reading metadata blocks from 939disk, but these buffer verifiers cannot provide any consistency checking 940between metadata structures. 941 942For more information, please see the documentation for 943Documentation/filesystems/xfs/xfs-self-describing-metadata.rst 944 945Reverse Mapping 946--------------- 947 948The original design of XFS (circa 1993) is an improvement upon 1980s Unix 949filesystem design. 950In those days, storage density was expensive, CPU time was scarce, and 951excessive seek time could kill performance. 952For performance reasons, filesystem authors were reluctant to add redundancy to 953the filesystem, even at the cost of data integrity. 954Filesystems designers in the early 21st century choose different strategies to 955increase internal redundancy -- either storing nearly identical copies of 956metadata, or more space-efficient encoding techniques. 957 958For XFS, a different redundancy strategy was chosen to modernize the design: 959a secondary space usage index that maps allocated disk extents back to their 960owners. 961By adding a new index, the filesystem retains most of its ability to scale 962well to heavily threaded workloads involving large datasets, since the primary 963file metadata (the directory tree, the file block map, and the allocation 964groups) remain unchanged. 965Like any system that improves redundancy, the reverse-mapping feature increases 966overhead costs for space mapping activities. 967However, it has two critical advantages: first, the reverse index is key to 968enabling online fsck and other requested functionality such as free space 969defragmentation, better media failure reporting, and filesystem shrinking. 970Second, the different ondisk storage format of the reverse mapping btree 971defeats device-level deduplication because the filesystem requires real 972redundancy. 973 974+--------------------------------------------------------------------------+ 975| **Sidebar**: | 976+--------------------------------------------------------------------------+ 977| A criticism of adding the secondary index is that it does nothing to | 978| improve the robustness of user data storage itself. | 979| This is a valid point, but adding a new index for file data block | 980| checksums increases write amplification by turning data overwrites into | 981| copy-writes, which age the filesystem prematurely. | 982| In keeping with thirty years of precedent, users who want file data | 983| integrity can supply as powerful a solution as they require. | 984| As for metadata, the complexity of adding a new secondary index of space | 985| usage is much less than adding volume management and storage device | 986| mirroring to XFS itself. | 987| Perfection of RAID and volume management are best left to existing | 988| layers in the kernel. | 989+--------------------------------------------------------------------------+ 990 991The information captured in a reverse space mapping record is as follows: 992 993.. code-block:: c 994 995 struct xfs_rmap_irec { 996 xfs_agblock_t rm_startblock; /* extent start block */ 997 xfs_extlen_t rm_blockcount; /* extent length */ 998 uint64_t rm_owner; /* extent owner */ 999 uint64_t rm_offset; /* offset within the owner */ 1000 unsigned int rm_flags; /* state flags */ 1001 }; 1002 1003The first two fields capture the location and size of the physical space, 1004in units of filesystem blocks. 1005The owner field tells scrub which metadata structure or file inode have been 1006assigned this space. 1007For space allocated to files, the offset field tells scrub where the space was 1008mapped within the file fork. 1009Finally, the flags field provides extra information about the space usage -- 1010is this an attribute fork extent? A file mapping btree extent? Or an 1011unwritten data extent? 1012 1013Online filesystem checking judges the consistency of each primary metadata 1014record by comparing its information against all other space indices. 1015The reverse mapping index plays a key role in the consistency checking process 1016because it contains a centralized alternate copy of all space allocation 1017information. 1018Program runtime and ease of resource acquisition are the only real limits to 1019what online checking can consult. 1020For example, a file data extent mapping can be checked against: 1021 1022* The absence of an entry in the free space information. 1023* The absence of an entry in the inode index. 1024* The absence of an entry in the reference count data if the file is not 1025 marked as having shared extents. 1026* The correspondence of an entry in the reverse mapping information. 1027 1028There are several observations to make about reverse mapping indices: 1029 10301. Reverse mappings can provide a positive affirmation of correctness if any of 1031 the above primary metadata are in doubt. 1032 The checking code for most primary metadata follows a path similar to the 1033 one outlined above. 1034 10352. Proving the consistency of secondary metadata with the primary metadata is 1036 difficult because that requires a full scan of all primary space metadata, 1037 which is very time intensive. 1038 For example, checking a reverse mapping record for a file extent mapping 1039 btree block requires locking the file and searching the entire btree to 1040 confirm the block. 1041 Instead, scrub relies on rigorous cross-referencing during the primary space 1042 mapping structure checks. 1043 10443. Consistency scans must use non-blocking lock acquisition primitives if the 1045 required locking order is not the same order used by regular filesystem 1046 operations. 1047 For example, if the filesystem normally takes a file ILOCK before taking 1048 the AGF buffer lock but scrub wants to take a file ILOCK while holding 1049 an AGF buffer lock, scrub cannot block on that second acquisition. 1050 This means that forward progress during this part of a scan of the reverse 1051 mapping data cannot be guaranteed if system load is heavy. 1052 1053In summary, reverse mappings play a key role in reconstruction of primary 1054metadata. 1055The details of how these records are staged, written to disk, and committed 1056into the filesystem are covered in subsequent sections. 1057 1058Checking and Cross-Referencing 1059------------------------------ 1060 1061The first step of checking a metadata structure is to examine every record 1062contained within the structure and its relationship with the rest of the 1063system. 1064XFS contains multiple layers of checking to try to prevent inconsistent 1065metadata from wreaking havoc on the system. 1066Each of these layers contributes information that helps the kernel to make 1067three decisions about the health of a metadata structure: 1068 1069- Is a part of this structure obviously corrupt (``XFS_SCRUB_OFLAG_CORRUPT``) ? 1070- Is this structure inconsistent with the rest of the system 1071 (``XFS_SCRUB_OFLAG_XCORRUPT``) ? 1072- Is there so much damage around the filesystem that cross-referencing is not 1073 possible (``XFS_SCRUB_OFLAG_XFAIL``) ? 1074- Can the structure be optimized to improve performance or reduce the size of 1075 metadata (``XFS_SCRUB_OFLAG_PREEN``) ? 1076- Does the structure contain data that is not inconsistent but deserves review 1077 by the system administrator (``XFS_SCRUB_OFLAG_WARNING``) ? 1078 1079The following sections describe how the metadata scrubbing process works. 1080 1081Metadata Buffer Verification 1082```````````````````````````` 1083 1084The lowest layer of metadata protection in XFS are the metadata verifiers built 1085into the buffer cache. 1086These functions perform inexpensive internal consistency checking of the block 1087itself, and answer these questions: 1088 1089- Does the block belong to this filesystem? 1090 1091- Does the block belong to the structure that asked for the read? 1092 This assumes that metadata blocks only have one owner, which is always true 1093 in XFS. 1094 1095- Is the type of data stored in the block within a reasonable range of what 1096 scrub is expecting? 1097 1098- Does the physical location of the block match the location it was read from? 1099 1100- Does the block checksum match the data? 1101 1102The scope of the protections here are very limited -- verifiers can only 1103establish that the filesystem code is reasonably free of gross corruption bugs 1104and that the storage system is reasonably competent at retrieval. 1105Corruption problems observed at runtime cause the generation of health reports, 1106failed system calls, and in the extreme case, filesystem shutdowns if the 1107corrupt metadata force the cancellation of a dirty transaction. 1108 1109Every online fsck scrubbing function is expected to read every ondisk metadata 1110block of a structure in the course of checking the structure. 1111Corruption problems observed during a check are immediately reported to 1112userspace as corruption; during a cross-reference, they are reported as a 1113failure to cross-reference once the full examination is complete. 1114Reads satisfied by a buffer already in cache (and hence already verified) 1115bypass these checks. 1116 1117Internal Consistency Checks 1118``````````````````````````` 1119 1120After the buffer cache, the next level of metadata protection is the internal 1121record verification code built into the filesystem. 1122These checks are split between the buffer verifiers, the in-filesystem users of 1123the buffer cache, and the scrub code itself, depending on the amount of higher 1124level context required. 1125The scope of checking is still internal to the block. 1126These higher level checking functions answer these questions: 1127 1128- Does the type of data stored in the block match what scrub is expecting? 1129 1130- Does the block belong to the owning structure that asked for the read? 1131 1132- If the block contains records, do the records fit within the block? 1133 1134- If the block tracks internal free space information, is it consistent with 1135 the record areas? 1136 1137- Are the records contained inside the block free of obvious corruptions? 1138 1139Record checks in this category are more rigorous and more time-intensive. 1140For example, block pointers and inumbers are checked to ensure that they point 1141within the dynamically allocated parts of an allocation group and within 1142the filesystem. 1143Names are checked for invalid characters, and flags are checked for invalid 1144combinations. 1145Other record attributes are checked for sensible values. 1146Btree records spanning an interval of the btree keyspace are checked for 1147correct order and lack of mergeability (except for file fork mappings). 1148For performance reasons, regular code may skip some of these checks unless 1149debugging is enabled or a write is about to occur. 1150Scrub functions, of course, must check all possible problems. 1151 1152Validation of Userspace-Controlled Record Attributes 1153```````````````````````````````````````````````````` 1154 1155Various pieces of filesystem metadata are directly controlled by userspace. 1156Because of this nature, validation work cannot be more precise than checking 1157that a value is within the possible range. 1158These fields include: 1159 1160- Superblock fields controlled by mount options 1161- Filesystem labels 1162- File timestamps 1163- File permissions 1164- File size 1165- File flags 1166- Names present in directory entries, extended attribute keys, and filesystem 1167 labels 1168- Extended attribute key namespaces 1169- Extended attribute values 1170- File data block contents 1171- Quota limits 1172- Quota timer expiration (if resource usage exceeds the soft limit) 1173 1174Cross-Referencing Space Metadata 1175```````````````````````````````` 1176 1177After internal block checks, the next higher level of checking is 1178cross-referencing records between metadata structures. 1179For regular runtime code, the cost of these checks is considered to be 1180prohibitively expensive, but as scrub is dedicated to rooting out 1181inconsistencies, it must pursue all avenues of inquiry. 1182The exact set of cross-referencing is highly dependent on the context of the 1183data structure being checked. 1184 1185The XFS btree code has keyspace scanning functions that online fsck uses to 1186cross reference one structure with another. 1187Specifically, scrub can scan the key space of an index to determine if that 1188keyspace is fully, sparsely, or not at all mapped to records. 1189For the reverse mapping btree, it is possible to mask parts of the key for the 1190purposes of performing a keyspace scan so that scrub can decide if the rmap 1191btree contains records mapping a certain extent of physical space without the 1192sparsenses of the rest of the rmap keyspace getting in the way. 1193 1194Btree blocks undergo the following checks before cross-referencing: 1195 1196- Does the type of data stored in the block match what scrub is expecting? 1197 1198- Does the block belong to the owning structure that asked for the read? 1199 1200- Do the records fit within the block? 1201 1202- Are the records contained inside the block free of obvious corruptions? 1203 1204- Are the name hashes in the correct order? 1205 1206- Do node pointers within the btree point to valid block addresses for the type 1207 of btree? 1208 1209- Do child pointers point towards the leaves? 1210 1211- Do sibling pointers point across the same level? 1212 1213- For each node block record, does the record key accurate reflect the contents 1214 of the child block? 1215 1216Space allocation records are cross-referenced as follows: 1217 12181. Any space mentioned by any metadata structure are cross-referenced as 1219 follows: 1220 1221 - Does the reverse mapping index list only the appropriate owner as the 1222 owner of each block? 1223 1224 - Are none of the blocks claimed as free space? 1225 1226 - If these aren't file data blocks, are none of the blocks claimed as space 1227 shared by different owners? 1228 12292. Btree blocks are cross-referenced as follows: 1230 1231 - Everything in class 1 above. 1232 1233 - If there's a parent node block, do the keys listed for this block match the 1234 keyspace of this block? 1235 1236 - Do the sibling pointers point to valid blocks? Of the same level? 1237 1238 - Do the child pointers point to valid blocks? Of the next level down? 1239 12403. Free space btree records are cross-referenced as follows: 1241 1242 - Everything in class 1 and 2 above. 1243 1244 - Does the reverse mapping index list no owners of this space? 1245 1246 - Is this space not claimed by the inode index for inodes? 1247 1248 - Is it not mentioned by the reference count index? 1249 1250 - Is there a matching record in the other free space btree? 1251 12524. Inode btree records are cross-referenced as follows: 1253 1254 - Everything in class 1 and 2 above. 1255 1256 - Is there a matching record in free inode btree? 1257 1258 - Do cleared bits in the holemask correspond with inode clusters? 1259 1260 - Do set bits in the freemask correspond with inode records with zero link 1261 count? 1262 12635. Inode records are cross-referenced as follows: 1264 1265 - Everything in class 1. 1266 1267 - Do all the fields that summarize information about the file forks actually 1268 match those forks? 1269 1270 - Does each inode with zero link count correspond to a record in the free 1271 inode btree? 1272 12736. File fork space mapping records are cross-referenced as follows: 1274 1275 - Everything in class 1 and 2 above. 1276 1277 - Is this space not mentioned by the inode btrees? 1278 1279 - If this is a CoW fork mapping, does it correspond to a CoW entry in the 1280 reference count btree? 1281 12827. Reference count records are cross-referenced as follows: 1283 1284 - Everything in class 1 and 2 above. 1285 1286 - Within the space subkeyspace of the rmap btree (that is to say, all 1287 records mapped to a particular space extent and ignoring the owner info), 1288 are there the same number of reverse mapping records for each block as the 1289 reference count record claims? 1290 1291Checking Extended Attributes 1292```````````````````````````` 1293 1294Extended attributes implement a key-value store that enable fragments of data 1295to be attached to any file. 1296Both the kernel and userspace can access the keys and values, subject to 1297namespace and privilege restrictions. 1298Most typically these fragments are metadata about the file -- origins, security 1299contexts, user-supplied labels, indexing information, etc. 1300 1301Names can be as long as 255 bytes and can exist in several different 1302namespaces. 1303Values can be as large as 64KB. 1304A file's extended attributes are stored in blocks mapped by the attr fork. 1305The mappings point to leaf blocks, remote value blocks, or dabtree blocks. 1306Block 0 in the attribute fork is always the top of the structure, but otherwise 1307each of the three types of blocks can be found at any offset in the attr fork. 1308Leaf blocks contain attribute key records that point to the name and the value. 1309Names are always stored elsewhere in the same leaf block. 1310Values that are less than 3/4 the size of a filesystem block are also stored 1311elsewhere in the same leaf block. 1312Remote value blocks contain values that are too large to fit inside a leaf. 1313If the leaf information exceeds a single filesystem block, a dabtree (also 1314rooted at block 0) is created to map hashes of the attribute names to leaf 1315blocks in the attr fork. 1316 1317Checking an extended attribute structure is not so straightforward due to the 1318lack of separation between attr blocks and index blocks. 1319Scrub must read each block mapped by the attr fork and ignore the non-leaf 1320blocks: 1321 13221. Walk the dabtree in the attr fork (if present) to ensure that there are no 1323 irregularities in the blocks or dabtree mappings that do not point to 1324 attr leaf blocks. 1325 13262. Walk the blocks of the attr fork looking for leaf blocks. 1327 For each entry inside a leaf: 1328 1329 a. Validate that the name does not contain invalid characters. 1330 1331 b. Read the attr value. 1332 This performs a named lookup of the attr name to ensure the correctness 1333 of the dabtree. 1334 If the value is stored in a remote block, this also validates the 1335 integrity of the remote value block. 1336 1337Checking and Cross-Referencing Directories 1338`````````````````````````````````````````` 1339 1340The filesystem directory tree is a directed acylic graph structure, with files 1341constituting the nodes, and directory entries (dirents) constituting the edges. 1342Directories are a special type of file containing a set of mappings from a 1343255-byte sequence (name) to an inumber. 1344These are called directory entries, or dirents for short. 1345Each directory file must have exactly one directory pointing to the file. 1346A root directory points to itself. 1347Directory entries point to files of any type. 1348Each non-directory file may have multiple directories point to it. 1349 1350In XFS, directories are implemented as a file containing up to three 32GB 1351partitions. 1352The first partition contains directory entry data blocks. 1353Each data block contains variable-sized records associating a user-provided 1354name with an inumber and, optionally, a file type. 1355If the directory entry data grows beyond one block, the second partition (which 1356exists as post-EOF extents) is populated with a block containing free space 1357information and an index that maps hashes of the dirent names to directory data 1358blocks in the first partition. 1359This makes directory name lookups very fast. 1360If this second partition grows beyond one block, the third partition is 1361populated with a linear array of free space information for faster 1362expansions. 1363If the free space has been separated and the second partition grows again 1364beyond one block, then a dabtree is used to map hashes of dirent names to 1365directory data blocks. 1366 1367Checking a directory is pretty straightforward: 1368 13691. Walk the dabtree in the second partition (if present) to ensure that there 1370 are no irregularities in the blocks or dabtree mappings that do not point to 1371 dirent blocks. 1372 13732. Walk the blocks of the first partition looking for directory entries. 1374 Each dirent is checked as follows: 1375 1376 a. Does the name contain no invalid characters? 1377 1378 b. Does the inumber correspond to an actual, allocated inode? 1379 1380 c. Does the child inode have a nonzero link count? 1381 1382 d. If a file type is included in the dirent, does it match the type of the 1383 inode? 1384 1385 e. If the child is a subdirectory, does the child's dotdot pointer point 1386 back to the parent? 1387 1388 f. If the directory has a second partition, perform a named lookup of the 1389 dirent name to ensure the correctness of the dabtree. 1390 13913. Walk the free space list in the third partition (if present) to ensure that 1392 the free spaces it describes are really unused. 1393 1394Checking operations involving :ref:`parents <dirparent>` and 1395:ref:`file link counts <nlinks>` are discussed in more detail in later 1396sections. 1397 1398Checking Directory/Attribute Btrees 1399^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1400 1401As stated in previous sections, the directory/attribute btree (dabtree) index 1402maps user-provided names to improve lookup times by avoiding linear scans. 1403Internally, it maps a 32-bit hash of the name to a block offset within the 1404appropriate file fork. 1405 1406The internal structure of a dabtree closely resembles the btrees that record 1407fixed-size metadata records -- each dabtree block contains a magic number, a 1408checksum, sibling pointers, a UUID, a tree level, and a log sequence number. 1409The format of leaf and node records are the same -- each entry points to the 1410next level down in the hierarchy, with dabtree node records pointing to dabtree 1411leaf blocks, and dabtree leaf records pointing to non-dabtree blocks elsewhere 1412in the fork. 1413 1414Checking and cross-referencing the dabtree is very similar to what is done for 1415space btrees: 1416 1417- Does the type of data stored in the block match what scrub is expecting? 1418 1419- Does the block belong to the owning structure that asked for the read? 1420 1421- Do the records fit within the block? 1422 1423- Are the records contained inside the block free of obvious corruptions? 1424 1425- Are the name hashes in the correct order? 1426 1427- Do node pointers within the dabtree point to valid fork offsets for dabtree 1428 blocks? 1429 1430- Do leaf pointers within the dabtree point to valid fork offsets for directory 1431 or attr leaf blocks? 1432 1433- Do child pointers point towards the leaves? 1434 1435- Do sibling pointers point across the same level? 1436 1437- For each dabtree node record, does the record key accurate reflect the 1438 contents of the child dabtree block? 1439 1440- For each dabtree leaf record, does the record key accurate reflect the 1441 contents of the directory or attr block? 1442 1443Cross-Referencing Summary Counters 1444`````````````````````````````````` 1445 1446XFS maintains three classes of summary counters: available resources, quota 1447resource usage, and file link counts. 1448 1449In theory, the amount of available resources (data blocks, inodes, realtime 1450extents) can be found by walking the entire filesystem. 1451This would make for very slow reporting, so a transactional filesystem can 1452maintain summaries of this information in the superblock. 1453Cross-referencing these values against the filesystem metadata should be a 1454simple matter of walking the free space and inode metadata in each AG and the 1455realtime bitmap, but there are complications that will be discussed in 1456:ref:`more detail <fscounters>` later. 1457 1458:ref:`Quota usage <quotacheck>` and :ref:`file link count <nlinks>` 1459checking are sufficiently complicated to warrant separate sections. 1460 1461Post-Repair Reverification 1462`````````````````````````` 1463 1464After performing a repair, the checking code is run a second time to validate 1465the new structure, and the results of the health assessment are recorded 1466internally and returned to the calling process. 1467This step is critical for enabling system administrator to monitor the status 1468of the filesystem and the progress of any repairs. 1469For developers, it is a useful means to judge the efficacy of error detection 1470and correction in the online and offline checking tools. 1471 1472Eventual Consistency vs. Online Fsck 1473------------------------------------ 1474 1475Complex operations can make modifications to multiple per-AG data structures 1476with a chain of transactions. 1477These chains, once committed to the log, are restarted during log recovery if 1478the system crashes while processing the chain. 1479Because the AG header buffers are unlocked between transactions within a chain, 1480online checking must coordinate with chained operations that are in progress to 1481avoid incorrectly detecting inconsistencies due to pending chains. 1482Furthermore, online repair must not run when operations are pending because 1483the metadata are temporarily inconsistent with each other, and rebuilding is 1484not possible. 1485 1486Only online fsck has this requirement of total consistency of AG metadata, and 1487should be relatively rare as compared to filesystem change operations. 1488Online fsck coordinates with transaction chains as follows: 1489 1490* For each AG, maintain a count of intent items targeting that AG. 1491 The count should be bumped whenever a new item is added to the chain. 1492 The count should be dropped when the filesystem has locked the AG header 1493 buffers and finished the work. 1494 1495* When online fsck wants to examine an AG, it should lock the AG header 1496 buffers to quiesce all transaction chains that want to modify that AG. 1497 If the count is zero, proceed with the checking operation. 1498 If it is nonzero, cycle the buffer locks to allow the chain to make forward 1499 progress. 1500 1501This may lead to online fsck taking a long time to complete, but regular 1502filesystem updates take precedence over background checking activity. 1503Details about the discovery of this situation are presented in the 1504:ref:`next section <chain_coordination>`, and details about the solution 1505are presented :ref:`after that<intent_drains>`. 1506 1507.. _chain_coordination: 1508 1509Discovery of the Problem 1510```````````````````````` 1511 1512Midway through the development of online scrubbing, the fsstress tests 1513uncovered a misinteraction between online fsck and compound transaction chains 1514created by other writer threads that resulted in false reports of metadata 1515inconsistency. 1516The root cause of these reports is the eventual consistency model introduced by 1517the expansion of deferred work items and compound transaction chains when 1518reverse mapping and reflink were introduced. 1519 1520Originally, transaction chains were added to XFS to avoid deadlocks when 1521unmapping space from files. 1522Deadlock avoidance rules require that AGs only be locked in increasing order, 1523which makes it impossible (say) to use a single transaction to free a space 1524extent in AG 7 and then try to free a now superfluous block mapping btree block 1525in AG 3. 1526To avoid these kinds of deadlocks, XFS creates Extent Freeing Intent (EFI) log 1527items to commit to freeing some space in one transaction while deferring the 1528actual metadata updates to a fresh transaction. 1529The transaction sequence looks like this: 1530 15311. The first transaction contains a physical update to the file's block mapping 1532 structures to remove the mapping from the btree blocks. 1533 It then attaches to the in-memory transaction an action item to schedule 1534 deferred freeing of space. 1535 Concretely, each transaction maintains a list of ``struct 1536 xfs_defer_pending`` objects, each of which maintains a list of ``struct 1537 xfs_extent_free_item`` objects. 1538 Returning to the example above, the action item tracks the freeing of both 1539 the unmapped space from AG 7 and the block mapping btree (BMBT) block from 1540 AG 3. 1541 Deferred frees recorded in this manner are committed in the log by creating 1542 an EFI log item from the ``struct xfs_extent_free_item`` object and 1543 attaching the log item to the transaction. 1544 When the log is persisted to disk, the EFI item is written into the ondisk 1545 transaction record. 1546 EFIs can list up to 16 extents to free, all sorted in AG order. 1547 15482. The second transaction contains a physical update to the free space btrees 1549 of AG 3 to release the former BMBT block and a second physical update to the 1550 free space btrees of AG 7 to release the unmapped file space. 1551 Observe that the physical updates are resequenced in the correct order 1552 when possible. 1553 Attached to the transaction is a an extent free done (EFD) log item. 1554 The EFD contains a pointer to the EFI logged in transaction #1 so that log 1555 recovery can tell if the EFI needs to be replayed. 1556 1557If the system goes down after transaction #1 is written back to the filesystem 1558but before #2 is committed, a scan of the filesystem metadata would show 1559inconsistent filesystem metadata because there would not appear to be any owner 1560of the unmapped space. 1561Happily, log recovery corrects this inconsistency for us -- when recovery finds 1562an intent log item but does not find a corresponding intent done item, it will 1563reconstruct the incore state of the intent item and finish it. 1564In the example above, the log must replay both frees described in the recovered 1565EFI to complete the recovery phase. 1566 1567There are subtleties to XFS' transaction chaining strategy to consider: 1568 1569* Log items must be added to a transaction in the correct order to prevent 1570 conflicts with principal objects that are not held by the transaction. 1571 In other words, all per-AG metadata updates for an unmapped block must be 1572 completed before the last update to free the extent, and extents should not 1573 be reallocated until that last update commits to the log. 1574 1575* AG header buffers are released between each transaction in a chain. 1576 This means that other threads can observe an AG in an intermediate state, 1577 but as long as the first subtlety is handled, this should not affect the 1578 correctness of filesystem operations. 1579 1580* Unmounting the filesystem flushes all pending work to disk, which means that 1581 offline fsck never sees the temporary inconsistencies caused by deferred 1582 work item processing. 1583 1584In this manner, XFS employs a form of eventual consistency to avoid deadlocks 1585and increase parallelism. 1586 1587During the design phase of the reverse mapping and reflink features, it was 1588decided that it was impractical to cram all the reverse mapping updates for a 1589single filesystem change into a single transaction because a single file 1590mapping operation can explode into many small updates: 1591 1592* The block mapping update itself 1593* A reverse mapping update for the block mapping update 1594* Fixing the freelist 1595* A reverse mapping update for the freelist fix 1596 1597* A shape change to the block mapping btree 1598* A reverse mapping update for the btree update 1599* Fixing the freelist (again) 1600* A reverse mapping update for the freelist fix 1601 1602* An update to the reference counting information 1603* A reverse mapping update for the refcount update 1604* Fixing the freelist (a third time) 1605* A reverse mapping update for the freelist fix 1606 1607* Freeing any space that was unmapped and not owned by any other file 1608* Fixing the freelist (a fourth time) 1609* A reverse mapping update for the freelist fix 1610 1611* Freeing the space used by the block mapping btree 1612* Fixing the freelist (a fifth time) 1613* A reverse mapping update for the freelist fix 1614 1615Free list fixups are not usually needed more than once per AG per transaction 1616chain, but it is theoretically possible if space is very tight. 1617For copy-on-write updates this is even worse, because this must be done once to 1618remove the space from a staging area and again to map it into the file! 1619 1620To deal with this explosion in a calm manner, XFS expands its use of deferred 1621work items to cover most reverse mapping updates and all refcount updates. 1622This reduces the worst case size of transaction reservations by breaking the 1623work into a long chain of small updates, which increases the degree of eventual 1624consistency in the system. 1625Again, this generally isn't a problem because XFS orders its deferred work 1626items carefully to avoid resource reuse conflicts between unsuspecting threads. 1627 1628However, online fsck changes the rules -- remember that although physical 1629updates to per-AG structures are coordinated by locking the buffers for AG 1630headers, buffer locks are dropped between transactions. 1631Once scrub acquires resources and takes locks for a data structure, it must do 1632all the validation work without releasing the lock. 1633If the main lock for a space btree is an AG header buffer lock, scrub may have 1634interrupted another thread that is midway through finishing a chain. 1635For example, if a thread performing a copy-on-write has completed a reverse 1636mapping update but not the corresponding refcount update, the two AG btrees 1637will appear inconsistent to scrub and an observation of corruption will be 1638recorded. This observation will not be correct. 1639If a repair is attempted in this state, the results will be catastrophic! 1640 1641Several other solutions to this problem were evaluated upon discovery of this 1642flaw and rejected: 1643 16441. Add a higher level lock to allocation groups and require writer threads to 1645 acquire the higher level lock in AG order before making any changes. 1646 This would be very difficult to implement in practice because it is 1647 difficult to determine which locks need to be obtained, and in what order, 1648 without simulating the entire operation. 1649 Performing a dry run of a file operation to discover necessary locks would 1650 make the filesystem very slow. 1651 16522. Make the deferred work coordinator code aware of consecutive intent items 1653 targeting the same AG and have it hold the AG header buffers locked across 1654 the transaction roll between updates. 1655 This would introduce a lot of complexity into the coordinator since it is 1656 only loosely coupled with the actual deferred work items. 1657 It would also fail to solve the problem because deferred work items can 1658 generate new deferred subtasks, but all subtasks must be complete before 1659 work can start on a new sibling task. 1660 16613. Teach online fsck to walk all transactions waiting for whichever lock(s) 1662 protect the data structure being scrubbed to look for pending operations. 1663 The checking and repair operations must factor these pending operations into 1664 the evaluations being performed. 1665 This solution is a nonstarter because it is *extremely* invasive to the main 1666 filesystem. 1667 1668.. _intent_drains: 1669 1670Intent Drains 1671````````````` 1672 1673Online fsck uses an atomic intent item counter and lock cycling to coordinate 1674with transaction chains. 1675There are two key properties to the drain mechanism. 1676First, the counter is incremented when a deferred work item is *queued* to a 1677transaction, and it is decremented after the associated intent done log item is 1678*committed* to another transaction. 1679The second property is that deferred work can be added to a transaction without 1680holding an AG header lock, but per-AG work items cannot be marked done without 1681locking that AG header buffer to log the physical updates and the intent done 1682log item. 1683The first property enables scrub to yield to running transaction chains, which 1684is an explicit deprioritization of online fsck to benefit file operations. 1685The second property of the drain is key to the correct coordination of scrub, 1686since scrub will always be able to decide if a conflict is possible. 1687 1688For regular filesystem code, the drain works as follows: 1689 16901. Call the appropriate subsystem function to add a deferred work item to a 1691 transaction. 1692 16932. The function calls ``xfs_defer_drain_bump`` to increase the counter. 1694 16953. When the deferred item manager wants to finish the deferred work item, it 1696 calls ``->finish_item`` to complete it. 1697 16984. The ``->finish_item`` implementation logs some changes and calls 1699 ``xfs_defer_drain_drop`` to decrease the sloppy counter and wake up any threads 1700 waiting on the drain. 1701 17025. The subtransaction commits, which unlocks the resource associated with the 1703 intent item. 1704 1705For scrub, the drain works as follows: 1706 17071. Lock the resource(s) associated with the metadata being scrubbed. 1708 For example, a scan of the refcount btree would lock the AGI and AGF header 1709 buffers. 1710 17112. If the counter is zero (``xfs_defer_drain_busy`` returns false), there are no 1712 chains in progress and the operation may proceed. 1713 17143. Otherwise, release the resources grabbed in step 1. 1715 17164. Wait for the intent counter to reach zero (``xfs_defer_drain_intents``), then go 1717 back to step 1 unless a signal has been caught. 1718 1719To avoid polling in step 4, the drain provides a waitqueue for scrub threads to 1720be woken up whenever the intent count drops to zero. 1721 1722.. _jump_labels: 1723 1724Static Keys (aka Jump Label Patching) 1725````````````````````````````````````` 1726 1727Online fsck for XFS separates the regular filesystem from the checking and 1728repair code as much as possible. 1729However, there are a few parts of online fsck (such as the intent drains, and 1730later, live update hooks) where it is useful for the online fsck code to know 1731what's going on in the rest of the filesystem. 1732Since it is not expected that online fsck will be constantly running in the 1733background, it is very important to minimize the runtime overhead imposed by 1734these hooks when online fsck is compiled into the kernel but not actively 1735running on behalf of userspace. 1736Taking locks in the hot path of a writer thread to access a data structure only 1737to find that no further action is necessary is expensive -- on the author's 1738computer, this have an overhead of 40-50ns per access. 1739Fortunately, the kernel supports dynamic code patching, which enables XFS to 1740replace a static branch to hook code with ``nop`` sleds when online fsck isn't 1741running. 1742This sled has an overhead of however long it takes the instruction decoder to 1743skip past the sled, which seems to be on the order of less than 1ns and 1744does not access memory outside of instruction fetching. 1745 1746When online fsck enables the static key, the sled is replaced with an 1747unconditional branch to call the hook code. 1748The switchover is quite expensive (~22000ns) but is paid entirely by the 1749program that invoked online fsck, and can be amortized if multiple threads 1750enter online fsck at the same time, or if multiple filesystems are being 1751checked at the same time. 1752Changing the branch direction requires taking the CPU hotplug lock, and since 1753CPU initialization requires memory allocation, online fsck must be careful not 1754to change a static key while holding any locks or resources that could be 1755accessed in the memory reclaim paths. 1756To minimize contention on the CPU hotplug lock, care should be taken not to 1757enable or disable static keys unnecessarily. 1758 1759Because static keys are intended to minimize hook overhead for regular 1760filesystem operations when xfs_scrub is not running, the intended usage 1761patterns are as follows: 1762 1763- The hooked part of XFS should declare a static-scoped static key that 1764 defaults to false. 1765 The ``DEFINE_STATIC_KEY_FALSE`` macro takes care of this. 1766 The static key itself should be declared as a ``static`` variable. 1767 1768- When deciding to invoke code that's only used by scrub, the regular 1769 filesystem should call the ``static_branch_unlikely`` predicate to avoid the 1770 scrub-only hook code if the static key is not enabled. 1771 1772- The regular filesystem should export helper functions that call 1773 ``static_branch_inc`` to enable and ``static_branch_dec`` to disable the 1774 static key. 1775 Wrapper functions make it easy to compile out the relevant code if the kernel 1776 distributor turns off online fsck at build time. 1777 1778- Scrub functions wanting to turn on scrub-only XFS functionality should call 1779 the ``xchk_fsgates_enable`` from the setup function to enable a specific 1780 hook. 1781 This must be done before obtaining any resources that are used by memory 1782 reclaim. 1783 Callers had better be sure they really need the functionality gated by the 1784 static key; the ``TRY_HARDER`` flag is useful here. 1785 1786Online scrub has resource acquisition helpers (e.g. ``xchk_perag_lock``) to 1787handle locking AGI and AGF buffers for all scrubber functions. 1788If it detects a conflict between scrub and the running transactions, it will 1789try to wait for intents to complete. 1790If the caller of the helper has not enabled the static key, the helper will 1791return -EDEADLOCK, which should result in the scrub being restarted with the 1792``TRY_HARDER`` flag set. 1793The scrub setup function should detect that flag, enable the static key, and 1794try the scrub again. 1795Scrub teardown disables all static keys obtained by ``xchk_fsgates_enable``. 1796 1797For more information, please see the kernel documentation of 1798Documentation/staging/static-keys.rst. 1799 1800.. _xfile: 1801 1802Pageable Kernel Memory 1803---------------------- 1804 1805Some online checking functions work by scanning the filesystem to build a 1806shadow copy of an ondisk metadata structure in memory and comparing the two 1807copies. 1808For online repair to rebuild a metadata structure, it must compute the record 1809set that will be stored in the new structure before it can persist that new 1810structure to disk. 1811Ideally, repairs complete with a single atomic commit that introduces 1812a new data structure. 1813To meet these goals, the kernel needs to collect a large amount of information 1814in a place that doesn't require the correct operation of the filesystem. 1815 1816Kernel memory isn't suitable because: 1817 1818* Allocating a contiguous region of memory to create a C array is very 1819 difficult, especially on 32-bit systems. 1820 1821* Linked lists of records introduce double pointer overhead which is very high 1822 and eliminate the possibility of indexed lookups. 1823 1824* Kernel memory is pinned, which can drive the system into OOM conditions. 1825 1826* The system might not have sufficient memory to stage all the information. 1827 1828At any given time, online fsck does not need to keep the entire record set in 1829memory, which means that individual records can be paged out if necessary. 1830Continued development of online fsck demonstrated that the ability to perform 1831indexed data storage would also be very useful. 1832Fortunately, the Linux kernel already has a facility for byte-addressable and 1833pageable storage: tmpfs. 1834In-kernel graphics drivers (most notably i915) take advantage of tmpfs files 1835to store intermediate data that doesn't need to be in memory at all times, so 1836that usage precedent is already established. 1837Hence, the ``xfile`` was born! 1838 1839+--------------------------------------------------------------------------+ 1840| **Historical Sidebar**: | 1841+--------------------------------------------------------------------------+ 1842| The first edition of online repair inserted records into a new btree as | 1843| it found them, which failed because filesystem could shut down with a | 1844| built data structure, which would be live after recovery finished. | 1845| | 1846| The second edition solved the half-rebuilt structure problem by storing | 1847| everything in memory, but frequently ran the system out of memory. | 1848| | 1849| The third edition solved the OOM problem by using linked lists, but the | 1850| memory overhead of the list pointers was extreme. | 1851+--------------------------------------------------------------------------+ 1852 1853xfile Access Models 1854``````````````````` 1855 1856A survey of the intended uses of xfiles suggested these use cases: 1857 18581. Arrays of fixed-sized records (space management btrees, directory and 1859 extended attribute entries) 1860 18612. Sparse arrays of fixed-sized records (quotas and link counts) 1862 18633. Large binary objects (BLOBs) of variable sizes (directory and extended 1864 attribute names and values) 1865 18664. Staging btrees in memory (reverse mapping btrees) 1867 18685. Arbitrary contents (realtime space management) 1869 1870To support the first four use cases, high level data structures wrap the xfile 1871to share functionality between online fsck functions. 1872The rest of this section discusses the interfaces that the xfile presents to 1873four of those five higher level data structures. 1874The fifth use case is discussed in the :ref:`realtime summary <rtsummary>` case 1875study. 1876 1877XFS is very record-based, which suggests that the ability to load and store 1878complete records is important. 1879To support these cases, a pair of ``xfile_load`` and ``xfile_store`` 1880functions are provided to read and persist objects into an xfile that treat any 1881error as an out of memory error. For online repair, squashing error conditions 1882in this manner is an acceptable behavior because the only reaction is to abort 1883the operation back to userspace. 1884 1885However, no discussion of file access idioms is complete without answering the 1886question, "But what about mmap?" 1887It is convenient to access storage directly with pointers, just like userspace 1888code does with regular memory. 1889Online fsck must not drive the system into OOM conditions, which means that 1890xfiles must be responsive to memory reclamation. 1891tmpfs can only push a pagecache folio to the swap cache if the folio is neither 1892pinned nor locked, which means the xfile must not pin too many folios. 1893 1894Short term direct access to xfile contents is done by locking the pagecache 1895folio and mapping it into kernel address space. Object load and store uses this 1896mechanism. Folio locks are not supposed to be held for long periods of time, so 1897long term direct access to xfile contents is done by bumping the folio refcount, 1898mapping it into kernel address space, and dropping the folio lock. 1899These long term users *must* be responsive to memory reclaim by hooking into 1900the shrinker infrastructure to know when to release folios. 1901 1902The ``xfile_get_folio`` and ``xfile_put_folio`` functions are provided to 1903retrieve the (locked) folio that backs part of an xfile and to release it. 1904The only code to use these folio lease functions are the xfarray 1905:ref:`sorting<xfarray_sort>` algorithms and the :ref:`in-memory 1906btrees<xfbtree>`. 1907 1908xfile Access Coordination 1909````````````````````````` 1910 1911For security reasons, xfiles must be owned privately by the kernel. 1912They are marked ``S_PRIVATE`` to prevent interference from the security system, 1913must never be mapped into process file descriptor tables, and their pages must 1914never be mapped into userspace processes. 1915 1916To avoid locking recursion issues with the VFS, all accesses to the shmfs file 1917are performed by manipulating the page cache directly. 1918xfile writers call the ``->write_begin`` and ``->write_end`` functions of the 1919xfile's address space to grab writable pages, copy the caller's buffer into the 1920page, and release the pages. 1921xfile readers call ``shmem_read_mapping_page_gfp`` to grab pages directly 1922before copying the contents into the caller's buffer. 1923In other words, xfiles ignore the VFS read and write code paths to avoid 1924having to create a dummy ``struct kiocb`` and to avoid taking inode and 1925freeze locks. 1926tmpfs cannot be frozen, and xfiles must not be exposed to userspace. 1927 1928If an xfile is shared between threads to stage repairs, the caller must provide 1929its own locks to coordinate access. 1930For example, if a scrub function stores scan results in an xfile and needs 1931other threads to provide updates to the scanned data, the scrub function must 1932provide a lock for all threads to share. 1933 1934.. _xfarray: 1935 1936Arrays of Fixed-Sized Records 1937````````````````````````````` 1938 1939In XFS, each type of indexed space metadata (free space, inodes, reference 1940counts, file fork space, and reverse mappings) consists of a set of fixed-size 1941records indexed with a classic B+ tree. 1942Directories have a set of fixed-size dirent records that point to the names, 1943and extended attributes have a set of fixed-size attribute keys that point to 1944names and values. 1945Quota counters and file link counters index records with numbers. 1946During a repair, scrub needs to stage new records during the gathering step and 1947retrieve them during the btree building step. 1948 1949Although this requirement can be satisfied by calling the read and write 1950methods of the xfile directly, it is simpler for callers for there to be a 1951higher level abstraction to take care of computing array offsets, to provide 1952iterator functions, and to deal with sparse records and sorting. 1953The ``xfarray`` abstraction presents a linear array for fixed-size records atop 1954the byte-accessible xfile. 1955 1956.. _xfarray_access_patterns: 1957 1958Array Access Patterns 1959^^^^^^^^^^^^^^^^^^^^^ 1960 1961Array access patterns in online fsck tend to fall into three categories. 1962Iteration of records is assumed to be necessary for all cases and will be 1963covered in the next section. 1964 1965The first type of caller handles records that are indexed by position. 1966Gaps may exist between records, and a record may be updated multiple times 1967during the collection step. 1968In other words, these callers want a sparse linearly addressed table file. 1969The typical use case are quota records or file link count records. 1970Access to array elements is performed programmatically via ``xfarray_load`` and 1971``xfarray_store`` functions, which wrap the similarly-named xfile functions to 1972provide loading and storing of array elements at arbitrary array indices. 1973Gaps are defined to be null records, and null records are defined to be a 1974sequence of all zero bytes. 1975Null records are detected by calling ``xfarray_element_is_null``. 1976They are created either by calling ``xfarray_unset`` to null out an existing 1977record or by never storing anything to an array index. 1978 1979The second type of caller handles records that are not indexed by position 1980and do not require multiple updates to a record. 1981The typical use case here is rebuilding space btrees and key/value btrees. 1982These callers can add records to the array without caring about array indices 1983via the ``xfarray_append`` function, which stores a record at the end of the 1984array. 1985For callers that require records to be presentable in a specific order (e.g. 1986rebuilding btree data), the ``xfarray_sort`` function can arrange the sorted 1987records; this function will be covered later. 1988 1989The third type of caller is a bag, which is useful for counting records. 1990The typical use case here is constructing space extent reference counts from 1991reverse mapping information. 1992Records can be put in the bag in any order, they can be removed from the bag 1993at any time, and uniqueness of records is left to callers. 1994The ``xfarray_store_anywhere`` function is used to insert a record in any 1995null record slot in the bag; and the ``xfarray_unset`` function removes a 1996record from the bag. 1997 1998Iterating Array Elements 1999^^^^^^^^^^^^^^^^^^^^^^^^ 2000 2001Most users of the xfarray require the ability to iterate the records stored in 2002the array. 2003Callers can probe every possible array index with the following: 2004 2005.. code-block:: c 2006 2007 xfarray_idx_t i; 2008 foreach_xfarray_idx(array, i) { 2009 xfarray_load(array, i, &rec); 2010 2011 /* do something with rec */ 2012 } 2013 2014All users of this idiom must be prepared to handle null records or must already 2015know that there aren't any. 2016 2017For xfarray users that want to iterate a sparse array, the ``xfarray_iter`` 2018function ignores indices in the xfarray that have never been written to by 2019calling ``xfile_seek_data`` (which internally uses ``SEEK_DATA``) to skip areas 2020of the array that are not populated with memory pages. 2021Once it finds a page, it will skip the zeroed areas of the page. 2022 2023.. code-block:: c 2024 2025 xfarray_idx_t i = XFARRAY_CURSOR_INIT; 2026 while ((ret = xfarray_iter(array, &i, &rec)) == 1) { 2027 /* do something with rec */ 2028 } 2029 2030.. _xfarray_sort: 2031 2032Sorting Array Elements 2033^^^^^^^^^^^^^^^^^^^^^^ 2034 2035During the fourth demonstration of online repair, a community reviewer remarked 2036that for performance reasons, online repair ought to load batches of records 2037into btree record blocks instead of inserting records into a new btree one at a 2038time. 2039The btree insertion code in XFS is responsible for maintaining correct ordering 2040of the records, so naturally the xfarray must also support sorting the record 2041set prior to bulk loading. 2042 2043Case Study: Sorting xfarrays 2044~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2045 2046The sorting algorithm used in the xfarray is actually a combination of adaptive 2047quicksort and a heapsort subalgorithm in the spirit of 2048`Sedgewick <https://algs4.cs.princeton.edu/23quicksort/>`_ and 2049`pdqsort <https://github.com/orlp/pdqsort>`_, with customizations for the Linux 2050kernel. 2051To sort records in a reasonably short amount of time, ``xfarray`` takes 2052advantage of the binary subpartitioning offered by quicksort, but it also uses 2053heapsort to hedge against performance collapse if the chosen quicksort pivots 2054are poor. 2055Both algorithms are (in general) O(n * lg(n)), but there is a wide performance 2056gulf between the two implementations. 2057 2058The Linux kernel already contains a reasonably fast implementation of heapsort. 2059It only operates on regular C arrays, which limits the scope of its usefulness. 2060There are two key places where the xfarray uses it: 2061 2062* Sorting any record subset backed by a single xfile page. 2063 2064* Loading a small number of xfarray records from potentially disparate parts 2065 of the xfarray into a memory buffer, and sorting the buffer. 2066 2067In other words, ``xfarray`` uses heapsort to constrain the nested recursion of 2068quicksort, thereby mitigating quicksort's worst runtime behavior. 2069 2070Choosing a quicksort pivot is a tricky business. 2071A good pivot splits the set to sort in half, leading to the divide and conquer 2072behavior that is crucial to O(n * lg(n)) performance. 2073A poor pivot barely splits the subset at all, leading to O(n\ :sup:`2`) 2074runtime. 2075The xfarray sort routine tries to avoid picking a bad pivot by sampling nine 2076records into a memory buffer and using the kernel heapsort to identify the 2077median of the nine. 2078 2079Most modern quicksort implementations employ Tukey's "ninther" to select a 2080pivot from a classic C array. 2081Typical ninther implementations pick three unique triads of records, sort each 2082of the triads, and then sort the middle value of each triad to determine the 2083ninther value. 2084As stated previously, however, xfile accesses are not entirely cheap. 2085It turned out to be much more performant to read the nine elements into a 2086memory buffer, run the kernel's in-memory heapsort on the buffer, and choose 2087the 4th element of that buffer as the pivot. 2088Tukey's ninthers are described in J. W. Tukey, `The ninther, a technique for 2089low-effort robust (resistant) location in large samples`, in *Contributions to 2090Survey Sampling and Applied Statistics*, edited by H. David, (Academic Press, 20911978), pp. 251–257. 2092 2093The partitioning of quicksort is fairly textbook -- rearrange the record 2094subset around the pivot, then set up the current and next stack frames to 2095sort with the larger and the smaller halves of the pivot, respectively. 2096This keeps the stack space requirements to log2(record count). 2097 2098As a final performance optimization, the hi and lo scanning phase of quicksort 2099keeps examined xfile pages mapped in the kernel for as long as possible to 2100reduce map/unmap cycles. 2101Surprisingly, this reduces overall sort runtime by nearly half again after 2102accounting for the application of heapsort directly onto xfile pages. 2103 2104.. _xfblob: 2105 2106Blob Storage 2107```````````` 2108 2109Extended attributes and directories add an additional requirement for staging 2110records: arbitrary byte sequences of finite length. 2111Each directory entry record needs to store entry name, 2112and each extended attribute needs to store both the attribute name and value. 2113The names, keys, and values can consume a large amount of memory, so the 2114``xfblob`` abstraction was created to simplify management of these blobs 2115atop an xfile. 2116 2117Blob arrays provide ``xfblob_load`` and ``xfblob_store`` functions to retrieve 2118and persist objects. 2119The store function returns a magic cookie for every object that it persists. 2120Later, callers provide this cookie to the ``xblob_load`` to recall the object. 2121The ``xfblob_free`` function frees a specific blob, and the ``xfblob_truncate`` 2122function frees them all because compaction is not needed. 2123 2124The details of repairing directories and extended attributes will be discussed 2125in a subsequent section about atomic file content exchanges. 2126However, it should be noted that these repair functions only use blob storage 2127to cache a small number of entries before adding them to a temporary ondisk 2128file, which is why compaction is not required. 2129 2130.. _xfbtree: 2131 2132In-Memory B+Trees 2133````````````````` 2134 2135The chapter about :ref:`secondary metadata<secondary_metadata>` mentioned that 2136checking and repairing of secondary metadata commonly requires coordination 2137between a live metadata scan of the filesystem and writer threads that are 2138updating that metadata. 2139Keeping the scan data up to date requires the ability to propagate 2140metadata updates from the filesystem into the data being collected by the scan. 2141This *can* be done by appending concurrent updates into a separate log file and 2142applying them before writing the new metadata to disk, but this leads to 2143unbounded memory consumption if the rest of the system is very busy. 2144Another option is to skip the side-log and commit live updates from the 2145filesystem directly into the scan data, which trades more overhead for a lower 2146maximum memory requirement. 2147In both cases, the data structure holding the scan results must support indexed 2148access to perform well. 2149 2150Given that indexed lookups of scan data is required for both strategies, online 2151fsck employs the second strategy of committing live updates directly into 2152scan data. 2153Because xfarrays are not indexed and do not enforce record ordering, they 2154are not suitable for this task. 2155Conveniently, however, XFS has a library to create and maintain ordered reverse 2156mapping records: the existing rmap btree code! 2157If only there was a means to create one in memory. 2158 2159Recall that the :ref:`xfile <xfile>` abstraction represents memory pages as a 2160regular file, which means that the kernel can create byte or block addressable 2161virtual address spaces at will. 2162The XFS buffer cache specializes in abstracting IO to block-oriented address 2163spaces, which means that adaptation of the buffer cache to interface with 2164xfiles enables reuse of the entire btree library. 2165Btrees built atop an xfile are collectively known as ``xfbtrees``. 2166The next few sections describe how they actually work. 2167 2168Using xfiles as a Buffer Cache Target 2169^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2170 2171Two modifications are necessary to support xfiles as a buffer cache target. 2172The first is to make it possible for the ``struct xfs_buftarg`` structure to 2173host the ``struct xfs_buf`` rhashtable, because normally those are held by a 2174per-AG structure. 2175The second change is to modify the buffer ``ioapply`` function to "read" cached 2176pages from the xfile and "write" cached pages back to the xfile. 2177Multiple access to individual buffers is controlled by the ``xfs_buf`` lock, 2178since the xfile does not provide any locking on its own. 2179With this adaptation in place, users of the xfile-backed buffer cache use 2180exactly the same APIs as users of the disk-backed buffer cache. 2181The separation between xfile and buffer cache implies higher memory usage since 2182they do not share pages, but this property could some day enable transactional 2183updates to an in-memory btree. 2184Today, however, it simply eliminates the need for new code. 2185 2186Space Management with an xfbtree 2187^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2188 2189Space management for an xfile is very simple -- each btree block is one memory 2190page in size. 2191These blocks use the same header format as an on-disk btree, but the in-memory 2192block verifiers ignore the checksums, assuming that xfile memory is no more 2193corruption-prone than regular DRAM. 2194Reusing existing code here is more important than absolute memory efficiency. 2195 2196The very first block of an xfile backing an xfbtree contains a header block. 2197The header describes the owner, height, and the block number of the root 2198xfbtree block. 2199 2200To allocate a btree block, use ``xfile_seek_data`` to find a gap in the file. 2201If there are no gaps, create one by extending the length of the xfile. 2202Preallocate space for the block with ``xfile_prealloc``, and hand back the 2203location. 2204To free an xfbtree block, use ``xfile_discard`` (which internally uses 2205``FALLOC_FL_PUNCH_HOLE``) to remove the memory page from the xfile. 2206 2207Populating an xfbtree 2208^^^^^^^^^^^^^^^^^^^^^ 2209 2210An online fsck function that wants to create an xfbtree should proceed as 2211follows: 2212 22131. Call ``xfile_create`` to create an xfile. 2214 22152. Call ``xfs_alloc_memory_buftarg`` to create a buffer cache target structure 2216 pointing to the xfile. 2217 22183. Pass the buffer cache target, buffer ops, and other information to 2219 ``xfbtree_init`` to initialize the passed in ``struct xfbtree`` and write an 2220 initial root block to the xfile. 2221 Each btree type should define a wrapper that passes necessary arguments to 2222 the creation function. 2223 For example, rmap btrees define ``xfs_rmapbt_mem_create`` to take care of 2224 all the necessary details for callers. 2225 22264. Pass the xfbtree object to the btree cursor creation function for the 2227 btree type. 2228 Following the example above, ``xfs_rmapbt_mem_cursor`` takes care of this 2229 for callers. 2230 22315. Pass the btree cursor to the regular btree functions to make queries against 2232 and to update the in-memory btree. 2233 For example, a btree cursor for an rmap xfbtree can be passed to the 2234 ``xfs_rmap_*`` functions just like any other btree cursor. 2235 See the :ref:`next section<xfbtree_commit>` for information on dealing with 2236 xfbtree updates that are logged to a transaction. 2237 22386. When finished, delete the btree cursor, destroy the xfbtree object, free the 2239 buffer target, and the destroy the xfile to release all resources. 2240 2241.. _xfbtree_commit: 2242 2243Committing Logged xfbtree Buffers 2244^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2245 2246Although it is a clever hack to reuse the rmap btree code to handle the staging 2247structure, the ephemeral nature of the in-memory btree block storage presents 2248some challenges of its own. 2249The XFS transaction manager must not commit buffer log items for buffers backed 2250by an xfile because the log format does not understand updates for devices 2251other than the data device. 2252An ephemeral xfbtree probably will not exist by the time the AIL checkpoints 2253log transactions back into the filesystem, and certainly won't exist during 2254log recovery. 2255For these reasons, any code updating an xfbtree in transaction context must 2256remove the buffer log items from the transaction and write the updates into the 2257backing xfile before committing or cancelling the transaction. 2258 2259The ``xfbtree_trans_commit`` and ``xfbtree_trans_cancel`` functions implement 2260this functionality as follows: 2261 22621. Find each buffer log item whose buffer targets the xfile. 2263 22642. Record the dirty/ordered status of the log item. 2265 22663. Detach the log item from the buffer. 2267 22684. Queue the buffer to a special delwri list. 2269 22705. Clear the transaction dirty flag if the only dirty log items were the ones 2271 that were detached in step 3. 2272 22736. Submit the delwri list to commit the changes to the xfile, if the updates 2274 are being committed. 2275 2276After removing xfile logged buffers from the transaction in this manner, the 2277transaction can be committed or cancelled. 2278 2279Bulk Loading of Ondisk B+Trees 2280------------------------------ 2281 2282As mentioned previously, early iterations of online repair built new btree 2283structures by creating a new btree and adding observations individually. 2284Loading a btree one record at a time had a slight advantage of not requiring 2285the incore records to be sorted prior to commit, but was very slow and leaked 2286blocks if the system went down during a repair. 2287Loading records one at a time also meant that repair could not control the 2288loading factor of the blocks in the new btree. 2289 2290Fortunately, the venerable ``xfs_repair`` tool had a more efficient means for 2291rebuilding a btree index from a collection of records -- bulk btree loading. 2292This was implemented rather inefficiently code-wise, since ``xfs_repair`` 2293had separate copy-pasted implementations for each btree type. 2294 2295To prepare for online fsck, each of the four bulk loaders were studied, notes 2296were taken, and the four were refactored into a single generic btree bulk 2297loading mechanism. 2298Those notes in turn have been refreshed and are presented below. 2299 2300Geometry Computation 2301```````````````````` 2302 2303The zeroth step of bulk loading is to assemble the entire record set that will 2304be stored in the new btree, and sort the records. 2305Next, call ``xfs_btree_bload_compute_geometry`` to compute the shape of the 2306btree from the record set, the type of btree, and any load factor preferences. 2307This information is required for resource reservation. 2308 2309First, the geometry computation computes the minimum and maximum records that 2310will fit in a leaf block from the size of a btree block and the size of the 2311block header. 2312Roughly speaking, the maximum number of records is:: 2313 2314 maxrecs = (block_size - header_size) / record_size 2315 2316The XFS design specifies that btree blocks should be merged when possible, 2317which means the minimum number of records is half of maxrecs:: 2318 2319 minrecs = maxrecs / 2 2320 2321The next variable to determine is the desired loading factor. 2322This must be at least minrecs and no more than maxrecs. 2323Choosing minrecs is undesirable because it wastes half the block. 2324Choosing maxrecs is also undesirable because adding a single record to each 2325newly rebuilt leaf block will cause a tree split, which causes a noticeable 2326drop in performance immediately afterwards. 2327The default loading factor was chosen to be 75% of maxrecs, which provides a 2328reasonably compact structure without any immediate split penalties:: 2329 2330 default_load_factor = (maxrecs + minrecs) / 2 2331 2332If space is tight, the loading factor will be set to maxrecs to try to avoid 2333running out of space:: 2334 2335 leaf_load_factor = enough space ? default_load_factor : maxrecs 2336 2337Load factor is computed for btree node blocks using the combined size of the 2338btree key and pointer as the record size:: 2339 2340 maxrecs = (block_size - header_size) / (key_size + ptr_size) 2341 minrecs = maxrecs / 2 2342 node_load_factor = enough space ? default_load_factor : maxrecs 2343 2344Once that's done, the number of leaf blocks required to store the record set 2345can be computed as:: 2346 2347 leaf_blocks = ceil(record_count / leaf_load_factor) 2348 2349The number of node blocks needed to point to the next level down in the tree 2350is computed as:: 2351 2352 n_blocks = (n == 0 ? leaf_blocks : node_blocks[n]) 2353 node_blocks[n + 1] = ceil(n_blocks / node_load_factor) 2354 2355The entire computation is performed recursively until the current level only 2356needs one block. 2357The resulting geometry is as follows: 2358 2359- For AG-rooted btrees, this level is the root level, so the height of the new 2360 tree is ``level + 1`` and the space needed is the summation of the number of 2361 blocks on each level. 2362 2363- For inode-rooted btrees where the records in the top level do not fit in the 2364 inode fork area, the height is ``level + 2``, the space needed is the 2365 summation of the number of blocks on each level, and the inode fork points to 2366 the root block. 2367 2368- For inode-rooted btrees where the records in the top level can be stored in 2369 the inode fork area, then the root block can be stored in the inode, the 2370 height is ``level + 1``, and the space needed is one less than the summation 2371 of the number of blocks on each level. 2372 This only becomes relevant when non-bmap btrees gain the ability to root in 2373 an inode, which is a future patchset and only included here for completeness. 2374 2375.. _newbt: 2376 2377Reserving New B+Tree Blocks 2378``````````````````````````` 2379 2380Once repair knows the number of blocks needed for the new btree, it allocates 2381those blocks using the free space information. 2382Each reserved extent is tracked separately by the btree builder state data. 2383To improve crash resilience, the reservation code also logs an Extent Freeing 2384Intent (EFI) item in the same transaction as each space allocation and attaches 2385its in-memory ``struct xfs_extent_free_item`` object to the space reservation. 2386If the system goes down, log recovery will use the unfinished EFIs to free the 2387unused space, the free space, leaving the filesystem unchanged. 2388 2389Each time the btree builder claims a block for the btree from a reserved 2390extent, it updates the in-memory reservation to reflect the claimed space. 2391Block reservation tries to allocate as much contiguous space as possible to 2392reduce the number of EFIs in play. 2393 2394While repair is writing these new btree blocks, the EFIs created for the space 2395reservations pin the tail of the ondisk log. 2396It's possible that other parts of the system will remain busy and push the head 2397of the log towards the pinned tail. 2398To avoid livelocking the filesystem, the EFIs must not pin the tail of the log 2399for too long. 2400To alleviate this problem, the dynamic relogging capability of the deferred ops 2401mechanism is reused here to commit a transaction at the log head containing an 2402EFD for the old EFI and new EFI at the head. 2403This enables the log to release the old EFI to keep the log moving forwards. 2404 2405EFIs have a role to play during the commit and reaping phases; please see the 2406next section and the section about :ref:`reaping<reaping>` for more details. 2407 2408Writing the New Tree 2409```````````````````` 2410 2411This part is pretty simple -- the btree builder (``xfs_btree_bulkload``) claims 2412a block from the reserved list, writes the new btree block header, fills the 2413rest of the block with records, and adds the new leaf block to a list of 2414written blocks:: 2415 2416 ┌────┐ 2417 │leaf│ 2418 │RRR │ 2419 └────┘ 2420 2421Sibling pointers are set every time a new block is added to the level:: 2422 2423 ┌────┐ ┌────┐ ┌────┐ ┌────┐ 2424 │leaf│→│leaf│→│leaf│→│leaf│ 2425 │RRR │←│RRR │←│RRR │←│RRR │ 2426 └────┘ └────┘ └────┘ └────┘ 2427 2428When it finishes writing the record leaf blocks, it moves on to the node 2429blocks 2430To fill a node block, it walks each block in the next level down in the tree 2431to compute the relevant keys and write them into the parent node:: 2432 2433 ┌────┐ ┌────┐ 2434 │node│──────→│node│ 2435 │PP │←──────│PP │ 2436 └────┘ └────┘ 2437 ↙ ↘ ↙ ↘ 2438 ┌────┐ ┌────┐ ┌────┐ ┌────┐ 2439 │leaf│→│leaf│→│leaf│→│leaf│ 2440 │RRR │←│RRR │←│RRR │←│RRR │ 2441 └────┘ └────┘ └────┘ └────┘ 2442 2443When it reaches the root level, it is ready to commit the new btree!:: 2444 2445 ┌─────────┐ 2446 │ root │ 2447 │ PP │ 2448 └─────────┘ 2449 ↙ ↘ 2450 ┌────┐ ┌────┐ 2451 │node│──────→│node│ 2452 │PP │←──────│PP │ 2453 └────┘ └────┘ 2454 ↙ ↘ ↙ ↘ 2455 ┌────┐ ┌────┐ ┌────┐ ┌────┐ 2456 │leaf│→│leaf│→│leaf│→│leaf│ 2457 │RRR │←│RRR │←│RRR │←│RRR │ 2458 └────┘ └────┘ └────┘ └────┘ 2459 2460The first step to commit the new btree is to persist the btree blocks to disk 2461synchronously. 2462This is a little complicated because a new btree block could have been freed 2463in the recent past, so the builder must use ``xfs_buf_delwri_queue_here`` to 2464remove the (stale) buffer from the AIL list before it can write the new blocks 2465to disk. 2466Blocks are queued for IO using a delwri list and written in one large batch 2467with ``xfs_buf_delwri_submit``. 2468 2469Once the new blocks have been persisted to disk, control returns to the 2470individual repair function that called the bulk loader. 2471The repair function must log the location of the new root in a transaction, 2472clean up the space reservations that were made for the new btree, and reap the 2473old metadata blocks: 2474 24751. Commit the location of the new btree root. 2476 24772. For each incore reservation: 2478 2479 a. Log Extent Freeing Done (EFD) items for all the space that was consumed 2480 by the btree builder. The new EFDs must point to the EFIs attached to 2481 the reservation to prevent log recovery from freeing the new blocks. 2482 2483 b. For unclaimed portions of incore reservations, create a regular deferred 2484 extent free work item to be free the unused space later in the 2485 transaction chain. 2486 2487 c. The EFDs and EFIs logged in steps 2a and 2b must not overrun the 2488 reservation of the committing transaction. 2489 If the btree loading code suspects this might be about to happen, it must 2490 call ``xrep_defer_finish`` to clear out the deferred work and obtain a 2491 fresh transaction. 2492 24933. Clear out the deferred work a second time to finish the commit and clean 2494 the repair transaction. 2495 2496The transaction rolling in steps 2c and 3 represent a weakness in the repair 2497algorithm, because a log flush and a crash before the end of the reap step can 2498result in space leaking. 2499Online repair functions minimize the chances of this occurring by using very 2500large transactions, which each can accommodate many thousands of block freeing 2501instructions. 2502Repair moves on to reaping the old blocks, which will be presented in a 2503subsequent :ref:`section<reaping>` after a few case studies of bulk loading. 2504 2505Case Study: Rebuilding the Inode Index 2506^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2507 2508The high level process to rebuild the inode index btree is: 2509 25101. Walk the reverse mapping records to generate ``struct xfs_inobt_rec`` 2511 records from the inode chunk information and a bitmap of the old inode btree 2512 blocks. 2513 25142. Append the records to an xfarray in inode order. 2515 25163. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number 2517 of blocks needed for the inode btree. 2518 If the free space inode btree is enabled, call it again to estimate the 2519 geometry of the finobt. 2520 25214. Allocate the number of blocks computed in the previous step. 2522 25235. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and 2524 generate the internal node blocks. 2525 If the free space inode btree is enabled, call it again to load the finobt. 2526 25276. Commit the location of the new btree root block(s) to the AGI. 2528 25297. Reap the old btree blocks using the bitmap created in step 1. 2530 2531Details are as follows. 2532 2533The inode btree maps inumbers to the ondisk location of the associated 2534inode records, which means that the inode btrees can be rebuilt from the 2535reverse mapping information. 2536Reverse mapping records with an owner of ``XFS_RMAP_OWN_INOBT`` marks the 2537location of the old inode btree blocks. 2538Each reverse mapping record with an owner of ``XFS_RMAP_OWN_INODES`` marks the 2539location of at least one inode cluster buffer. 2540A cluster is the smallest number of ondisk inodes that can be allocated or 2541freed in a single transaction; it is never smaller than 1 fs block or 4 inodes. 2542 2543For the space represented by each inode cluster, ensure that there are no 2544records in the free space btrees nor any records in the reference count btree. 2545If there are, the space metadata inconsistencies are reason enough to abort the 2546operation. 2547Otherwise, read each cluster buffer to check that its contents appear to be 2548ondisk inodes and to decide if the file is allocated 2549(``xfs_dinode.i_mode != 0``) or free (``xfs_dinode.i_mode == 0``). 2550Accumulate the results of successive inode cluster buffer reads until there is 2551enough information to fill a single inode chunk record, which is 64 consecutive 2552numbers in the inumber keyspace. 2553If the chunk is sparse, the chunk record may include holes. 2554 2555Once the repair function accumulates one chunk's worth of data, it calls 2556``xfarray_append`` to add the inode btree record to the xfarray. 2557This xfarray is walked twice during the btree creation step -- once to populate 2558the inode btree with all inode chunk records, and a second time to populate the 2559free inode btree with records for chunks that have free non-sparse inodes. 2560The number of records for the inode btree is the number of xfarray records, 2561but the record count for the free inode btree has to be computed as inode chunk 2562records are stored in the xfarray. 2563 2564Case Study: Rebuilding the Space Reference Counts 2565^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2566 2567Reverse mapping records are used to rebuild the reference count information. 2568Reference counts are required for correct operation of copy on write for shared 2569file data. 2570Imagine the reverse mapping entries as rectangles representing extents of 2571physical blocks, and that the rectangles can be laid down to allow them to 2572overlap each other. 2573From the diagram below, it is apparent that a reference count record must start 2574or end wherever the height of the stack changes. 2575In other words, the record emission stimulus is level-triggered:: 2576 2577 █ ███ 2578 ██ █████ ████ ███ ██████ 2579 ██ ████ ███████████ ████ █████████ 2580 ████████████████████████████████ ███████████ 2581 ^ ^ ^^ ^^ ^ ^^ ^^^ ^^^^ ^ ^^ ^ ^ ^ 2582 2 1 23 21 3 43 234 2123 1 01 2 3 0 2583 2584The ondisk reference count btree does not store the refcount == 0 cases because 2585the free space btree already records which blocks are free. 2586Extents being used to stage copy-on-write operations should be the only records 2587with refcount == 1. 2588Single-owner file blocks aren't recorded in either the free space or the 2589reference count btrees. 2590 2591The high level process to rebuild the reference count btree is: 2592 25931. Walk the reverse mapping records to generate ``struct xfs_refcount_irec`` 2594 records for any space having more than one reverse mapping and add them to 2595 the xfarray. 2596 Any records owned by ``XFS_RMAP_OWN_COW`` are also added to the xfarray 2597 because these are extents allocated to stage a copy on write operation and 2598 are tracked in the refcount btree. 2599 2600 Use any records owned by ``XFS_RMAP_OWN_REFC`` to create a bitmap of old 2601 refcount btree blocks. 2602 26032. Sort the records in physical extent order, putting the CoW staging extents 2604 at the end of the xfarray. 2605 This matches the sorting order of records in the refcount btree. 2606 26073. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number 2608 of blocks needed for the new tree. 2609 26104. Allocate the number of blocks computed in the previous step. 2611 26125. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and 2613 generate the internal node blocks. 2614 26156. Commit the location of new btree root block to the AGF. 2616 26177. Reap the old btree blocks using the bitmap created in step 1. 2618 2619Details are as follows; the same algorithm is used by ``xfs_repair`` to 2620generate refcount information from reverse mapping records. 2621 2622- Until the reverse mapping btree runs out of records: 2623 2624 - Retrieve the next record from the btree and put it in a bag. 2625 2626 - Collect all records with the same starting block from the btree and put 2627 them in the bag. 2628 2629 - While the bag isn't empty: 2630 2631 - Among the mappings in the bag, compute the lowest block number where the 2632 reference count changes. 2633 This position will be either the starting block number of the next 2634 unprocessed reverse mapping or the next block after the shortest mapping 2635 in the bag. 2636 2637 - Remove all mappings from the bag that end at this position. 2638 2639 - Collect all reverse mappings that start at this position from the btree 2640 and put them in the bag. 2641 2642 - If the size of the bag changed and is greater than one, create a new 2643 refcount record associating the block number range that we just walked to 2644 the size of the bag. 2645 2646The bag-like structure in this case is a type 2 xfarray as discussed in the 2647:ref:`xfarray access patterns<xfarray_access_patterns>` section. 2648Reverse mappings are added to the bag using ``xfarray_store_anywhere`` and 2649removed via ``xfarray_unset``. 2650Bag members are examined through ``xfarray_iter`` loops. 2651 2652Case Study: Rebuilding File Fork Mapping Indices 2653^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2654 2655The high level process to rebuild a data/attr fork mapping btree is: 2656 26571. Walk the reverse mapping records to generate ``struct xfs_bmbt_rec`` 2658 records from the reverse mapping records for that inode and fork. 2659 Append these records to an xfarray. 2660 Compute the bitmap of the old bmap btree blocks from the ``BMBT_BLOCK`` 2661 records. 2662 26632. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number 2664 of blocks needed for the new tree. 2665 26663. Sort the records in file offset order. 2667 26684. If the extent records would fit in the inode fork immediate area, commit the 2669 records to that immediate area and skip to step 8. 2670 26715. Allocate the number of blocks computed in the previous step. 2672 26736. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and 2674 generate the internal node blocks. 2675 26767. Commit the new btree root block to the inode fork immediate area. 2677 26788. Reap the old btree blocks using the bitmap created in step 1. 2679 2680There are some complications here: 2681First, it's possible to move the fork offset to adjust the sizes of the 2682immediate areas if the data and attr forks are not both in BMBT format. 2683Second, if there are sufficiently few fork mappings, it may be possible to use 2684EXTENTS format instead of BMBT, which may require a conversion. 2685Third, the incore extent map must be reloaded carefully to avoid disturbing 2686any delayed allocation extents. 2687 2688.. _reaping: 2689 2690Reaping Old Metadata Blocks 2691--------------------------- 2692 2693Whenever online fsck builds a new data structure to replace one that is 2694suspect, there is a question of how to find and dispose of the blocks that 2695belonged to the old structure. 2696The laziest method of course is not to deal with them at all, but this slowly 2697leads to service degradations as space leaks out of the filesystem. 2698Hopefully, someone will schedule a rebuild of the free space information to 2699plug all those leaks. 2700Offline repair rebuilds all space metadata after recording the usage of 2701the files and directories that it decides not to clear, hence it can build new 2702structures in the discovered free space and avoid the question of reaping. 2703 2704As part of a repair, online fsck relies heavily on the reverse mapping records 2705to find space that is owned by the corresponding rmap owner yet truly free. 2706Cross referencing rmap records with other rmap records is necessary because 2707there may be other data structures that also think they own some of those 2708blocks (e.g. crosslinked trees). 2709Permitting the block allocator to hand them out again will not push the system 2710towards consistency. 2711 2712For space metadata, the process of finding extents to dispose of generally 2713follows this format: 2714 27151. Create a bitmap of space used by data structures that must be preserved. 2716 The space reservations used to create the new metadata can be used here if 2717 the same rmap owner code is used to denote all of the objects being rebuilt. 2718 27192. Survey the reverse mapping data to create a bitmap of space owned by the 2720 same ``XFS_RMAP_OWN_*`` number for the metadata that is being preserved. 2721 27223. Use the bitmap disunion operator to subtract (1) from (2). 2723 The remaining set bits represent candidate extents that could be freed. 2724 The process moves on to step 4 below. 2725 2726Repairs for file-based metadata such as extended attributes, directories, 2727symbolic links, quota files and realtime bitmaps are performed by building a 2728new structure attached to a temporary file and exchanging all mappings in the 2729file forks. 2730Afterward, the mappings in the old file fork are the candidate blocks for 2731disposal. 2732 2733The process for disposing of old extents is as follows: 2734 27354. For each candidate extent, count the number of reverse mapping records for 2736 the first block in that extent that do not have the same rmap owner for the 2737 data structure being repaired. 2738 2739 - If zero, the block has a single owner and can be freed. 2740 2741 - If not, the block is part of a crosslinked structure and must not be 2742 freed. 2743 27445. Starting with the next block in the extent, figure out how many more blocks 2745 have the same zero/nonzero other owner status as that first block. 2746 27476. If the region is crosslinked, delete the reverse mapping entry for the 2748 structure being repaired and move on to the next region. 2749 27507. If the region is to be freed, mark any corresponding buffers in the buffer 2751 cache as stale to prevent log writeback. 2752 27538. Free the region and move on. 2754 2755However, there is one complication to this procedure. 2756Transactions are of finite size, so the reaping process must be careful to roll 2757the transactions to avoid overruns. 2758Overruns come from two sources: 2759 2760a. EFIs logged on behalf of space that is no longer occupied 2761 2762b. Log items for buffer invalidations 2763 2764This is also a window in which a crash during the reaping process can leak 2765blocks. 2766As stated earlier, online repair functions use very large transactions to 2767minimize the chances of this occurring. 2768 2769Case Study: Reaping After a Regular Btree Repair 2770```````````````````````````````````````````````` 2771 2772Old reference count and inode btrees are the easiest to reap because they have 2773rmap records with special owner codes: ``XFS_RMAP_OWN_REFC`` for the refcount 2774btree, and ``XFS_RMAP_OWN_INOBT`` for the inode and free inode btrees. 2775Creating a list of extents to reap the old btree blocks is quite simple, 2776conceptually: 2777 27781. Lock the relevant AGI/AGF header buffers to prevent allocation and frees. 2779 27802. For each reverse mapping record with an rmap owner corresponding to the 2781 metadata structure being rebuilt, set the corresponding range in a bitmap. 2782 27833. Walk the current data structures that have the same rmap owner. 2784 For each block visited, clear that range in the above bitmap. 2785 27864. Each set bit in the bitmap represents a block that could be a block from the 2787 old data structures and hence is a candidate for reaping. 2788 In other words, ``(rmap_records_owned_by & ~blocks_reachable_by_walk)`` 2789 are the blocks that might be freeable. 2790 2791If it is possible to maintain the AGF lock throughout the repair (which is the 2792common case), then step 2 can be performed at the same time as the reverse 2793mapping record walk that creates the records for the new btree. 2794 2795Case Study: Rebuilding the Free Space Indices 2796````````````````````````````````````````````` 2797 2798The high level process to rebuild the free space indices is: 2799 28001. Walk the reverse mapping records to generate ``struct xfs_alloc_rec_incore`` 2801 records from the gaps in the reverse mapping btree. 2802 28032. Append the records to an xfarray. 2804 28053. Use the ``xfs_btree_bload_compute_geometry`` function to compute the number 2806 of blocks needed for each new tree. 2807 28084. Allocate the number of blocks computed in the previous step from the free 2809 space information collected. 2810 28115. Use ``xfs_btree_bload`` to write the xfarray records to btree blocks and 2812 generate the internal node blocks for the free space by length index. 2813 Call it again for the free space by block number index. 2814 28156. Commit the locations of the new btree root blocks to the AGF. 2816 28177. Reap the old btree blocks by looking for space that is not recorded by the 2818 reverse mapping btree, the new free space btrees, or the AGFL. 2819 2820Repairing the free space btrees has three key complications over a regular 2821btree repair: 2822 2823First, free space is not explicitly tracked in the reverse mapping records. 2824Hence, the new free space records must be inferred from gaps in the physical 2825space component of the keyspace of the reverse mapping btree. 2826 2827Second, free space repairs cannot use the common btree reservation code because 2828new blocks are reserved out of the free space btrees. 2829This is impossible when repairing the free space btrees themselves. 2830However, repair holds the AGF buffer lock for the duration of the free space 2831index reconstruction, so it can use the collected free space information to 2832supply the blocks for the new free space btrees. 2833It is not necessary to back each reserved extent with an EFI because the new 2834free space btrees are constructed in what the ondisk filesystem thinks is 2835unowned space. 2836However, if reserving blocks for the new btrees from the collected free space 2837information changes the number of free space records, repair must re-estimate 2838the new free space btree geometry with the new record count until the 2839reservation is sufficient. 2840As part of committing the new btrees, repair must ensure that reverse mappings 2841are created for the reserved blocks and that unused reserved blocks are 2842inserted into the free space btrees. 2843Deferrred rmap and freeing operations are used to ensure that this transition 2844is atomic, similar to the other btree repair functions. 2845 2846Third, finding the blocks to reap after the repair is not overly 2847straightforward. 2848Blocks for the free space btrees and the reverse mapping btrees are supplied by 2849the AGFL. 2850Blocks put onto the AGFL have reverse mapping records with the owner 2851``XFS_RMAP_OWN_AG``. 2852This ownership is retained when blocks move from the AGFL into the free space 2853btrees or the reverse mapping btrees. 2854When repair walks reverse mapping records to synthesize free space records, it 2855creates a bitmap (``ag_owner_bitmap``) of all the space claimed by 2856``XFS_RMAP_OWN_AG`` records. 2857The repair context maintains a second bitmap corresponding to the rmap btree 2858blocks and the AGFL blocks (``rmap_agfl_bitmap``). 2859When the walk is complete, the bitmap disunion operation ``(ag_owner_bitmap & 2860~rmap_agfl_bitmap)`` computes the extents that are used by the old free space 2861btrees. 2862These blocks can then be reaped using the methods outlined above. 2863 2864.. _rmap_reap: 2865 2866Case Study: Reaping After Repairing Reverse Mapping Btrees 2867`````````````````````````````````````````````````````````` 2868 2869Old reverse mapping btrees are less difficult to reap after a repair. 2870As mentioned in the previous section, blocks on the AGFL, the two free space 2871btree blocks, and the reverse mapping btree blocks all have reverse mapping 2872records with ``XFS_RMAP_OWN_AG`` as the owner. 2873The full process of gathering reverse mapping records and building a new btree 2874are described in the case study of 2875:ref:`live rebuilds of rmap data <rmap_repair>`, but a crucial point from that 2876discussion is that the new rmap btree will not contain any records for the old 2877rmap btree, nor will the old btree blocks be tracked in the free space btrees. 2878The list of candidate reaping blocks is computed by setting the bits 2879corresponding to the gaps in the new rmap btree records, and then clearing the 2880bits corresponding to extents in the free space btrees and the current AGFL 2881blocks. 2882The result ``(new_rmapbt_gaps & ~(agfl | bnobt_records))`` are reaped using the 2883methods outlined above. 2884 2885The rest of the process of rebuildng the reverse mapping btree is discussed 2886in a separate :ref:`case study<rmap_repair>`. 2887 2888Case Study: Rebuilding the AGFL 2889``````````````````````````````` 2890 2891The allocation group free block list (AGFL) is repaired as follows: 2892 28931. Create a bitmap for all the space that the reverse mapping data claims is 2894 owned by ``XFS_RMAP_OWN_AG``. 2895 28962. Subtract the space used by the two free space btrees and the rmap btree. 2897 28983. Subtract any space that the reverse mapping data claims is owned by any 2899 other owner, to avoid re-adding crosslinked blocks to the AGFL. 2900 29014. Once the AGFL is full, reap any blocks leftover. 2902 29035. The next operation to fix the freelist will right-size the list. 2904 2905See `fs/xfs/scrub/agheader_repair.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/xfs/scrub/agheader_repair.c>`_ for more details. 2906 2907Inode Record Repairs 2908-------------------- 2909 2910Inode records must be handled carefully, because they have both ondisk records 2911("dinodes") and an in-memory ("cached") representation. 2912There is a very high potential for cache coherency issues if online fsck is not 2913careful to access the ondisk metadata *only* when the ondisk metadata is so 2914badly damaged that the filesystem cannot load the in-memory representation. 2915When online fsck wants to open a damaged file for scrubbing, it must use 2916specialized resource acquisition functions that return either the in-memory 2917representation *or* a lock on whichever object is necessary to prevent any 2918update to the ondisk location. 2919 2920The only repairs that should be made to the ondisk inode buffers are whatever 2921is necessary to get the in-core structure loaded. 2922This means fixing whatever is caught by the inode cluster buffer and inode fork 2923verifiers, and retrying the ``iget`` operation. 2924If the second ``iget`` fails, the repair has failed. 2925 2926Once the in-memory representation is loaded, repair can lock the inode and can 2927subject it to comprehensive checks, repairs, and optimizations. 2928Most inode attributes are easy to check and constrain, or are user-controlled 2929arbitrary bit patterns; these are both easy to fix. 2930Dealing with the data and attr fork extent counts and the file block counts is 2931more complicated, because computing the correct value requires traversing the 2932forks, or if that fails, leaving the fields invalid and waiting for the fork 2933fsck functions to run. 2934 2935Quota Record Repairs 2936-------------------- 2937 2938Similar to inodes, quota records ("dquots") also have both ondisk records and 2939an in-memory representation, and hence are subject to the same cache coherency 2940issues. 2941Somewhat confusingly, both are known as dquots in the XFS codebase. 2942 2943The only repairs that should be made to the ondisk quota record buffers are 2944whatever is necessary to get the in-core structure loaded. 2945Once the in-memory representation is loaded, the only attributes needing 2946checking are obviously bad limits and timer values. 2947 2948Quota usage counters are checked, repaired, and discussed separately in the 2949section about :ref:`live quotacheck <quotacheck>`. 2950 2951.. _fscounters: 2952 2953Freezing to Fix Summary Counters 2954-------------------------------- 2955 2956Filesystem summary counters track availability of filesystem resources such 2957as free blocks, free inodes, and allocated inodes. 2958This information could be compiled by walking the free space and inode indexes, 2959but this is a slow process, so XFS maintains a copy in the ondisk superblock 2960that should reflect the ondisk metadata, at least when the filesystem has been 2961unmounted cleanly. 2962For performance reasons, XFS also maintains incore copies of those counters, 2963which are key to enabling resource reservations for active transactions. 2964Writer threads reserve the worst-case quantities of resources from the 2965incore counter and give back whatever they don't use at commit time. 2966It is therefore only necessary to serialize on the superblock when the 2967superblock is being committed to disk. 2968 2969The lazy superblock counter feature introduced in XFS v5 took this even further 2970by training log recovery to recompute the summary counters from the AG headers, 2971which eliminated the need for most transactions even to touch the superblock. 2972The only time XFS commits the summary counters is at filesystem unmount. 2973To reduce contention even further, the incore counter is implemented as a 2974percpu counter, which means that each CPU is allocated a batch of blocks from a 2975global incore counter and can satisfy small allocations from the local batch. 2976 2977The high-performance nature of the summary counters makes it difficult for 2978online fsck to check them, since there is no way to quiesce a percpu counter 2979while the system is running. 2980Although online fsck can read the filesystem metadata to compute the correct 2981values of the summary counters, there's no way to hold the value of a percpu 2982counter stable, so it's quite possible that the counter will be out of date by 2983the time the walk is complete. 2984Earlier versions of online scrub would return to userspace with an incomplete 2985scan flag, but this is not a satisfying outcome for a system administrator. 2986For repairs, the in-memory counters must be stabilized while walking the 2987filesystem metadata to get an accurate reading and install it in the percpu 2988counter. 2989 2990To satisfy this requirement, online fsck must prevent other programs in the 2991system from initiating new writes to the filesystem, it must disable background 2992garbage collection threads, and it must wait for existing writer programs to 2993exit the kernel. 2994Once that has been established, scrub can walk the AG free space indexes, the 2995inode btrees, and the realtime bitmap to compute the correct value of all 2996four summary counters. 2997This is very similar to a filesystem freeze, though not all of the pieces are 2998necessary: 2999 3000- The final freeze state is set one higher than ``SB_FREEZE_COMPLETE`` to 3001 prevent other threads from thawing the filesystem, or other scrub threads 3002 from initiating another fscounters freeze. 3003 3004- It does not quiesce the log. 3005 3006With this code in place, it is now possible to pause the filesystem for just 3007long enough to check and correct the summary counters. 3008 3009+--------------------------------------------------------------------------+ 3010| **Historical Sidebar**: | 3011+--------------------------------------------------------------------------+ 3012| The initial implementation used the actual VFS filesystem freeze | 3013| mechanism to quiesce filesystem activity. | 3014| With the filesystem frozen, it is possible to resolve the counter values | 3015| with exact precision, but there are many problems with calling the VFS | 3016| methods directly: | 3017| | 3018| - Other programs can unfreeze the filesystem without our knowledge. | 3019| This leads to incorrect scan results and incorrect repairs. | 3020| | 3021| - Adding an extra lock to prevent others from thawing the filesystem | 3022| required the addition of a ``->freeze_super`` function to wrap | 3023| ``freeze_fs()``. | 3024| This in turn caused other subtle problems because it turns out that | 3025| the VFS ``freeze_super`` and ``thaw_super`` functions can drop the | 3026| last reference to the VFS superblock, and any subsequent access | 3027| becomes a UAF bug! | 3028| This can happen if the filesystem is unmounted while the underlying | 3029| block device has frozen the filesystem. | 3030| This problem could be solved by grabbing extra references to the | 3031| superblock, but it felt suboptimal given the other inadequacies of | 3032| this approach. | 3033| | 3034| - The log need not be quiesced to check the summary counters, but a VFS | 3035| freeze initiates one anyway. | 3036| This adds unnecessary runtime to live fscounter fsck operations. | 3037| | 3038| - Quiescing the log means that XFS flushes the (possibly incorrect) | 3039| counters to disk as part of cleaning the log. | 3040| | 3041| - A bug in the VFS meant that freeze could complete even when | 3042| sync_filesystem fails to flush the filesystem and returns an error. | 3043| This bug was fixed in Linux 5.17. | 3044+--------------------------------------------------------------------------+ 3045 3046Full Filesystem Scans 3047--------------------- 3048 3049Certain types of metadata can only be checked by walking every file in the 3050entire filesystem to record observations and comparing the observations against 3051what's recorded on disk. 3052Like every other type of online repair, repairs are made by writing those 3053observations to disk in a replacement structure and committing it atomically. 3054However, it is not practical to shut down the entire filesystem to examine 3055hundreds of billions of files because the downtime would be excessive. 3056Therefore, online fsck must build the infrastructure to manage a live scan of 3057all the files in the filesystem. 3058There are two questions that need to be solved to perform a live walk: 3059 3060- How does scrub manage the scan while it is collecting data? 3061 3062- How does the scan keep abreast of changes being made to the system by other 3063 threads? 3064 3065.. _iscan: 3066 3067Coordinated Inode Scans 3068``````````````````````` 3069 3070In the original Unix filesystems of the 1970s, each directory entry contained 3071an index number (*inumber*) which was used as an index into on ondisk array 3072(*itable*) of fixed-size records (*inodes*) describing a file's attributes and 3073its data block mapping. 3074This system is described by J. Lions, `"inode (5659)" 3075<http://www.lemis.com/grog/Documentation/Lions/>`_ in *Lions' Commentary on 3076UNIX, 6th Edition*, (Dept. of Computer Science, the University of New South 3077Wales, November 1977), pp. 18-2; and later by D. Ritchie and K. Thompson, 3078`"Implementation of the File System" 3079<https://archive.org/details/bstj57-6-1905/page/n8/mode/1up>`_, from *The UNIX 3080Time-Sharing System*, (The Bell System Technical Journal, July 1978), pp. 30811913-4. 3082 3083XFS retains most of this design, except now inumbers are search keys over all 3084the space in the data section filesystem. 3085They form a continuous keyspace that can be expressed as a 64-bit integer, 3086though the inodes themselves are sparsely distributed within the keyspace. 3087Scans proceed in a linear fashion across the inumber keyspace, starting from 3088``0x0`` and ending at ``0xFFFFFFFFFFFFFFFF``. 3089Naturally, a scan through a keyspace requires a scan cursor object to track the 3090scan progress. 3091Because this keyspace is sparse, this cursor contains two parts. 3092The first part of this scan cursor object tracks the inode that will be 3093examined next; call this the examination cursor. 3094Somewhat less obviously, the scan cursor object must also track which parts of 3095the keyspace have already been visited, which is critical for deciding if a 3096concurrent filesystem update needs to be incorporated into the scan data. 3097Call this the visited inode cursor. 3098 3099Advancing the scan cursor is a multi-step process encapsulated in 3100``xchk_iscan_iter``: 3101 31021. Lock the AGI buffer of the AG containing the inode pointed to by the visited 3103 inode cursor. 3104 This guarantee that inodes in this AG cannot be allocated or freed while 3105 advancing the cursor. 3106 31072. Use the per-AG inode btree to look up the next inumber after the one that 3108 was just visited, since it may not be keyspace adjacent. 3109 31103. If there are no more inodes left in this AG: 3111 3112 a. Move the examination cursor to the point of the inumber keyspace that 3113 corresponds to the start of the next AG. 3114 3115 b. Adjust the visited inode cursor to indicate that it has "visited" the 3116 last possible inode in the current AG's inode keyspace. 3117 XFS inumbers are segmented, so the cursor needs to be marked as having 3118 visited the entire keyspace up to just before the start of the next AG's 3119 inode keyspace. 3120 3121 c. Unlock the AGI and return to step 1 if there are unexamined AGs in the 3122 filesystem. 3123 3124 d. If there are no more AGs to examine, set both cursors to the end of the 3125 inumber keyspace. 3126 The scan is now complete. 3127 31284. Otherwise, there is at least one more inode to scan in this AG: 3129 3130 a. Move the examination cursor ahead to the next inode marked as allocated 3131 by the inode btree. 3132 3133 b. Adjust the visited inode cursor to point to the inode just prior to where 3134 the examination cursor is now. 3135 Because the scanner holds the AGI buffer lock, no inodes could have been 3136 created in the part of the inode keyspace that the visited inode cursor 3137 just advanced. 3138 31395. Get the incore inode for the inumber of the examination cursor. 3140 By maintaining the AGI buffer lock until this point, the scanner knows that 3141 it was safe to advance the examination cursor across the entire keyspace, 3142 and that it has stabilized this next inode so that it cannot disappear from 3143 the filesystem until the scan releases the incore inode. 3144 31456. Drop the AGI lock and return the incore inode to the caller. 3146 3147Online fsck functions scan all files in the filesystem as follows: 3148 31491. Start a scan by calling ``xchk_iscan_start``. 3150 31512. Advance the scan cursor (``xchk_iscan_iter``) to get the next inode. 3152 If one is provided: 3153 3154 a. Lock the inode to prevent updates during the scan. 3155 3156 b. Scan the inode. 3157 3158 c. While still holding the inode lock, adjust the visited inode cursor 3159 (``xchk_iscan_mark_visited``) to point to this inode. 3160 3161 d. Unlock and release the inode. 3162 31638. Call ``xchk_iscan_teardown`` to complete the scan. 3164 3165There are subtleties with the inode cache that complicate grabbing the incore 3166inode for the caller. 3167Obviously, it is an absolute requirement that the inode metadata be consistent 3168enough to load it into the inode cache. 3169Second, if the incore inode is stuck in some intermediate state, the scan 3170coordinator must release the AGI and push the main filesystem to get the inode 3171back into a loadable state. 3172 3173Inode Management 3174```````````````` 3175 3176In regular filesystem code, references to allocated XFS incore inodes are 3177always obtained (``xfs_iget``) outside of transaction context because the 3178creation of the incore context for an existing file does not require metadata 3179updates. 3180However, it is important to note that references to incore inodes obtained as 3181part of file creation must be performed in transaction context because the 3182filesystem must ensure the atomicity of the ondisk inode btree index updates 3183and the initialization of the actual ondisk inode. 3184 3185References to incore inodes are always released (``xfs_irele``) outside of 3186transaction context because there are a handful of activities that might 3187require ondisk updates: 3188 3189- The VFS may decide to kick off writeback as part of a ``DONTCACHE`` inode 3190 release. 3191 3192- Speculative preallocations need to be unreserved. 3193 3194- An unlinked file may have lost its last reference, in which case the entire 3195 file must be inactivated, which involves releasing all of its resources in 3196 the ondisk metadata and freeing the inode. 3197 3198These activities are collectively called inode inactivation. 3199Inactivation has two parts -- the VFS part, which initiates writeback on all 3200dirty file pages, and the XFS part, which cleans up XFS-specific information 3201and frees the inode if it was unlinked. 3202If the inode is unlinked (or unconnected after a file handle operation), the 3203kernel drops the inode into the inactivation machinery immediately. 3204 3205During normal operation, resource acquisition for an update follows this order 3206to avoid deadlocks: 3207 32081. Inode reference (``iget``). 3209 32102. Filesystem freeze protection, if repairing (``mnt_want_write_file``). 3211 32123. Inode ``IOLOCK`` (VFS ``i_rwsem``) lock to control file IO. 3213 32144. Inode ``MMAPLOCK`` (page cache ``invalidate_lock``) lock for operations that 3215 can update page cache mappings. 3216 32175. Log feature enablement. 3218 32196. Transaction log space grant. 3220 32217. Space on the data and realtime devices for the transaction. 3222 32238. Incore dquot references, if a file is being repaired. 3224 Note that they are not locked, merely acquired. 3225 32269. Inode ``ILOCK`` for file metadata updates. 3227 322810. AG header buffer locks / Realtime metadata inode ILOCK. 3229 323011. Realtime metadata buffer locks, if applicable. 3231 323212. Extent mapping btree blocks, if applicable. 3233 3234Resources are often released in the reverse order, though this is not required. 3235However, online fsck differs from regular XFS operations because it may examine 3236an object that normally is acquired in a later stage of the locking order, and 3237then decide to cross-reference the object with an object that is acquired 3238earlier in the order. 3239The next few sections detail the specific ways in which online fsck takes care 3240to avoid deadlocks. 3241 3242iget and irele During a Scrub 3243^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 3244 3245An inode scan performed on behalf of a scrub operation runs in transaction 3246context, and possibly with resources already locked and bound to it. 3247This isn't much of a problem for ``iget`` since it can operate in the context 3248of an existing transaction, as long as all of the bound resources are acquired 3249before the inode reference in the regular filesystem. 3250 3251When the VFS ``iput`` function is given a linked inode with no other 3252references, it normally puts the inode on an LRU list in the hope that it can 3253save time if another process re-opens the file before the system runs out 3254of memory and frees it. 3255Filesystem callers can short-circuit the LRU process by setting a ``DONTCACHE`` 3256flag on the inode to cause the kernel to try to drop the inode into the 3257inactivation machinery immediately. 3258 3259In the past, inactivation was always done from the process that dropped the 3260inode, which was a problem for scrub because scrub may already hold a 3261transaction, and XFS does not support nesting transactions. 3262On the other hand, if there is no scrub transaction, it is desirable to drop 3263otherwise unused inodes immediately to avoid polluting caches. 3264To capture these nuances, the online fsck code has a separate ``xchk_irele`` 3265function to set or clear the ``DONTCACHE`` flag to get the required release 3266behavior. 3267 3268.. _ilocking: 3269 3270Locking Inodes 3271^^^^^^^^^^^^^^ 3272 3273In regular filesystem code, the VFS and XFS will acquire multiple IOLOCK locks 3274in a well-known order: parent → child when updating the directory tree, and 3275in numerical order of the addresses of their ``struct inode`` object otherwise. 3276For regular files, the MMAPLOCK can be acquired after the IOLOCK to stop page 3277faults. 3278If two MMAPLOCKs must be acquired, they are acquired in numerical order of 3279the addresses of their ``struct address_space`` objects. 3280Due to the structure of existing filesystem code, IOLOCKs and MMAPLOCKs must be 3281acquired before transactions are allocated. 3282If two ILOCKs must be acquired, they are acquired in inumber order. 3283 3284Inode lock acquisition must be done carefully during a coordinated inode scan. 3285Online fsck cannot abide these conventions, because for a directory tree 3286scanner, the scrub process holds the IOLOCK of the file being scanned and it 3287needs to take the IOLOCK of the file at the other end of the directory link. 3288If the directory tree is corrupt because it contains a cycle, ``xfs_scrub`` 3289cannot use the regular inode locking functions and avoid becoming trapped in an 3290ABBA deadlock. 3291 3292Solving both of these problems is straightforward -- any time online fsck 3293needs to take a second lock of the same class, it uses trylock to avoid an ABBA 3294deadlock. 3295If the trylock fails, scrub drops all inode locks and use trylock loops to 3296(re)acquire all necessary resources. 3297Trylock loops enable scrub to check for pending fatal signals, which is how 3298scrub avoids deadlocking the filesystem or becoming an unresponsive process. 3299However, trylock loops means that online fsck must be prepared to measure the 3300resource being scrubbed before and after the lock cycle to detect changes and 3301react accordingly. 3302 3303.. _dirparent: 3304 3305Case Study: Finding a Directory Parent 3306^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 3307 3308Consider the directory parent pointer repair code as an example. 3309Online fsck must verify that the dotdot dirent of a directory points up to a 3310parent directory, and that the parent directory contains exactly one dirent 3311pointing down to the child directory. 3312Fully validating this relationship (and repairing it if possible) requires a 3313walk of every directory on the filesystem while holding the child locked, and 3314while updates to the directory tree are being made. 3315The coordinated inode scan provides a way to walk the filesystem without the 3316possibility of missing an inode. 3317The child directory is kept locked to prevent updates to the dotdot dirent, but 3318if the scanner fails to lock a parent, it can drop and relock both the child 3319and the prospective parent. 3320If the dotdot entry changes while the directory is unlocked, then a move or 3321rename operation must have changed the child's parentage, and the scan can 3322exit early. 3323 3324.. _fshooks: 3325 3326Filesystem Hooks 3327````````````````` 3328 3329The second piece of support that online fsck functions need during a full 3330filesystem scan is the ability to stay informed about updates being made by 3331other threads in the filesystem, since comparisons against the past are useless 3332in a dynamic environment. 3333Two pieces of Linux kernel infrastructure enable online fsck to monitor regular 3334filesystem operations: filesystem hooks and :ref:`static keys<jump_labels>`. 3335 3336Filesystem hooks convey information about an ongoing filesystem operation to 3337a downstream consumer. 3338In this case, the downstream consumer is always an online fsck function. 3339Because multiple fsck functions can run in parallel, online fsck uses the Linux 3340notifier call chain facility to dispatch updates to any number of interested 3341fsck processes. 3342Call chains are a dynamic list, which means that they can be configured at 3343run time. 3344Because these hooks are private to the XFS module, the information passed along 3345contains exactly what the checking function needs to update its observations. 3346 3347The current implementation of XFS hooks uses SRCU notifier chains to reduce the 3348impact to highly threaded workloads. 3349Regular blocking notifier chains use a rwsem and seem to have a much lower 3350overhead for single-threaded applications. 3351However, it may turn out that the combination of blocking chains and static 3352keys are a more performant combination; more study is needed here. 3353 3354The following pieces are necessary to hook a certain point in the filesystem: 3355 3356- A ``struct xfs_hooks`` object must be embedded in a convenient place such as 3357 a well-known incore filesystem object. 3358 3359- Each hook must define an action code and a structure containing more context 3360 about the action. 3361 3362- Hook providers should provide appropriate wrapper functions and structs 3363 around the ``xfs_hooks`` and ``xfs_hook`` objects to take advantage of type 3364 checking to ensure correct usage. 3365 3366- A callsite in the regular filesystem code must be chosen to call 3367 ``xfs_hooks_call`` with the action code and data structure. 3368 This place should be adjacent to (and not earlier than) the place where 3369 the filesystem update is committed to the transaction. 3370 In general, when the filesystem calls a hook chain, it should be able to 3371 handle sleeping and should not be vulnerable to memory reclaim or locking 3372 recursion. 3373 However, the exact requirements are very dependent on the context of the hook 3374 caller and the callee. 3375 3376- The online fsck function should define a structure to hold scan data, a lock 3377 to coordinate access to the scan data, and a ``struct xfs_hook`` object. 3378 The scanner function and the regular filesystem code must acquire resources 3379 in the same order; see the next section for details. 3380 3381- The online fsck code must contain a C function to catch the hook action code 3382 and data structure. 3383 If the object being updated has already been visited by the scan, then the 3384 hook information must be applied to the scan data. 3385 3386- Prior to unlocking inodes to start the scan, online fsck must call 3387 ``xfs_hooks_setup`` to initialize the ``struct xfs_hook``, and 3388 ``xfs_hooks_add`` to enable the hook. 3389 3390- Online fsck must call ``xfs_hooks_del`` to disable the hook once the scan is 3391 complete. 3392 3393The number of hooks should be kept to a minimum to reduce complexity. 3394Static keys are used to reduce the overhead of filesystem hooks to nearly 3395zero when online fsck is not running. 3396 3397.. _liveupdate: 3398 3399Live Updates During a Scan 3400`````````````````````````` 3401 3402The code paths of the online fsck scanning code and the :ref:`hooked<fshooks>` 3403filesystem code look like this:: 3404 3405 other program 3406 ↓ 3407 inode lock ←────────────────────┐ 3408 ↓ │ 3409 AG header lock │ 3410 ↓ │ 3411 filesystem function │ 3412 ↓ │ 3413 notifier call chain │ same 3414 ↓ ├─── inode 3415 scrub hook function │ lock 3416 ↓ │ 3417 scan data mutex ←──┐ same │ 3418 ↓ ├─── scan │ 3419 update scan data │ lock │ 3420 ↑ │ │ 3421 scan data mutex ←──┘ │ 3422 ↑ │ 3423 inode lock ←────────────────────┘ 3424 ↑ 3425 scrub function 3426 ↑ 3427 inode scanner 3428 ↑ 3429 xfs_scrub 3430 3431These rules must be followed to ensure correct interactions between the 3432checking code and the code making an update to the filesystem: 3433 3434- Prior to invoking the notifier call chain, the filesystem function being 3435 hooked must acquire the same lock that the scrub scanning function acquires 3436 to scan the inode. 3437 3438- The scanning function and the scrub hook function must coordinate access to 3439 the scan data by acquiring a lock on the scan data. 3440 3441- Scrub hook function must not add the live update information to the scan 3442 observations unless the inode being updated has already been scanned. 3443 The scan coordinator has a helper predicate (``xchk_iscan_want_live_update``) 3444 for this. 3445 3446- Scrub hook functions must not change the caller's state, including the 3447 transaction that it is running. 3448 They must not acquire any resources that might conflict with the filesystem 3449 function being hooked. 3450 3451- The hook function can abort the inode scan to avoid breaking the other rules. 3452 3453The inode scan APIs are pretty simple: 3454 3455- ``xchk_iscan_start`` starts a scan 3456 3457- ``xchk_iscan_iter`` grabs a reference to the next inode in the scan or 3458 returns zero if there is nothing left to scan 3459 3460- ``xchk_iscan_want_live_update`` to decide if an inode has already been 3461 visited in the scan. 3462 This is critical for hook functions to decide if they need to update the 3463 in-memory scan information. 3464 3465- ``xchk_iscan_mark_visited`` to mark an inode as having been visited in the 3466 scan 3467 3468- ``xchk_iscan_teardown`` to finish the scan 3469 3470.. _quotacheck: 3471 3472Case Study: Quota Counter Checking 3473^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 3474 3475It is useful to compare the mount time quotacheck code to the online repair 3476quotacheck code. 3477Mount time quotacheck does not have to contend with concurrent operations, so 3478it does the following: 3479 34801. Make sure the ondisk dquots are in good enough shape that all the incore 3481 dquots will actually load, and zero the resource usage counters in the 3482 ondisk buffer. 3483 34842. Walk every inode in the filesystem. 3485 Add each file's resource usage to the incore dquot. 3486 34873. Walk each incore dquot. 3488 If the incore dquot is not being flushed, add the ondisk buffer backing the 3489 incore dquot to a delayed write (delwri) list. 3490 34914. Write the buffer list to disk. 3492 3493Like most online fsck functions, online quotacheck can't write to regular 3494filesystem objects until the newly collected metadata reflect all filesystem 3495state. 3496Therefore, online quotacheck records file resource usage to a shadow dquot 3497index implemented with a sparse ``xfarray``, and only writes to the real dquots 3498once the scan is complete. 3499Handling transactional updates is tricky because quota resource usage updates 3500are handled in phases to minimize contention on dquots: 3501 35021. The inodes involved are joined and locked to a transaction. 3503 35042. For each dquot attached to the file: 3505 3506 a. The dquot is locked. 3507 3508 b. A quota reservation is added to the dquot's resource usage. 3509 The reservation is recorded in the transaction. 3510 3511 c. The dquot is unlocked. 3512 35133. Changes in actual quota usage are tracked in the transaction. 3514 35154. At transaction commit time, each dquot is examined again: 3516 3517 a. The dquot is locked again. 3518 3519 b. Quota usage changes are logged and unused reservation is given back to 3520 the dquot. 3521 3522 c. The dquot is unlocked. 3523 3524For online quotacheck, hooks are placed in steps 2 and 4. 3525The step 2 hook creates a shadow version of the transaction dquot context 3526(``dqtrx``) that operates in a similar manner to the regular code. 3527The step 4 hook commits the shadow ``dqtrx`` changes to the shadow dquots. 3528Notice that both hooks are called with the inode locked, which is how the 3529live update coordinates with the inode scanner. 3530 3531The quotacheck scan looks like this: 3532 35331. Set up a coordinated inode scan. 3534 35352. For each inode returned by the inode scan iterator: 3536 3537 a. Grab and lock the inode. 3538 3539 b. Determine that inode's resource usage (data blocks, inode counts, 3540 realtime blocks) and add that to the shadow dquots for the user, group, 3541 and project ids associated with the inode. 3542 3543 c. Unlock and release the inode. 3544 35453. For each dquot in the system: 3546 3547 a. Grab and lock the dquot. 3548 3549 b. Check the dquot against the shadow dquots created by the scan and updated 3550 by the live hooks. 3551 3552Live updates are key to being able to walk every quota record without 3553needing to hold any locks for a long duration. 3554If repairs are desired, the real and shadow dquots are locked and their 3555resource counts are set to the values in the shadow dquot. 3556 3557.. _nlinks: 3558 3559Case Study: File Link Count Checking 3560^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 3561 3562File link count checking also uses live update hooks. 3563The coordinated inode scanner is used to visit all directories on the 3564filesystem, and per-file link count records are stored in a sparse ``xfarray`` 3565indexed by inumber. 3566During the scanning phase, each entry in a directory generates observation 3567data as follows: 3568 35691. If the entry is a dotdot (``'..'``) entry of the root directory, the 3570 directory's parent link count is bumped because the root directory's dotdot 3571 entry is self referential. 3572 35732. If the entry is a dotdot entry of a subdirectory, the parent's backref 3574 count is bumped. 3575 35763. If the entry is neither a dot nor a dotdot entry, the target file's parent 3577 count is bumped. 3578 35794. If the target is a subdirectory, the parent's child link count is bumped. 3580 3581A crucial point to understand about how the link count inode scanner interacts 3582with the live update hooks is that the scan cursor tracks which *parent* 3583directories have been scanned. 3584In other words, the live updates ignore any update about ``A → B`` when A has 3585not been scanned, even if B has been scanned. 3586Furthermore, a subdirectory A with a dotdot entry pointing back to B is 3587accounted as a backref counter in the shadow data for A, since child dotdot 3588entries affect the parent's link count. 3589Live update hooks are carefully placed in all parts of the filesystem that 3590create, change, or remove directory entries, since those operations involve 3591bumplink and droplink. 3592 3593For any file, the correct link count is the number of parents plus the number 3594of child subdirectories. 3595Non-directories never have children of any kind. 3596The backref information is used to detect inconsistencies in the number of 3597links pointing to child subdirectories and the number of dotdot entries 3598pointing back. 3599 3600After the scan completes, the link count of each file can be checked by locking 3601both the inode and the shadow data, and comparing the link counts. 3602A second coordinated inode scan cursor is used for comparisons. 3603Live updates are key to being able to walk every inode without needing to hold 3604any locks between inodes. 3605If repairs are desired, the inode's link count is set to the value in the 3606shadow information. 3607If no parents are found, the file must be :ref:`reparented <orphanage>` to the 3608orphanage to prevent the file from being lost forever. 3609 3610.. _rmap_repair: 3611 3612Case Study: Rebuilding Reverse Mapping Records 3613^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 3614 3615Most repair functions follow the same pattern: lock filesystem resources, 3616walk the surviving ondisk metadata looking for replacement metadata records, 3617and use an :ref:`in-memory array <xfarray>` to store the gathered observations. 3618The primary advantage of this approach is the simplicity and modularity of the 3619repair code -- code and data are entirely contained within the scrub module, 3620do not require hooks in the main filesystem, and are usually the most efficient 3621in memory use. 3622A secondary advantage of this repair approach is atomicity -- once the kernel 3623decides a structure is corrupt, no other threads can access the metadata until 3624the kernel finishes repairing and revalidating the metadata. 3625 3626For repairs going on within a shard of the filesystem, these advantages 3627outweigh the delays inherent in locking the shard while repairing parts of the 3628shard. 3629Unfortunately, repairs to the reverse mapping btree cannot use the "standard" 3630btree repair strategy because it must scan every space mapping of every fork of 3631every file in the filesystem, and the filesystem cannot stop. 3632Therefore, rmap repair foregoes atomicity between scrub and repair. 3633It combines a :ref:`coordinated inode scanner <iscan>`, :ref:`live update hooks 3634<liveupdate>`, and an :ref:`in-memory rmap btree <xfbtree>` to complete the 3635scan for reverse mapping records. 3636 36371. Set up an xfbtree to stage rmap records. 3638 36392. While holding the locks on the AGI and AGF buffers acquired during the 3640 scrub, generate reverse mappings for all AG metadata: inodes, btrees, CoW 3641 staging extents, and the internal log. 3642 36433. Set up an inode scanner. 3644 36454. Hook into rmap updates for the AG being repaired so that the live scan data 3646 can receive updates to the rmap btree from the rest of the filesystem during 3647 the file scan. 3648 36495. For each space mapping found in either fork of each file scanned, 3650 decide if the mapping matches the AG of interest. 3651 If so: 3652 3653 a. Create a btree cursor for the in-memory btree. 3654 3655 b. Use the rmap code to add the record to the in-memory btree. 3656 3657 c. Use the :ref:`special commit function <xfbtree_commit>` to write the 3658 xfbtree changes to the xfile. 3659 36606. For each live update received via the hook, decide if the owner has already 3661 been scanned. 3662 If so, apply the live update into the scan data: 3663 3664 a. Create a btree cursor for the in-memory btree. 3665 3666 b. Replay the operation into the in-memory btree. 3667 3668 c. Use the :ref:`special commit function <xfbtree_commit>` to write the 3669 xfbtree changes to the xfile. 3670 This is performed with an empty transaction to avoid changing the 3671 caller's state. 3672 36737. When the inode scan finishes, create a new scrub transaction and relock the 3674 two AG headers. 3675 36768. Compute the new btree geometry using the number of rmap records in the 3677 shadow btree, like all other btree rebuilding functions. 3678 36799. Allocate the number of blocks computed in the previous step. 3680 368110. Perform the usual btree bulk loading and commit to install the new rmap 3682 btree. 3683 368411. Reap the old rmap btree blocks as discussed in the case study about how 3685 to :ref:`reap after rmap btree repair <rmap_reap>`. 3686 368712. Free the xfbtree now that it not needed. 3688 3689Staging Repairs with Temporary Files on Disk 3690-------------------------------------------- 3691 3692XFS stores a substantial amount of metadata in file forks: directories, 3693extended attributes, symbolic link targets, free space bitmaps and summary 3694information for the realtime volume, and quota records. 3695File forks map 64-bit logical file fork space extents to physical storage space 3696extents, similar to how a memory management unit maps 64-bit virtual addresses 3697to physical memory addresses. 3698Therefore, file-based tree structures (such as directories and extended 3699attributes) use blocks mapped in the file fork offset address space that point 3700to other blocks mapped within that same address space, and file-based linear 3701structures (such as bitmaps and quota records) compute array element offsets in 3702the file fork offset address space. 3703 3704Because file forks can consume as much space as the entire filesystem, repairs 3705cannot be staged in memory, even when a paging scheme is available. 3706Therefore, online repair of file-based metadata createas a temporary file in 3707the XFS filesystem, writes a new structure at the correct offsets into the 3708temporary file, and atomically exchanges all file fork mappings (and hence the 3709fork contents) to commit the repair. 3710Once the repair is complete, the old fork can be reaped as necessary; if the 3711system goes down during the reap, the iunlink code will delete the blocks 3712during log recovery. 3713 3714**Note**: All space usage and inode indices in the filesystem *must* be 3715consistent to use a temporary file safely! 3716This dependency is the reason why online repair can only use pageable kernel 3717memory to stage ondisk space usage information. 3718 3719Exchanging metadata file mappings with a temporary file requires the owner 3720field of the block headers to match the file being repaired and not the 3721temporary file. 3722The directory, extended attribute, and symbolic link functions were all 3723modified to allow callers to specify owner numbers explicitly. 3724 3725There is a downside to the reaping process -- if the system crashes during the 3726reap phase and the fork extents are crosslinked, the iunlink processing will 3727fail because freeing space will find the extra reverse mappings and abort. 3728 3729Temporary files created for repair are similar to ``O_TMPFILE`` files created 3730by userspace. 3731They are not linked into a directory and the entire file will be reaped when 3732the last reference to the file is lost. 3733The key differences are that these files must have no access permission outside 3734the kernel at all, they must be specially marked to prevent them from being 3735opened by handle, and they must never be linked into the directory tree. 3736 3737+--------------------------------------------------------------------------+ 3738| **Historical Sidebar**: | 3739+--------------------------------------------------------------------------+ 3740| In the initial iteration of file metadata repair, the damaged metadata | 3741| blocks would be scanned for salvageable data; the extents in the file | 3742| fork would be reaped; and then a new structure would be built in its | 3743| place. | 3744| This strategy did not survive the introduction of the atomic repair | 3745| requirement expressed earlier in this document. | 3746| | 3747| The second iteration explored building a second structure at a high | 3748| offset in the fork from the salvage data, reaping the old extents, and | 3749| using a ``COLLAPSE_RANGE`` operation to slide the new extents into | 3750| place. | 3751| | 3752| This had many drawbacks: | 3753| | 3754| - Array structures are linearly addressed, and the regular filesystem | 3755| codebase does not have the concept of a linear offset that could be | 3756| applied to the record offset computation to build an alternate copy. | 3757| | 3758| - Extended attributes are allowed to use the entire attr fork offset | 3759| address space. | 3760| | 3761| - Even if repair could build an alternate copy of a data structure in a | 3762| different part of the fork address space, the atomic repair commit | 3763| requirement means that online repair would have to be able to perform | 3764| a log assisted ``COLLAPSE_RANGE`` operation to ensure that the old | 3765| structure was completely replaced. | 3766| | 3767| - A crash after construction of the secondary tree but before the range | 3768| collapse would leave unreachable blocks in the file fork. | 3769| This would likely confuse things further. | 3770| | 3771| - Reaping blocks after a repair is not a simple operation, and | 3772| initiating a reap operation from a restarted range collapse operation | 3773| during log recovery is daunting. | 3774| | 3775| - Directory entry blocks and quota records record the file fork offset | 3776| in the header area of each block. | 3777| An atomic range collapse operation would have to rewrite this part of | 3778| each block header. | 3779| Rewriting a single field in block headers is not a huge problem, but | 3780| it's something to be aware of. | 3781| | 3782| - Each block in a directory or extended attributes btree index contains | 3783| sibling and child block pointers. | 3784| Were the atomic commit to use a range collapse operation, each block | 3785| would have to be rewritten very carefully to preserve the graph | 3786| structure. | 3787| Doing this as part of a range collapse means rewriting a large number | 3788| of blocks repeatedly, which is not conducive to quick repairs. | 3789| | 3790| This lead to the introduction of temporary file staging. | 3791+--------------------------------------------------------------------------+ 3792 3793Using a Temporary File 3794`````````````````````` 3795 3796Online repair code should use the ``xrep_tempfile_create`` function to create a 3797temporary file inside the filesystem. 3798This allocates an inode, marks the in-core inode private, and attaches it to 3799the scrub context. 3800These files are hidden from userspace, may not be added to the directory tree, 3801and must be kept private. 3802 3803Temporary files only use two inode locks: the IOLOCK and the ILOCK. 3804The MMAPLOCK is not needed here, because there must not be page faults from 3805userspace for data fork blocks. 3806The usage patterns of these two locks are the same as for any other XFS file -- 3807access to file data are controlled via the IOLOCK, and access to file metadata 3808are controlled via the ILOCK. 3809Locking helpers are provided so that the temporary file and its lock state can 3810be cleaned up by the scrub context. 3811To comply with the nested locking strategy laid out in the :ref:`inode 3812locking<ilocking>` section, it is recommended that scrub functions use the 3813xrep_tempfile_ilock*_nowait lock helpers. 3814 3815Data can be written to a temporary file by two means: 3816 38171. ``xrep_tempfile_copyin`` can be used to set the contents of a regular 3818 temporary file from an xfile. 3819 38202. The regular directory, symbolic link, and extended attribute functions can 3821 be used to write to the temporary file. 3822 3823Once a good copy of a data file has been constructed in a temporary file, it 3824must be conveyed to the file being repaired, which is the topic of the next 3825section. 3826 3827Logged File Content Exchanges 3828----------------------------- 3829 3830Once repair builds a temporary file with a new data structure written into 3831it, it must commit the new changes into the existing file. 3832It is not possible to swap the inumbers of two files, so instead the new 3833metadata must replace the old. 3834This suggests the need for the ability to swap extents, but the existing extent 3835swapping code used by the file defragmenting tool ``xfs_fsr`` is not sufficient 3836for online repair because: 3837 3838a. When the reverse-mapping btree is enabled, the swap code must keep the 3839 reverse mapping information up to date with every exchange of mappings. 3840 Therefore, it can only exchange one mapping per transaction, and each 3841 transaction is independent. 3842 3843b. Reverse-mapping is critical for the operation of online fsck, so the old 3844 defragmentation code (which swapped entire extent forks in a single 3845 operation) is not useful here. 3846 3847c. Defragmentation is assumed to occur between two files with identical 3848 contents. 3849 For this use case, an incomplete exchange will not result in a user-visible 3850 change in file contents, even if the operation is interrupted. 3851 3852d. Online repair needs to swap the contents of two files that are by definition 3853 *not* identical. 3854 For directory and xattr repairs, the user-visible contents might be the 3855 same, but the contents of individual blocks may be very different. 3856 3857e. Old blocks in the file may be cross-linked with another structure and must 3858 not reappear if the system goes down mid-repair. 3859 3860These problems are overcome by creating a new deferred operation and a new type 3861of log intent item to track the progress of an operation to exchange two file 3862ranges. 3863The new exchange operation type chains together the same transactions used by 3864the reverse-mapping extent swap code, but records intermedia progress in the 3865log so that operations can be restarted after a crash. 3866This new functionality is called the file contents exchange (xfs_exchrange) 3867code. 3868The underlying implementation exchanges file fork mappings (xfs_exchmaps). 3869The new log item records the progress of the exchange to ensure that once an 3870exchange begins, it will always run to completion, even there are 3871interruptions. 3872The new ``XFS_SB_FEAT_INCOMPAT_EXCHRANGE`` incompatible feature flag 3873in the superblock protects these new log item records from being replayed on 3874old kernels. 3875 3876+--------------------------------------------------------------------------+ 3877| **Sidebar: Using Log-Incompatible Feature Flags** | 3878+--------------------------------------------------------------------------+ 3879| Starting with XFS v5, the superblock contains a | 3880| ``sb_features_log_incompat`` field to indicate that the log contains | 3881| records that might not readable by all kernels that could mount this | 3882| filesystem. | 3883| In short, log incompat features protect the log contents against kernels | 3884| that will not understand the contents. | 3885| Unlike the other superblock feature bits, log incompat bits are | 3886| ephemeral because an empty (clean) log does not need protection. | 3887| The log cleans itself after its contents have been committed into the | 3888| filesystem, either as part of an unmount or because the system is | 3889| otherwise idle. | 3890| Because upper level code can be working on a transaction at the same | 3891| time that the log cleans itself, it is necessary for upper level code to | 3892| communicate to the log when it is going to use a log incompatible | 3893| feature. | 3894| | 3895| The log coordinates access to incompatible features through the use of | 3896| one ``struct rw_semaphore`` for each feature. | 3897| The log cleaning code tries to take this rwsem in exclusive mode to | 3898| clear the bit; if the lock attempt fails, the feature bit remains set. | 3899| The code supporting a log incompat feature should create wrapper | 3900| functions to obtain the log feature and call | 3901| ``xfs_add_incompat_log_feature`` to set the feature bits in the primary | 3902| superblock. | 3903| The superblock update is performed transactionally, so the wrapper to | 3904| obtain log assistance must be called just prior to the creation of the | 3905| transaction that uses the functionality. | 3906| For a file operation, this step must happen after taking the IOLOCK | 3907| and the MMAPLOCK, but before allocating the transaction. | 3908| When the transaction is complete, the ``xlog_drop_incompat_feat`` | 3909| function is called to release the feature. | 3910| The feature bit will not be cleared from the superblock until the log | 3911| becomes clean. | 3912| | 3913| Log-assisted extended attribute updates and file content exchanges bothe | 3914| use log incompat features and provide convenience wrappers around the | 3915| functionality. | 3916+--------------------------------------------------------------------------+ 3917 3918Mechanics of a Logged File Content Exchange 3919``````````````````````````````````````````` 3920 3921Exchanging contents between file forks is a complex task. 3922The goal is to exchange all file fork mappings between two file fork offset 3923ranges. 3924There are likely to be many extent mappings in each fork, and the edges of 3925the mappings aren't necessarily aligned. 3926Furthermore, there may be other updates that need to happen after the exchange, 3927such as exchanging file sizes, inode flags, or conversion of fork data to local 3928format. 3929This is roughly the format of the new deferred exchange-mapping work item: 3930 3931.. code-block:: c 3932 3933 struct xfs_exchmaps_intent { 3934 /* Inodes participating in the operation. */ 3935 struct xfs_inode *xmi_ip1; 3936 struct xfs_inode *xmi_ip2; 3937 3938 /* File offset range information. */ 3939 xfs_fileoff_t xmi_startoff1; 3940 xfs_fileoff_t xmi_startoff2; 3941 xfs_filblks_t xmi_blockcount; 3942 3943 /* Set these file sizes after the operation, unless negative. */ 3944 xfs_fsize_t xmi_isize1; 3945 xfs_fsize_t xmi_isize2; 3946 3947 /* XFS_EXCHMAPS_* log operation flags */ 3948 uint64_t xmi_flags; 3949 }; 3950 3951The new log intent item contains enough information to track two logical fork 3952offset ranges: ``(inode1, startoff1, blockcount)`` and ``(inode2, startoff2, 3953blockcount)``. 3954Each step of an exchange operation exchanges the largest file range mapping 3955possible from one file to the other. 3956After each step in the exchange operation, the two startoff fields are 3957incremented and the blockcount field is decremented to reflect the progress 3958made. 3959The flags field captures behavioral parameters such as exchanging attr fork 3960mappings instead of the data fork and other work to be done after the exchange. 3961The two isize fields are used to exchange the file sizes at the end of the 3962operation if the file data fork is the target of the operation. 3963 3964When the exchange is initiated, the sequence of operations is as follows: 3965 39661. Create a deferred work item for the file mapping exchange. 3967 At the start, it should contain the entirety of the file block ranges to be 3968 exchanged. 3969 39702. Call ``xfs_defer_finish`` to process the exchange. 3971 This is encapsulated in ``xrep_tempexch_contents`` for scrub operations. 3972 This will log an extent swap intent item to the transaction for the deferred 3973 mapping exchange work item. 3974 39753. Until ``xmi_blockcount`` of the deferred mapping exchange work item is zero, 3976 3977 a. Read the block maps of both file ranges starting at ``xmi_startoff1`` and 3978 ``xmi_startoff2``, respectively, and compute the longest extent that can 3979 be exchanged in a single step. 3980 This is the minimum of the two ``br_blockcount`` s in the mappings. 3981 Keep advancing through the file forks until at least one of the mappings 3982 contains written blocks. 3983 Mutual holes, unwritten extents, and extent mappings to the same physical 3984 space are not exchanged. 3985 3986 For the next few steps, this document will refer to the mapping that came 3987 from file 1 as "map1", and the mapping that came from file 2 as "map2". 3988 3989 b. Create a deferred block mapping update to unmap map1 from file 1. 3990 3991 c. Create a deferred block mapping update to unmap map2 from file 2. 3992 3993 d. Create a deferred block mapping update to map map1 into file 2. 3994 3995 e. Create a deferred block mapping update to map map2 into file 1. 3996 3997 f. Log the block, quota, and extent count updates for both files. 3998 3999 g. Extend the ondisk size of either file if necessary. 4000 4001 h. Log a mapping exchange done log item for th mapping exchange intent log 4002 item that was read at the start of step 3. 4003 4004 i. Compute the amount of file range that has just been covered. 4005 This quantity is ``(map1.br_startoff + map1.br_blockcount - 4006 xmi_startoff1)``, because step 3a could have skipped holes. 4007 4008 j. Increase the starting offsets of ``xmi_startoff1`` and ``xmi_startoff2`` 4009 by the number of blocks computed in the previous step, and decrease 4010 ``xmi_blockcount`` by the same quantity. 4011 This advances the cursor. 4012 4013 k. Log a new mapping exchange intent log item reflecting the advanced state 4014 of the work item. 4015 4016 l. Return the proper error code (EAGAIN) to the deferred operation manager 4017 to inform it that there is more work to be done. 4018 The operation manager completes the deferred work in steps 3b-3e before 4019 moving back to the start of step 3. 4020 40214. Perform any post-processing. 4022 This will be discussed in more detail in subsequent sections. 4023 4024If the filesystem goes down in the middle of an operation, log recovery will 4025find the most recent unfinished mapping exchange log intent item and restart 4026from there. 4027This is how atomic file mapping exchanges guarantees that an outside observer 4028will either see the old broken structure or the new one, and never a mismash of 4029both. 4030 4031Preparation for File Content Exchanges 4032`````````````````````````````````````` 4033 4034There are a few things that need to be taken care of before initiating an 4035atomic file mapping exchange operation. 4036First, regular files require the page cache to be flushed to disk before the 4037operation begins, and directio writes to be quiesced. 4038Like any filesystem operation, file mapping exchanges must determine the 4039maximum amount of disk space and quota that can be consumed on behalf of both 4040files in the operation, and reserve that quantity of resources to avoid an 4041unrecoverable out of space failure once it starts dirtying metadata. 4042The preparation step scans the ranges of both files to estimate: 4043 4044- Data device blocks needed to handle the repeated updates to the fork 4045 mappings. 4046- Change in data and realtime block counts for both files. 4047- Increase in quota usage for both files, if the two files do not share the 4048 same set of quota ids. 4049- The number of extent mappings that will be added to each file. 4050- Whether or not there are partially written realtime extents. 4051 User programs must never be able to access a realtime file extent that maps 4052 to different extents on the realtime volume, which could happen if the 4053 operation fails to run to completion. 4054 4055The need for precise estimation increases the run time of the exchange 4056operation, but it is very important to maintain correct accounting. 4057The filesystem must not run completely out of free space, nor can the mapping 4058exchange ever add more extent mappings to a fork than it can support. 4059Regular users are required to abide the quota limits, though metadata repairs 4060may exceed quota to resolve inconsistent metadata elsewhere. 4061 4062Special Features for Exchanging Metadata File Contents 4063`````````````````````````````````````````````````````` 4064 4065Extended attributes, symbolic links, and directories can set the fork format to 4066"local" and treat the fork as a literal area for data storage. 4067Metadata repairs must take extra steps to support these cases: 4068 4069- If both forks are in local format and the fork areas are large enough, the 4070 exchange is performed by copying the incore fork contents, logging both 4071 forks, and committing. 4072 The atomic file mapping exchange mechanism is not necessary, since this can 4073 be done with a single transaction. 4074 4075- If both forks map blocks, then the regular atomic file mapping exchange is 4076 used. 4077 4078- Otherwise, only one fork is in local format. 4079 The contents of the local format fork are converted to a block to perform the 4080 exchange. 4081 The conversion to block format must be done in the same transaction that 4082 logs the initial mapping exchange intent log item. 4083 The regular atomic mapping exchange is used to exchange the metadata file 4084 mappings. 4085 Special flags are set on the exchange operation so that the transaction can 4086 be rolled one more time to convert the second file's fork back to local 4087 format so that the second file will be ready to go as soon as the ILOCK is 4088 dropped. 4089 4090Extended attributes and directories stamp the owning inode into every block, 4091but the buffer verifiers do not actually check the inode number! 4092Although there is no verification, it is still important to maintain 4093referential integrity, so prior to performing the mapping exchange, online 4094repair builds every block in the new data structure with the owner field of the 4095file being repaired. 4096 4097After a successful exchange operation, the repair operation must reap the old 4098fork blocks by processing each fork mapping through the standard :ref:`file 4099extent reaping <reaping>` mechanism that is done post-repair. 4100If the filesystem should go down during the reap part of the repair, the 4101iunlink processing at the end of recovery will free both the temporary file and 4102whatever blocks were not reaped. 4103However, this iunlink processing omits the cross-link detection of online 4104repair, and is not completely foolproof. 4105 4106Exchanging Temporary File Contents 4107`````````````````````````````````` 4108 4109To repair a metadata file, online repair proceeds as follows: 4110 41111. Create a temporary repair file. 4112 41132. Use the staging data to write out new contents into the temporary repair 4114 file. 4115 The same fork must be written to as is being repaired. 4116 41173. Commit the scrub transaction, since the exchange resource estimation step 4118 must be completed before transaction reservations are made. 4119 41204. Call ``xrep_tempexch_trans_alloc`` to allocate a new scrub transaction with 4121 the appropriate resource reservations, locks, and fill out a ``struct 4122 xfs_exchmaps_req`` with the details of the exchange operation. 4123 41245. Call ``xrep_tempexch_contents`` to exchange the contents. 4125 41266. Commit the transaction to complete the repair. 4127 4128.. _rtsummary: 4129 4130Case Study: Repairing the Realtime Summary File 4131^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4132 4133In the "realtime" section of an XFS filesystem, free space is tracked via a 4134bitmap, similar to Unix FFS. 4135Each bit in the bitmap represents one realtime extent, which is a multiple of 4136the filesystem block size between 4KiB and 1GiB in size. 4137The realtime summary file indexes the number of free extents of a given size to 4138the offset of the block within the realtime free space bitmap where those free 4139extents begin. 4140In other words, the summary file helps the allocator find free extents by 4141length, similar to what the free space by count (cntbt) btree does for the data 4142section. 4143 4144The summary file itself is a flat file (with no block headers or checksums!) 4145partitioned into ``log2(total rt extents)`` sections containing enough 32-bit 4146counters to match the number of blocks in the rt bitmap. 4147Each counter records the number of free extents that start in that bitmap block 4148and can satisfy a power-of-two allocation request. 4149 4150To check the summary file against the bitmap: 4151 41521. Take the ILOCK of both the realtime bitmap and summary files. 4153 41542. For each free space extent recorded in the bitmap: 4155 4156 a. Compute the position in the summary file that contains a counter that 4157 represents this free extent. 4158 4159 b. Read the counter from the xfile. 4160 4161 c. Increment it, and write it back to the xfile. 4162 41633. Compare the contents of the xfile against the ondisk file. 4164 4165To repair the summary file, write the xfile contents into the temporary file 4166and use atomic mapping exchange to commit the new contents. 4167The temporary file is then reaped. 4168 4169Case Study: Salvaging Extended Attributes 4170^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4171 4172In XFS, extended attributes are implemented as a namespaced name-value store. 4173Values are limited in size to 64KiB, but there is no limit in the number of 4174names. 4175The attribute fork is unpartitioned, which means that the root of the attribute 4176structure is always in logical block zero, but attribute leaf blocks, dabtree 4177index blocks, and remote value blocks are intermixed. 4178Attribute leaf blocks contain variable-sized records that associate 4179user-provided names with the user-provided values. 4180Values larger than a block are allocated separate extents and written there. 4181If the leaf information expands beyond a single block, a directory/attribute 4182btree (``dabtree``) is created to map hashes of attribute names to entries 4183for fast lookup. 4184 4185Salvaging extended attributes is done as follows: 4186 41871. Walk the attr fork mappings of the file being repaired to find the attribute 4188 leaf blocks. 4189 When one is found, 4190 4191 a. Walk the attr leaf block to find candidate keys. 4192 When one is found, 4193 4194 1. Check the name for problems, and ignore the name if there are. 4195 4196 2. Retrieve the value. 4197 If that succeeds, add the name and value to the staging xfarray and 4198 xfblob. 4199 42002. If the memory usage of the xfarray and xfblob exceed a certain amount of 4201 memory or there are no more attr fork blocks to examine, unlock the file and 4202 add the staged extended attributes to the temporary file. 4203 42043. Use atomic file mapping exchange to exchange the new and old extended 4205 attribute structures. 4206 The old attribute blocks are now attached to the temporary file. 4207 42084. Reap the temporary file. 4209 4210Fixing Directories 4211------------------ 4212 4213Fixing directories is difficult with currently available filesystem features, 4214since directory entries are not redundant. 4215The offline repair tool scans all inodes to find files with nonzero link count, 4216and then it scans all directories to establish parentage of those linked files. 4217Damaged files and directories are zapped, and files with no parent are 4218moved to the ``/lost+found`` directory. 4219It does not try to salvage anything. 4220 4221The best that online repair can do at this time is to read directory data 4222blocks and salvage any dirents that look plausible, correct link counts, and 4223move orphans back into the directory tree. 4224The salvage process is discussed in the case study at the end of this section. 4225The :ref:`file link count fsck <nlinks>` code takes care of fixing link counts 4226and moving orphans to the ``/lost+found`` directory. 4227 4228Case Study: Salvaging Directories 4229````````````````````````````````` 4230 4231Unlike extended attributes, directory blocks are all the same size, so 4232salvaging directories is straightforward: 4233 42341. Find the parent of the directory. 4235 If the dotdot entry is not unreadable, try to confirm that the alleged 4236 parent has a child entry pointing back to the directory being repaired. 4237 Otherwise, walk the filesystem to find it. 4238 42392. Walk the first partition of data fork of the directory to find the directory 4240 entry data blocks. 4241 When one is found, 4242 4243 a. Walk the directory data block to find candidate entries. 4244 When an entry is found: 4245 4246 i. Check the name for problems, and ignore the name if there are. 4247 4248 ii. Retrieve the inumber and grab the inode. 4249 If that succeeds, add the name, inode number, and file type to the 4250 staging xfarray and xblob. 4251 42523. If the memory usage of the xfarray and xfblob exceed a certain amount of 4253 memory or there are no more directory data blocks to examine, unlock the 4254 directory and add the staged dirents into the temporary directory. 4255 Truncate the staging files. 4256 42574. Use atomic file mapping exchange to exchange the new and old directory 4258 structures. 4259 The old directory blocks are now attached to the temporary file. 4260 42615. Reap the temporary file. 4262 4263**Future Work Question**: Should repair revalidate the dentry cache when 4264rebuilding a directory? 4265 4266*Answer*: Yes, it should. 4267 4268In theory it is necessary to scan all dentry cache entries for a directory to 4269ensure that one of the following apply: 4270 42711. The cached dentry reflects an ondisk dirent in the new directory. 4272 42732. The cached dentry no longer has a corresponding ondisk dirent in the new 4274 directory and the dentry can be purged from the cache. 4275 42763. The cached dentry no longer has an ondisk dirent but the dentry cannot be 4277 purged. 4278 This is the problem case. 4279 4280Unfortunately, the current dentry cache design doesn't provide a means to walk 4281every child dentry of a specific directory, which makes this a hard problem. 4282There is no known solution. 4283 4284Parent Pointers 4285``````````````` 4286 4287A parent pointer is a piece of file metadata that enables a user to locate the 4288file's parent directory without having to traverse the directory tree from the 4289root. 4290Without them, reconstruction of directory trees is hindered in much the same 4291way that the historic lack of reverse space mapping information once hindered 4292reconstruction of filesystem space metadata. 4293The parent pointer feature, however, makes total directory reconstruction 4294possible. 4295 4296XFS parent pointers contain the information needed to identify the 4297corresponding directory entry in the parent directory. 4298In other words, child files use extended attributes to store pointers to 4299parents in the form ``(dirent_name) → (parent_inum, parent_gen)``. 4300The directory checking process can be strengthened to ensure that the target of 4301each dirent also contains a parent pointer pointing back to the dirent. 4302Likewise, each parent pointer can be checked by ensuring that the target of 4303each parent pointer is a directory and that it contains a dirent matching 4304the parent pointer. 4305Both online and offline repair can use this strategy. 4306 4307+--------------------------------------------------------------------------+ 4308| **Historical Sidebar**: | 4309+--------------------------------------------------------------------------+ 4310| Directory parent pointers were first proposed as an XFS feature more | 4311| than a decade ago by SGI. | 4312| Each link from a parent directory to a child file is mirrored with an | 4313| extended attribute in the child that could be used to identify the | 4314| parent directory. | 4315| Unfortunately, this early implementation had major shortcomings and was | 4316| never merged into Linux XFS: | 4317| | 4318| 1. The XFS codebase of the late 2000s did not have the infrastructure to | 4319| enforce strong referential integrity in the directory tree. | 4320| It did not guarantee that a change in a forward link would always be | 4321| followed up with the corresponding change to the reverse links. | 4322| | 4323| 2. Referential integrity was not integrated into offline repair. | 4324| Checking and repairs were performed on mounted filesystems without | 4325| taking any kernel or inode locks to coordinate access. | 4326| It is not clear how this actually worked properly. | 4327| | 4328| 3. The extended attribute did not record the name of the directory entry | 4329| in the parent, so the SGI parent pointer implementation cannot be | 4330| used to reconnect the directory tree. | 4331| | 4332| 4. Extended attribute forks only support 65,536 extents, which means | 4333| that parent pointer attribute creation is likely to fail at some | 4334| point before the maximum file link count is achieved. | 4335| | 4336| The original parent pointer design was too unstable for something like | 4337| a file system repair to depend on. | 4338| Allison Henderson, Chandan Babu, and Catherine Hoang are working on a | 4339| second implementation that solves all shortcomings of the first. | 4340| During 2022, Allison introduced log intent items to track physical | 4341| manipulations of the extended attribute structures. | 4342| This solves the referential integrity problem by making it possible to | 4343| commit a dirent update and a parent pointer update in the same | 4344| transaction. | 4345| Chandan increased the maximum extent counts of both data and attribute | 4346| forks, thereby ensuring that the extended attribute structure can grow | 4347| to handle the maximum hardlink count of any file. | 4348| | 4349| For this second effort, the ondisk parent pointer format as originally | 4350| proposed was ``(parent_inum, parent_gen, dirent_pos) → (dirent_name)``. | 4351| The format was changed during development to eliminate the requirement | 4352| of repair tools needing to ensure that the ``dirent_pos`` field always | 4353| matched when reconstructing a directory. | 4354| | 4355| There were a few other ways to have solved that problem: | 4356| | 4357| 1. The field could be designated advisory, since the other three values | 4358| are sufficient to find the entry in the parent. | 4359| However, this makes indexed key lookup impossible while repairs are | 4360| ongoing. | 4361| | 4362| 2. We could allow creating directory entries at specified offsets, which | 4363| solves the referential integrity problem but runs the risk that | 4364| dirent creation will fail due to conflicts with the free space in the | 4365| directory. | 4366| | 4367| These conflicts could be resolved by appending the directory entry | 4368| and amending the xattr code to support updating an xattr key and | 4369| reindexing the dabtree, though this would have to be performed with | 4370| the parent directory still locked. | 4371| | 4372| 3. Same as above, but remove the old parent pointer entry and add a new | 4373| one atomically. | 4374| | 4375| 4. Change the ondisk xattr format to | 4376| ``(parent_inum, name) → (parent_gen)``, which would provide the attr | 4377| name uniqueness that we require, without forcing repair code to | 4378| update the dirent position. | 4379| Unfortunately, this requires changes to the xattr code to support | 4380| attr names as long as 263 bytes. | 4381| | 4382| 5. Change the ondisk xattr format to ``(parent_inum, hash(name)) → | 4383| (name, parent_gen)``. | 4384| If the hash is sufficiently resistant to collisions (e.g. sha256) | 4385| then this should provide the attr name uniqueness that we require. | 4386| Names shorter than 247 bytes could be stored directly. | 4387| | 4388| 6. Change the ondisk xattr format to ``(dirent_name) → (parent_ino, | 4389| parent_gen)``. This format doesn't require any of the complicated | 4390| nested name hashing of the previous suggestions. However, it was | 4391| discovered that multiple hardlinks to the same inode with the same | 4392| filename caused performance problems with hashed xattr lookups, so | 4393| the parent inumber is now xor'd into the hash index. | 4394| | 4395| In the end, it was decided that solution #6 was the most compact and the | 4396| most performant. A new hash function was designed for parent pointers. | 4397+--------------------------------------------------------------------------+ 4398 4399 4400Case Study: Repairing Directories with Parent Pointers 4401^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4402 4403Directory rebuilding uses a :ref:`coordinated inode scan <iscan>` and 4404a :ref:`directory entry live update hook <liveupdate>` as follows: 4405 44061. Set up a temporary directory for generating the new directory structure, 4407 an xfblob for storing entry names, and an xfarray for stashing the fixed 4408 size fields involved in a directory update: ``(child inumber, add vs. 4409 remove, name cookie, ftype)``. 4410 44112. Set up an inode scanner and hook into the directory entry code to receive 4412 updates on directory operations. 4413 44143. For each parent pointer found in each file scanned, decide if the parent 4415 pointer references the directory of interest. 4416 If so: 4417 4418 a. Stash the parent pointer name and an addname entry for this dirent in the 4419 xfblob and xfarray, respectively. 4420 4421 b. When finished scanning that file or the kernel memory consumption exceeds 4422 a threshold, flush the stashed updates to the temporary directory. 4423 44244. For each live directory update received via the hook, decide if the child 4425 has already been scanned. 4426 If so: 4427 4428 a. Stash the parent pointer name an addname or removename entry for this 4429 dirent update in the xfblob and xfarray for later. 4430 We cannot write directly to the temporary directory because hook 4431 functions are not allowed to modify filesystem metadata. 4432 Instead, we stash updates in the xfarray and rely on the scanner thread 4433 to apply the stashed updates to the temporary directory. 4434 44355. When the scan is complete, replay any stashed entries in the xfarray. 4436 44376. When the scan is complete, atomically exchange the contents of the temporary 4438 directory and the directory being repaired. 4439 The temporary directory now contains the damaged directory structure. 4440 44417. Reap the temporary directory. 4442 4443Case Study: Repairing Parent Pointers 4444^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4445 4446Online reconstruction of a file's parent pointer information works similarly to 4447directory reconstruction: 4448 44491. Set up a temporary file for generating a new extended attribute structure, 4450 an xfblob for storing parent pointer names, and an xfarray for stashing the 4451 fixed size fields involved in a parent pointer update: ``(parent inumber, 4452 parent generation, add vs. remove, name cookie)``. 4453 44542. Set up an inode scanner and hook into the directory entry code to receive 4455 updates on directory operations. 4456 44573. For each directory entry found in each directory scanned, decide if the 4458 dirent references the file of interest. 4459 If so: 4460 4461 a. Stash the dirent name and an addpptr entry for this parent pointer in the 4462 xfblob and xfarray, respectively. 4463 4464 b. When finished scanning the directory or the kernel memory consumption 4465 exceeds a threshold, flush the stashed updates to the temporary file. 4466 44674. For each live directory update received via the hook, decide if the parent 4468 has already been scanned. 4469 If so: 4470 4471 a. Stash the dirent name and an addpptr or removepptr entry for this dirent 4472 update in the xfblob and xfarray for later. 4473 We cannot write parent pointers directly to the temporary file because 4474 hook functions are not allowed to modify filesystem metadata. 4475 Instead, we stash updates in the xfarray and rely on the scanner thread 4476 to apply the stashed parent pointer updates to the temporary file. 4477 44785. When the scan is complete, replay any stashed entries in the xfarray. 4479 44806. Copy all non-parent pointer extended attributes to the temporary file. 4481 44827. When the scan is complete, atomically exchange the mappings of the attribute 4483 forks of the temporary file and the file being repaired. 4484 The temporary file now contains the damaged extended attribute structure. 4485 44868. Reap the temporary file. 4487 4488Digression: Offline Checking of Parent Pointers 4489^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4490 4491Examining parent pointers in offline repair works differently because corrupt 4492files are erased long before directory tree connectivity checks are performed. 4493Parent pointer checks are therefore a second pass to be added to the existing 4494connectivity checks: 4495 44961. After the set of surviving files has been established (phase 6), 4497 walk the surviving directories of each AG in the filesystem. 4498 This is already performed as part of the connectivity checks. 4499 45002. For each directory entry found, 4501 4502 a. If the name has already been stored in the xfblob, then use that cookie 4503 and skip the next step. 4504 4505 b. Otherwise, record the name in an xfblob, and remember the xfblob cookie. 4506 Unique mappings are critical for 4507 4508 1. Deduplicating names to reduce memory usage, and 4509 4510 2. Creating a stable sort key for the parent pointer indexes so that the 4511 parent pointer validation described below will work. 4512 4513 c. Store ``(child_ag_inum, parent_inum, parent_gen, name_hash, name_len, 4514 name_cookie)`` tuples in a per-AG in-memory slab. The ``name_hash`` 4515 referenced in this section is the regular directory entry name hash, not 4516 the specialized one used for parent pointer xattrs. 4517 45183. For each AG in the filesystem, 4519 4520 a. Sort the per-AG tuple set in order of ``child_ag_inum``, ``parent_inum``, 4521 ``name_hash``, and ``name_cookie``. 4522 Having a single ``name_cookie`` for each ``name`` is critical for 4523 handling the uncommon case of a directory containing multiple hardlinks 4524 to the same file where all the names hash to the same value. 4525 4526 b. For each inode in the AG, 4527 4528 1. Scan the inode for parent pointers. 4529 For each parent pointer found, 4530 4531 a. Validate the ondisk parent pointer. 4532 If validation fails, move on to the next parent pointer in the 4533 file. 4534 4535 b. If the name has already been stored in the xfblob, then use that 4536 cookie and skip the next step. 4537 4538 c. Record the name in a per-file xfblob, and remember the xfblob 4539 cookie. 4540 4541 d. Store ``(parent_inum, parent_gen, name_hash, name_len, 4542 name_cookie)`` tuples in a per-file slab. 4543 4544 2. Sort the per-file tuples in order of ``parent_inum``, ``name_hash``, 4545 and ``name_cookie``. 4546 4547 3. Position one slab cursor at the start of the inode's records in the 4548 per-AG tuple slab. 4549 This should be trivial since the per-AG tuples are in child inumber 4550 order. 4551 4552 4. Position a second slab cursor at the start of the per-file tuple slab. 4553 4554 5. Iterate the two cursors in lockstep, comparing the ``parent_ino``, 4555 ``name_hash``, and ``name_cookie`` fields of the records under each 4556 cursor: 4557 4558 a. If the per-AG cursor is at a lower point in the keyspace than the 4559 per-file cursor, then the per-AG cursor points to a missing parent 4560 pointer. 4561 Add the parent pointer to the inode and advance the per-AG 4562 cursor. 4563 4564 b. If the per-file cursor is at a lower point in the keyspace than 4565 the per-AG cursor, then the per-file cursor points to a dangling 4566 parent pointer. 4567 Remove the parent pointer from the inode and advance the per-file 4568 cursor. 4569 4570 c. Otherwise, both cursors point at the same parent pointer. 4571 Update the parent_gen component if necessary. 4572 Advance both cursors. 4573 45744. Move on to examining link counts, as we do today. 4575 4576Rebuilding directories from parent pointers in offline repair would be very 4577challenging because xfs_repair currently uses two single-pass scans of the 4578filesystem during phases 3 and 4 to decide which files are corrupt enough to be 4579zapped. 4580This scan would have to be converted into a multi-pass scan: 4581 45821. The first pass of the scan zaps corrupt inodes, forks, and attributes 4583 much as it does now. 4584 Corrupt directories are noted but not zapped. 4585 45862. The next pass records parent pointers pointing to the directories noted 4587 as being corrupt in the first pass. 4588 This second pass may have to happen after the phase 4 scan for duplicate 4589 blocks, if phase 4 is also capable of zapping directories. 4590 45913. The third pass resets corrupt directories to an empty shortform directory. 4592 Free space metadata has not been ensured yet, so repair cannot yet use the 4593 directory building code in libxfs. 4594 45954. At the start of phase 6, space metadata have been rebuilt. 4596 Use the parent pointer information recorded during step 2 to reconstruct 4597 the dirents and add them to the now-empty directories. 4598 4599This code has not yet been constructed. 4600 4601.. _dirtree: 4602 4603Case Study: Directory Tree Structure 4604^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4605 4606As mentioned earlier, the filesystem directory tree is supposed to be a 4607directed acylic graph structure. 4608However, each node in this graph is a separate ``xfs_inode`` object with its 4609own locks, which makes validating the tree qualities difficult. 4610Fortunately, non-directories are allowed to have multiple parents and cannot 4611have children, so only directories need to be scanned. 4612Directories typically constitute 5-10% of the files in a filesystem, which 4613reduces the amount of work dramatically. 4614 4615If the directory tree could be frozen, it would be easy to discover cycles and 4616disconnected regions by running a depth (or breadth) first search downwards 4617from the root directory and marking a bitmap for each directory found. 4618At any point in the walk, trying to set an already set bit means there is a 4619cycle. 4620After the scan completes, XORing the marked inode bitmap with the inode 4621allocation bitmap reveals disconnected inodes. 4622However, one of online repair's design goals is to avoid locking the entire 4623filesystem unless it's absolutely necessary. 4624Directory tree updates can move subtrees across the scanner wavefront on a live 4625filesystem, so the bitmap algorithm cannot be applied. 4626 4627Directory parent pointers enable an incremental approach to validation of the 4628tree structure. 4629Instead of using one thread to scan the entire filesystem, multiple threads can 4630walk from individual subdirectories upwards towards the root. 4631For this to work, all directory entries and parent pointers must be internally 4632consistent, each directory entry must have a parent pointer, and the link 4633counts of all directories must be correct. 4634Each scanner thread must be able to take the IOLOCK of an alleged parent 4635directory while holding the IOLOCK of the child directory to prevent either 4636directory from being moved within the tree. 4637This is not possible since the VFS does not take the IOLOCK of a child 4638subdirectory when moving that subdirectory, so instead the scanner stabilizes 4639the parent -> child relationship by taking the ILOCKs and installing a dirent 4640update hook to detect changes. 4641 4642The scanning process uses a dirent hook to detect changes to the directories 4643mentioned in the scan data. 4644The scan works as follows: 4645 46461. For each subdirectory in the filesystem, 4647 4648 a. For each parent pointer of that subdirectory, 4649 4650 1. Create a path object for that parent pointer, and mark the 4651 subdirectory inode number in the path object's bitmap. 4652 4653 2. Record the parent pointer name and inode number in a path structure. 4654 4655 3. If the alleged parent is the subdirectory being scrubbed, the path is 4656 a cycle. 4657 Mark the path for deletion and repeat step 1a with the next 4658 subdirectory parent pointer. 4659 4660 4. Try to mark the alleged parent inode number in a bitmap in the path 4661 object. 4662 If the bit is already set, then there is a cycle in the directory 4663 tree. 4664 Mark the path as a cycle and repeat step 1a with the next subdirectory 4665 parent pointer. 4666 4667 5. Load the alleged parent. 4668 If the alleged parent is not a linked directory, abort the scan 4669 because the parent pointer information is inconsistent. 4670 4671 6. For each parent pointer of this alleged ancestor directory, 4672 4673 a. Record the parent pointer name and inode number in the path object 4674 if no parent has been set for that level. 4675 4676 b. If an ancestor has more than one parent, mark the path as corrupt. 4677 Repeat step 1a with the next subdirectory parent pointer. 4678 4679 c. Repeat steps 1a3-1a6 for the ancestor identified in step 1a6a. 4680 This repeats until the directory tree root is reached or no parents 4681 are found. 4682 4683 7. If the walk terminates at the root directory, mark the path as ok. 4684 4685 8. If the walk terminates without reaching the root, mark the path as 4686 disconnected. 4687 46882. If the directory entry update hook triggers, check all paths already found 4689 by the scan. 4690 If the entry matches part of a path, mark that path and the scan stale. 4691 When the scanner thread sees that the scan has been marked stale, it deletes 4692 all scan data and starts over. 4693 4694Repairing the directory tree works as follows: 4695 46961. Walk each path of the target subdirectory. 4697 4698 a. Corrupt paths and cycle paths are counted as suspect. 4699 4700 b. Paths already marked for deletion are counted as bad. 4701 4702 c. Paths that reached the root are counted as good. 4703 47042. If the subdirectory is either the root directory or has zero link count, 4705 delete all incoming directory entries in the immediate parents. 4706 Repairs are complete. 4707 47083. If the subdirectory has exactly one path, set the dotdot entry to the 4709 parent and exit. 4710 47114. If the subdirectory has at least one good path, delete all the other 4712 incoming directory entries in the immediate parents. 4713 47145. If the subdirectory has no good paths and more than one suspect path, delete 4715 all the other incoming directory entries in the immediate parents. 4716 47176. If the subdirectory has zero paths, attach it to the lost and found. 4718 4719.. _orphanage: 4720 4721The Orphanage 4722------------- 4723 4724Filesystems present files as a directed, and hopefully acyclic, graph. 4725In other words, a tree. 4726The root of the filesystem is a directory, and each entry in a directory points 4727downwards either to more subdirectories or to non-directory files. 4728Unfortunately, a disruption in the directory graph pointers result in a 4729disconnected graph, which makes files impossible to access via regular path 4730resolution. 4731 4732Without parent pointers, the directory parent pointer online scrub code can 4733detect a dotdot entry pointing to a parent directory that doesn't have a link 4734back to the child directory and the file link count checker can detect a file 4735that isn't pointed to by any directory in the filesystem. 4736If such a file has a positive link count, the file is an orphan. 4737 4738With parent pointers, directories can be rebuilt by scanning parent pointers 4739and parent pointers can be rebuilt by scanning directories. 4740This should reduce the incidence of files ending up in ``/lost+found``. 4741 4742When orphans are found, they should be reconnected to the directory tree. 4743Offline fsck solves the problem by creating a directory ``/lost+found`` to 4744serve as an orphanage, and linking orphan files into the orphanage by using the 4745inumber as the name. 4746Reparenting a file to the orphanage does not reset any of its permissions or 4747ACLs. 4748 4749This process is more involved in the kernel than it is in userspace. 4750The directory and file link count repair setup functions must use the regular 4751VFS mechanisms to create the orphanage directory with all the necessary 4752security attributes and dentry cache entries, just like a regular directory 4753tree modification. 4754 4755Orphaned files are adopted by the orphanage as follows: 4756 47571. Call ``xrep_orphanage_try_create`` at the start of the scrub setup function 4758 to try to ensure that the lost and found directory actually exists. 4759 This also attaches the orphanage directory to the scrub context. 4760 47612. If the decision is made to reconnect a file, take the IOLOCK of both the 4762 orphanage and the file being reattached. 4763 The ``xrep_orphanage_iolock_two`` function follows the inode locking 4764 strategy discussed earlier. 4765 47663. Use ``xrep_adoption_trans_alloc`` to reserve resources to the repair 4767 transaction. 4768 47694. Call ``xrep_orphanage_compute_name`` to compute the new name in the 4770 orphanage. 4771 47725. If the adoption is going to happen, call ``xrep_adoption_reparent`` to 4773 reparent the orphaned file into the lost and found and invalidate the dentry 4774 cache. 4775 47766. Call ``xrep_adoption_finish`` to commit any filesystem updates, release the 4777 orphanage ILOCK, and clean the scrub transaction. Call 4778 ``xrep_adoption_commit`` to commit the updates and the scrub transaction. 4779 47807. If a runtime error happens, call ``xrep_adoption_cancel`` to release all 4781 resources. 4782 47836. Userspace Algorithms and Data Structures 4784=========================================== 4785 4786This section discusses the key algorithms and data structures of the userspace 4787program, ``xfs_scrub``, that provide the ability to drive metadata checks and 4788repairs in the kernel, verify file data, and look for other potential problems. 4789 4790.. _scrubcheck: 4791 4792Checking Metadata 4793----------------- 4794 4795Recall the :ref:`phases of fsck work<scrubphases>` outlined earlier. 4796That structure follows naturally from the data dependencies designed into the 4797filesystem from its beginnings in 1993. 4798In XFS, there are several groups of metadata dependencies: 4799 4800a. Filesystem summary counts depend on consistency within the inode indices, 4801 the allocation group space btrees, and the realtime volume space 4802 information. 4803 4804b. Quota resource counts depend on consistency within the quota file data 4805 forks, inode indices, inode records, and the forks of every file on the 4806 system. 4807 4808c. The naming hierarchy depends on consistency within the directory and 4809 extended attribute structures. 4810 This includes file link counts. 4811 4812d. Directories, extended attributes, and file data depend on consistency within 4813 the file forks that map directory and extended attribute data to physical 4814 storage media. 4815 4816e. The file forks depends on consistency within inode records and the space 4817 metadata indices of the allocation groups and the realtime volume. 4818 This includes quota and realtime metadata files. 4819 4820f. Inode records depends on consistency within the inode metadata indices. 4821 4822g. Realtime space metadata depend on the inode records and data forks of the 4823 realtime metadata inodes. 4824 4825h. The allocation group metadata indices (free space, inodes, reference count, 4826 and reverse mapping btrees) depend on consistency within the AG headers and 4827 between all the AG metadata btrees. 4828 4829i. ``xfs_scrub`` depends on the filesystem being mounted and kernel support 4830 for online fsck functionality. 4831 4832Therefore, a metadata dependency graph is a convenient way to schedule checking 4833operations in the ``xfs_scrub`` program: 4834 4835- Phase 1 checks that the provided path maps to an XFS filesystem and detect 4836 the kernel's scrubbing abilities, which validates group (i). 4837 4838- Phase 2 scrubs groups (g) and (h) in parallel using a threaded workqueue. 4839 4840- Phase 3 scans inodes in parallel. 4841 For each inode, groups (f), (e), and (d) are checked, in that order. 4842 4843- Phase 4 repairs everything in groups (i) through (d) so that phases 5 and 6 4844 may run reliably. 4845 4846- Phase 5 starts by checking groups (b) and (c) in parallel before moving on 4847 to checking names. 4848 4849- Phase 6 depends on groups (i) through (b) to find file data blocks to verify, 4850 to read them, and to report which blocks of which files are affected. 4851 4852- Phase 7 checks group (a), having validated everything else. 4853 4854Notice that the data dependencies between groups are enforced by the structure 4855of the program flow. 4856 4857Parallel Inode Scans 4858-------------------- 4859 4860An XFS filesystem can easily contain hundreds of millions of inodes. 4861Given that XFS targets installations with large high-performance storage, 4862it is desirable to scrub inodes in parallel to minimize runtime, particularly 4863if the program has been invoked manually from a command line. 4864This requires careful scheduling to keep the threads as evenly loaded as 4865possible. 4866 4867Early iterations of the ``xfs_scrub`` inode scanner naïvely created a single 4868workqueue and scheduled a single workqueue item per AG. 4869Each workqueue item walked the inode btree (with ``XFS_IOC_INUMBERS``) to find 4870inode chunks and then called bulkstat (``XFS_IOC_BULKSTAT``) to gather enough 4871information to construct file handles. 4872The file handle was then passed to a function to generate scrub items for each 4873metadata object of each inode. 4874This simple algorithm leads to thread balancing problems in phase 3 if the 4875filesystem contains one AG with a few large sparse files and the rest of the 4876AGs contain many smaller files. 4877The inode scan dispatch function was not sufficiently granular; it should have 4878been dispatching at the level of individual inodes, or, to constrain memory 4879consumption, inode btree records. 4880 4881Thanks to Dave Chinner, bounded workqueues in userspace enable ``xfs_scrub`` to 4882avoid this problem with ease by adding a second workqueue. 4883Just like before, the first workqueue is seeded with one workqueue item per AG, 4884and it uses INUMBERS to find inode btree chunks. 4885The second workqueue, however, is configured with an upper bound on the number 4886of items that can be waiting to be run. 4887Each inode btree chunk found by the first workqueue's workers are queued to the 4888second workqueue, and it is this second workqueue that queries BULKSTAT, 4889creates a file handle, and passes it to a function to generate scrub items for 4890each metadata object of each inode. 4891If the second workqueue is too full, the workqueue add function blocks the 4892first workqueue's workers until the backlog eases. 4893This doesn't completely solve the balancing problem, but reduces it enough to 4894move on to more pressing issues. 4895 4896.. _scrubrepair: 4897 4898Scheduling Repairs 4899------------------ 4900 4901During phase 2, corruptions and inconsistencies reported in any AGI header or 4902inode btree are repaired immediately, because phase 3 relies on proper 4903functioning of the inode indices to find inodes to scan. 4904Failed repairs are rescheduled to phase 4. 4905Problems reported in any other space metadata are deferred to phase 4. 4906Optimization opportunities are always deferred to phase 4, no matter their 4907origin. 4908 4909During phase 3, corruptions and inconsistencies reported in any part of a 4910file's metadata are repaired immediately if all space metadata were validated 4911during phase 2. 4912Repairs that fail or cannot be repaired immediately are scheduled for phase 4. 4913 4914In the original design of ``xfs_scrub``, it was thought that repairs would be 4915so infrequent that the ``struct xfs_scrub_metadata`` objects used to 4916communicate with the kernel could also be used as the primary object to 4917schedule repairs. 4918With recent increases in the number of optimizations possible for a given 4919filesystem object, it became much more memory-efficient to track all eligible 4920repairs for a given filesystem object with a single repair item. 4921Each repair item represents a single lockable object -- AGs, metadata files, 4922individual inodes, or a class of summary information. 4923 4924Phase 4 is responsible for scheduling a lot of repair work in as quick a 4925manner as is practical. 4926The :ref:`data dependencies <scrubcheck>` outlined earlier still apply, which 4927means that ``xfs_scrub`` must try to complete the repair work scheduled by 4928phase 2 before trying repair work scheduled by phase 3. 4929The repair process is as follows: 4930 49311. Start a round of repair with a workqueue and enough workers to keep the CPUs 4932 as busy as the user desires. 4933 4934 a. For each repair item queued by phase 2, 4935 4936 i. Ask the kernel to repair everything listed in the repair item for a 4937 given filesystem object. 4938 4939 ii. Make a note if the kernel made any progress in reducing the number 4940 of repairs needed for this object. 4941 4942 iii. If the object no longer requires repairs, revalidate all metadata 4943 associated with this object. 4944 If the revalidation succeeds, drop the repair item. 4945 If not, requeue the item for more repairs. 4946 4947 b. If any repairs were made, jump back to 1a to retry all the phase 2 items. 4948 4949 c. For each repair item queued by phase 3, 4950 4951 i. Ask the kernel to repair everything listed in the repair item for a 4952 given filesystem object. 4953 4954 ii. Make a note if the kernel made any progress in reducing the number 4955 of repairs needed for this object. 4956 4957 iii. If the object no longer requires repairs, revalidate all metadata 4958 associated with this object. 4959 If the revalidation succeeds, drop the repair item. 4960 If not, requeue the item for more repairs. 4961 4962 d. If any repairs were made, jump back to 1c to retry all the phase 3 items. 4963 49642. If step 1 made any repair progress of any kind, jump back to step 1 to start 4965 another round of repair. 4966 49673. If there are items left to repair, run them all serially one more time. 4968 Complain if the repairs were not successful, since this is the last chance 4969 to repair anything. 4970 4971Corruptions and inconsistencies encountered during phases 5 and 7 are repaired 4972immediately. 4973Corrupt file data blocks reported by phase 6 cannot be recovered by the 4974filesystem. 4975 4976Checking Names for Confusable Unicode Sequences 4977----------------------------------------------- 4978 4979If ``xfs_scrub`` succeeds in validating the filesystem metadata by the end of 4980phase 4, it moves on to phase 5, which checks for suspicious looking names in 4981the filesystem. 4982These names consist of the filesystem label, names in directory entries, and 4983the names of extended attributes. 4984Like most Unix filesystems, XFS imposes the sparest of constraints on the 4985contents of a name: 4986 4987- Slashes and null bytes are not allowed in directory entries. 4988 4989- Null bytes are not allowed in userspace-visible extended attributes. 4990 4991- Null bytes are not allowed in the filesystem label. 4992 4993Directory entries and attribute keys store the length of the name explicitly 4994ondisk, which means that nulls are not name terminators. 4995For this section, the term "naming domain" refers to any place where names are 4996presented together -- all the names in a directory, or all the attributes of a 4997file. 4998 4999Although the Unix naming constraints are very permissive, the reality of most 5000modern-day Linux systems is that programs work with Unicode character code 5001points to support international languages. 5002These programs typically encode those code points in UTF-8 when interfacing 5003with the C library because the kernel expects null-terminated names. 5004In the common case, therefore, names found in an XFS filesystem are actually 5005UTF-8 encoded Unicode data. 5006 5007To maximize its expressiveness, the Unicode standard defines separate control 5008points for various characters that render similarly or identically in writing 5009systems around the world. 5010For example, the character "Cyrillic Small Letter A" U+0430 "а" often renders 5011identically to "Latin Small Letter A" U+0061 "a". 5012 5013The standard also permits characters to be constructed in multiple ways -- 5014either by using a defined code point, or by combining one code point with 5015various combining marks. 5016For example, the character "Angstrom Sign U+212B "Å" can also be expressed 5017as "Latin Capital Letter A" U+0041 "A" followed by "Combining Ring Above" 5018U+030A "◌̊". 5019Both sequences render identically. 5020 5021Like the standards that preceded it, Unicode also defines various control 5022characters to alter the presentation of text. 5023For example, the character "Right-to-Left Override" U+202E can trick some 5024programs into rendering "moo\\xe2\\x80\\xaegnp.txt" as "mootxt.png". 5025A second category of rendering problems involves whitespace characters. 5026If the character "Zero Width Space" U+200B is encountered in a file name, the 5027name will render identically to a name that does not have the zero width 5028space. 5029 5030If two names within a naming domain have different byte sequences but render 5031identically, a user may be confused by it. 5032The kernel, in its indifference to upper level encoding schemes, permits this. 5033Most filesystem drivers persist the byte sequence names that are given to them 5034by the VFS. 5035 5036Techniques for detecting confusable names are explained in great detail in 5037sections 4 and 5 of the 5038`Unicode Security Mechanisms <https://unicode.org/reports/tr39/>`_ 5039document. 5040When ``xfs_scrub`` detects UTF-8 encoding in use on a system, it uses the 5041Unicode normalization form NFD in conjunction with the confusable name 5042detection component of 5043`libicu <https://github.com/unicode-org/icu>`_ 5044to identify names with a directory or within a file's extended attributes that 5045could be confused for each other. 5046Names are also checked for control characters, non-rendering characters, and 5047mixing of bidirectional characters. 5048All of these potential issues are reported to the system administrator during 5049phase 5. 5050 5051Media Verification of File Data Extents 5052--------------------------------------- 5053 5054The system administrator can elect to initiate a media scan of all file data 5055blocks. 5056This scan after validation of all filesystem metadata (except for the summary 5057counters) as phase 6. 5058The scan starts by calling ``FS_IOC_GETFSMAP`` to scan the filesystem space map 5059to find areas that are allocated to file data fork extents. 5060Gaps between data fork extents that are smaller than 64k are treated as if 5061they were data fork extents to reduce the command setup overhead. 5062When the space map scan accumulates a region larger than 32MB, a media 5063verification request is sent to the disk as a directio read of the raw block 5064device. 5065 5066If the verification read fails, ``xfs_scrub`` retries with single-block reads 5067to narrow down the failure to the specific region of the media and recorded. 5068When it has finished issuing verification requests, it again uses the space 5069mapping ioctl to map the recorded media errors back to metadata structures 5070and report what has been lost. 5071For media errors in blocks owned by files, parent pointers can be used to 5072construct file paths from inode numbers for user-friendly reporting. 5073 50747. Conclusion and Future Work 5075============================= 5076 5077It is hoped that the reader of this document has followed the designs laid out 5078in this document and now has some familiarity with how XFS performs online 5079rebuilding of its metadata indices, and how filesystem users can interact with 5080that functionality. 5081Although the scope of this work is daunting, it is hoped that this guide will 5082make it easier for code readers to understand what has been built, for whom it 5083has been built, and why. 5084Please feel free to contact the XFS mailing list with questions. 5085 5086XFS_IOC_EXCHANGE_RANGE 5087---------------------- 5088 5089As discussed earlier, a second frontend to the atomic file mapping exchange 5090mechanism is a new ioctl call that userspace programs can use to commit updates 5091to files atomically. 5092This frontend has been out for review for several years now, though the 5093necessary refinements to online repair and lack of customer demand mean that 5094the proposal has not been pushed very hard. 5095 5096File Content Exchanges with Regular User Files 5097`````````````````````````````````````````````` 5098 5099As mentioned earlier, XFS has long had the ability to swap extents between 5100files, which is used almost exclusively by ``xfs_fsr`` to defragment files. 5101The earliest form of this was the fork swap mechanism, where the entire 5102contents of data forks could be exchanged between two files by exchanging the 5103raw bytes in each inode fork's immediate area. 5104When XFS v5 came along with self-describing metadata, this old mechanism grew 5105some log support to continue rewriting the owner fields of BMBT blocks during 5106log recovery. 5107When the reverse mapping btree was later added to XFS, the only way to maintain 5108the consistency of the fork mappings with the reverse mapping index was to 5109develop an iterative mechanism that used deferred bmap and rmap operations to 5110swap mappings one at a time. 5111This mechanism is identical to steps 2-3 from the procedure above except for 5112the new tracking items, because the atomic file mapping exchange mechanism is 5113an iteration of an existing mechanism and not something totally novel. 5114For the narrow case of file defragmentation, the file contents must be 5115identical, so the recovery guarantees are not much of a gain. 5116 5117Atomic file content exchanges are much more flexible than the existing swapext 5118implementations because it can guarantee that the caller never sees a mix of 5119old and new contents even after a crash, and it can operate on two arbitrary 5120file fork ranges. 5121The extra flexibility enables several new use cases: 5122 5123- **Atomic commit of file writes**: A userspace process opens a file that it 5124 wants to update. 5125 Next, it opens a temporary file and calls the file clone operation to reflink 5126 the first file's contents into the temporary file. 5127 Writes to the original file should instead be written to the temporary file. 5128 Finally, the process calls the atomic file mapping exchange system call 5129 (``XFS_IOC_EXCHANGE_RANGE``) to exchange the file contents, thereby 5130 committing all of the updates to the original file, or none of them. 5131 5132.. _exchrange_if_unchanged: 5133 5134- **Transactional file updates**: The same mechanism as above, but the caller 5135 only wants the commit to occur if the original file's contents have not 5136 changed. 5137 To make this happen, the calling process snapshots the file modification and 5138 change timestamps of the original file before reflinking its data to the 5139 temporary file. 5140 When the program is ready to commit the changes, it passes the timestamps 5141 into the kernel as arguments to the atomic file mapping exchange system call. 5142 The kernel only commits the changes if the provided timestamps match the 5143 original file. 5144 A new ioctl (``XFS_IOC_COMMIT_RANGE``) is provided to perform this. 5145 5146- **Emulation of atomic block device writes**: Export a block device with a 5147 logical sector size matching the filesystem block size to force all writes 5148 to be aligned to the filesystem block size. 5149 Stage all writes to a temporary file, and when that is complete, call the 5150 atomic file mapping exchange system call with a flag to indicate that holes 5151 in the temporary file should be ignored. 5152 This emulates an atomic device write in software, and can support arbitrary 5153 scattered writes. 5154 5155(This functionality was merged into mainline as of 2025) 5156 5157Vectorized Scrub 5158---------------- 5159 5160As it turns out, the :ref:`refactoring <scrubrepair>` of repair items mentioned 5161earlier was a catalyst for enabling a vectorized scrub system call. 5162Since 2018, the cost of making a kernel call has increased considerably on some 5163systems to mitigate the effects of speculative execution attacks. 5164This incentivizes program authors to make as few system calls as possible to 5165reduce the number of times an execution path crosses a security boundary. 5166 5167With vectorized scrub, userspace pushes to the kernel the identity of a 5168filesystem object, a list of scrub types to run against that object, and a 5169simple representation of the data dependencies between the selected scrub 5170types. 5171The kernel executes as much of the caller's plan as it can until it hits a 5172dependency that cannot be satisfied due to a corruption, and tells userspace 5173how much was accomplished. 5174It is hoped that ``io_uring`` will pick up enough of this functionality that 5175online fsck can use that instead of adding a separate vectored scrub system 5176call to XFS. 5177 5178(This functionality was merged into mainline as of 2025) 5179 5180Quality of Service Targets for Scrub 5181------------------------------------ 5182 5183One serious shortcoming of the online fsck code is that the amount of time that 5184it can spend in the kernel holding resource locks is basically unbounded. 5185Userspace is allowed to send a fatal signal to the process which will cause 5186``xfs_scrub`` to exit when it reaches a good stopping point, but there's no way 5187for userspace to provide a time budget to the kernel. 5188Given that the scrub codebase has helpers to detect fatal signals, it shouldn't 5189be too much work to allow userspace to specify a timeout for a scrub/repair 5190operation and abort the operation if it exceeds budget. 5191However, most repair functions have the property that once they begin to touch 5192ondisk metadata, the operation cannot be cancelled cleanly, after which a QoS 5193timeout is no longer useful. 5194 5195Defragmenting Free Space 5196------------------------ 5197 5198Over the years, many XFS users have requested the creation of a program to 5199clear a portion of the physical storage underlying a filesystem so that it 5200becomes a contiguous chunk of free space. 5201Call this free space defragmenter ``clearspace`` for short. 5202 5203The first piece the ``clearspace`` program needs is the ability to read the 5204reverse mapping index from userspace. 5205This already exists in the form of the ``FS_IOC_GETFSMAP`` ioctl. 5206The second piece it needs is a new fallocate mode 5207(``FALLOC_FL_MAP_FREE_SPACE``) that allocates the free space in a region and 5208maps it to a file. 5209Call this file the "space collector" file. 5210The third piece is the ability to force an online repair. 5211 5212To clear all the metadata out of a portion of physical storage, clearspace 5213uses the new fallocate map-freespace call to map any free space in that region 5214to the space collector file. 5215Next, clearspace finds all metadata blocks in that region by way of 5216``GETFSMAP`` and issues forced repair requests on the data structure. 5217This often results in the metadata being rebuilt somewhere that is not being 5218cleared. 5219After each relocation, clearspace calls the "map free space" function again to 5220collect any newly freed space in the region being cleared. 5221 5222To clear all the file data out of a portion of the physical storage, clearspace 5223uses the FSMAP information to find relevant file data blocks. 5224Having identified a good target, it uses the ``FICLONERANGE`` call on that part 5225of the file to try to share the physical space with a dummy file. 5226Cloning the extent means that the original owners cannot overwrite the 5227contents; any changes will be written somewhere else via copy-on-write. 5228Clearspace makes its own copy of the frozen extent in an area that is not being 5229cleared, and uses ``FIEDEUPRANGE`` (or the :ref:`atomic file content exchanges 5230<exchrange_if_unchanged>` feature) to change the target file's data extent 5231mapping away from the area being cleared. 5232When all other mappings have been moved, clearspace reflinks the space into the 5233space collector file so that it becomes unavailable. 5234 5235There are further optimizations that could apply to the above algorithm. 5236To clear a piece of physical storage that has a high sharing factor, it is 5237strongly desirable to retain this sharing factor. 5238In fact, these extents should be moved first to maximize sharing factor after 5239the operation completes. 5240To make this work smoothly, clearspace needs a new ioctl 5241(``FS_IOC_GETREFCOUNTS``) to report reference count information to userspace. 5242With the refcount information exposed, clearspace can quickly find the longest, 5243most shared data extents in the filesystem, and target them first. 5244 5245**Future Work Question**: How might the filesystem move inode chunks? 5246 5247*Answer*: To move inode chunks, Dave Chinner constructed a prototype program 5248that creates a new file with the old contents and then locklessly runs around 5249the filesystem updating directory entries. 5250The operation cannot complete if the filesystem goes down. 5251That problem isn't totally insurmountable: create an inode remapping table 5252hidden behind a jump label, and a log item that tracks the kernel walking the 5253filesystem to update directory entries. 5254The trouble is, the kernel can't do anything about open files, since it cannot 5255revoke them. 5256 5257**Future Work Question**: Can static keys be used to minimize the cost of 5258supporting ``revoke()`` on XFS files? 5259 5260*Answer*: Yes. 5261Until the first revocation, the bailout code need not be in the call path at 5262all. 5263 5264The relevant patchsets are the 5265`kernel freespace defrag 5266<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=defrag-freespace>`_ 5267and 5268`userspace freespace defrag 5269<https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=defrag-freespace>`_ 5270series. 5271 5272Shrinking Filesystems 5273--------------------- 5274 5275Removing the end of the filesystem ought to be a simple matter of evacuating 5276the data and metadata at the end of the filesystem, and handing the freed space 5277to the shrink code. 5278That requires an evacuation of the space at end of the filesystem, which is a 5279use of free space defragmentation!
~/
Read-only file preview.