#Fs

Status: standard library specification draft.

This chapter defines std.fs, which lib/std writes in ordinary hd over the language tier:

the host capability traits FsRead and FsWrite;
the error enum FsError, and the Entry value that describes a file;
a free helper for each method of FsRead and FsWrite;
walk! and glob!, which find entries below a directory;
MemoryFs, the deterministic provider of both traits.

Paths are Path values. Which provider a command binds is CLI tier (Host Capabilities).

#File System Errors

Every file system operation reports one error enum:

pub enum FsError:
    NotFound(path: Path)
    PermissionDenied(path: Path)
    NotGranted(path: Path)
    AlreadyExists(path: Path)
    NotADirectory(path: Path)
    IsADirectory(path: Path)
    InvalidUtf8(path: Path)
    Other(message: string)

pub enum EntryKind:
    File
    Directory
    Symlink

pub struct Entry:
    pub path: Path
    pub kind: EntryKind
    pub size: u64
    pub modified: Timestamp? = .None

std-fs.types.entry-metadatastd.fs declares FsError, EntryKind, and Entry as above. Entry is a struct value: a copy is independent of the entry it was copied from. Code imports them, as in use std.fs.{Entry, FsError}.
std-fs.error.one-enumEvery method of FsRead and FsWrite reports its failure as an FsError.
std-fs.error.otherOther holds a failure that no other variant names, with the host's message, or the message of a std call such as an invalid glob pattern.
std-fs.error.traitsFsError, EntryKind, and Entry implement Eq, and FsError implements Display.
std-fs.entry.sizeAn Entry's size is the file's length in bytes.
std-fs.entry.modifiedEntry.modified is the entry's modification time as a Timestamp?, defaulting to .None.
std-fs.entry.modified-hostHost providers fill modified from filesystem metadata when its time is available and representable as a Timestamp.
std-fs.entry.modified-unknownA provider that cannot report the modification time returns .None for that field.
std-fs.error.not-grantedNotGranted(path) reports that the program's capability grant does not cover path, by Partial Deny.
std-fs.error.permission-deniedPermissionDenied(path) reports that the operating system refused the access.
std-fs.error.not-granted.plainThe Display text of NotGranted(path) is access to PATH is not granted, with the path for PATH.
std-fs.error.eq.variant-payloadTwo FsError values compare equal only when their variants and corresponding payloads are equal.

Why. NotGranted and PermissionDenied have different fixes: a flag or an hd.toml entry for the first, file modes for the second.

#Reading

pub trait FsRead:
    fn read_bytes!(self, path: Path) -> Result[List[u8], FsError]
    fn read_text!(self, path: Path) -> Result[string, FsError]
    fn list_dir!(self, path: Path) -> Result[List[Entry], FsError]
    fn stat!(self, path: Path) -> Result[Entry?, FsError]

std-fs.read.declstd.fs declares the host capability trait FsRead with the methods above. Code imports it, as in use std.fs.FsRead.
std-fs.read.bytesread_bytes! returns the whole content of the file at path.
std-fs.read.textread_text! returns that content as a string. Content that is not valid UTF-8 is .Err(FsError.InvalidUtf8(path)).
std-fs.read.list-dirlist_dir! returns one Entry for each entry of the directory at path.
std-fs.read.statstat! returns the Entry for path, or .None when nothing is there.

#Writing

pub trait FsWrite:
    fn write_bytes!(&mut self, path: Path, bytes: List[u8]) -> Result[void, FsError]
    fn write_text!(&mut self, path: Path, text: string) -> Result[void, FsError]
    fn append_text!(&mut self, path: Path, text: string) -> Result[void, FsError]
    fn create_dir_all!(&mut self, path: Path) -> Result[void, FsError]
    fn remove!(&mut self, path: Path) -> Result[void, FsError]
    fn rename!(&mut self, from: Path, to: Path) -> Result[void, FsError]
    fn copy!(&mut self, from: Path, to: Path) -> Result[void, FsError]

std-fs.write.copy-declstd.fs declares the host capability trait FsWrite with the methods above. Code imports it, as in use std.fs.FsWrite.
std-fs.write.replacewrite_bytes! and write_text! create the file at path, or replace its whole content.
std-fs.write.appendappend_text! adds text at the end of the file at path, and creates the file when there is none.
std-fs.write.create-dir-allcreate_dir_all! creates the directory at path and every missing directory above it.
std-fs.write.removeremove! removes the file or the empty directory at path.
std-fs.write.renamerename! moves the entry at from to to.
std-fs.write.mut-ref.reference-valuesEach FsWrite method takes &mut self, so FsWrite is a mutable requirement trait and $.use(FsWrite) is a &mut dyn FsWrite, by req.ref.use.reference-values. Each FsRead method takes self, so FsRead is a readonly one, and a reference installed as its provider reads the place it refers to, by req.ref.share.reference-values.
std-fs.cancel.writeCancelling the suspension of write_bytes!, write_text!, or append_text! may leave a partial file at path.

#Copying Files

std-fs.copy.bytesFsWrite.copy!(from: Path, to: Path) -> Result[void, FsError] copies the source file's bytes to the destination.
std-fs.copy.replaceCopying creates a missing destination file or replaces its entire content.
std-fs.copy.hostA host provider copies inside the host, without transferring the file's bytes into the program.
std-fs.copy.read-grantThe host checks the source against the FsRead grant, as for read_bytes!.
std-fs.copy.write-grantThe host checks the destination against the FsWrite grant, as for write_bytes!.
std-fs.copy.refusedA refused endpoint returns NotGranted naming that endpoint, before copying any bytes.
std-fs.copy.source-missingA missing source returns NotFound(from).
std-fs.copy.source-directoryA directory source returns IsADirectory(from).
std-fs.copy.destination-directoryA destination directory returns IsADirectory(to).
std-fs.copy.destination-parent-fileA destination below a file returns NotADirectory(to).
std-fs.copy.aliasCopying a file to another path for the same file succeeds without changing its bytes.
std-fs.memory.copyMemoryFs.copy! reads the source bytes and writes them to the destination with its existing read and write errors.
std-fs.helper.copypub fn copy!(from: Path, to: Path) -> Result[void, FsError] $ FsWrite calls copy!(from, to) on the covering FsWrite provider.

Why. Copying is one host operation: a program need not read bytes only to send them straight back. Separate source and destination grants preserve the authority of the corresponding read and write operations.

#Suspension

std-fs.suspendsFile system access is I/O, so every method of FsRead and FsWrite is a bang method.
std-fs.special-fileA read of a special file, such as a FIFO, a device, or a named pipe, lets other tasks run while it waits, by std-host.op.wait.
std-fs.special-file.deadlineSuch a read has no other deadline guarantee.

use std.fs.FsRead
use std.path.Path

fn show!(path: Path) -> string $ FsRead:
    match $.use(FsRead).read_text!(path):   # a bang call: it suspends on I/O
        .Ok(text) => text
        .Err(_) => ""

Why. Reading and writing are separate traits, so a row says which one a function needs. A program that only reads gains no authority to change files.

#File Helpers

use std.fs.{FsError, FsRead, FsWrite, read_text, write_text}
use std.path.Path

fn copy_notes!() -> Result[void, FsError] $ FsRead + FsWrite:
    text := read_text!(Path("notes.txt"))?
    write_text!(Path("notes.bak"), text)

std-fs.helper.read-textstd.fs declares pub fn read_text!(path: Path) -> Result[string, FsError] $ FsRead, which calls read_text!(path) on the FsRead provider that covers the call.
std-fs.helper.write-textstd.fs declares pub fn write_text!(path: Path, text: string) -> Result[void, FsError] $ FsWrite, which calls write_text!(path, text) on the FsWrite provider that covers the call.
std-fs.helper.suspendsEvery helper in this section is a bang function. std declares no helper that reads or writes a file without suspending.
std-fs.helper.read-bytesstd.fs declares pub fn read_bytes!(path: Path) -> Result[List[u8], FsError] $ FsRead, which calls read_bytes!(path) on the covering FsRead provider.
std-fs.helper.list-dirstd.fs declares pub fn list_dir!(path: Path) -> Result[List[Entry], FsError] $ FsRead, which calls list_dir!(path) on the covering FsRead provider.
std-fs.helper.statstd.fs declares pub fn stat!(path: Path) -> Result[Entry?, FsError] $ FsRead, which calls stat!(path) on the covering FsRead provider.
std-fs.helper.write-bytesstd.fs declares pub fn write_bytes!(path: Path, bytes: List[u8]) -> Result[void, FsError] $ FsWrite, which calls write_bytes!(path, bytes) on the covering FsWrite provider.
std-fs.helper.append-textstd.fs declares pub fn append_text!(path: Path, text: string) -> Result[void, FsError] $ FsWrite, which calls append_text!(path, text) on the covering FsWrite provider.
std-fs.helper.create-dir-allstd.fs declares pub fn create_dir_all!(path: Path) -> Result[void, FsError] $ FsWrite, which calls create_dir_all!(path) on the covering FsWrite provider.
std-fs.helper.removestd.fs declares pub fn remove!(path: Path) -> Result[void, FsError] $ FsWrite, which calls remove!(path) on the covering FsWrite provider.
std-fs.helper.renamestd.fs declares pub fn rename!(from: Path, to: Path) -> Result[void, FsError] $ FsWrite, which calls rename!(from, to) on the covering FsWrite provider.
std-fs.helper.rowA helper's row names exactly the capability of the method it calls: FsRead for read_text!, read_bytes!, list_dir!, and stat!, and FsWrite for the others.

use std.fs.{FsError, append_text, list_dir, stat}
use std.fs.{EntryKind, FsRead, FsWrite}
use std.path.Path

fn log_line!(path: Path, line: string) -> Result[bool, FsError] $ FsRead + FsWrite:
    append_text!(path, "${line}\n")?
    .Ok(stat!(path)?.is_some())

fn count_files!(dir: Path) -> Result[usize, FsError] $ FsRead:
    let mut count: usize = 0
    for entry in list_dir!(dir)?:
        if entry.kind == EntryKind.File:
            count = count + 1
    .Ok(count)

Why. A non-suspending wrapper would give each file operation two spellings. A program that reads files declares main!.

No exists!. stat! already answers a missing path with .None, so stat!(path)?.is_some() says whether anything is there, and keeps a failed check, such as a denied path, apart from "missing". One atom, not two.

#Walking And Globbing

use std.fs.{EntryKind, FsError, FsRead, glob, walk}
use std.path.Path

fn markdown!(root: Path) -> Result[List[Path], FsError] $ FsRead:
    let mut found: List[Path] = []
    for entry in walk!(root)?:
        if entry.kind == EntryKind.File && entry.path.extension() == .Some("md"):
            found.push(entry.path)
    .Ok(found)

fn markdown_again!() -> Result[List[Path], FsError] $ FsRead:
    glob!("docs/**/*.md")

walk! lists a whole tree and glob! lists the paths that fit a pattern. Both read only through list_dir! and stat! of the covering FsRead provider.

std-fs.walk.declstd.fs declares pub fn walk!(root: Path) -> Result[List[Entry], FsError] $ FsRead.
std-fs.walk.entrieswalk! returns one Entry for every entry below root, at any depth, and none for root itself. Each Entry is the one list_dir! returns for it.
std-fs.walk.orderThe order is depth first: the entries of one directory come in the byte order of their path text, and a directory's entry comes directly before the entries under it.
std-fs.walk.symlinksA symlink is one entry, with the kind Symlink. walk! does not look into it.
std-fs.walk.emptyA directory with no entries adds only its own entry, and an empty root gives an empty list.
std-fs.walk.errorWhen a list_dir! call fails, walk! returns that call's .Err and no list. So a missing root is NotFound(root), and a file as root is NotADirectory(root) on a provider that reports it.
std-fs.glob.declstd.fs declares pub fn glob!(pattern: string) -> Result[List[Path], FsError] $ FsRead.
std-fs.glob.componentsglob! splits pattern at / into components and ignores empty ones. A pattern with no component matches nothing.
std-fs.glob.starWithin a component, * matches any run of characters, the empty run included.
std-fs.glob.one? matches exactly one character.
std-fs.glob.class[ starts a class that ends at the next ] and matches one character: one of the listed characters, or one in a range low-high. A ! right after [ makes the class match any character it does not list. A - is a range only between two characters, and is a plain character first or last in the class.
std-fs.glob.literalAny other character matches itself, and there is no escape. To match *, ?, or [ itself, write it in a class, as in [*].
std-fs.glob.dot*, ?, and a class match a leading . of a name like any other character.
std-fs.glob.deepA component that is exactly ** matches zero or more directories. The pieces ** inside a longer component are two *.
std-fs.glob.deep-lastA ** as the last component matches every entry below the directory before it, at any depth, and not that directory itself.
std-fs.glob.directoriesA component before the last matches only a directory. The last component matches a file, a directory, or a symlink.
std-fs.glob.symlinksNo component, ** included, looks into a symlink.
std-fs.glob.relativeA pattern that does not start with / is relative: glob! matches below the current directory, and its results do not start with /.
std-fs.glob.absoluteA pattern that starts with / is absolute: glob! matches below /, and its results start with /.
std-fs.glob.resultEach result is the matched components and entry names joined by /. The list is in the byte order of the path text, with no path twice.
std-fs.glob.noneA pattern that matches nothing gives .Ok([]). A directory that is missing or is a file matches nothing, and is not an error.
std-fs.glob.errorA pattern with an unclosed [, an empty class ([] or [!]), or a range whose low end is after its high end is .Err(FsError.Other(message)), where message is invalid glob pattern PATTERN: REASON. REASON is unclosed [, empty class, or range LOW-HIGH runs backwards.
std-fs.glob.error.firstglob! checks every component before it reads the file system, so a bad pattern fails even when an earlier component matches nothing. A list_dir! or stat! failure ends glob! with that call's .Err.
std-fs.glob.rowwalk! and glob! have the row FsRead, and they are bang functions by std-fs.helper.suspends.

Why. The tree walk is the loop every file tool writes by hand, and a pattern is how people name a set of files. Both build on FsRead, so MemoryFs tests them with no host support. Go's filepath.WalkDir and Rust's walkdir also visit a directory before its contents. A fixed order makes the result the same on every host.

Why symlinks stay closed. A symlink to a parent directory would make a walk run forever. walkdir follows links only on request, and Go's WalkDir never follows them. A program that wants links followed reads the Symlink entry and walks its target.

Why glob! is relative to the current directory. Python's glob and the glob crate resolve a relative pattern the same way, and ! negates a class in both.

#Memory File System

MemoryFs is the deterministic provider of FsRead and FsWrite. It holds its files in memory, and a test supplies it with $.with:

use std.fs.{FsError, FsRead, FsWrite, MemoryFs, read_text, write_text}
use std.path.Path
fn copy_notes!() -> Result[void, FsError] $ FsRead + FsWrite:
    text := read_text!(Path("notes.txt"))?
    write_text!(Path("notes.bak"), text)

tests:
    use std.testing.assert

    it("copies in memory"):
        files := MemoryFs::new()
        assert(files.write_text!(Path("notes.txt"), "tea").is_ok(), reason="seeded")
        $.with(FsRead=files, FsWrite=files):
            assert(copy_notes!().is_ok(), reason="copied")
        assert(files.read_text!(Path("notes.bak")).unwrap_or("") == "tea", reason="the copy")

std-fs.memory.declstd.fs declares MemoryFs, which implements FsRead and FsWrite, with private fields. Code imports it, as in use std.fs.MemoryFs.
std-fs.memory.new-refMemoryFs::new() -> *mut MemoryFs returns a new object holding a file system with no files and no directories. MemoryFs is a struct used behind that reference: installing it as FsRead=files, FsWrite=files makes both providers refer to the one object, so a test reads back through files what the code under test wrote.
std-fs.memory.normalized-keysEvery MemoryFs operation drops empty and . components from its path keys, so ./a.txt and a.txt name one entry.
std-fs.memory.rooted-keysA normalized key keeps a leading / when the supplied path starts with /.
std-fs.memory.literal-parentNormalization leaves .. components unchanged, without resolving parent directories.
std-fs.memory.normalized-rootA relative path with no remaining components names ., and an absolute one names /.
std-fs.memory.supplied-errorsErrors retain the caller's supplied path, including its original spelling.
std-fs.memory.dirsA directory exists when create_dir_all! made it, or when its path is the text before a / in the path of a file. So writing a file makes the directories above it.
std-fs.memory.create-dir-allcreate_dir_all! records path and every directory above it, so each stays after the entries inside it are removed.
std-fs.memory.entryA file's Entry has the kind File and its length in bytes. A directory's has the kind Directory and the size 0. No entry is a Symlink.
std-fs.memory.normalized-list-orderlist_dir! returns the entries directly inside the directory, in the byte order of their path text. The returned entry paths use normalized keys.
std-fs.memory.normalized-root-directoryThe roots . and / always exist, and stat! returns a Directory entry for either.
std-fs.memory.relative-root-listlist_dir!(Path(".")) lists direct relative children with bare paths, such as a.md.
std-fs.memory.renamerename! of a directory moves every entry under it as well.
std-fs.memory.no-hostA MemoryFs never reads or changes the host's file system.

Its failures are these, and it reports no other. Each error holds the path that fails, which for a rename is to unless from is missing:

Rule	Operation	Error
std-fs.memory.error.not-foundNothing there	reading, listing, removing, or renaming a path where no entry is	`NotFound(path)`, with the missing path
std-fs.memory.error.is-directoryA directory as a file	reading a directory, or writing, appending, or renaming a file to a directory's path	`IsADirectory(path)`
std-fs.memory.error.not-directoryA file as a directory	listing a file, creating a directory at a file's path, renaming a directory onto a file, or writing, appending, creating, or renaming to a path below a file	`NotADirectory(path)`
std-fs.memory.error.not-emptyA directory with entries	removing it	`Other(message)`
std-fs.memory.error.utf8Not text	`read_text!` of content that is not valid UTF-8	`InvalidUtf8(path)`, as `std-fs.read.text` states

Why. A test states the files its code reads, and reads back what the code wrote, with nothing on disk to set up or clean. Go's fstest.MapFS serves tests the same way.

#Scripted Modification Times

MemoryFs records only the times a test supplies; it has no clock.

std-fs.memory.modified-noneA MemoryFs entry has modified equal to .None until its time is scripted.
std-fs.memory.set-modifiedset_modified(&mut self, path: Path, time: Timestamp) -> Result[void, FsError] sets an existing file's or directory's modification time.
std-fs.memory.modified-missingSetting a time on a missing path returns .Err(FsError.NotFound(path)) with the supplied path.
std-fs.memory.modified-writeA successful write_bytes!, write_text!, or append_text! clears that entry's scripted time to .None.
std-fs.memory.modified-renameRenaming an entry moves its scripted time, including the times of descendants moved with a directory.
std-fs.memory.modified-removeRemoving an entry removes its scripted time.

Why. A test can script deterministic ordering without reading a clock. After a write, retaining an old scripted time could hide stale metadata, so the new time is unknown until the test supplies it again.