Standard Library

#Text

Status: standard library specification draft.

This chapter defines the part of std.text that lib/std writes in ordinary hd over the language tier:

• the string methods above the representation intrinsics: trim, lower, split, replace, starts_with, lines, and repeat;
• the splitting and padding methods split_once, split_whitespace, pad_start, pad_end, and count;
• the char classification methods and to_digit;
• the UTF-8 conversions to_utf8 and string::from_utf8, and Utf8Error;
• the Parse trait, string.parse::[T](), and the Parse impls of bool, char, and string;
• the raw-text string prefix r.

The language tier keeps the representation of a string and the methods that expose it (Strings, String Indexing, String Methods):

The prefix mechanism, @str_prefix and std.ops.Template, is language tier too (Prefixed Strings).

#String Methods

std gives string these methods, beside the language-tier ones:

1. std-text.string.lowerlower uses Unicode Default Case Conversion with full mappings.
2. std-text.string.trimtrim removes the Unicode White_Space property at both ends.
3. std-text.string.splitsplit(separator) retains empty pieces between adjacent separators and at either end.
4. std-text.string.split.empty-separatorAn empty separator splits into one-scalar strings, with an empty input producing an empty list.
5. std-text.string.split.absentWith a non-empty separator, an input without that separator, including the empty string, yields one piece, so "".split(",") is [""].
6. std-text.string.replacereplace replaces non-overlapping matches from left to right.
7. std-text.string.replace.emptyAn empty old inserts the replacement at scalar boundaries.
8. std-text.string.starts-withstarts_with compares scalar sequences exactly and performs no normalization or case folding.
9. std-text.string.lineslines returns the pieces between \n separators, and removes a \r directly before each \n.
10. std-text.string.lines.finalA final \n ends the last line and starts no empty one, so "a\nb\n".lines() is ["a", "b"] and "".lines() is [].
11. std-text.string.repeatrepeat(count) joins count copies of the string, so a count of 0 gives "".

fn rows(text: string) -> usize:
    text.lines().len()  # 2 for "a\r\nb\n", as Rust's str::lines

fn rule(width: usize) -> string:
    "-".repeat(width)

#Splitting And Padding

std also gives string methods that split text a bounded number of times, split it into words, pad it to a width, and count matches:

fn setting(line: string) -> string:
    match line.split_once("="):
        .Some((key, value)) => "$key is $value"  # "port is 80=x" for "port=80=x"
        .None => "no setting"

fn first_two(path: string) -> List[string]:
    path.splitn("/", 2)  # ["a", "b/c"] for "a/b/c"

fn cell(text: string) -> string:
    text.pad_start(6)  # "    42" for "42"

fn word_count(text: string) -> usize:
    text.split_whitespace().len()  # 2 for " a\tb "

1. std-text.split-oncesplit_once(separator) returns .Some((before, after)), with the text before and after the first occurrence of separator.
2. std-text.split-once.absentIt returns .None when separator does not occur, so "".split_once("=") is .None.
3. std-text.split-once.emptyAn empty separator occurs at offset 0, so "ab".split_once("") is .Some(("", "ab")).
4. std-text.splitnsplitn(separator, n) returns the first n - 1 pieces that split(separator) returns, followed by the rest of the string after the last of those separators as the final piece.
5. std-text.splitn.fewA string with at most n - 1 separators gives the pieces of split(separator), so "a=b".splitn("=", 5) is ["a", "b"].
6. std-text.splitn.smallAn n of 1 gives the whole string as one piece, and an n of 0 gives [].
7. std-text.splitn.empty-separatorAn empty separator takes one scalar value per piece, as split does, so "héy".splitn("", 2) is ["h", "éy"] and "".splitn("", 2) is [].
8. std-text.split-whitespacesplit_whitespace returns, in order, each maximal run of scalar values that lack the Unicode White_Space property.
9. std-text.split-whitespace.no-emptyIt returns no empty piece, so an empty string, or one of whitespace only, gives [].
10. std-text.pad.lengthpad_start and pad_end measure a string's length in scalar values, not in bytes or display columns.
11. std-text.pad.startpad_start(width, fill) puts copies of fill before the string until its length is width.
12. std-text.pad.endpad_end(width, fill) puts the copies after the string instead.
13. std-text.pad.width-at-most-lengthA width at or below the string's length gives the string unchanged.
14. std-text.pad.fill-defaultfill defaults to the space U+0020, so "7".pad_end(3) is "7 ".
15. std-text.countcount(needle) returns the number of non-overlapping matches of needle, found from left to right, so "aaaa".count("aa") is 2.
16. std-text.count.emptyAn empty needle matches at every scalar boundary, so it gives the number of scalar values plus one, and "".count("") is 1.

Why. The names and results are Rust's splitn, split_once and split_whitespace, Python's str.count, and JavaScript's padStart and padEnd. A width in scalar values is the unit chars yields; display columns would need the East Asian Width tables. An empty needle counts the boundaries that replace with an empty old fills, as in Python.

#Character Classification

std gives char four classification methods and to_digit, as Rust does:

fn word_start(c: char) -> bool:
    c.is_alphabetic() || c == '_'

fn hex_value(c: char) -> u32?:
    c.to_digit(16)  # .Some(15) for 'f' and for 'F'

1. std-text.char.unicode-versionThe Unicode properties and general categories that this section names are those of Unicode 17.0.
2. std-text.char.to-digit.valueto_digit gives '0' to '9' the values 0 to 9, and gives 'a' to 'z' and 'A' to 'Z' the values 10 to 35.
3. std-text.char.to-digit.resultIt returns .Some of that value when the value is less than radix, and .None otherwise.
4. std-text.char.to-digit.ascii-onlyEvery other char gives .None, so '٣'.to_digit(10) is .None, though '٣'.is_alphanumeric() is true.
5. std-text.char.to-digit.radixA radix below 2 or above 36 panics. Panic: explicit-panic.

Why. The names and results are Rust's, so agents and readers already know them. to_digit reads ASCII only, so a number parser built on it accepts no digit of another script by accident.

#UTF-8 Conversion

to_utf8 copies a string's bytes, and string::from_utf8 checks bytes and makes a string of them, as types.string.from-bytes requires:

use std.text.Utf8Error

fn decode(bytes: List[u8]) -> string:
    match string::from_utf8(bytes):
        .Ok(text) => text
        .Err(Utf8Error.InvalidSequence(position)) => "bad byte at $position"
        .Err(Utf8Error.Truncated(position)) => "cut short at $position"

1. std-text.utf8.to-utf8to_utf8 returns a new list of the string's bytes, in order.
2. std-text.utf8.from-utf8string::from_utf8(bytes) returns .Ok of the string whose bytes are bytes when they are well-formed UTF-8, and .Err otherwise.
3. std-text.utf8.well-formedWell-formed UTF-8 is that of the Unicode Standard, so an overlong encoding, a surrogate code point, and a value above U+10FFFF are not well formed.
4. std-text.utf8.round-tripFor every string s, string::from_utf8(s.to_utf8()) is .Ok(s).
5. std-text.utf8.error.declared-positionsstd.text declares the enum Utf8Error, with the variants InvalidSequence(position: usize) and Truncated(position: usize). Code imports it, as in use std.text.Utf8Error.
6. std-text.utf8.error.firstThe error describes the first sequence, from the start of bytes, that is not well formed.
7. std-text.utf8.error.truncated-positionIt is Truncated(position) when the bytes end before that sequence has the length its first byte gives, where position is the byte offset at which that sequence starts. That holds only when each byte of it after the first is a continuation byte, 0x80 to 0xBF.
8. std-text.utf8.error.invalidOtherwise it is InvalidSequence(position), where position is the byte offset at which that sequence starts.
9. std-text.utf8.error.traitsUtf8Error implements Eq, Debug, and Display. Two errors are equal when they are the same variant with the same position.

Note. So [0xE2, 0x82] is Truncated(0), [0x68, 0xE2, 0x82] is Truncated(1), [0xC3, 0x28] is InvalidSequence(0), and the overlong [0xC0, 0xAF] is InvalidSequence(0).

#Parsing Text

Parse is the trait of a type that reads itself from text, as Rust's FromStr. string.parse::[T]() calls it:

use std.text.Parse

fn port(text: string) -> i32:
    match text.parse::[i32]():
        .Ok(value) => value
        .Err(_) => 8080

fn read[T < Parse](text: string) -> Result[T, T::Error]:
    T::parse(text)

fn flag(text: string) -> bool:
    text.parse::[bool]().unwrap_or(false)

The trait and the bool, char, and string impls are declared here. The number impls are in Integer Parsing.

pub trait Parse:
    type Error
    fn parse(text: string) -> Result[Self, Self::Error]

pub enum ParseBoolError:
    Invalid

pub enum ParseCharError:
    Empty
    TooMany

1. std-text.parse.traitstd.text declares the trait Parse as above. It is not a prelude name; code imports it, as in use std.text.Parse.
2. std-text.parse.wholeSelf::parse(text) reads the whole of text. An impl never trims, skips, or ignores any part of it.
3. std-text.parse.methodstd gives string the method parse[T < Parse](self) -> Result[T, T::Error], which returns T::parse(self). Calling it needs no import, and the call names the type, as in text.parse::[i32]().
4. std-text.parse.stringstring implements Parse with Error set to never, and parse returns .Ok of the same text.
5. std-text.parse.boolbool implements Parse with Error set to ParseBoolError. The text "true" gives .Ok(true), and "false" gives .Ok(false).
6. std-text.parse.bool-otherAny other text gives .Err(ParseBoolError.Invalid). That includes "", "True", "1", and text with a space.
7. std-text.parse.bool-errorstd.text declares ParseBoolError as above. It implements Eq, Debug, Display, and Error. Its Display text is provided string was not `true` or `false` .
8. std-text.parse.charchar implements Parse with Error set to ParseCharError. Text of exactly one Unicode scalar value gives .Ok of it.
9. std-text.parse.char-emptyEmpty text gives .Err(ParseCharError.Empty).
10. std-text.parse.char-manyText of two or more scalar values gives .Err(ParseCharError.TooMany).
11. std-text.parse.char-errorstd.text declares ParseCharError as above. It implements Eq, Debug, Display, and Error. Its Display texts are cannot parse a character from empty text for Empty and text has more than one character for TooMany.
12. std-text.parse.never-traitsnever implements Eq, Debug, Display, and Error, so Result[string, never] works wherever Result[T, ParseNumberError] does, and a generic reader over T < Parse[Error = E] and E < Display accepts string. No value of never exists, so none of their methods runs.

Why. Parse is Rust's FromStr: one trait gives std.cli and std.host a typed read without a function per type, and a user type can join it. An impl reads the whole text so that the same text means the same value everywhere, as numeric Parse does.

#Raw Text Prefix

The standard library declares one prefix, in std.text:

1. std-text.prefix.std.r-meaningSo r"\d+ $n" is the text \d+ followed by the Display text of n, and r"a\"b" keeps its backslash.
2. std-text.prefix.std.only-rr is the only standard prefix. std declares no b, so b"..." names nothing until a bytes type exists.
3. std-text.prefix.std.import-textstd.text declares the string prefix r. It is not a prelude name; code imports it, as in use std.text.r.

use std.text.r

fn digits(count: i32) -> string:
    r"\d{$count}"  # the text \d{ then count, then }

See also: Prefixed Strings, which specifies how a prefixed string calls its prefix function.

#More String Methods

std also gives string methods that test, search, and trim text:

fn extension(name: string) -> string?:
    match name.rfind("."):
        .Some(at) => .Some(name.slice(at + 1, name.len()))  # "gz" for "a.tar.gz"
        .None => .None

fn header(line: string) -> string?:
    if line.is_empty() || !line.contains(":"):
        return .None
    match line.strip_prefix("# "):
        .Some(rest) => .Some(rest.trim_end().upper())
        .None => .None

1. std-text.more.is-emptyis_empty is true exactly when the string has no bytes.
2. std-text.more.ends-withends_with compares scalar sequences exactly, as starts_with does.
3. std-text.more.containscontains(needle) is true when needle occurs in the string. An empty needle occurs in every string.
4. std-text.more.findfind(needle) returns the byte offset of the first occurrence of needle in .Some, or .None. An empty needle is found at 0.
5. std-text.rfindrfind(needle) returns the byte offset of the last occurrence of needle in .Some, or .None. Occurrences may overlap, so "aaaa".rfind("aa") is .Some(2).
6. std-text.rfind.emptyAn empty needle is found at the length of the string in bytes, as in Rust, so "abc".rfind("") is .Some(3).
7. std-text.rfind.linearrfind takes time linear in the lengths of the string and the needle together.
8. std-text.more.trim-endstrim_start removes the Unicode White_Space property at the start only, and trim_end at the end only.
9. std-text.more.upperupper uses Unicode Default Case Conversion with full mappings, as lower does.
10. std-text.more.stripstrip_prefix(prefix) returns the rest of the string after prefix in .Some when the string starts with it, and .None otherwise. strip_suffix does the same at the end.

Why. The names are Rust's str methods. find gives a byte offset, so its result is a valid slice bound.

#Joining And Building

The join method of List[string] and StringBuilder join many pieces of text:

use std.text.StringBuilder

fn csv(cells: List[string]) -> string:
    cells.join(",")

fn report(names: List[string]) -> string:
    out := StringBuilder::new()
    for name in names:
        out.push("- ")
        out.push(name)
        out.push_char('\n')
    out.build()

1. std-text.join.list-methodstd.text gives List[string] the method join(self, separator: string) -> string. Calling it needs no import.
2. std-text.join.list-resultjoin returns the parts in order with separator between each two, so one part gives that part and no part gives "".
3. std-text.builder.decl-structstd.text declares the struct StringBuilder, with private fields, which implements Debug. Code imports it, as in use std.text.StringBuilder.
4. std-text.builder.new-refStringBuilder::new() -> *mut StringBuilder returns an empty builder, a new object. Every copy of the reference pushes to that one builder.
5. std-text.builder.push-localStringBuilder.push(&mut self, text: string) -> void appends text.
6. std-text.builder.push-char-localStringBuilder.push_char(&mut self, c: char) -> void appends the UTF-8 encoding of c, one to four bytes.
7. std-text.builder.push-ascii-localStringBuilder.push_ascii(&mut self, byte: u8) -> void appends the byte byte as an ASCII character.
8. std-text.builder.push-ascii.rangeA byte of 128 or more panics. Panic: explicit-panic.
9. std-text.builder.reserve-localStringBuilder.reserve(&mut self, count: usize) -> void makes room for at least count more bytes without appending text.
10. std-text.builder.build-localStringBuilder.build(&mut self) -> string returns every text pushed since the builder was made or last built, in push order, joined with nothing between, and empties the builder.

Why. The method is Rust's [String].join(sep), and agents wrote it in usability probes. There is no free join(parts, separator), since it was a second spelling of the method. trait.inherent.unique-unifying.disjoint already allows an inherent implementation for List[string] alone.

A builder is one buffer of bytes. build hands it over and starts again, as Go's strings.Builder shares its buffer with the string it returns, so the string takes the bytes with no copy. push_ascii takes only ASCII, so every string the builder makes is valid UTF-8 by construction and build checks nothing; other characters go through push_char or push.

#Prefix Helpers

std.text gives a custom string prefix the helpers that r uses:

use std.ops.{Template, str_prefix}
use std.text.{interpolate, process_escapes}

@str_prefix
fn cooked(t: Template[i32]) -> string:
    let mut pieces: List[string] = []
    for piece in t.raw_parts:
        match process_escapes(piece):
            .Ok(text) => pieces.push(text)
            .Err(error) => pieces.push("?${error.position}")
    interpolate(Template { raw_parts: pieces, values: t.values })

1. std-text.prefix-helper.interpolatestd.text declares pub fn interpolate[T < Display](t: Template[T]) -> string, which joins the raw pieces and the values' Display text, alternating, starting with the first piece.
2. std-text.prefix-helper.escapesstd.text declares pub fn process_escapes(text: string) -> Result[string, EscapeError], which replaces each escape sequence of Escape Sequences with its meaning.
3. std-text.prefix-helper.escape-error-valuestd.text declares pub struct EscapeError, a value with one public field, position: usize, the byte offset of the first invalid escape's backslash. It implements Eq and Debug.
4. std-text.prefix-helper.final-backslashA lone final backslash is an invalid escape.

Why. The field is position, as in every other std error. Its byte unit is the one slice takes.