hd-lang

#Regex

Status: standard library specification draft.

This chapter defines std.regex, which lib/std writes in ordinary hd over the language tier:

  • Regex, a compiled pattern, and Regex::new, which compiles one;
  • the pattern syntax, a subset of RE2's;
  • is_match, find, and find_all, which search a text in linear time;
  • captures and captures_all, which report the text of each group;
  • replace, replace_all, and split;
  • Match, where a match is, Captures, the groups of a match, and RegexError, why a pattern does not compile.

A pattern is ordinary text, so a backslash in it is written \\ in a plain string. The prefix r of std.text keeps backslashes as written (Raw Text Prefix), and \$ keeps a dollar sign from starting an interpolation (lex.interp.escaped-dollar).

#Compiling A Pattern

use std.regex.{Regex, RegexError}
use std.text.r

fn digits() -> Result[Regex, RegexError]:
    Regex::new(r"\d+")
Rule Item Meaning
std-regex.regex.newnew pub fn new(pattern: string) -> Result[Regex, RegexError] the compiled pattern, or the first error in it
std-regex.regex.patternpattern pub fn pattern(self) -> string the pattern as written
  1. std-regex.regex.decl-structstd.regex declares the struct Regex, a value with private fields, and the methods in this chapter. Code imports it, as in use std.regex.Regex.
  2. std-regex.regex.plainEvery function and method in this chapter is a plain call with the empty requirement row.
  3. std-regex.regex.immutableA Regex never changes after new, and searching with it leaves it unchanged.
  4. std-regex.regex.debugRegex implements Debug, which shows the pattern.

Note. Write a pattern as a raw string, r"...". Then \d needs no second backslash, and a $ before ", ), or | stays text (lex.interp.dollar-text). A $ before a letter or { still starts an interpolation, in a raw string too.

use std.regex.Regex
use std.text.r

fn whole_number(line: string) -> bool:
    match Regex::new(r"^\d+$"):
        .Ok(pattern) => pattern.is_match(line)
        .Err(_) => false

#Pattern Syntax

A pattern is read as a sequence of characters, each a Unicode scalar value. The forms below are the whole syntax.

#Literals And Escapes

Rule Form Matches
std-regex.syntax.literalLiteral a character with no meaning below that character
std-regex.syntax.dotAny character . any one character except \n
std-regex.syntax.escape.metaEscaped punctuation \ and an ASCII punctuation character, as in \. or \\ that character
std-regex.syntax.escape.controlControl escape \n, \t, \r a line feed, a tab, a carriage return
std-regex.syntax.escape.digitDigit \d; \D an ASCII digit 0 to 9; any other character
std-regex.syntax.escape.wordWord \w; \W an ASCII letter, digit, or _; any other character
std-regex.syntax.escape.spaceSpace \s; \S a character with the Unicode White_Space property; any other character
  1. std-regex.syntax.punctuationThe ASCII punctuation characters are those from ! to /, from : to @, from [ to `, and from { to ~.
  2. std-regex.syntax.escape.otherAny other escape, such as \b, \q, or \1, or a \ that ends the pattern, is an error. Error: BadEscape.
  3. std-regex.syntax.brace-literalA { that does not begin a counted repetition, as in a{ or a{x}, is a literal. So are } and ] outside a class.
use std.regex.{Regex, Match}
use std.text.r

fn found(pattern: string, text: string) -> string?:
    match Regex::new(pattern):
        .Ok(compiled) => compiled.find(text).map(fn(m: Match) -> string: m.text)
        .Err(_) => .None

fn demo() -> List[string]:
    [found(r"a\.b", "axb a.b").unwrap_or(""),
     found(r"a\tb", "a\tb").unwrap_or(""),
     found(r"\d+", "abc42").unwrap_or("")]
    # ["a.b", "a<TAB>b", "42"]

#Classes

Rule Form Matches
std-regex.syntax.classClass [ items ] a character that some item matches
std-regex.syntax.class.negatedNegated class [^ items ] a character that no item matches, \n included
  1. std-regex.syntax.class.itemAn item is a character, a range a-z of two characters, or one of \d, \D, \w, \W, \s, and \S.
  2. std-regex.syntax.class.charAn item's character is any character but \, or an escape of Literals And Escapes that stands for one character.
  3. std-regex.syntax.class.rangeA range a-z matches every character from a to z, compared by scalar value.
  4. std-regex.syntax.class.close-firstA ] right after [ or [^ is an item, not the end, so []a] matches ] or a.
  5. std-regex.syntax.class.dashA - first, last, or right after a range is an item, as in [-a] or [a-].
  6. std-regex.syntax.class.unclosedA class with no closing ] is an error, at its [. Error: MissingBracket.
  7. std-regex.syntax.class.bad-rangeA range whose end is below its start is an error, at the range's first character. So is one whose start or end is \d, \w, \s, or their negations, at the same place. Error: BadRange.
use std.regex.Regex
use std.text.r

fn hex_digit(text: string) -> bool:
    match Regex::new(r"^[0-9a-f]$"):
        .Ok(pattern) => pattern.is_match(text)
        .Err(_) => false

fn demo() -> List[bool]:
    [hex_digit("f"), hex_digit("G")]   # [true, false]

#Anchors, Groups, And Alternation

Rule Form Matches
std-regex.syntax.beginBegin ^ the empty text at the start of the searched text
std-regex.syntax.endEnd $ the empty text at the end of the searched text
std-regex.syntax.groupGroup ( pattern ) what the inner pattern matches
std-regex.syntax.group.plainPlain group (?: pattern ) what the inner pattern matches
std-regex.syntax.group.namedNamed group (?P<name> pattern ) or (?<name> pattern ) what the inner pattern matches
std-regex.syntax.alternationAlternation pattern | pattern what either side matches
std-regex.syntax.emptyEmpty nothing, as in (), a|, or the pattern "" the empty text
  1. std-regex.syntax.anchor.text^ and $ see only the ends of the whole text. A \n inside it is no start or end.
  2. std-regex.syntax.group.numberA ( ) group and a named group are capture groups, numbered from 1 in the order of their (. A (?: ) group has no number.
  3. std-regex.syntax.group.nameA group's name is an ASCII letter or _, then any number of ASCII letters, digits, and _.
  4. std-regex.syntax.group.bad-nameA named group whose name breaks that rule, or that has no > after <, is an error, at the (. Error: BadGroupName.
  5. std-regex.syntax.group.duplicate-nameA name that an earlier group already has is an error, at the later group's (. Error: DuplicateGroupName.
  6. std-regex.syntax.group.unsupportedA (? that begins none of (?:, (?P<, and (?< is an error, at the (. So are (?<= and (?<!. Error: UnsupportedGroup.
  7. std-regex.syntax.group.unclosedA ( with no matching ) is an error, at the (. Error: MissingParen.
  8. std-regex.syntax.group.unopenedA ) with no matching ( is an error, at the ). Error: UnmatchedParen.
  9. std-regex.syntax.precedenceRepetition binds tighter than concatenation, and concatenation binds tighter than |. So ab|cd* is (?:ab)|(?:c(?:d*)).
use std.regex.Regex
use std.text.r

fn yes_or_no(line: string) -> bool:
    match Regex::new(r"^(yes|no)$"):
        .Ok(pattern) => pattern.is_match(line)
        .Err(_) => false

fn demo() -> List[bool]:
    [yes_or_no("yes"), yes_or_no("yesno"), yes_or_no("maybe")]
    # [true, false, false]

#Repetition

A quantifier follows the item it repeats: a character, ., an escape, a class, an anchor, or a group.

Rule Quantifier Repeats the item Lazy form
std-regex.repeat.starStar * zero or more times *?
std-regex.repeat.plusPlus + one or more times +?
std-regex.repeat.optionalOptional ? zero times or once ??
std-regex.repeat.exactExact {m} m times {m}?
std-regex.repeat.at-leastAt least {m,} m or more times {m,}?
std-regex.repeat.betweenBetween {m,n} from m to n times {m,n}?
  1. std-regex.repeat.countsm and n are runs of ASCII digits.
  2. std-regex.repeat.greedyA quantifier prefers more repetitions, and its lazy form prefers fewer. The preference picks among matches that start at one position; see Searching.
  3. std-regex.repeat.bad-countA count above 1000, or an n below m, is an error, at the {. Error: BadRepeat.
  4. std-regex.repeat.nothingA quantifier with no item before it is an error, at the quantifier. That covers one at the start of the pattern or a group, and one after |. It also covers one right after another quantifier, as in a** or a{2}{3}. Error: NothingToRepeat.
use std.regex.{Regex, Match}
use std.text.r

fn year(line: string) -> string?:
    match Regex::new(r"(?:19|20)\d{2}"):
        .Ok(pattern) => pattern.find(line).map(fn(found: Match) -> string: found.text)
        .Err(_) => .None

Why. This is the syntax that RE2 and Go's regexp share, without flags, \b, lookaround, and Unicode classes. Each left-out form is an error today, not a literal, so adding it later changes no pattern's meaning.

#Searching

Rule Method Result
std-regex.is-matchis_match pub fn is_match(self, text: string) -> bool whether some part of text matches
std-regex.findfind pub fn find(self, text: string) -> Match? the first match in text, or .None
  1. std-regex.find.charsA search steps through text one character, a Unicode scalar value, at a time. A match starts and ends at character boundaries.
  2. std-regex.find.leftmostfind returns a match that starts at the smallest offset where any match starts.
  3. std-regex.find.leftmost-firstAmong the matches that start there, it returns the one the pattern prefers. That is the left side of | before the right, and each quantifier's preferred count, earlier choices first.
  4. std-regex.find.emptyA match may be empty. So x* finds the empty match at offset 0 of "abc".
  5. std-regex.is-match.findis_match(text) is find(text).is_some().
use std.regex.{Regex, Match}

fn first() -> string?:
    match Regex::new("a|ab"):
        .Ok(pattern) => pattern.find("ab").map(fn(found: Match) -> string: found.text)  # "a"
        .Err(_) => .None

Why. Leftmost-first is what RE2, Go, Rust, Perl, and JavaScript return, so a pattern copied from one of them finds the same text. POSIX leftmost-longest would return "ab" here.

#Matches

pub struct Match:
    pub start: usize
    pub end: usize
    pub text: string
  1. std-regex.match.decl-structstd.regex declares the struct Match with the public fields above.
  2. std-regex.match.offsetsstart and end are byte offsets into the searched text, with start <= end, as string.slice takes them.
  3. std-regex.match.texttext is text.slice(start, end) of the searched text.
  4. std-regex.match.traitsMatch implements Eq and Debug.
  5. std-regex.match.valueA Match is a value: copying it copies its fields, so a change to one copy leaves the other as it was.

#All Matches

Rule Method Result
std-regex.find-allfind_all pub fn find_all(self, text: string) -> List[Match] every match in text, left to right, none overlapping
  1. std-regex.find-all.nextfind_all runs a series of searches. The first starts at offset 0, and each finds the match find would, among those that start at or after its offset.
  2. std-regex.find-all.emptyAfter a match that is not empty, the next search starts at its end. After an empty match, it starts one character later, and none follows an empty match at the end of the text.
  3. std-regex.find-all.dropAn empty match that starts where the previous match ended is dropped.
  4. std-regex.find-all.anchors^ and $ still see only the ends of the whole text, wherever a search starts.
use std.regex.Regex

fn spots() -> List[usize]:
    let mut starts: List[usize] = []
    for found in Regex::new("a*").expect("compiles").find_all("baaac"):
        starts.push(found.start)
    starts  # [0, 1, 5]

Why. Stepping one character past an empty match keeps the loop finite, and dropping an empty match right after another match follows Go and Rust. So a* finds "", "aaa", and "" in "baaac", not a second "" at offset 4.

#Time Bound

  1. std-regex.time.linearA search takes time in O(m × n). Here n is the length of the text, and m is the size of the pattern with each counted repetition written out. No pattern makes it slower: there is no backtracking.
  2. std-regex.time.memoryA search uses memory in O(m + n).
  3. std-regex.time.linear.groupsA search that also reports groups, as captures and the replacements do, takes time in O(m × g × n). Here g is the number of groups plus one.
  4. std-regex.time.memory.groupsSuch a search uses memory in O(m × g + n).
  5. std-regex.time.allfind_all, captures_all, replace_all, and split run at most 2k + 1 searches for k matches.
  6. std-regex.time.no-backrefsSo the syntax has no backreference and no lookaround, which no linear-time matcher supports.
  7. std-regex.time.sizeThe written-out size of a pattern is given by the table below, where s is the size of the repeated item.
  8. std-regex.error.too-largeA pattern whose written-out size is above 10,000 is an error, at offset 0. Error: TooLarge.
Form Written-out size
a character, ., an escape, a class, or an anchor 1
the empty pattern 0
a group the size of its inner pattern
a concatenation the sum of its parts
an alternation of k branches the sum of the branches, plus 2 × (k − 1)
e*, e{0,}, and their lazy forms s + 2
e+, and e{m,} with m at least 1, and their lazy forms m × s + 1, with m = 1 for e+
e?, e{m}, e{m,n}, and their lazy forms m × s + (n − m) × (s + 1), with m = 0 and n = 1 for e?, and n = m for e{m}
use std.regex.Regex

fn quick(text: string) -> bool:
    match Regex::new("(a*)*b"):
        .Ok(pattern) => pattern.is_match(text)  # linear, even on a long run of "a"
        .Err(_) => false

Note. lib/std compiles a pattern to a Pike VM. The VM runs every thread in step over the text and keeps at most one thread for each instruction, in priority order. Each thread carries the positions of the groups it has passed. The written-out size is the number of instructions that VM runs, without the final one and the two that mark each group's ends.

Why. A script runs patterns on input it does not control. RE2's guarantee keeps a pattern like (a*)*b, which takes exponential time in a backtracking engine, linear. A search may read on past its match to the end of the text. So find_all can take O(m × n²), as Rust documents for its iterators.

#Captures

Rule Method Result
std-regex.capturescaptures pub fn captures(self, text: string) -> Captures? the groups of the match find returns, or .None
std-regex.captures-allcaptures_all pub fn captures_all(self, text: string) -> List[Captures] the groups of each match find_all returns, in order

Captures has these methods:

Rule Method Result
std-regex.captures.lenlen pub fn len(self) -> usize the number of capture groups in the pattern, plus one
std-regex.captures.getget pub fn get(self, index: usize) -> Match? group index, where 0 is the whole match
std-regex.captures.namename pub fn name(self, name: string) -> Match? the group named name
  1. std-regex.captures.decl-structstd.regex declares the struct Captures, a value with private fields.
  2. std-regex.captures.wholeget(0) is the whole match, the one find or find_all returns.
  3. std-regex.captures.absentget(index) is .None when group index took no part in the match, or when index is not below len().
  4. std-regex.captures.name.absentname(name) is get of the group with that name, or .None when no group has it.
  5. std-regex.captures.pathA group's positions are those along the path the match takes, chosen by the preferences of std-regex.find.leftmost-first.
  6. std-regex.captures.lastA group that the path passes more than once reports its last pass. So (a|b)+ on "ab" gives group 1 the text "b".
  7. std-regex.captures.empty-loopAn optional iteration of *, +, or {m,} that would match the empty text is not taken. So (a*)* on "b" leaves group 1 out, and (a*)+ does not.
use std.regex.Regex
use std.text.r

fn year_and_month(line: string) -> (string, string)?:
    date := Regex::new(r"(?<year>\d{4})-(\d{2})").expect("compiles")
    found := date.captures(line)?
    .Some((found.name("year")?.text, found.get(2)?.text))  # "2026-10" gives ("2026", "10")

Why. Group numbers are what RE2, Go, and Rust report. So are .None for a group that took no part, and the last pass of a repeated group. Both spellings of a named group are accepted, as RE2, Go 1.22, and Rust accept them. (?P<name>) is Python's spelling, and (?<name>) is JavaScript's, Java's, and .NET's.

#Typed Decoding

Typed decoding builds a record from the named groups of one match through its Deserialize implementation. pattern.decode::[Hit](line) combines captures with Captures.decode::[Hit]().

  1. std-regex.decode.regexRegex declares pub fn decode[T < Deserialize](self, text: string) -> Result[T, RegexError].
  2. std-regex.decode.capturesCaptures declares pub fn decode[T < Deserialize](self) -> Result[T, RegexError].
  3. std-regex.decode.searchRegex.decode runs captures once, then decodes that capture set.
  4. std-regex.decode.no-matchWhen no match exists, Regex.decode returns .Err(RegexError.NoMatch).
  5. std-regex.decode.reuseCaptures.decode decodes its existing groups without another search.
  6. std-regex.decode.rootThe decoded root must be a record; other roots return WrongType with group "0".
  7. std-regex.decode.unskipped-members-wire-nameEach unskipped record member reads the capture group with its serde wire name, including a Rename fact.
  8. std-regex.decode.missing-nameA member without a named group returns .Err(RegexError.MissingGroup(name)).
  9. std-regex.decode.optionalA T? member is .None when its named group took no part in the match.
  10. std-regex.decode.requiredA required member whose named group took no part returns .Err(RegexError.MissingGroup(name)).
  11. std-regex.decode.stringA string member is a slice of its group's matched text.
  12. std-regex.decode.scalarsInteger, float, bool, and char members parse the group text, then apply the type's Deserialize checks.
  13. std-regex.decode.f32An f32 member must be finite and within the f32 range, as its Deserialize implementation requires.
  14. std-regex.decode.variantA group naming a payload-free variant decodes that variant, including a unit variant of a mixed enum.
  15. std-regex.decode.payloadA selected variant with payload members returns WrongType for that group.
  16. std-regex.decode.otherA list, map, or nested record member returns WrongType for that group.
  17. std-regex.decode.wrong-typeA refused conversion returns .Err(RegexError.WrongType(group, expected)), with the group name and the expected form.
  18. std-regex.decode.no-match.textThe Display text of NoMatch is the regex did not match.
  19. std-regex.decode.missing-group.textThe Display text of MissingGroup(name) is missing regex group NAME, with name for NAME.
  20. std-regex.decode.wrong-type.textThe Display text of WrongType(group, expected) is cannot decode regex group GROUP as EXPECTED, substituting the two payloads.

Why. A type opts into Deserialize once; a format supplies its reader. A caller can skip NoMatch lines while keeping malformed captured values visible as errors.

#Replacing And Splitting

Rule Method Result
std-regex.replacereplace pub fn replace(self, text: string, replacement: string) -> string text with the match find returns replaced
std-regex.replace-allreplace_all pub fn replace_all(self, text: string, replacement: string) -> string text with each match find_all returns replaced
std-regex.splitsplit pub fn split(self, text: string) -> List[string] the pieces of text around each match find_all returns
  1. std-regex.replace.noneWith no match, replace and replace_all return text unchanged.
  2. std-regex.replace.expandA match is replaced by replacement, with each reference in the table below replaced by the text it names in that match's groups.
  3. std-regex.replace.oncereplacement is read once, from left to right, for each match. A group's text is never read for references.
Rule Reference Replaced by
std-regex.replace.syntax.dollarDollar $$ one $
std-regex.replace.syntax.numberNumber $ and ASCII digits, as in $1 the text of the group with that number
std-regex.replace.syntax.bracedBraced number ${ ASCII digits }, as in ${1} the text of the group with that number
std-regex.replace.syntax.nameName ${ name }, as in ${year} the text of the group with that name
  1. std-regex.replace.longestA number takes every digit that follows the $, so $12 is group 12. Write ${1}2 for group 1 then 2.
  2. std-regex.replace.absentA reference to a group that took no part, or that the pattern does not have, is replaced by the empty text.
  3. std-regex.replace.literalA $ that begins no reference in the table is itself. So $x, ${}, an unclosed ${1, and a final $ stay as written.
  4. std-regex.split.piecesWith k matches, split returns k + 1 pieces. They are the text before the first match, the text between each pair of matches, and the text after the last.
  5. std-regex.split.endsSo a match at the start of text gives a first piece "", and one at its end a last piece "". Splitting "" with no match gives [""].
use std.regex.Regex
use std.text.r

fn iso(dates: string) -> string:
    us := Regex::new(r"(?<month>\d\d)/(?<day>\d\d)/(\d{4})").expect("compiles")
    us.replace_all(dates, "$3-\${month}-\${day}")  # "12/31/2025" gives "2025-12-31"
use std.regex.Regex
use std.text.r

fn fields(line: string) -> List[string]:
    Regex::new(r"\s*,\s*").expect("compiles").split(line)  # ",a , b" gives ["", "a", "b"]

Note. A replacement is a string literal, so its $ follows Interpolation. $1 and $$ are text as written, by lex.interp.dollar-text. A braced reference needs \$, as in "\${month}", or the string interpolates month. A raw string does not help: r"\${month}" keeps the backslash.

Why. $1, ${name}, and $$ are the replacement syntax of Go and Rust. A bare $name is left out, since an hd string would interpolate it. So $1a is group 1 then a, where Go and Rust read a group named 1a.

#Pattern Errors

pub enum RegexError:
    MissingParen(position: usize)
    UnmatchedParen(position: usize)
    MissingBracket(position: usize)
    BadEscape(position: usize)
    BadRange(position: usize)
    NothingToRepeat(position: usize)
    BadRepeat(position: usize)
    UnsupportedGroup(position: usize)
    BadGroupName(position: usize)
    DuplicateGroupName(position: usize)
    TooLarge(position: usize)
    NoMatch
    MissingGroup(name: string)
    WrongType(group: string, expected: string)
Rule Variant When Display text
std-regex.error.kind.missing-parenMissing paren MissingParen std-regex.syntax.group.unclosed missing closing )
std-regex.error.kind.unmatched-parenUnmatched paren UnmatchedParen std-regex.syntax.group.unopened unexpected )
std-regex.error.kind.missing-bracketMissing bracket MissingBracket std-regex.syntax.class.unclosed missing closing ]
std-regex.error.kind.bad-escapeBad escape BadEscape std-regex.syntax.escape.other invalid escape
std-regex.error.kind.bad-rangeBad range BadRange std-regex.syntax.class.bad-range invalid character class range
std-regex.error.kind.nothing-to-repeatNothing to repeat NothingToRepeat std-regex.repeat.nothing missing argument to repetition operator
std-regex.error.kind.bad-repeatBad repeat BadRepeat std-regex.repeat.bad-count invalid repeat count
std-regex.error.kind.unsupported-groupUnsupported group UnsupportedGroup std-regex.syntax.group.unsupported unsupported group
std-regex.error.kind.bad-group-nameBad group name BadGroupName std-regex.syntax.group.bad-name invalid named capture
std-regex.error.kind.duplicate-group-nameDuplicate group name DuplicateGroupName std-regex.syntax.group.duplicate-name duplicate capture group name
std-regex.error.kind.too-largeToo large TooLarge std-regex.error.too-large expression too large
  1. std-regex.error.variants-with-decodestd.regex declares the enum RegexError with the variants shown above.
  2. std-regex.error.positionposition is the byte offset in the pattern where the rule for the variant puts the error.
  3. std-regex.error.firstnew reads the pattern from left to right and returns the first error it meets. A group or class is checked for its closing character after its contents.
  4. std-regex.error.size-lastTooLarge is checked only once the whole pattern has no other error.
  5. std-regex.error.compile-decode-traitsRegexError implements Eq, Debug, Display, and std.error.Error.
  6. std-regex.error.equalityTwo RegexError values are equal when their variants and corresponding payloads are equal.
  7. std-regex.error.pattern-textA pattern error's Display text is its table text, then at byte, then position.
use std.regex.{Regex, RegexError}

fn broken() -> RegexError?:
    Regex::new("a(b").err()  # .Some(RegexError.MissingParen(1))

Why. The variants and texts follow Go's regexp/syntax error codes, and Rust's text for a duplicate name, so an error reads the same as there. A position lets a tool point at the spot in a pattern the user wrote.

#Not Yet Specified

These are not in this version of std.regex:

  • flags, such as (?i), (?m), and (?s), which are UnsupportedGroup errors today;
  • \b and the Unicode classes \p{...}, which are BadEscape errors;
  • a replacement computed by a function from each match's Captures.