#Standard Library Specification
This directory holds the stdlib tier of the specification. The numbered
chapters in lang/ are the language tier.
Both tiers are one specification, written to one
Specification Style, with one conformance suite.
#Tiers
| Tier | Holds | Where |
|---|---|---|
| language | syntax, static and dynamic semantics, intrinsics, and every std item the compiler must know by name | the numbered chapters |
| stdlib | std APIs that lib/std writes in ordinary hd over the language tier |
this directory |
| CLI | how the hd command finds and runs a package or a single file |
Command Line |
The language tier names a std item only when the compiler must know it:
- a lang item, such as
Iterableforfor,std.process.Termination, or astd.opsoperator trait; - an intrinsic, such as
List.push,string.len, orfacts_of(f); - a prelude name and its signature;
- the functions of the test registration calls
it,it_each,it_prop, andit_prop_with, with their position rules and diagnostics; - the conformance harness:
it,assert,assert_equal, andprintln.
This tier keeps what those registration functions do when a test runs: table rows, property generation and shrinking, the runner capabilities, reporting, and snapshots (Testing).
Every other decided std API belongs in this directory.
#The Tier Test
Could
lib/stdimplement the item in ordinary hd, over language-tier items only, with no compiler knowledge of its name, and keep the same observable behavior?
Yes puts the item in the stdlib tier. A diagnostic code about the item, a language rule that names it, or a position rule that lists it each mean no.
Only std may add inherent methods to built-in types
(trait.own.inherent.std).
That alone does not make a method language-tier.
#Standard Library Primitives
lib/std is ordinary hd over the language tier, plus the primitives
below. A primitive is a function or a method whose body the
implementation supplies. A primitive function is declared once: in the
std-only runtime module std.rt.bytes when several std modules call it
(module.rt.bytes), else
privately in the one module that calls it. So no program calls it. The specification has no
syntax for declaring one; the reference implementation writes
@intrinsic("name") before it. A primitive operation is an
intrinsic method, which any
program may call.
Any primitive not in these tables needs the owner's approval.
Representation. These read or build the run-time layout of a string,
a char, a list or a map, or are one machine operation on a number.
| Primitive | Signature | Why it is a primitive |
|---|---|---|
bytes_len |
(s: string) -> usize |
reads the byte length of a string's representation |
bytes_at |
(s: string, i: usize) -> u8 |
reads one byte of a string's representation |
bytes_slice |
(s: string, start: usize, end: usize) -> string |
shares the bytes in constant time, as module.string.slice.shared requires |
string_from_bytes |
(bytes: List[u8]) -> string; owned raw form (bytes: Buffer[u8], len: usize) -> string; initialized string-room form (room: string, start: usize, len: usize) -> string |
builds text from initialized UTF-8 bytes; an unshared list or owned raw room transfers to the string without a release-build copy |
string_to_bytes |
string.to_utf8(self) -> List[u8], an intrinsic method of std.core |
builds a list of exactly a string's bytes in one sized allocation and one copy, which an hd loop over bytes_at and push cannot do; std.core declares it, since Hasher.write_str writes a string's bytes |
format_int |
[N < Integer](value: N, destination: Ptr[u8]) -> (usize, usize) |
initializes twenty bytes of caller-owned room and writes decimal digits, including a negative sign, into its tail; returns the digit offset and length, allocates nothing, and has an ordinary-hd string-returning wrapper |
char_scalar, char_from_scalar |
(c: char) -> u32, (point: u32) -> char? |
a char is its scalar value at run time; char_from_scalar gives .None for a value that is not a Unicode scalar value, a surrogate or one above 0x10FFFF |
f64_to_bits, f64_from_bits |
(value: f64) -> u64, (bits: u64) -> f64 |
the bit cast between an f64 and its IEEE 754 pattern, one reinterpret instruction; hd has no other way to read a NaN's sign, which std-num.float-bits.to and total_cmp need |
f32_to_bits, f32_from_bits |
(value: f32) -> u32, (bits: u32) -> f32 |
the same bit cast for f32 |
int_min, int_max |
[N < Integer]() -> N |
each integer type's bounds as constants, which a generic body over Integer cannot name otherwise |
int_wrapping_add, int_wrapping_sub, int_wrapping_mul |
the methods wrapping_add, wrapping_sub, wrapping_mul of impl[N < Integer] N: (self, other: N) -> N |
one machine instruction that wraps in every profile, where +, - and * panic on overflow in a debug or test build |
int_count_ones, int_leading_zeros, int_trailing_zeros |
the methods count_ones, leading_zeros, trailing_zeros of impl[N < Integer] N: (self) -> u32 |
one machine instruction (popcnt, clz, ctz) at the type's width |
list_truncate |
[T](items: &mut List[T], len: usize) -> void |
shortens the list items refers to, to its first len elements, the one hook under List.pop, insert, remove_at and clear; panics when len is greater than the size. Panic: index-out-of-bounds |
list_extend |
[T](items: &mut List[T], other: List[T]) -> void |
appends other's elements to the list items refers to with one capacity check and one copy of all of them, where push in a loop checks and may grow once per element; List.extend is this call. other may be the list itself |
list_reserve |
[T](items: &mut List[T], count: usize) -> void |
makes room in the list items refers to for at least count more elements with at most one new buffer, where pushing and truncating elements would grow it once per doubling; count 0 does nothing. list_extend and List.reserve reuse it. |
map_probe |
the body of Map.get_or_insert(&mut self, key: K, default: V) -> &mut V |
finds or inserts a key's entry in one probe of the map's table and returns a reference into it (module.map.probe.get-or-insert.decl) |
map_steps |
[K < Eq & Hash, V](entries: Map[K, V]) -> fn(usize) -> (usize, (K, V))? |
the step function of a map cursor: from entry number at on, the first live entry and its number, or .None past the last; it reads the table's entry order, which only the compiler lays out |
Scans. These search the bytes of a string or a byte list. Each has an hd reference body that defines its result, and an implementation may supply a faster body that gives the same result. The owner approved them in #428.
| Primitive | Signature | Why it is a primitive |
|---|---|---|
bytes_index_set |
(s: string, from: usize, w0: u64, w1: u64, w2: u64, w3: u64) -> usize |
the offset of the first byte at or after from in the 256-bit set w0..w3 (byte b is bit b % 64 of word b / 64), or bytes_len(s) when there is none; SIMD tests 16 bytes per step where an hd loop tests one, and JSON encoding and parsing scan every string byte for their escape set |
bytes_index |
(s: string, from: usize, byte: u8) -> usize |
the offset of the first byte at or after from equal to byte, or bytes_len(s) when there is none; SIMD compares 16 bytes per step where an hd loop compares one, and one-byte find, split, count, replace and lines, the regex literal prefilter and the JSON pointer scan are all this search |
bytes_mismatch |
(left: string, right: string) -> usize |
the offset of the first position where left and right differ; when one is a prefix of the other, the shorter length, and when they are equal, their common length; SIMD compares 16 bytes of each per step where an hd loop compares one, and string == and ordering, starts_with and ends_with are all this comparison |
bytes_index_pair |
(s: string, from: usize, first: u8, last: u8, gap: usize) -> usize |
the offset of the first i >= from with i + gap < bytes_len(s), byte i equal to first and byte i + gap equal to last, or bytes_len(s) when there is none; SIMD tests 16 offsets per step where an hd loop tests one, and find, split and the other multi-byte substring searches use it to skip to a window whose first and last bytes are the needle's before checking the whole needle |
utf8_valid_len |
(bytes: List[u8]) -> usize |
the length of the longest prefix of bytes that is well-formed UTF-8 and ends at a scalar boundary, so bytes.len() exactly when all of it is valid; SIMD passes 16 ASCII bytes per step where an hd loop decodes one, and string::from_utf8, and through it HTTP bodies, file reads and std.encoding, validates every byte |
Panic. One primitive raises every checked runtime panic that lib/std
raises with a category other than explicit-panic.
| Primitive | Signature | Why it is a primitive |
|---|---|---|
panic |
(category: string, message: string) -> void |
the prelude panic always has the category explicit-panic; this one names a stable category, and an empty message shows none |
Compiler-level. The compiler lowers these at each use.
| Primitive | Why it is a primitive |
|---|---|
facts_of |
the compiler builds each call's facts value (Function Facts) |
task_race_frame, task_all_frame |
the polling frames of race! and homogeneous task-list driving, which are compiler intrinsics (req.combinator.intrinsic) |
task_idle_hook |
std.rt.wait.idle_hook(hook: fn() -> void) installs the function a driver runs when every task waits on the host; the driver's loop is compiler-supplied |
fn_call, fn_mut_call |
the bodies of Fn.call and FnMut.call_mut: a call through a function value's code and environment, whose layout only the compiler knows (fn.trait.decl, fn.trait.mut.decl) |
ref_identity |
the identity of a reference's place, which deep_equal keys its visited pairs by; the checker reads it from the reference |
mem_ref_address |
the address of the place a shared reference refers to, which MembersRecord.id returns for a member directory's static record (annot.structure.members.id) |
caller_location, caller_args |
Location::caller() and Location::caller_args() in std.debug: the call site of the enclosing @track_caller function, which only the compiler knows (fn.caller.location) |
descriptor_glue_mask |
the checked report's mask of a block descriptor's glue bits, which the compiler lays out |
Runtime. The memory intrinsics of std.rt.memory and its typed
memory are language tier, because the compiler knows them by name, so
Memory Intrinsics and
Typed Memory define them.
They are pub, but only std may use a
runtime module, so still no
program calls them.
Host. The host supplies these, for now.
| Primitive | Signature | Why it is a primitive |
|---|---|---|
format_f64, format_f32 |
(value: f64) -> string, (value: f32) -> string |
the shortest round-trip decimal text of a float (Numeric Display) |
string_lower, string_upper |
(text: string) -> string |
Unicode case mapping, which needs the Unicode tables |
parse_f64 |
(text: string) -> f64 |
the correctly rounded f64 of an unsigned decimal number text, which needs arbitrary-precision arithmetic |
format_f64_fixed |
(value: f64, digits: usize) -> string |
the correctly rounded fixed-point text of a float, which needs arbitrary-precision arithmetic |
map_hash_seed in std.rt |
() -> u64 |
a fresh map seed mixed from the run seed and a per-run counter (std-hash.map.seed-new) |
snapshot_update |
(text: string) -> bool |
the update run's side of snapshot: in an update run of an it test case it records text and the call's place and returns true, else it returns false; only std.testing calls it |
A host primitive takes and returns each f32 and f64 as a raw IEEE
754 value, as a host capability call does by
module.profile.host-float.raw.
So a NaN, both infinities, and -0.0 reach a hook unchanged.
format_f64 and format_f32 write the whole Display text of a float,
by Numeric Display. The hook,
not lib/std, writes the text of
types.display.special:
NaN, inf, -inf, and the - of -0.0.
parse_f64 takes only an unsigned
decimal number, with no sign and no
special word. lib/std checks the grammar and reads the sign before the
call (std-num.parse-f64.hook). The
hook returns the f64 nearest the text's decimal value, rounding a tie to
the one with an even significand, as IEEE 754 rounds to nearest. So a
value past the finite f64 range is positive infinity, and a value too
small for the smallest subnormal is 0.0.
parse_f64 has no error or refusal signal. A call with text outside the
grammar is a bug in lib/std, not a case the host handles.
format_f64_fixed writes the text that
to_fixed returns, with every rule of that
section but the check of digits. Its caller passes a digits from 0 to
100. For a finite value, it writes the multiple of 10 to the power
-digits nearest the value's exact binary value, and a tie goes to the
multiple whose last digit is even. A NaN gives NaN, and the infinities
give inf and -inf. The hook, not lib/std, writes that text, and the
- of a negative value or of -0.0.
Test-runner hooks. std.testing reaches the test runner through the
host capabilities TestRunner and PropertyRunner
(Runner Capabilities), so table rows,
timeouts, snapshot files, and property draws need no primitive.
Operations. One intrinsic method per primitive operation. lib/std
writes each once, mostly in a numeric-family implementation such as
impl[N < Num] Add for N
(trait.target.numeric-family),
and the compiler specializes it for each type.
| Intrinsic method | Implemented for | Why it is a primitive |
|---|---|---|
Add.add |
every number type | machine addition, with the integer overflow panic of a debug or test build |
Sub.sub |
every number type | machine subtraction, with the integer overflow panic of a debug or test build |
Mul.mul |
every number type | machine multiplication, with the integer overflow panic of a debug or test build |
Div.div |
every number type | truncating integer division with its panics, or IEEE 754 division |
Rem.rem |
every number type | the truncated remainder, with its panic, or the floating remainder |
Neg.neg |
the signed integer and floating-point types | machine negation; negating the minimum integer panics in a debug or test build |
BitAnd.bit_and |
every integer type | one machine instruction on the bits |
BitOr.bit_or |
every integer type | one machine instruction on the bits |
BitXor.bit_xor |
every integer type | one machine instruction on the bits |
Not.not |
every integer type | one machine instruction on the bits |
Shl.shl |
every integer type, with any unsigned integer count | a fixed-width shift, with the shift count panic |
Shr.shr |
every integer type, with any unsigned integer count | a fixed-width shift, arithmetic for a signed value, with the shift count panic |
Eq.eq |
every number type, char, and bool |
compares the machine values; floats follow IEEE 754 |
PartialOrd.partial_cmp |
every number type and char |
compares the machine values; a NaN operand is unordered |
Ord.cmp |
every integer type and char |
compares the machine values |
sqrt |
f32 and f64 |
the correctly rounded IEEE 754 square root, one machine instruction (std-num.float-sqrt) |
string's Add, Eq, PartialOrd, and Ord are not intrinsic: lib/std
writes them in hd.
Why. Each primitive is a promise that every implementation must keep, and code that the compiler supplies cannot be read or tested as hd. A short explicit list keeps the rest of
lib/stdportable.
#Chapters
Each chapter is one unnumbered file named after its std module. The Spec Tiers migration plan added the files below, and it is complete. A new decided std module gets a file of its own.
| File | Module | Rule ID prefix | Scope |
|---|---|---|---|
testing.md |
std.testing |
std-testing |
what it_each, it_prop, and it_prop_with cases do when they run, property tests, the draw limit, derived Arbitrary, table-test rows, snapshot files, the timeout option, hd_run!, the runner capabilities |
iter.md |
std.iter |
std-iter |
iterator adapters, including skip, take_while, zip, chain, flat_map, any, all, find, and count; what collect builds, FromIterator and its impls, map on a list or an optional |
text.md |
std.text |
std-text |
string methods above the intrinsics, including lines and repeat; split_once, split_whitespace, padding, and count; the r prefix |
format.md |
std.format |
std-format |
the text debug returns, Debug builders and layout, derived builder calls, Debug for the public std types |
time.md |
std.time |
std-time |
Duration, its suffixes, its arithmetic, and its Display text; Timestamp + Duration; the host trait Clock, Timestamp, Instant, now, and sleep!; the provider ManualClock; the UTC Date, RFC 3339 text, and ParseTimeError; the serialization opt-in of the time types |
task.md |
std.task |
std-task |
Layer and apply!; retry, timeout, fallback, breaker and hedge policies; TimedOut and CircuitOpen; map_join! |
ops.md |
std.ops |
std-ops |
the Default trait and its standard implementations |
cmp.md |
std.cmp |
std-cmp |
what derived Eq, PartialOrd, and Ord compare; tuple comparison; clamp; Reverse |
hash.md |
std.hash |
std-hash |
what derived Hash hashes; tuple hashing; DefaultHasher and hash_of; the stable hasher Fnv1a |
collections.md |
std.collections |
std-collections |
the List methods view and chunks, the ListView type, the List helpers such as sorted_by_key, group_by, and windows, the Map methods contains_key, keys, values, and get_or_insert, counts, Set, Deque, and Heap |
console.md |
std.console |
std-console |
eprintln, when host console output becomes visible, the host trait ConsoleInput, and read_line! and read_all! |
contract.md |
std.contract |
std-contract |
the always-checked precondition helper require |
host.md |
std.host |
std-host |
the host traits Args, Env, and Sys, the helpers args and env, the providers ScriptedArgs, ScriptedEnv, and ScriptedSys, HostError, and the rules every host operation follows |
fs.md |
std.fs |
std-fs |
the host traits FsRead and FsWrite, FsError, Entry, the helpers read_text! and write_text!, and the provider MemoryFs |
path.md |
std.path |
std-path |
the Path newtype |
process.md |
std.process |
std-process |
the Display text of ProcessError; Eq and Debug for ExitCode and ProcessOutput; the provider ScriptedProcess; child process inheritance, signal status, and cancellation |
http.md |
std.http |
std-http |
the host trait Http, request sending and checked get!, Request, Response, Method, HttpError, the provider ScriptedHttp, and the playground's Http |
server-sdk.md |
server_sdk.server (SDK package) |
std-server-sdk |
typed request leases, the SDK-owned HttpServer capability, and joined worker loops |
net.md |
std.net |
std-net |
the host trait Net for DNS lookup, TCP, and UDP; the handles TcpStream, TcpListener, and UdpSocket; Datagram and NetError |
random.md |
std.random |
std-random |
the host trait Random, the provider SeededRandom, and the seeded generator Rng with rng |
option.md |
std.option |
std-option |
the methods and_then, unwrap_or, ok_or, is_some, is_none, and expect of T? |
result.md |
std.result |
std-result |
the methods and_then, map_err, unwrap_or, map, ok, err, is_ok, is_err, and expect of Result[T, E] |
num.md |
std.num |
std-num |
checked, wrapping, and saturating integer methods, abs_diff, and the bit counts on every integer type; is_nan and is_finite; integer parsing with ParseNumberError; parse_f64; to_fixed |
error.md |
std.error |
std-error |
the cause chain chain, what root_cause and find return, the boundary-safe ErrorReport, report_of, and Result.context with private error wrappers |
encoding.md |
std.encoding |
std-encoding |
hex and base64 text for bytes: hex_encode, hex_decode, the enum Base64 with encode and decode, and DecodeError |
digest.md |
std.digest |
std-digest |
Digest with constant-time equality, the streaming Sha256 with Sha256::hmac, and sha256 |
json.md |
std.json |
std-json |
the untyped Json value, Number, parse, JsonError, pretty, and the Display text; JSON as a format over the serialization opt-in: to_json, from_json, encode, and decode |
serde.md |
std.serde |
std-serde |
the data model of Serializer and Deserializer, the calls of derived Serialize and Deserialize, and the standard implementations |
cli.md |
std.cli |
std-cli |
argument parsing for scripts: the Cli builder, parse, parse_args, Parsed, CliError, and usage |
regex.md |
std.regex |
std-regex |
regular expressions in the RE2 subset: Regex, is_match, find, Match, and RegexError |
validate.md |
std.validate |
std-validate |
post-reader value validation, typed rule facts, structural paths, issue caps, and reader-selected path names |
#Test Provider Names
Each host trait has a test provider in its own module. The name's first word says how the provider answers:
| Prefix | The provider | Providers |
|---|---|---|
Scripted |
answers from data the test supplies | ScriptedInput, ScriptedProcess, ScriptedHttp, ScriptedNet, ScriptedStream, ScriptedListener, ScriptedUdp, ScriptedArgs, ScriptedEnv, ScriptedSys |
Memory |
keeps its state in memory, as a real host would | MemoryFs, MemoryConsole |
Manual |
holds a state that the test advances | ManualClock |
Seeded |
draws from a generator that a seed starts | SeededRandom |
#Glossary
This glossary lists the terms of the stdlib chapters. Each entry links to the rule, or the section, that defines the term. The language terms are in the language glossary.
| Term | Definition |
|---|---|
| Debug builders | The DebugWriter methods that describe a value as a struct, tuple, list, or map. See Debug Builders. |
| draw limit | The per-case limit on draws from Choices; once it is spent, every draw returns its simplest value. See std-testing.limit. |
| flag | A command-line argument that std.cli reads as set or not set; it takes no value. See std-cli.parser.arg. |
| iterator adapters | Methods of the prelude Iterator[T] that wrap an iterator in a new one, or drain it. See Iterator Adapters. |
| option | A command-line argument that std.cli reads with one value. See std-cli.parser.arg. |
| pad bits | The low bits of the last base64 symbol that encode no byte; they are zero. See std-encoding.base64.encode.pad-bits. |
| padding | The one or two = characters that end a base64 text. See std-encoding.base64.decode.padding. |
| positional | A command-line argument that is not a flag or an option; std.cli reads it by position. See std-cli.parser.arg. |
| property test | A test case whose body runs on inputs drawn from a Choices source. See Property Tests. |
#Rule IDs
- A rule ID here starts with its file's prefix:
std-and the module name. - A rule that moves here from a numbered chapter is retired there and
keeps its tail under the new prefix.
module.testing.prop.debugbecomesstd-testing.prop.debug. - The retired ID is deleted, as Stability requires.
#Links Between Tiers
- A stdlib chapter may cite any language rule.
- A language chapter links here only from a Note or a See also, never from a numbered rule. No language rule depends on stdlib behavior.
- A moved heading is deleted, and every link to it is fixed when it moves.
#Conformance And Diagnostics
- A case's tier is the tier of its
specificationcolumn incases.tsv. A path understd/is stdlib. - Every diagnostic code stays in the Diagnostics table, and every panic category stays in Control Flow. Each is a compiler check.