Benchmark TypeScript Parsers: Demystify Rust Tooling Performance

TL;DR: Native parsers used in JavaScript are not always faster due to extra work across languages. Avoiding these overhead and using multi-core are crucial for performance.

Rust is rapidly becoming a language of choice within the JavaScript ecosystem for its performance and safety features. However, integrating Rust into JavaScript tooling presents unique challenges, particularly when it comes to designing an efficient and portable plugin system.

“Rewriting JavaScript tooling in Rust is advantageous for speed-focused projects that do not require extensive external contributions.” - Nicholas C. Zakas, creator of ESLint

Learning Rust can be daunting due to its steep learning curve, and distributing compiled binaries across different platforms is not straightforward.
A Rust based plugins necessitates either static compilation of all plugins or a carefully designed application binary interface for dynamic loading.
These considerations, however, are beyond the scope of this article. Instead, we’ll concentrate on how to provide robust tooling for writing plugins in JavaScript.

A critical component of JavaScript tooling is the parsing of source code into an Abstract Syntax Tree (AST). Plugins commonly inspect and manipulate the AST to transform the source code. Therefore, it’s not sufficient to parse in Rust alone; we must also make the AST accessible to JavaScript.

This post will benchmark several popular TypeScript parsers implemented in JavaScript, Rust, and C.

Parser Choices

While there are numerous JavaScript parsers available, we focus on TypeScript parsers for this benchmark. Modern bundlers must support TypeScript out-of-the-box, and TypeScript is a superset of JavaScript. Benchmarking TypeScript is a sensible choice to emulate the real-world bundler workload.

The parsers we’re evaluating include:

• Babel: The Babel parser (previously Babylon) is a JavaScript parser used in Babel compiler.
• TypeScript: The official parser implementation from the TypeScript team.
• Tree-sitter: An incremental parsing library that can build and update concrete syntax trees for source files, aiming to parse any programming language quickly enough for text editor use.
• ast-grep: A CLI tool for code structural search, lint, and rewriting based on abstract syntax trees. We are using its napi binding here.
• swc: A super-fast TypeScript/JavaScript compiler written in Rust, with a focus on performance and being a library for both Rust and JavaScript users.
• oxc: The Oxidation Compiler is a suite of high-performance tools for JS/TS, claiming to have the fastest and most conformant parser written in Rust.

Native Addon Performance Characteristics

Before diving into the benchmarks, let’s first review the performance characteristics of Node-API based solutions.

Node-API Pros:

• Better Compiler Optimization: Code in native languages have compact data layouts, leading to fewer CPU instructions.
• No Garbage Collector Runtime Overhead: This allows for more predictable performance.

However, Node-API is not a silver bullet.

Node-API Cons:

• FFI Overhead: The cost of interfacing between different programming languages.
• Serde Overhead: Serialization and deserialization of Rust data structures can be costly.
• Encoding Overhead: Converting JS string in utf-16 to Rust’s utf-8 string can introduce significant delays.

We need to understand the pros and cons of using native node addons in order to design an insightful benchmark.

Benchmark Design

We consider two main factors:

1. File Size: Different file sizes reveal distinct performance characteristics. The parsing time of an N-API based parser consists of actual parsing and cross-language overhead. While parsing time is proportional to file size, the growth of cross-language overhead depends on the parser’s implementation.

2. Concurrency Level: Parallel parsing is not possible in JavaScript’s single main thread. However, N-API based parsers can run in separate threads, either using libuv’s thread pool or their own threading model. That said, thread spawning also incurs overhead.

We are not considering these factors in this post.

• Warmup and JIT: No significant difference observed between warmup and non-warmup runs.
• GC, Memory Usage: Not evaluated in this benchmark.
• Node.js CLI arguments: To make the benchmark representative, default Node.js arguments were used, although tuning could potentially improve performance.

Benchmark Setup

The benchmark code is hosted in ast-grep’s repo https://github.com/ast-grep/ast-grep/blob/main/benches/bench.ts.

Testing Environment

The benchmarks were executed on a system equipped with the following specifications:

• Operating System: macOS 12.6
• Processor: arm64 Apple M1
• Memory: 16.00 GB
• Benchmarking Tool: Benny

File Size Categories

To assess parser performance across a variety of codebases, we categorized file sizes as follows:

• Single Line: A minimal TypeScript snippet, let a = 123;, to measure baseline overhead.
• Small File: A concise 24-line TypeScript module, representing a common utility file.
• Medium File: A typical 400-line TypeScript file, reflecting average development workloads.
• Large File: The extensive 2.79MB checker.ts from the TypeScript repository, challenging parsers with a complex and sizable codebase.

Concurrency Level

For this benchmark, we simulate a realistic workload by parsing five files concurrently. This number is an arbitrary but reasonable proxy to the actual JavaScript tooling.

It’s worth noting, to seasoned Node.js developers, that this setup may influence asynchronous parsing performance. However it does not disproportionately favor Rust-based parsers. The rationale behind this is left as an exercise for the reader. :)

This post aims to provide a general overview of the benchmarking for TypeScript parsers, focusing on the performance characteristics of N-API based solutions and the trade-offs involved. Feel free to adjust the benchmark setup to better fit your workload.

Now, let’s delve into the results of TypeScript parser benchmarking!

Results

Raw data can be found in this Google Sheet.

Synchronous Parsing

The performance of each parser is quantified in operations per second—a metric provided by the Benny benchmarking framework. For ease of comparison, we’ve normalized the results:

• The fastest parser is designated as the benchmark, set at 100% efficiency.
• Other parsers are evaluated relative to this benchmark, with their performance expressed as a percentage of the benchmark’s speed.

TypeScript consistently outperforms the competition across all file sizes, being twice as fast as Babel.
Native language parsers show improved performance for larger files due to the reduced relative impact of FFI overhead.
Nevertheless, the performance gains are not as pronounced due to serialization and deserialization (serde) overhead, which is proportional to the input file size.

Asynchronous Parsing

In the asynchronous parsing scenario, we observe the following:

ast-grep excels when handling multiple medium to large files simultaneously, effectively utilizing multi-core capabilities. TypeScript and Tree-sitter, however, experience a decline in performance with larger files. SWC and Oxc maintain consistent performance, indicating efficient use of multi-core processing.

Parse Time Breakdown

When benchmarking a Node-API based program, it’s crucial to understand the time spent not only executing Rust code but also the Node.js glue code that binds everything together. The parsing time can be dissected into three main components:

1

time = ffi_time + parse_time + serde_time

Here’s a closer look at each term:

• ffi_time (Foreign Function Interface Time): This represents the overhead associated with invoking functions across different programming languages. Typically, ffi_time is a fixed cost and remains constant regardless of the input file size.

• parse_time (Parse Time): The core duration required for the parser to analyze the source code and generate an Abstract Syntax Tree (AST). parse_time scales with the size of the input, making it a variable cost in the parsing process.

• serde_time (Serialization/Deserialization Time): The time needed to serialize Rust data structures into a format compatible with JavaScript, and vice versa. As with parse_time, serde_time increases as the input file size grows.

In essence, benchmarking a parser involves measuring the time for the actual parsing (parse_time) and accounting for the extra overhead from cross-language function calls (ffi_time) and data format conversion (serde_time). Understanding these elements helps us evaluate the efficiency and scalability of the parser in question.

Result Interpretation

This section offers a detailed and technical analysis of the benchmark results based on the parse time framework above. Readers seeking a high-level overview may prefer to skip ahead to the summary.

FFI Overhead

In both sync parsing and async parsing scenario, the “one line” test case, which is predominant FFI overhead with minimal parsing or serialization, shows TypeScript’s superior performance. Surprisingly, Babel, expected to excel in this one-line scenario, demonstrates its own peculiar overhead.

As file size increases, FFI overhead becomes less significant, as it’s largely size-independent. For instance, ast-grep’s relative speed is 78% for a large file compared to 72% for a single line, suggesting an approximate 6% FFI overhead in synchronous parsing.

FFI overhead is more pronounced in asynchronous parsing. ast-grep’s performance drops from 72% to 60% when comparing synchronous to asynchronous parsing of a single line. The absence of a notable difference in performance for swc/oxc may be due to their unique implementation details.

Serde Overhead
Unfortunately, we failed to replicate swc/oxc’s blazing performance we witnessed in other applications.
Despite minimal FFI impact in “Large file” test cases, swc and oxc underperform compared to the TypeScript compiler. This can be attributed to their reliance on calling JSON.parse on strings returned from Rust, which is, to our disappointment, still more efficient than direct data structure returns.

Tree-sitter and ast-grep avoid serde overhead by returning a tree object rather than a full AST structure. Accessing tree nodes requires invoking Rust methods from JavaScript, which distributes the cost over the reading process.

Parallel

Except tree-sitter, all native TS parsers have parallel support. Contrary to JS parsers, native parsers performance will not degrade when concurrently parsing larger files. This is thanks to the power of multiple cores. JS parsers suffer from CPU bound because they have to parse file one by one.

Perf summary for parsers

The performance of each parser is summarized in the table below, which outlines the time complexity for different operations.

In the table, constant denotes a constant time cost that does not change with input size, while proportional indicates a variable cost that grows proportionally with the input size. An N/A signifies that the cost is not applicable.

JS-based parsers operate entirely within the JavaScript environment, thus avoiding any FFI or serde overhead. Their performance is solely dependent on the parsing time, which scales with the size of the input file.

The performance of Rust-based parsers is influenced by a fixed FFI overhead and a parsing time that grows with input size. However, their serde overhead varies depending on the implementation:

For ast-grep and tree-sitter, they have a fixed serialization cost of one tree object, regardless of the input size.
For swc and oxc, the serialization and deserialization costs increase linearly with the input size, impacting overall performance.

Discussion

Transform vs. Parse

While Rust-based tools are renowned for their speed in transpiling code, our benchmarks reveal a different narrative when it comes to converting code into an AST that’s usable in JavaScript.
This discrepancy highlights a critical consideration for Rust tooling authors: the process of passing Rust data structures to JavaScript is a complex task that can significantly affect performance.
It’s essential to optimize this data exchange to maintain the high efficiency expected from Rust tooling.

Criteria for Parser Inclusion

In our benchmark, we focused on parsers that offer a JavaScript API, which influenced our selection:

• Sucrase: Excluded due to its lack of a parsing API and inability to produce a complete AST, which are crucial for our evaluation criteria.
• Esbuild/Biome: Not included because esbuild functions primarily as a bundler, not a standalone parser. It offers transformation and build capabilities but does not expose an AST to JavaScript. Similarly, biome is a CLI application without a JavaScript API.
• Esprima: Not considered for this benchmark as it lacks TypeScript support, which is a key requirement for the modern JavaScript development ecosystem.

JS Parser Review

Babel:
Babel is divided into two main packages: @babel/core and @babel/parser. It’s noteworthy that @babel/core exhibits lower performance compared to @babel/parser. This is because the additional entry and hook code that surrounds the parser in the core package. Furthermore, the parseAsync function in Babel core is not genuinely asynchronous; it’s essentially a synchronous parser method wrapped in an asynchronous function. This wrapper provides extra hooks but does not enhance performance for CPU-intensive tasks due to JavaScript’s single-threaded nature. In fact, the overhead of managing asynchronous tasks can further burden the performance of @babel/core.

TypeScript:

The parsing capabilities of TypeScript defy the common perception of the TypeScript compiler (TSC) being slow. The benchmark results suggest that the primary bottleneck for TSC is not in parsing but in the subsequent type checking phase.

Native Parser Review

SWC:
As the first Rust parser to make its mark, SWC adopts a direct approach by serializing the entire AST for use in JavaScript. It stands out for offering a broad range of APIs, making it a top choice for those seeking Rust-based tooling solutions. Despite some inherent overhead, SWC’s robustness and pioneering status continue to make it a preferred option.

Oxc::
Oxc is a contender for the title of the fastest parser available, but its performance is tempered by serialization and deserialization (serde) overhead. The inclusion of JSON parsing in our benchmarks reflects real-world usage, although omitting this step could significantly boost Oxc’s speed.

Tree-sitter
Tree-sitter serves as a versatile parser suitable for a variety of languages, not specifically optimized for TypeScript. Consequently, its performance aligns closely with that of Babel, a JavaScript-focused parser implemented in JavaScript. Alas, a Rust parser is not inherently faster by default, even without any N-API overhead.
A general purpose parser in Rust may not beat a carefully hand-crafted parser in JavaScript.

ast-grep

ast-grep is powered by tree-sitter. Its performance is marginally faster than tree-sitter, indicating napi.rs is a faster binding than manual using C++ nan.h.
I cannot tell whether the performance gain is from napi or napi.rs but
Leveraging the capabilities of tree-sitter, ast-grep achieves slightly better performance, suggesting that napi.rs offers a more efficient binding than traditional C++ nan.h methods. While the exact source of this performance gain—whether from napi or napi.rs—is unclear, the results speak to the effectiveness of the implementation. Or put it in another way, Broooooklyn is 🐐.

Native Parser Performance Tricks

tree-sitter & ast-grep’ Edge

These parsers manage to bypass serde costs post-parsing by returning a Rust object wrapper to Node.js. This strategy, while efficient, can lead to slower AST access in JavaScript as the cost is amortized over the reading phase.

ast-grep’s async advantage:

ast-grep’s performance in concurrent parsing scenarios is largely due to its utilization of multiple libuv threads. By default, the libuv thread pool size is set to four, but there’s potential to enhance performance further by expanding the thread pool size, thus fully leveraging the available CPU cores.

Future Outlook

As we look to the future, several promising avenues could further refine TypeScript parser performance:

• Minimizing Serde Overhead: By optimizing serialization and deserialization processes, such as employing Rust object wrappers, we can reduce the performance toll these operations take.

• Harnessing Multi-core Capabilities: Effective utilization of multi-core architectures can lead to substantial gains in parsing speeds, transforming the efficiency of our tooling.

• Promoting AST Reusability: Facilitating the reuse of Abstract Syntax Trees within JavaScript can diminish the frequency of costly parsing operations.

• Shifting Workloads to Rust: The creation of a domain-specific language (DSL) tailored for AST node querying could shift a greater portion of computational work to the Rust side, enhancing overall efficiency.

These potential improvements represent exciting opportunities to push the boundaries of Rust tooling in parsing performance.

Hope this article helps you! We can continue to innovate and deliver even more powerful tools to the developer community!

Using open source software can be a double-edged sword: We enjoy the latest features and innovations, but we hate frequent and sometimes tedious upgrades.

Bevy is a fast and flexible game engine written in Rust. It aims to provide a modern and modular architecture, notably Entity Component System(ECS), that allows developers to craft rich and interactive experiences.
However, the shiny new engine is also an evolving project that periodically introduces breaking changes in its API.
Bevy’s migration guide is comprehensive, but daunting. It is sometimes overwhelmingly long because it covers many topics and scenarios.

In this article, we will show you how to make migration easier by using some command line tools such as git, cargo and ast-grep. These tools can help you track the changes, search for specific patterns in your code, and automate API migration. Hope you can migrate your Bevy projects with less hassle and more confidence by following our tips.

We will use the utility AI library big-brain, the second most starred Bevy project on GitHub, as an example to illustrate bumping Bevy version from 0.9 to 0.10.
Upgrading consists of four big steps: make a clean git branch, updating the dependencies, running fix commands, and fixing failing tests. And here is a list of commands used in the migration.

• git: Manage code history, keep code snapshot, and help you revert changes if needed.
• cargo check: Quickly check code for errors and warnings without building it.
• ast-grep: Search for ASTs in source and automate code rewrite using patterns or expressions.
• cargo fmt: Format the rewritten code according to Rust style guidelines.
• cargo test: Run tests in the project and report the results to ensure the program still works.

Preparation

Before we start, we need to make sure that we have the following tools installed: Rust, git and ast-grep.

Compared to the other two tools, ast-grep is lesser-known. In short it can do search and replace based on abstract syntax trees. You can install it via cargo or brew.

1
2
3
4

# install the binary `sg`/`ast-grep`
cargo install ast-grep
# or use brew
brew install ast-grep

Clone

The first step is to clone your repository to your local machine. You can use the following command to clone the big-brain project:

1

git clone git@github.com:HerringtonDarkholme/big-brain.git

Note that the big-brain project is not the official repository of the game, but a fork that has not updated its dependencies yet. We use this fork for illustration purposes only.

Check out a new branch

Next, you need to create a new branch for the migration. This will allow you to keep track of your changes and revert them if something goes wrong. You can use the following command to create and switch to a new branch called upgrade-bevy:

1

git checkout -b upgrade-bevy

Key take away: make sure you have a clean git history and create a new branch for upgrading.

Update Dependency

Now it’s time for us to kick off the real migration! First big step is to update dependencies. It can be a little bit tricker than you think because of transitive dependencies.

Update dependencies

Let’s change the dependency file Cargo.toml. Luckily big-brain has clean dependencies.

Here is the diff:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

diff --git a/Cargo.toml b/Cargo.toml
index c495381..9e99a3b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -14,11 +14,11 @@ homepage = "https://github.com/zkat/big-brain"
 [workspace]

 [dependencies]
-bevy = { version = "0.9.0", default-features = false }
+bevy = { version = "0.10.0", default-features = false }
 big-brain-derive = { version = "=0.16.0", path = "./derive" }

 [dev-dependencies]
-bevy = { version = "0.9.0", default-features = true }
+bevy = { version = "0.10.0", default-features = true }
 rand = { version = "0.8.5", features = ["small_rng"] }

 [features]

Update lock-file

After you have updated your dependencies, you need to build a new lock-file that reflects the changes. You can do this by running the following command:

1

cargo check

This will check your code for errors and generate a new Cargo.lock file that contains the exact versions of your dependencies.

Check Cargo.lock, return to step 3 if necessary

You should inspect your Cargo.lock file to make sure that all your dependencies are compatible and use the same version of Bevy. Bevy is more a bazaar than a cathedral. You may install third-party plugins and extensions from the ecosystem besides the core library. This means that some of these crates may not be updated or compatible with the latest version of Bevy or may have different dependencies themselves, causing errors or unexpected behavior in your code.
If you find any inconsistencies, you can go back to step 3 and modify your dependencies accordingly. Repeat this process until your Cargo.lock file is clean and consistent.

A tip here is to search bevy 0.9 in the lock file. Cargo.lock will list library with different version numbers.

Fortunately, Bevy is the only dependency in big-brain. So we are good to go now!

Key take away: take advantage of Cargo.lock to find transitive dependencies that need updating.

(Semi-)Automate Migration

cargo check and ast-grep --rewrite

We will use compiler to spot breaking changes and use AST rewrite tool to repeatedly fix these issues.
This is a semi-automated process because we need to manually check the results and fix the remaining errors.

The mantra here is to use automation that maximize your productivity. Write codemod that is straightforward to you and fix remaining issues by hand.

1. CoreSet

The first error is quite easy. The compiler outputs the following error.

1
2
3
4
5

error[E0432]: unresolved import `CoreStage`
   --> src/lib.rs:226:13
    |
226 |         use CoreStage::*;
    |             ^^^^^^^^^ use of undeclared type `CoreStage`

From migration guide:

The CoreStage (… more omitted) enums have been replaced with CoreSet (… more omitted). The same scheduling guarantees have been preserved.

So we just need to change the import name. Using ast-grep is trivial here.
We need to provide a pattern, -p, for it to search as well as a rewrite string, -r to replace the old API with the new one. The command should be quite self-explanatory.

1

sg -p 'CoreStage' -r CoreSet -i

We suggest to add -i flag for --interactive editing. ast-grep will display the changed code diff and ask your decision to accept or not.

1
2
3
4
5
6
7
8

--- a/src/lib.rs
+++ b/src/lib.rs
@@ -223,7 +223,7 @@ pub struct BigBrainPlugin;

 impl Plugin for BigBrainPlugin {
     fn build(&self, app: &mut App) {
-        use CoreStage::*;
+        use CoreSet::*;

2. StageLabel

Our next error is also easy-peasy.

1
2
3
4
5

error: cannot find derive macro `StageLabel` in this scope
   --> src/lib.rs:269:45
    |
269 | #[derive(Clone, Debug, Hash, Eq, PartialEq, StageLabel, Reflect)]
    |

The doc:

System labels have been renamed to systems sets and unified with stage labels. The StageLabel trait should be replaced by a system set, using the SystemSet trait as dicussed immediately below.

The command:

1

sg -p 'StageLabel' -r SystemSet -i

3. SystemStage

The next error is much harder. First, the error complains two breaking changes.

1
2
3
4
5

error[E0599]: no method named `add_stage_after` found for mutable reference `&mut bevy::prelude::App` in the current scope
   --> src/lib.rs:228:13
    |                                                            ↓↓↓↓↓↓↓↓↓↓↓ use of undeclared type `SystemStage`
228 |         app.add_stage_after(First, BigBrainStage::Scorers, SystemStage::parallel());
    |             ^^^^^^^^^^^^^^^ help: there is a method with a similar name: `add_state`

Let’s see what migration guide said. This time we will give the code example.

1
2
3
4
5
6
7
8
9

// before
app.add_stage_after(CoreStage::Update, AfterUpdate, SystemStage::parallel());

// after
app.configure_set(
    AfterUpdate
        .after(CoreSet::UpdateFlush)
        .before(CoreSet::PostUpdate),
);

add_stage_after is removed and SystemStage is renamed. We should use configure_set and before/after methods.

Let’s write a command for this code migration.

1
2
3

sg \
  -p '$APP.add_stage_after($STAGE, $OWN_STAGE, SystemStage::parallel())' \
  -r '$APP.configure_set($OWN_STAGE.after($STAGE))' -i

This pattern deserves some explanation.

$STAGE and $OWN_STAGE are meta-variables.

meta-variable is a wildcard expression that can match any single AST node. So we effectively find all add_stage_after call. We can also use meta-variables in the rewrite string and ast-grep will replace them with the captured AST nodes. ast-grep’s meta-variables are very similar to regular expression’s dot ., except they are not textual.

However, I found some add_stage_afters are not replaced. Nah, ast-grep is quite dumb that it cannot handle the optional comma after the last argument. So I used another query with a trailing comma.

1
2
3

sg \
  -p 'app.add_stage_after($STAGE, $OWN_STAGE, SystemStage::parallel(),)' \
  -r 'app.configure_set($OWN_STAGE.after($STAGE))' -i

Cool! Now it replaced all add_stage_after calls!

1
2
3
4
5
6
7
8
9
10
11

--- a/src/lib.rs
+++ b/src/lib.rs
@@ -225,7 +225,7 @@ impl Plugin for BigBrainPlugin {
-        app.add_stage_after(First, BigBrainStage::Scorers, SystemStage::parallel());
+        app.configure_set(BigBrainStage::Scorers.after(First));
@@ -245,7 +245,7 @@ impl Plugin for BigBrainPlugin {
-        app.add_stage_after(PreUpdate, BigBrainStage::Actions, SystemStage::parallel());
+        app.configure_set(BigBrainStage::Actions.after(PreUpdate));
@@ -253,7 +253,7 @@ impl Plugin for BigBrainPlugin {
-        app.add_stage_after(Last, BigBrainStage::Cleanup, SystemStage::parallel());
+        app.configure_set(BigBrainStage::Cleanup.after(Last));

4. Stage

Our next error is about add_system_to_stage. The migration guide told us:

1
2
3
4

// Before:
app.add_system_to_stage(CoreStage::PostUpdate, my_system)
// After:
app.add_system(my_system.in_base_set(CoreSet::PostUpdate))

Let’s also write a pattern for it.

1
2
3

sg \
  -p '$APP.add_system_to_stage($STAGE, $SYS)' \
  -r '$APP.add_system($SYS.in_base_set($STAGE))' -i

Example diff:

1
2
3
4
5

--- a/src/lib.rs
+++ b/src/lib.rs
@@ -243,7 +243,7 @@ impl Plugin for BigBrainPlugin {
-        app.add_system_to_stage(BigBrainStage::Thinkers, thinker::thinker_system);
+        app.add_system(thinker::thinker_system.in_base_set(BigBrainStage::Thinkers));

5. system_sets

The next error corresponds to the system_sets in migration guide.

1
2
3
4
5
6
7
8
9

// Before:
app.add_system_set(
  SystemSet::new()
    .with_system(a)
    .with_system(b)
    .with_run_criteria(my_run_criteria)
);
// After:
app.add_systems((a, b).run_if(my_run_condition));

We need to change SystemSet::new().with_system(a).with_system(b) to (a, b).
Alas, I don’t know how to write a pattern to fix that. Maybe ast-grep is not strong enough to support this. I just change with_system manually.
It is still faster than me scratching my head about how to automate everything.

Another change is to use add_systems instead of add_system_set. This is a simple pattern!

1
2
3

sg \
  -p '$APP.add_system_set_to_stage($STAGE, $SYS,)' \
  -r '$APP.add_systems($SYS.in_set($STAGE))' -i

This should fix system_sets!

6. Last error

Our last error is about in_base_set‘s type.

1
2
3
4
5
6
7
8
9
10
11
12

error[E0277]: the trait bound `BigBrainStage: BaseSystemSet` is not satisfied
   --> src/lib.rs:238:60
    |
238 |         app.add_system(thinker::thinker_system.in_base_set(BigBrainStage::Thinkers));
    |                                                ----------- ^^^^^^^^^^^^^^^^^^^^^^^ the trait `BaseSystemSet` is not implemented for `BigBrainStage`
    |                                                |
    |                                                required by a bound introduced by this call
    |
    = help: the following other types implement trait `BaseSystemSet`:
              StartupSet
              bevy::prelude::CoreSet
note: required by a bound in `bevy::prelude::IntoSystemConfig::in_base_set`

Okay, BigBrainStage::Thinkers is not a base set in Bevy, so we should change it to in_set.

1
2

-        .add_system(one_off_action_system.in_base_set(BigBrainStage::Actions))
+        .add_system(one_off_action_system.in_set(BigBrainStage::Actions))

Hoooray! Finally the program compiles! ship it Now let’s test it.

Key take away: Automation saves your time! But you don’t have to automate everything.

cargo fmt

Congrats! You have automated code refactoring! But ast-grep’s rewrite can be messy and hard to read. Most code-rewriting tool does not support pretty-print, sadly.
A simple solution is to run cargo fmt and make the repository neat and tidy.

1

cargo fmt

A good practice is to run this command every time after a code rewrite.

Key take away: Format code rewrite as much as you want.

Test Our Refactor

cargo test

Let’s use Rust’s standard test command to verify our changes: cargo test.

Oops. we have one test error, not too bad!

1
2
3
4
5
6
7
8
9

running 1 test
test steps ... FAILED

failures:

---- steps stdout ----
steps test
thread 'steps' panicked at '`"Update"` and `"Cleanup"` have a `before`-`after` relationship (which may be transitive) but share systems.'
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Okay, it complains that Update and Cleanup have a conflicting running order. This is probably caused by configure_set.

I should have caught the bug during diff review but I missed that. It is not too late to change it manually.

1
2
3
4
5
6
7
8

--- a/src/lib.rs
+++ b/src/lib.rs
@@ -225,7 +225,7 @@ impl Plugin for BigBrainPlugin {
-        app.configure_set(BigBrainStage::Scorers.after(First));
+        app.configure_set(BigBrainStage::Scorers.in_base_set(First));
@@ -242,12 +242,12 @@ impl Plugin for BigBrainPlugin {
-        app.configure_set(BigBrainStage::Actions.after(PreUpdate));
+        app.configure_set(BigBrainStage::Actions.in_base_set(PreUpdate));

Run cargo test again?

1
2
3
4
5
6
7
8
9
10

   Doc-tests big-brain

failures:

---- src/lib.rs - (line 127) stdout ----
error[E0599]:
  no method named `add_system_to_stage` found for mutable reference
    `&mut bevy::prelude::App`
  in the current scope

We failed doc-test!

Because our ast based tool does not process comments. Lame. :(
We need manually fix them.

1
2
3
4
5
6
7

--- a/src/lib.rs
+++ b/src/lib.rs
@@ -137,8 +137,8 @@
-//!         .add_system_to_stage(BigBrainStage::Actions, drink_action_system)
-//!         .add_system_to_stage(BigBrainStage::Scorers, thirsty_scorer_system)
+//!         .add_system(drink_action_system.in_set(BigBrainStage::Actions))
+//!         .add_system(thirsty_scorer_system.in_set(BigBrainStage::Scorers))

Finally we passed all tests!

1

test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.68s

Conclusion

Now we can commit and push our version upgrade to the upstream. It is not a too long battle, is it?

I have created a pull request for reference. https://github.com/HerringtonDarkholme/big-brain/pull/1/files

Reading a long migration guide is not easy, and fixing compiler errors is even harder.

It would be nice if the official guide can contain some automated command to ease the burden. For example, yew.rs did a great job by providing automation in every release note!

To recap our semi-automated refactoring, this is our four steps:

• Keep a clean git branch for upgrading
• Update all dependencies in the project and check lock files.
• Compile, Rewrite, Verify and Format. Repeat this process until the project compiles.
• Run Test and fix the remaining bugs.

I hope this workflow will help you and other programming language developers in the future!

React Server Component is a new React architecture that the React team introduced at the end of 2020, which enables developers to render components on the server side, thereby boosting performance and streamlining code.

However, this innovation, though already more than two years old, still poses some novel challenges and issues, and many React experts are baffled or perplexed by it.
I believe it’s not only me how are confused by the heated discussion on Twitter and hours long video explaining the new paradigm.
Dan Abramov presented three quizzes about React Server Component, arguably the hottest topic in the React community, to help the audience to understand the new technology better.

This article will analyze the three quizzes that half of the React experts on Twitter didn’t get it right. I hope this can help you understand the RSC better.

TLDR: React will render Client Component as a reference to the script on the server side, and Server Component will be streamed and rendered as a JSON-like UI. The references and JSON will be passed to the browser for coordination and view updates.

The Three RSC Quizzes

First Quiz

1
2
3
4
5
6

function Note({ note }) {
  return (
    <Toggle>
      <Details note={note} />
    </Toggle>
}

the only Client Component out of these is Toggle. It has state (isOn, initially false). It returns <>{isOn ? children : null}</>.

what happens when you setIsOn(true)?

• Details gets fetched
• Details appears instantly

Second Quiz

Now say isOn is true. You’ve edited the note and told the router to “refresh” the route. This refetches the RSC tree for this route, and your Note server component receives a note prop with latest DB content.

(1) does Toggle state get reset?
(2) does Details show fresh content?

• (1) yes and (2) yes
• (1) yes and (2) no
• (1) no and (2) yes
• (1) no and (2) no

Third Quiz

Here’s a little twist.

1
2
3
4

<Layout
  left={<Sidebar />}
  right={<Content />}
/>

All are Server components. But now your want to add a bit of state to Layout, like column width, that changes on mouse drag.

Can you make Layout a Client component? If yes, what happens on drag?

I believe some readers who don’t know RSC may be completely confused after reading these three questions and don’t understand what they are asking. So at first, we will briefly introduce what is RSC for those who are new here. If you already know the purpose of RSC, you can skip the section safely.

What is React Server Component?

React Server Component is a special React component that does not run on the browser side, but instead on the server side. So that it can directly access the server’s data and resources, without obtain them through indirection APIs like REST or GraphQL, etc.

React Server Component is a pattern that can help us reduce the number of network requests and the size of data, thereby improving the page loading speed and user experience. React Server Component can also serve dynamic content to users according to different requests and parameters, without having to recompile or deploy.

The purpose of React Server Component is to let developers build applications that span the server and client, combining the rich interactivity of client-side applications and the optimized performance of traditional server rendering.
React Server Component can solve some problems that existing technologies cannot solve or solve well, such as:

• Zero package size: React Server Component’s code only runs on the server side and will never be downloaded to the client side, so it does not affect the client’s package size and startup time. The client only receives the rendered results of RSC.

• Full access to backend: React Server Component can directly access backend data sources, such as databases, file systems or microservices without additional API endpoints.

• Automatic code splitting: React Server Component can dynamically choose which client components to render, so that the client only downloads the necessary code.

• No client-server waterfall: React Server Component can load data on the server and pass it as props to client components, thus avoiding the client-server waterfall problem.

• Avoid abstraction tax: React Server Component can use native JavaScript syntax and features, such as async and await, without having to use specific libraries or frameworks to implement data fetching or rendering logic.

Server Component and Client Component

Before understanding how RSC works, we must first understand two big concepts in RSC, server-side components (Server Component) and client-side components (Client Component).

Server Component

As the name suggests, server components run only once per request on the server, so they have no state and cannot use features that only exist on the client. Specifically:

• ❌ You cannot use state and side effects, because they (conceptually) run only once per request on the server. So useState(), useReducer(), useEffect() and useLayoutEffect() are not supported. You also cannot use custom hooks that depend on state or side effects.
• ❌ You cannot use browser-specific APIs, such as DOM (unless you polyfill them on the server).
• ✅ You can use async/await to access server data sources, such as databases, internal (micro) services, file systems, etc.
• ✅ You can render other server components, native elements (div, span, etc.) or client components.

Developers can also create some custom hooks or libraries designed for the server. All rules for server components apply. For example, a use case for a server hook is to provide some helper functions for accessing server data sources.

Client Component

Client Component is a standard React component. It obeys all the rules we learnt about React before. The new rules to consider are mainly what they can’t import server components.

• ❌ Cannot not import server components or call server hooks/libraries, because they only work on the server. However, server components can pass another server component as children to a client component.
• ❌ Cannot not use server-only data sources.
• ✅ You can use state and side effects, as well as custom React hooks.
• ✅ You can use browser APIs.

Here we need to emphasize the nesting of server components and client components. Although client components cannot directly import server components, they can use server components as children. For example, you can write code like <ClientTabBar><ServerTabContent/></ClientTabBar>. From the perspective of the client component, its child component will be a rendered tree, such as the output of ServerTabContent. This means that server and client components can be nested and interleaved at any level. We will explain this design in later quizzes.

How RSC works?

After understanding server components and client components, we can now start to learn how RSC works. RSC rendering is divided into two major phases: initial loading and view updating. There are also two environments for RSC: server and browser. Note that although server components only run on the server, the browser also needs to be aware of them for actual view creation or updating. Client components are similar.

Initial loading

Server

• [Framework] The framework’s routing matches the requested URL with a server component, passing route parameters as props to the component. Then it calls React to render the component and its props.
• [React] React renders the root server component, and recursively renders any child components that are also server components.
• [React] Rendering stops at native components (div, span, etc.) and client components. Native components are streamed in a JSON description of the UI, and client components are streamed in a serialized props plus a reference to the component code.
• [Framework] The framework is responsible for streaming the rendered output to the client as React renders each UI unit.

By default React returns a description of the rendered UI, which is a JSON-like data structure, rather than HTML. Using JSON data will allow new data to be more easily reconciled with existing client components. Of course, frameworks can choose to combine server components with “server-side rendering” (SSR) so that the initial render is also streamed as HTML, which will speed up the initial non-interactive display of the page.

On the server, if any server component suspends, React will pause rendering that subtree and send client a placeholder value. When the component is able to continue (unsuspend), React will re-render the component and stream the actual result of the component to the client. You can think of the data being streamed to the browser as JSON, but with slots for suspended components, where the values for those slots are provided as additional items in the response stream later.

Let’s dive into the RSC protocol a little bit by looking at an example of rendering a UI description to help us understand this paragraph. Note this paragraph might not be precise and RSC implementation might change in the future.

Suppose we want to render a div.

1
2
3
4

<div>
  Hello World
  <ClientComponent/>
</div>

After calling React.createElement, a data structure similar to the following is generated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

{
  $$typeof: Symbol(react.element),
  // element tag
  type: "div",
  props: {
    children: [
      // static text
      "Hello World",
      // client compoennt
      {
        $$typeof: Symbol(react.module.reference),
        type: {
          name: "ClientComponent",
          filename: "./src/ClientComponent.js"
        },
      },
    ]
  },
}

This data structure will be streamed to browser, in format like this:

1
2
3
4

// Sever Component Ouptut
["$","div",null,{"children":["Hello World", ["$","$L1",null,{}]]}]
// Client Component Reference
1:I{"id":"./src/ClientComponent.js","chunks":["client1"],"name":"ClientComponent","async":false}

The first array represents the output of a Server Component, and you can see that its output is similar to React’s data structure, except that the structure is not an object but an array.
The array element $ represents createElement, and the following element {children: xxx} represents props. In children, the first child is directly transmitted with a string. L1 is a placeholder, and the 1 in 1:I below corresponds to it, and 1:I’s data will be filled into L1‘s position.

You might be curious about what does the I mean. In react server component’s protocol, I represents ClientReferenceMetadata, a data structure helping browser to find the correct script entry to the client component.
1:I’s output is a reference to a client component, which contains the script name, chunk name and export name, for the browser runtime (such as webpack) to dynamically import client component code. This structure is the streaming structure mentioned above.

In summary, a Server Component will be rendered into a JSON-like data that represents UI, while client components will be converted into a JSON data that expresses script references.

Browser

• [Framework] On the client side, the framework receives the streaming React response and uses React to render it on the page
• [React] React deserializes the response and renders native elements and client components.
• [React] Once all client components and all server component outputs have been loaded, the final UI state will be displayed to the user. By then all Suspense boundaries have been revealed.

Note that browser rendering is gradual. React does not need to wait for the entire stream to complete before displaying some content. Suspense allows developers to display meaningful loading states while loading client component code and server components are fetching remaining data.

View Updating

Server components also support reloading to see the latest data. Note that developers do not fetch server components individually: one component by one request.
The idea is that given some starting server components and props, the entire subtree will be refetched at once. As with initial loading, this typically involves integration with routing and script bundling:

On Browser

• [App] When the application changes state or changes routes, it requests the server to refetch the new UI for the changed Server Component.
• [Framework] The framework coordinates sending the new route and props to the appropriate API endpoint, requesting the rendering result.

On Server

• [Framework] The interface receives the request and matches it with the requested server component. And it calls React to render the component and props, and handles the streaming of the rendering result.
• [React] React renders the component to the destination, with different rendering strategies for components and initial loading.
• [Framework] The framework is responsible for gradually returning the streaming response data to the client.

On Browser

• [Framework] The framework receives the streaming response and triggers a rerender of the route with the new rendering output.
• [React] React reconciles the new rendering output with the existing components on the screen. Because the description of UI is data, not HTML, React can merge new props into existing components, preserving important UI state such as focus or input input, or triggering CSS transitions on top of existing content. This is a key reason why server components return UI output as data (“virtual DOM”) rather than HTML.

Summary So Far…

This section is very long, but we can summarize the working principle of RSC in one sentence.

Client Component will be rendered into a script reference, Server Component will be streamed into a JSON-like UI, Server Component with async/await will be replaced by a placeholder first, and then streamed to the browser after resolving.

The table below has more details and principles analysis.

Three Quizzes, Three Features

Now let’s see how the above rendering process is applied in the RSC quizzes?

This article will combine these three questions to explain the three major features of RSC: rendering completeness, state consistency, and commutative client/server component.

Rendering Completeness

The first quiz:

1
2
3
4
5
6

function Note({ note }) {
  return (
    <Toggle>
      <Details note={note} />
    </Toggle>
}

Let’s write some more components to provide context for this question. Our Toggle component and Details component looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

"use client";

import { useState } from "react"

export function Toggle(props) {
  const [isOn, setIsOn] = useState(false)
  return (
    <div>
      <button onClick={() => setIsOn(on => !on)}>Toggle</button>
      <div>{isOn ? "on" : "off"}</div>
      <div>{isOn ? props.children : <p>not showing children</p>}</div>
    </div>
  )
}

1
2
3
4
5
6
7
8
9

export async function Details(props) {
  const details = await getDetails(props.note);
  return <div>{details}</div>;
}

async function getDetails(note: string) {
  await new Promise((resolve) => setTimeout(resolve, 2000));
  return `Details for ${note}`;
}

In this example, Note and Details are server-side components. Toggle is a client-side component, but its children Details appears directly under the server-side component `Note. So when rendering Note, it will roughly be rendered into

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{
  $$typeof: Symbol(react.element),
  type: {
    $$typeof: Symbol(react.module.reference),
    name: "default",
    filename: "./Toggle.js"
  },
  props: { children: [
   // children, note the
   {
      $$typeof: Symbol(react.element),
      type: Details,  // Details is rendered!
      props: { note: note },
   }
  ] },
}

Notice that Details is always rendered on the server side and delivered to the client.

When Toggle is rendered on the client side, Details is not used, but its rendering result is still sent to the client side.

Even though Details is an asynchronous server component that uses async/await, it can still be sent to the front-end after it finishes asynchronously due to the streaming process of React Server Component.

And when the user changes state, the client can directly use the server’s pre-rendered results for dom operations because Details props are the same as those rendered by the server. Therefore, the answer to this question is that Details will appear immediately.

This question reveals the “completeness” of React Server Component: as long as the component appears under the render function of the server-side component, it will be rendered regardless of its usage in client side.

State Consistency

Now assume that isOn is true. You edit the note and tell the router to “refresh” the route. This will re-fetch the RSC tree for this route, and your Note server component will receive a note attribute with the latest database content.

The second question reveals the consistency of the RSC. When the Toggle component changes props on the client side, this change is synchronized between both the server-side component and the client-side component and remains consistent on both ends.
<Details note={note} /> When a note changes, React detects the change in the note and sends a request to the server for the new Details rendering data.

Also, the state of the client component Toggle itself is not reset or lost in the browser.

Thus, the design of RSC ensures that the state of the application is consistent across both server and browser.

3. Commutative Client/Server Component

For the third question, let’s expand the question code for context as well.

1
2
3
4
5
6
7
8

function App() {
  return (
    <Layout
      left={<Sidebar />}
      right={<Content />}
    />
  )
}

Layout component is a server component.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// Server Component
export function Layout(props: {
  left: React.ReactNode
  right: React.ReactNode
}) {
  return (
    <div>
      <div>
        <div style={{ width: `${width}px` }}>{props.left}</div>
        <div style={{ width: `${500 - width}px` }}>{props.right}</div>
      </div>
    </div>
  )
}

Let’s rewrite it to a client component that useState. In this example, the width is changed on the client side by changing the input slider. (The implementation detail is inconsequential here).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

"use client"

import { useState } from "react"

export function Layout(props: {
  left: React.ReactNode
  right: React.ReactNode
}) {
  const [width, setWidth] = useState(200)
  return (
    <div>
      <input
        type="range"
        step={1}
        value={width}
        onChange={(e) => setWidth(Number(e.target.value))}
      />
      <div>
        <div style={{ width: `${width}px` }}>{props.left}</div>
        <div style={{ width: `${500 - width}px` }}>{props.right}</div>
      </div>
    </div>
  )
}

Note, in this case we can change the Layout from server component to client component without the App component.
The only thing changed is how the App is rendered on server side.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

 {
   $$typeof: Symbol(react.element),
-  type: Layout,
+  type: {
+    $$typeof: Symbol(react.module.reference),
+    name: "default",
+    filename: "./Layout.js"
+  },
   props: {
    left: {
       $$typeof: Symbol(react.element),
       type: Sidebar,
    },
    right: {
       $$typeof: Symbol(react.element),
       type: Content,
    }
   },
 }

As you can see, during serialization, the Layout is transformed from a server-side component to a client-side component. The type field is change from a direct import of Layout component to a module.reference. Meanwhile its child components remain unchanged.

Before we change Layout to client component, the process of rendering Layout happens completely on the server side, and its children are also rendered on the server side. The rendered results are sent to the browser to be transformed into DOM.

After we change Layout to a client component, the process of rendering Layout happens in the browser, but the child components are still rendered on the server side. When the browser renders the server’s JSON UI output, Layout inserts the results of the server-side child components into the browser DOM.

Since the props of the child components are not changed when user changes the layout width on client-side (because they have no props), so the rendering result on the server side does not need to be recaptured.

Therefore, the answer to this question is “it can be converted to a client-side component and the child components will not be recaptured”.

We can rewrite server-side components as client-side components in RSC projects without rewriting component composition at use site. We can call this interchangeability as “commutative” server/client components.

Conclusion

The documentation and RFC for React Server Component is relatively obscure and does not give practical examples, leading many people to wonder what it really is.

In this article, I tried to explain the design ideas and principles of React Server Component by explaining it with Dan’s quizzes.
I hope it can help you understand this new feature and become one of the few materials that can let you learn RSC without watching Youtube or following Twitter threads. I hope this will let you understand more about the principle of RSC and the three performance characteristics! Complete Rendering, Consistent State, and Commutative Server/Client Components.

It is not easy to create, if you think this article is helpful to you, please follow me on Medium or treat me a cup of coffee.

Reference

1. React 18: React Server Components | Next.js. https://nextjs.org/docs/advanced-features/react-18/server-components.
2. What you need to know about React Server Components. https://blog.logrocket.com/what-you-need-to-know-about-react-server-components/.
3. React Server Components. - It’s not server-side rendering. | by Nathan …. https://blog.bitsrc.io/react-server-components-1ca621ac2519.
4. What are React Server Components? - FreeCodecamp. https://www.freecodecamp.org/news/what-are-react-server-components/.
5. How React Server Compoents Wors. https://www.plasmic.app/blog/how-react-server-components-work#the-high-level-picture
6. 45% failed Dan’s Server Component Quiz https://www.youtube.com/watch?v=AGAax7WzStc

This is a rewrite of the zhihu article. Permission has been granted.
The original author of this article is finding a job, find him on GitHub.

What’s Type level programming, a.k.a type gymnastics?

TypeScript is a powerful and expressive language that allows you to write complex yet elegant code. Some TypeScript users love to explore the possibilities of the type system and love to encode logic at type level.
This practice, as we have a slang for it, is known as type gymnastics. Type gymnastics can help you to perform various tasks such as parsing, validating, transforming, or computing values based on types. Type gymnastics often requires the users mastering a wide range of advanced TypeScript features such as conditional types, recursive types, template literal types, mapped types and more. The community also helps users to learn type gymnastics by creating fun and challenges such as type challenges.

You can proudly call yourself a TypeScript magician if you can solve most of the type challenges!

The type-level programming community has been creating all kinds of amazing tricks ever since the introduction of template literal type in version 4.1. Some enthusiasts have pushed this hobby to the extreme, and even tried to reimplement TypeScript at compile-time using pure types!

However, there are still three major obstacles that prevent TypeScript becoming a true turing complete machine (#14833). They are:

• tail recursion count: the maximum number of recursive calls in a conditional type.
• type instantiation depth: the maximum depth when we instantiate a generic type alias.
• type instantiation count: the maximum number of type instantiations.

These limits are not arbitrary. The compiler has set a maximum limit for these three values and will throw an error TS2589 if we exceed them to prevent the compiler from hanging or crashing.
They are the walls that stop us from breaking through the limits of TypeScript compiler. But, there is actually a hidden layer behind the wall, if we explore the compiler source code carefully.

Doing type level programming is like casting magic, but breaking through the wall of type gymnastics is like finding the hidden chamber of secrets that contains the most powerful and dangerous type of all.

Some digression for background…

Back then three years ago, I wrote an article about how to implement a recursive descent parser. At that time, there was no template literal type, and a lot of type system limitations have been lifted since then. Crafting a code interpreter using recursive descent parsing would easily exceed the compiler’s limit.

So I decided to switch to a bottom-up parsing method to more easily bypass the limit, with some tricks :). Later, as TypeScript gradually evolves, newly added features opened unprecedented possibilities of type gymnastics. This finally gave me a chance to implement a type gymnastics parser generator for LALR(1) grammar to generate a parser for a subset of JavaScript syntax.
I designed a special purpose “bytecode” for the ease of type level execution and directly compiled the input code into the bytecode during the parsing process(I still have some unfinished work left, procrastinating now…). My ultimate goal is to implement a type level scripting language that can execute sufficiently complex code.

To achieve this goal, I need to solve all the three limitations mentioned above to run lunatic code patterns, to some extent, at compile time!

Tail Recursion Count

Let’s look at the first limitation, the type tail recursion count (tailCount). Type level programming often uses recursive types. Before the previous versions supported recursive types, we usually implemented recursion by combining mapped types, recursive type definitions and index types. Ever since we had recursive types and tail recursion of type instantiation, this kind of type gymnastics became much more elegant, but at the same time we also got this tail recursion count limitation.

Searching for tailCount in the TypeScript source code, we can see that the implementation is related to computing conditional types. Let’s simplify the code a bit and take a look (sorry folks, I cannot get a link to GitHub source code due to checker.ts is too large):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

// transitively used in instantiateType
function getConditionalType(root: ConditionalRoot) {
    let result;
    let tailCount = 0;

    while (true) {
        if (tailCount === 1000) {
            error(2589);
            result = errorType;
            break;
        }
        // false branch
        if (!isTypeAssignableTo(checkType, extendsType)) {
            const falseType = getTypeFromTypeNode(root.node.falseType);

            if (canTailRecurse(falseType)) {
                // root = falseType;
                continue;
            }
            // tail call terminates, creating new type
            result = instantiateType(falseType);
            break;
        }
        // true branch
        if (isTypeAssignableTo(checkType, extendsType)) {
            const trueType = getTypeFromTypeNode(root.node.trueType);

            if (canTailRecurse(trueType)) {
                // root = trueType;
                continue;
            }
            // end of tail recursion, instantiate new type
            result = instantiateType(trueType);
            break;
        }
    }
    return result;

    function canTailRecurse(newType: Type) {
        if (newType.flags & TypeFlags.Conditional) {
            const newRoot = (newType as ConditionalType).root;
            const newCheckType = newRoot.isDistributive
                ? newRoot.checkType
                : undefined;

            if (
                !newCheckType ||
                !(newCheckType.flags & (TypeFlags.Union | TypeFlags.Never))
            ) {
                root = newRoot;

                if (newRoot.aliasSymbol) {
                    tailCount++;
                }
                return true;
            }
        }
    }
}

We can see that as long as the branch of the conditional type that matches is another conditional type, and the new conditional type is neither a distributive conditional type nor a union type or never, then the new conditional type will be substituted into the next round of computation.

The only problem is that it will trigger an error TS2589 when the loop reaches 1000 times, TS Playground:

1
2
3
4
5
6
7

type ZeroFill<
    Output extends 0[],
    Count extends number
> = Output["length"] extends Count ? Output : ZeroFill<[...Output, 0], Count>;

type Sized999 = ZeroFill<[], 999>; // Okay.
type Sized1K = ZeroFill<[], 1000>; // Oops, TS2589.

To avoid reaching the 1000 times recursion limit, we can deliberately break the tail recursion after a certain number of recursions (using an extra parameter to count), as long as we return a type that does not satisfy the above conditions, and does not affect the result of the type we return, such as returning an intersection type (Playground Link):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// Hard-code a number array, array index corresponds to value plus 1
type Incr = [
   1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
  21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
  41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
  61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
  81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100,
  0,
];

type ZeroFill<
    Output extends 0[],
    Count extends number,
    Iterations extends Incr[number] = 0
> = Iterations extends 100
    ? // Change the recursive type to an intersection type, break the tail recursion and reset the iteration
      ZeroFill<Output, Count, 0> & {}
    : // Recursion terminates
    Output["length"] extends Count
    ? // Final Result
      Output
    : // Increment the count for each recursion
      ZeroFill<[...Output, 0], Count, Incr[Iterations]>;

type Sized3K = ZeroFill<[], 3000>; // Okay.
type Sized4K = ZeroFill<[], 4000>; // Okay, but this is not the end of story...

Type Instantiation Depth

The type above has now exceeded the limit of 1000 recursions, but if you comment out Sized3K, you will see that Sized4K will trigger TS2589. This is related to the second limitation, which is the type instantiation depth (instantiationDepth). The reason why commenting out has an effect is because TypeScript will cache all instantiated types, which we will discuss later. Again, we can search for instantiationDepth in the TS source code. You can see that it is implemented with all types that have type mapping. There are several kinds of type mapping. For example, types that use generic parameters will have type mapping to map parameters to actual types. The relevant code is simplified as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

function instantiateType(type?: Type, mapper?: TypeMapper): Type | undefined {
    return type && mapper ? instantiateTypeWithAlias(type, mapper) : type;
}

function instantiateTypeWithAlias(
    type: Type,
    mapper: TypeMapper,
    aliasSymbol?: Symbol,
    aliasTypeArguments?: readonly Type[]
) {
    if (instantiationDepth === 100 || instantiationCount >= 5000000) {
        error(2589);
        return errorType;
    }
    totalInstantiationCount++;
    instantiationCount++; // note: `count` only increments
    instantiationDepth++;
    const result = instantiateTypeWorker(
        type,
        mapper,
        aliasSymbol,
        aliasTypeArguments
    );
    instantiationDepth--; // note: only depth got decremented
    return result;
}

We also see the instantiationCount mentioned at the beginning, but we will first focus on the instantiationDepth for now. We can see that before and after each type alias instantiation, the instantiation depth will increase and decrease, and any call to instantiateType in the middle process will indirectly call the above type, which will make the depth count continue to increase.
To avoid reaching the maximum depth, one is to use tail recursion as much as possible, and the other is to reduce the dependence on intermediate types in the recursive process if you use the technique of breaking tail recursion before.
Here is an example code (to reduce interference we simplify it to only have two generic parameters, only keep the recursive logic itself, and use intersection type to prevent tail recursion instantiation):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// Simulate a type that keeps calculating values and sets the finish state based on the results
type BadWay<State, Value> = State extends `finish`
    ? Value
    : // Use conditional type and infer type to save the new result
    Calculate<Value> extends infer NewValue
    ? // Use conditional type and infer type to save the new state
      CheckValue<NewValue> extends infer NewState
        ? // Compared to direct recursion, this adds two levels of type instantiation depth
          BadWay<NewState, NewValue>
        : never
    : never;

// In fact, repeating “calling” type aliases does not lose anything, TS will always cache all types instances.
type GoodWay<State, Value> = State extends `finish`
    ? Value
    : // Direct recursion reduces the instantiation depth as much as possible
      GoodWay<
          // The intermediate result is instantiated when passing parameters, which can be instantiated before recursion, releasing the increased depth.
          CheckValue<Calculate<Value>>,
          // The repeated alias calls will only be truly instantiated once due to the caching mechanism, so you don’t have to worry about various restrictions and overheads.
          Calculate<Value>
      > & {};

In the example above, BadWay increases the depth by two levels more than GoodWay for each recursion, which will reach the recursion depth limit faster.

With the instantiation level limit, we should start recursion as early as possible, putting all kinds of intermediate type calculations into parameters. And let each iteration type complete instantiation as soon as possible, releasing the increased depth.

Another way to release the depth count in advance is to manually split the iteration process into multiple parts, storing intermediate result as a type alias.
Let’s take the first type as an example:

1
2
3

type Sized3K = ZeroFill<[], 3000>; // Okay.
type Sized4K = ZeroFill<Sized3K, 4000>; // Okay.
type Sized5K = ZeroFill<Sized4K, 5000>; // Okay.

One thing to keep in mind is that as the type becomes more complex, the tsc compiler may handle it fine, but the editor may not give any feedback due to the long response time. This is because they use different methods and caches for type checking. For example, in the code above, if we comment out Sized4K (which has a cache effect), and directly generate Sized5K from Sized3K, we will get a TS2589 error again. See the playground link.This is not caused by the two limitations we mentioned before, because this time we only have 2000 iterations, much less than Sized3K. The actual cause of this compilation error leads us to the third limitation: the type instantiation count (instantiationCount).

Type Instantiation Count

We have encountered the variable instantiationCount before in the function instantiateTypeWithAlias. It has a current limit of 5,000,000, which is incremented together with instantiationDepth every time a type is instantiated. Unlike instantiationDepth, it does not decrease when the instantiation is done because it represents the number of type instances that are created in one instantiation process. If we continue to search the source code, we can see that it is set to 0 when the compiler checks some types of syntax nodes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

function checkSourceElement(node) {
    // ...
    instantiationCount = 0;
    // ...
}

function checkDeferredNode(node) {
    // ...
    instantiationCount = 0;
    // ...
}

function checkExpression(node, checkMode, forceTuple) {
    // ...
    instantiationCount = 0;
    // ...
}

If we debug tsc and set a breakpoint after the condition instantiationCount >= 5000000, we will see in the call stack that the error is thrown when instantiating an object type with more than 3700 members. This object type is actually the array type that we generated. We will not analyze step by step why it produces so many (5 million) type instances here. We just need to know that this array is created by many generic parameters, which leads to a large number of type instances. Even if our type gymnastics did not generate such a large array, it would slow down the speed when producing so many type instances. So how can we avoid this situation?

The direct conclusion is to use other types to achieve the purpose according to the needs. For example, if you use an array type to simulate a stack structure, and only need to access the last one or a few elements, then you can define a List object type, which is concatenated, but the structure of the object is fixed:

1
2
3
4
5
6
7
8
9
10
11

type List<T> = { data: T; prev: List<T> };

type Append<P extends List<unknown>, T> = { data: T; prev: P };

type Stack = Append<Append<Append<never, 1>, 2>, 3>; // [1, 2, 3]

type Popped = Stack["prev"]; // [1, 2]

type Last = Stack["data"]; // 3

type SecondToLast = Stack["prev"]["data"]; // 2

Alternatively we can encode the logic in string by constructing template string of a specific format.

1
2
3
4
5
6
7
8

type Append<S extends string, T extends string> = `${T},${S}`;

type Stack = Append<Append<Append<"", "alpha">, "beta">, "gamma">;

type Popped = Stack extends `${string},${infer S}` ? S : never;

type Last = Stack extends `${infer T},${string}` ? T : never;

You can even use union types. However, you need to pay attention that when union types are used in distributive conditional types, they will generate the same number of type instances as the number of elements, but they are still a better choice in many cases.

Usual type level programming will not trigger this limitation, so we will not describe it too much. To optimize the number of instantiations, you inevitably need to read the compiler source code, or you can add the --diagnostics parameter when calling tsc, and observe the Instantiations in the output, which can show how many type instances are generated in total.

Conclusion

One caveat is that all tricks in the article are probably the byproduct of a specific compiler implementation and are not officially supported by the TypeScript team. And they may even be broken in the future!
Actually, most of these dark magics are manipulating type instantiation to make time-space trade-offs. We are nudging compiler to forget intermediate type caches (Obliviate!), prompting it to compute more types (Imperio!) and forcing it to work longer for a compilation (Incarcerous!). All the grueling rituals for the holy grail of turing completeness!

However, they are still very useful in some cases and help us break the limitations of TypeScript! The snake is in the chamber of secrets, so use these lost spells wisely, my great mage!

If you find this article useful, I would be more than happy if you can treat me some coffee.

Translation, rewrite and title image generation are all assisted by MicroSoft Bing Chat.

In this post I will discuss how to optimize the Rust CLI tool ast-grep to become 10 times faster.

Rust itself usually runs fast enough, but it is not a silver bullet to all performance issues.

In this case, I did not pay enough attention to runtime details or opted for naive implementation for a quick prototype. And these inadvertent mistakes and deliberate slacking off became ast-grep’s bottleneck.

Context

ast-grep is my hobby project to help you search and rewrite code using abstract syntax tree.

Conceptually, ast-grep takes a piece of pattern code (think it like a regular expression but for AST), matches the pattern against your codebase and gives a list of matched AST nodes back to you. See the playground for a live demo.

I designed ast-grpe’s architecture with performance in mind. Here are a few performance related highlights:

• it is written in Rust, a native language compiled to machine code.
• it uses the venerable C library tree-sitter to parse code, which is the same library powering GitHub’s codesearch.
• its command line interface is built upon ignore, the same crates used by the blazing fast ripgrep.

Okay, enough self-promotion BS. If it is designed to be fast, how comes this blog? Let’s dive into the performance bottleneck I found in my bad code.

Spoiler. It’s my bad to write slow Rust.

Profiling

The first thing to optimize a program is to profile it. I am lazy this time and just uses the flamegraph tool.

Installing it is simple.

1

cargo install flamegraph

Then run it against ast-grep! No other setup is needed, compared to other profiling tools!

This time I’m using an ast-grep port of es-lint against TypeScript‘s src folder.

This is the profiling command I used.

1

sudo flamegraph -- sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null

The flamegraph looks like this.

Optimizing the program is a matter of finding the hotspots in the flamegraph and fix them.

For a more intuitive feeling about performance, I used the old command time to measure the wall time to run the command. The result is not good.

1
2

time sg scan -c eslint/sgconfig.yml TypeScript/src
17.63s user, 0.46s system, 167% cpu, 10.823 total

The time before user is the actual CPU time spent on my program. The time before total represents the wall time. The ratio between them is the CPU utilization. In this case, it is 167%. It means my program is not fully utilizing the CPU.

It only runs six rules against the codebase and it costs about 10 whole seconds!

In contrast, running one ast-grep pattern agasint the TypeScript source only costs 0.5 second and the CPU utilization is decent.

1
2
3

time sg run -p '$A && $A()' TypeScript/src --json > /dev/null

1.96s user, 0.11s system, 329% cpu, 0.628 total

Expensive Regex Cloning

The first thing I noticed is that the regex::Regex type is cloned a lot. I do know it is expensive to compile a regex, but I did not expect cloning one will be the bottleneck.
Much to my limited understanding, dropping Regex is also expensive!

Fortunately the fix is simple: I can use a reference to the regex instead of cloning it.

This optimzation alone shaves about 50% of execution time.

1
2

time sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
13.89s user, 0.74s system, 274% cpu 5.320 total

The new flamegraph looks like this.

Matching Rule can be Avoided

The second thing I noticed is that the match_node function is called a lot. It is the function that matches a pattern against an AST node.
ast-grep can match an AST node by rules, and those rules can be composed together into more complex rules.
For example, the rule any: [rule1, rule2] is a composite rule that consists of two sub-rules and the composite rule matches a node when either one of the sub-rules matches the node.
This can be expensive since multiple rules must be tried for every node to see if they actually make a match.

I have already forsee it so every rule in ast-grep has an optimzation called potential_kinds. AST node in tree-sitter has its own type encoded in a unsigned number called kind.
If a rule can only match nodes with specific kinds, then we can avoid calling match_node for nodes if its kind is not in the potential_kinds set.
I used a BitSet to encode the set of potential kinds. Naturally the potential_kinds of composite rules can be constructed by merging the potential_kinds of its sub-rules, according to their logic nature.
For example, any‘s potential_kinds is the union of its sub-rules’ potential_kinds, and all‘s potential_kinds is the intersection of its sub-rules’ potential_kinds.

Using this optimization, I can avoid calling match_node for nodes that can never match a rule. This optimization shaves another 40% of execution time!

1
2

sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
11.57s user, 0.48s system, 330% cpu, 3.644 total

The new flamegraph.

Duplicate Tree Traversal

Finally, the function call ts_tree_cursor_child_iterator_next caught my eyes. It meant that a lot of time was spent on traversing the AST tree.

Well, I dumbly iterating through all the six rules and matching the whole AST tree for each rule. This is a lot of duplicated work!

So I used a data structure to combine these rules, according to their potential_kinds. When I’m traversing the AST tree, I will first retrieve the rules with potential_kinds containing the kind of the current node. Then I will only run these rules against the node. And nodes without any potential_kinds hit will be naturally skipped during the traversal.

This is a huge optimization! The ending result is less than 1 second! And the CPU utilization is pretty good.

1
2

sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
2.82s user, 0.12s system, 301% cpu, 0.975 total

Conclusion

The final flamegraph looks like this. I’m too lazy to optimize more. I’m happy with the sub-second result for now.

Optimizing ast-grep is a fun journey. I learned a lot about Rust and performance tuning. I hope you enjoyed this post as well.

Angular recently announced a new render engine called Ivy.
Ivy is an amazing present from Angular team! It produces hello-world app in mere 3.2KB, on a par with minimal framework like preact. Unfortunately little documentation, if any, exists to explain how Ivy works.

This blog post strives to fill this gap, by translating an amazing answer from zhihu, a superior Q&A website to Quora. I hope this post can communicate Angular’s innovation to broader front-end world by breaking language barrier.

TL;DR: Ivy is an extremely building-tool friendly, web-first render engine that paves Angular’s way to web components.

Opinion is not mine. Words in parentheses are from the original author unless explicitly annotated as from translator. Translator notes are added for audience with no Angular background.

Orignal post:

How to comment on the fact that Angular’s new Ivy render engine produces 3.2KB compiled JavaScript? (traslator note: how to comment is a jargon on zhihu, meaning what's your opinion)

Let’s first answer several frequent questions:

1. 3.2KB is the result after minification + gzip (a convention to compare framework size), which is the exact payload size we send to browser. (if we use more advanced compression like brotli, the size will be smaller on modern browsers).

2. Ivy renderer won’t be default in Angular v6. You need to manually enable it by switching on certain compiler option. It might be default in v7 if no further issues are found.

3. Code completion isn’t proportional to feature completion. 50% code completed doesn’t imply 50% feature completed. For now Ivy has seemingly decent code completion rate, but it still has a long way before ready for production. It can’t even complete a simple app for benchmark.

4. Compared with previous Angular’s compiled code, Ivy’s result is almost like hand-written. And actually you can write it by hand! (Though in practice you won’t.)

5. NgModule is challenged, again.

6. Ivy won’t promise you a plunge in code weight. Your bundle will still be bulky if a lot of CSS is inlined in your components.

Since the question is to comment on 3.2KB compiled file, I will focus on payload size optimization, thus, answering the question “What optimization has Ivy Renderer done?”.

It can never be overstated that the extreme size isn’t achieved by the ADVANCED mode of Closure Compiler. Rollup can achieve the same level optimization (within less than 1KB difference). Therefore Ivy is arguably the true building-tool friendly renderer, not Closure-Compiler-only renderer.

Disclaimer: all the below is based on current (Feb, 2018) implementation, published roadmap. If following version changes leads to the following content outdated or incorrect, please pardon our no updating.

A. Removing dependence on platform-browser

As a platform independent framework, can we run application without platform specific code? The answer is NO, of course. Ivy just inlines DOM Rendere to its core. If you only need run your application on browser (taking no account of WebWorker), you can run Angular without including platform specific code. For example, the code for binding text is:

1
2
3
4

value !== NO_CHANGE &&
  ((renderer as ProceduralRenderer3).setValue ?
    (renderer as ProceduralRenderer3).setValue(existingNode.native, stringify(value)) :
    existingNode.native.textContent = stringify(value));

Fallback code is obvious: if no renderer exists (translator note: the condition is better explained as renderer isn’t a ProceduralRenderer3), Ivy will directly modify dom element’s textContent.

Currently no Ivy code relies on platform, thus common problems arise:

• No support for Event Plugin. e.g. syntactic sugar for keyboard event (like keydown.enter) and events from Hammer, which are all provided by platform-browser.

• No Sanitizer. Though Angular isn’t string template by itself and is naturally immune to XSS, it still cannot survive abhorrent abuse of innerHTML. Sanitizer, again currently bestowed by platform-browser, comes to rescue by filtering user content. Without platform hardly can we achieve the same level security.

• No support for Component Style; No View Encapsulation. They are all implemented by different Renderers in platform-browser package. Only inline style is available for component styling (Another better way is independent CSS and dynamic class).

So the current Ivy renderer demoed in hello-world app isn’t a full featured Angular for some users.

B. No NgFactory file anymore

Under the new Ivy mode, compiled template will be stored in the static fields in class, instead of generating new wrapper class (NgFactory). For example:

Component -> ngComponentDef

1
2
3
4
5
6
7
8

@Component({ /*...*/ })
class MyComponent { }

// ->

class MyComponent {
  static ngComponentDef = defineComponent({ /*...*/ })
}

Directive -> ngDirectiveDef

1
2
3
4
5
6
7
8

@Directive({ /*...*/ })
class MyDirective { }

// ->

class MyDirective {
  static ngDirectiveDef = defineDirective({ /*...*/ })
}

NgModule -> ngInjectorDef

1
2
3
4
5
6
7
8

@NgModule({ /*...*/ })
class MyModule { }

// ->

class MyModule {
  static ngInjectorDef = defineInjector({ /*...*/ })
}

WAT? How can Your Majesty NgModule compiled to one single Injector?(

Due to the colocation of compiled template and class, the new compilation can fully enjoy building tool’s normal optimization. (Tree-Shaking for the most part), waiving most special process in build-optimizer.

In fact, this improvment is more significant for building than for size optimization. The new style compiler enables single file compilation, one ts file to one js file. We no longer need to worry about mapping from source file to ngfactory.

Further more, this unifies library consumption in JIT and AOT. Once the compilation is stabilized, all libraries can compile code before publication, rather than at end uses’ bundling phase. This can lead to faster building.

C. Greatly simplify bootstrap code

All Angular applications need to configure bootstrap component (actually it isn’t mandatory, but alternatives are more complex), and initialize one NgModule[Factory]. Something like:

1
2
3
4
5
6
7
8
9
10

import { NgModule } from '@angular/core'
import { platformBrowser } from '@angular/platform-browser'
import { AppComponent } from './app.component'

@NgModule({
  bootstrap: [AppComponent]
})
class AppModule { }

platformBrowserDynamic().bootstrapModule(AppModule)

New bootstrapping code is based on component:

1
2
3
4
5

// not available yet
import { renderComponent } from '@angular/core'
import { AppComponent } from './app.component'

renderComponent(AppComponent)

Hey, why we still need AppModule?

Ivy is based on NgModule-less bootstrapping for now. Though not completed yet, compilers can make bootstrapping accept NgModule (if it still exists) to provide Injector. In other word, renderComponent can accept an optional configuration argument.

By the way, this is also friendlier to bootstrapping multiple Angular instances in one page.

D. Redesigned DevMode configuration and checking

In the previous Angular, DevMode is disabled by function call dynamically.

1
2
3
4
5

import { enableProdMode } from '@angular/core'

enableProdMode()

platformFoo().bootstrapBar(baz)

In other word, whether DevMode is on is totally Angular’s internal state. Thus all debug related code is dynamically used and cannot be excluded in compile time. (Even closure compiler is ineffective).

But in Ivy DevMode is purely compile time configuration, all debugging code will be executed after checking the global variable ngDevMode. For example:

1

ngDevMode && assertEqual((state as LView).node, null, 'lView.node');

So building tool can replace the corresponding variable (e.g Webpack’s DefinePlugin), and corresponding debug code can be detected as dead code, and then optimizer (e.g UglifyJS) can remove it.

E. Feature named as Feature

Ivy mode has added a new feature called Feature. Or, Ivy introduces a new concept called Feature. Simply put, Feature is a pre-processor for DirectiveDef, which can be thought as a “Decorator pattern” tailored for DirectiveDef. (Translator note: More simply, you can provide a custom function to Angular, and Angular will apply it against ComponentDef metadata. So you can extend ComponentDef‘s Feature in your own function. Source).

Let’s take an example in Ivy. OnChanges isn’t implemented by Renderer but a predefined Feature. The Feature uses defineProperty to listen on the property decorated by @Input, and automatically stores previousValue in the instance to generate SimpleChanges for change detection, and finally triggers OnChanges by intercepting OnInit and DoCheck without Angular core’s help.

This means OnChanges‘ code is also tree-shakable! If no component ever declares the lifecycle ngOnChanges, no DirectiveDef will import NgOnChangesFeature. So optimizer can reasonably eliminate dead code.

WAT? How about the dirty check promise? From now on we have exception in Angular’s change detection?!

So it is notable that OnChanges in Ivy is not a real lifecycle any more (Translator note: it is not listed in the ComponentDef).
Put differently, users can extend lifecycle themselves, do as they please.

But the direct aim of Feature concept is for the incoming LifeCycle as Observables. Users don’t need to declare methods (ngOnInt) in class, but rather use the lifecycle observables directly. Indeed. by intercepting DirectiveDef, users and third-party library authors can modify lifecycle hooks in runtime. We can even use decorators to declare lifecycles.

In summary, Feature can be thought as a variant of Higher Order Component (compared with class factory): it modifies component type itself based on runtime requirement, instead of changing component’s internal state according to component logic.

F. New Injectable API

Actually this isn’t related to the hello-world demo, nor related to even Ivy (since it can be used in non Ivy mode). However, the new Injectable API has a huge impact on Angular’s size.

Besides the XXXDef properties listed above, we have another one called ngInjectableDef:

1
2
3
4
5
6
7
8

@Injectable()
class MyService { }

// ->

class MyService {
  static ngInjectableDef = defineInjectable({ /*...*/ })
}

In classical sense (if anyone ever cared), we can easily find dependency injection and dead code elimination are contradictory in priciple.

• DI’s nature is side effect: Provider changes execution context via configuration, and Consumer retrieves content from context.

• DCE’s assumption is side-effect-free: Consumer should import Provider directly, and if Consumer doesn’t exist in application, Provider can be removed at all.

Apparently, all DI based code cannot be effectively DCE optimized.

In current Angular pattern, we will configure Provider in NgModule:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// lib.service.ts
@Injectable()
class LibService { }

// lib.module.ts
import { LibService } from './lib.service'

@NgModule({
  providers: [LibService],
})
class LibModule {}

// app.component.ts
import { LibService } from './lib.service'

@Component({ /*...*/ })
class AppComponent {
  constructor(libService: LibService) {}
}

// app.module.ts
import { LibModule } from './lib.module'
import { AppComponent } from './app.component'

@NgModule({
  declarations: [AppComponent],
  imports: [LibModule],
})
class AppModule {}

The dependence flow is: (translator note: in the below items parentheses after is added by translator for Angular newcomer)

• Library Module imports Library Service (for exposing interface and providing implementation)
• Application Component imports Library Service (for consuming interface)
• Application Module imports Application Component
• Application Module imports Library Module (for configuring which implementation to inject)

Even if service isn’t used in application component, service cannot be removed. This is because we introduce additional dependence during DI configuration. (translator note: consider app component doesn’t import libService. It is desirable libService is excluded from final build. But we cannot eliminate it because libModule imports it, and libModule is further imported by application module)

To solve this problem, Angular allows to invert the dependence of configuration. The new API is like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// lib.module.ts
@NgModule({ /*...*/ })
class LibModule {}

// lib.service.ts
import { LibModule } from './lib.module'

@Injectable({ scope: LibModule })
class LibService { }

// app.component.ts
import { LibService } from './lib.service'

@Component({ /*...*/ })
class AppComponent {
  constructor(libService: LibService) {}
}

// app.module.ts
import { LibModule } from './lib.module'
import { AppComponent } from './app.component'

@NgModule({
  declarations: [AppComponent],
  import: [LibModule],
})
class AppModule {}

The new dependence flow is:

• Library Service imports Library Module
• Application Component imports Library Service
• Application Module imports Application Component
• Application Module imports Library Module

Thus, as long as application component is removed, library service is not depended at all and can be safely removed together.

WAT? So why is libModule there? As a cosmetic?

We can optionally choose providers like useClass, useFactory to set implementation to other values instead of current class.
(translator note: they are Angular’s alternative DI functions and are DCE ready)

1
2
3
4
5
6
7

@Injectable({
  scope: SomeModule,
  useValue: { valueOfLife: 42 },
})
abstract class MyService {
  abstract valueOfLife: number
}

Of course, the new Injectable API only handles one ideal situation where dependency declared is dependency used. Nevertheless this is the most common situation. If intermediate Injector nodes overrides Provider, side effect is still ineluctable and thus adds code size. (But the injected dependency is probably used when one does override)

G. New template compilation

Template compilation, at macro level, has only two types:

• (structural) data
• (operational) instructions

Take a simple example. If we compile template (or equivalent) to some render method and if:

• calling render doesn’t make view changed, but its return value is what to be rendered. This is the former type.
• calling render does update view, and thus no return value is needed. This is the latter type.

The most early Angular, v2 version, compiles template to instructions. One can refer to this zhihu answer for compilation behavior. This style is very similar to Svelte: compile as much detail as possible, in order to reduce common runtime dependency (shared code).

But the problem of this approach is obvious. With more and more template authored, compiled code size will easily exceed shared code (Off-topic: the 0kb-boasting framework Svelte also provides a shared compile option to use library).

Later Angular v4 compiles template to data and introduces View Engine as common dependency. Its compilation is explained here. This style is very close to virtual dom, except that dom-like data is stored per type, not per instance.

Ivy renderer in v6 re-chooses compiling to instruction approach. Different from v2, v6 employs a strategy to minimize compiled code size.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

export class AppComponent  {
  static ngComponentDef = defineComponent({
    type: AppComponent,
    tag: 'my-app',
    template: function AppComponent_Template(comp: AppComponent, cm: boolean) {
      if (cm) {
        E(0, 'div', ['class', 'container'])
          E(1, 'div', ['class', 'jumbotron'])
            E(2, 'div', ['class', 'row'])
              E(3, 'div', ['class', 'col-md-6'])
                E(4, 'h1')
                  T(5, 'Angular v6.x.x (Ivy Renderer)')
                e()
              e()
              E(6, 'div', ['class', 'col-md-6'])
                E(7, 'div', ['class', 'col-sm-6 smallpad'])
                  E(8, 'button', ['type', 'button', 'id', 'run', 'class', 'btn btn-primary btn-block'])
                    L('click', () => {
                      comp.run()
                      detectChanges(comp)
                    })
                    T(9, 'Create 1,000 rows')
                  e()
                  //...
                e()
              e()
            e()
          e()
          E(20, 'table', ['class', 'table table-hover table-striped test-data'])
            E(21, 'tbody')
              C(22, [NgForOf], trTemplate)
            e()
          e()
          E(24, 'span', ['aria-hidden', 'true', 'class', 'preloadicon glyphicon glyphicon-remove'])
          e()
        e()
      }
      p(22, 'ngForOf', b(comp.data))
      p(22, 'ngForTrackBy', b(comp.itemById))
      cR(22)
        r(23, 0)
      cr()

      function trTemplate(row: NgForOfContext<Data>, cm: boolean) {
        if (cm) {
          E(0, 'tr')
            E(1, 'td', ['class', 'col-md-1'])
              T(2)
            e()
            E(3, 'td', ['class', 'col-md-4'])
              E(4, 'a', ['href', '#'])
                L('click', (e: MouseEvent) => {
                  comp.select(row.$implicit, e)
                  detectChanges(comp)
                })
                T(5)
              e()
            e()
            //...
          e()
        }
        p(0, 'class.danger', b(row.$implicit.id === comp.selected))
        t(2, b(row.$implicit.id))
        t(5, b(row.$implicit.label))
      }
    },
    factory: () => new AppComponent()
  })
}

(The density of information is almost as high as HTML.)

Compiling to instructions has another advantage over compiling to data: common dependency is still DCE friendly.
Used instructions will be directly depended, and non-used instructions will be removed by optimizer.
On the other hand compiling to data requires all operations used by template processor, and thus cannot be optimized statically.

Therefore, Ivy’s new compilation strategy should be the one with minimal code size (not regarding of the cost to implement compiler).

Memory and Speed

• New view layer employs compact binary frame design (and more bitwise operations).
• New view layer uses as many sequences (arrays) as possible rather than key-value pairs (objects) to store data.
• New view layer prefers adding expando properties on exposing types rather than new wrapping type.
• DI uses bloom filter to speed up Directive searching

(View layer is less friendly to pull request)

Summary

The new Ivy mode push “building-friendliness” to extreme, but you can also say it is not “non-building-friendly”. It won’t be useful for projects built with bare <script> tags.

Ivy’s most crucial target is cooperation with Angular Element. By encapsulating Angular components to custom elements (web components), we can achieve standalone publication, independent importing and independent usage of Angular as widgets. (To some degree this strangulates Svelte?)

The most important part of this strategy is code size. Even wihout common runtime library, Angular should work with minial size.
Components used as Angular Elements will not depended on external packages like forms and router, hence creating great value for size reduction.

Whether should NgModule still exist? It certainly has value on organizing code. That said, Dart version didn’t ever has NgModule. It is sure that application structure can be well formed without NgModule (at least for Googler).
With decreasing usage of NgModule in real world API, it might be optional in future. But I’m sure v6 won’t change NgModule. (I personally favor NgModule, but oppose enforcing it in app).

For applications plumbing many third party libraries, the main size problem might not come from Angular but rather from, for example, incorrect use of RxJS, importing moment.js inadvertently, or writing all styles in “comonent styles”. For specific size problem one needs to analyze it specifically. Don’t pray to Angular’s advanced optimizer.

The worst part of Ivy is OnChanges. It is now implemented by Object.defineProperty! It is no longer right to say Angular is based solely on dirty check. All articles about change detection need updating! And every section needs separate explaination!

Lifecycles might have big change in near future. But it will await Ivy being the default, no earlier than v7. In fact, Angular’s overall extensibility and runtime preprocessing has been greatly improved.

Vue 2.5 improves TypeScript definition! Before that, TS users will have to use class component API to get proper typing, but now canonical API is both precise and concise with few compromises!

For ordinary users, Vue’s official blog and updated documentation will guide you to upgrade or create projects.
But curious audience might wonder how the improvement is done and why TS support isn’t integrated in Vue2.0 at first place.

This blog post will deep dive into the technical details of Vue2.5 typing, which seems daunting at first glance. Don’t worry! We will show how TypeScript’s advanced types can be used in a popular framework.

Note: Reader’s familiarity with Vue and TypeScript is assumed in this post. If you are new to these two, checkout their official website!

TL;DR;

Vue2.5 exploits ThisType, mapped type, generic defaults and a clever trick to cover most APIs.

We will also list some limitations in current typing schema.

this is Vue

Let’s examine a basic Vue usage. We pass an object literal as component option to Vue constructor.

1
2
3
4
5
6
7
8

new Vue({
  methods: {
    greet() {
      this.$el // `this` is Vue
      console.log('Hello World!')
    }
  }
})

this keyword is bound to Vue instance in component option. Prior to Vue2.5, we declare this as a plain Vue type. Here is a simplified ComponentOption type.

1
2
3
4

interface ComponentOption {
  methods: { [key: string]: (this: Vue) => any }
  // other fields ...
}

However, we cannot access our custom methods/data via the declaration above since this is nothing but Vue. The typing doesn’t capture the fact that the VM injected into methods is instantiated with our custom methods/data/props.

A new type parameter V can allow users to specify their custom properties. So a better solution will be:

1
2
3

interface ComponentOption<V extends Vue> {
  methods: { [key: string]: (this: V) => void }
}

And users can use it like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

declare function newVue<V extends Vue>(option: ComponentOption<V>): V

interface MyComponent extends Vue {
  greet(str: string): void
  hello(): void
}
newVue<MyComponent>({
  methods: {
    greet(str) {
      console.log(str)
    },
    hello() {
      this.greet('hello world')
    }
  }
})

It works, but also requires one interface declaration and one explicit type annotation.
Can compiler be smarter and infer this for us?

ThisType<Vue>

We can strongly type this by a special marker interface ThisType. It is introduced in TypeScript 2.3, which is the very reason why we didn’t have strong type until Vue2.5.

The original pull request has a detailed introduction and example for ThisType.

The most important rule is quoted here.

(If) the containing object literal has a contextual type that includes a ThisType<T>, this has type T

What does this mean? Let’s break this rule down to several pieces.

object literal means the component option in Vue’s case; contextual type means the component option is passed to a function as argument and the component option is typed via function declaration, without caller’s annotation; and finally ThisType<T> needs to be used in the function parameter declaration. The type parameter T refers to the type of this in the component option. In simple terms, this rule says we can change this keyword’s type according to the component option passed to new Vue or so.

Combining these together, we can write a simple declaration that understands our Vue component option.

Note, you will need noImplicitThis compiler flag to enable this new type checking.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

interface ComponentOption<Method> {
  methods: Method
}

declare function newVue<Method>(
  option: ComponentOption<Method> & ThisType<Method & Vue>
): Vue&Method

// Method is inferred as
// { greet(str): void, hello(): void }
newVue({
  methods: {
    greet(str) {
      console.log(str)
    },
    hello() {
      // this is typed as Method & Vue
      this.greet('hello world')   // custom methods works!
      this.$el // vue property also works!
    }
  }
})

This code needs some explanation. First we define an ComponentOption and it takes a type parameter Method, which acts as a “stub” for compiler to infer custom properties on this.
Then in the function we declare a type parameter Method again and pass it to ComponentOption and ThisType.
Finally, ThisType<Method & Vue> means the type of this inside option will be an intersection of Vue and Method.

When we call newVue, compiler will first infer Method from ComponentOption object we pass to the function. Then the Method will flow into this keyword, resulting a type that has both Vue property and our own methods.

Mapping Computed

Typing methods alone is so far so good. However fields like computed have a different story.
The object in methods field has the same shape as part of this type. Say, methods has a hello function property and this also has a function property with the same name (in algebraic terms, endomorphism). But a property in computed is a function that returns a value and this has a namesake property with the same value type. For example.

1
2
3
4
5
6
7
8
9
10
11

newVue({
  computed: {
    myname: () => 'world' // a function returns string
  },
  methods: {
    greet() {
      console.log(this.myname)
      // myname is a string, not a function returning string
    }
  }
})

How can we get a new type from computed definition object?

Here comes the mapped type, a new kind of object type that maps a type representing property names over a property declaration template. In other words, we can create computed type in Vue instance based on that in component option. (algebraically, homomorphism)

In a mapped type, the new type transforms each property in the old type in the same way.

The official documentation is crystal clear. Let’s see how we integrate this awesomeness into Vue.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// we map a plain type to a type of which the property is a function
// e.g. { myname: string } will be mapped to { myname: () => string }
// note this process can also be reversed during type inference
type Accessors<T> = { [K in keyof T]: () => T[K] }

interface ComponentOption<Method, Computed> {
  methods: Method
  computed: Accessors<Computed>
}

type ThisTypedOption<Method, Computed> =
  ComponentOption<Method, Computed> & ThisType<Method & Computed & Vue>

declare function newVue<Method, Computed>(
  option: ThisTypedOption<Method, Computed>
): Method & Computed & Vue

Accessors<T> will map the type T to a new type with same property names. But property value type is a function returning the type in the original T. This process is reversed during type inference. When we pass computed field as {myname: () => string} to newVue function, compiler will try to map the type to Accessors<T>, which results in Computed being {myname: string}.

And Computed is mixed into this, so we can access myname as string from this.

We skipped here computed setter style declaration for a more lucid demonstration. Supporting setter in computed is similar.

Prop Types Trick

props has a subtle difference from computed: we define a prop by giving a constructor of that value type.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

type PropDef<T> = { new(...args: any[]): T }
type Props<T> = { [K in keyof T]: PropDef<T[K]> }
interface ComponentOption<Prop> {
  props: Props<Prop>
}
type ThisTypedOption<Prop> =
  ComponentOption<Prop> & ThisType<Prop & Vue>

declare function newVue<Prop>(option: ThisTypedOption<Prop>): Prop & Vue

class User {}
newVue({
  props: {
    user: User,
    name: String
  }
})

One would naturally expect newVue will infer Prop as { user: User, name: string }. Sadly, it is not.

The problem lies in PropDef, which uses constructor type new(): T. Custom constructor is fine. For example User‘s constructor returns User. But primitive value’s constructor doesn’t work because String has the signature new(): String.

Alas! The return value is String, rather than string. Their difference is listed in the first rule of TypeScript Do’s and Don’ts. A string type is what we use and String refers to non-primitive boxed objects that are almost never used.

We can use another signature to type primitive constructor and union custom constructor together. Note every primitive constructor has a call signature, that is, String(value) will return a primitive string value rather than a wrapper object.

1

type PropDef<T> = { (): T } | { new(...args: any[]): T }

It should work, shouldn’t it? Sadly again, NO.

1
2

declare function propTest<T>(t: PropDef<T>): T
propTest(String) // return String, not string!

Because String satisfy both call and constructor signature in PropDef, compiler will prefer returning String.

How can we nudge compiler to prefer primitive type? Here is an undocumented trick.
The main idea is to exploit type inference priority. If a type parameter is single naked, that is, not in intersection type nor in union type, compiler will prefer to infer from that single naked position over intersection/union position. So we can add an intersection to constructor signature and then compiler will first infer call signature. Exactly what we want! To make the signature more self explanatory, we can use the object type to flag constructor type should not return primitive type.

1
2
3

type PropDef<T> = { (): T } | { new(...args: any[]): T & object }
declare function propTest<T>(t: PropDef<T>): T
propTest(String) // return string, yay!

Now we can happily infer props without manual annotation!

Compatibility

For better inference, our new type has many more type parameters than original ComponentOption<V> which only has one parameter. Nevertheless, it will be a catastrophic breaking change if we ship the new type without proper fallback. Generic defaults introduced in TS2.3 gives us a chance to bring about a more smooth upgrade.

1
2
3
4
5
6
7
8
9
10
11
12
13

interface ComponentOption<V extends Vue, Method=any, Data=any, Prop=any, Computed=any> {
  // ....
}

interface MyVue extends Vue {
 // ...
}

// users can use ComponentOption without changing their code
// parameter with default can be skipped
var option: ComponentOption<MyVue> = {
  // ...
}

Happy ending!

Limitation

The “No silver bullet” rule also applies to typing. The more advanced types we use, the more complex error messages will be generated. Hope this blog post will help you to understand the new typing better and help you to debug your own application.

There are also some type system limitations in Vue typing. Let’s see some examples.

• functions in computed need return type annotation

Return type annotation is required if computed method uses this. It turns out that using mapped type and ThisType at the same time without explicit annotation will cause cyclic inference error in current compiler.

1
2
3
4
5
6
7
8
9
10

new Vue({
  computed: {
    foo() {
      return 123
    },
    bar(): number { // required
      return this.foo
    }
  }
})

TypeScript has already opened an issue tracking this.

• Prop types’ union declaration requires manual type cast

Vue accepts an array of constructors in prop’s definition as union type. However, PropDef cannot unify primitive constructors and custom constructors which have two heterogeneous signatures.

1
2
3
4
5
6
7

Vue.component('union-prop', {
  props: {
    primitive: [String, Number], // both primitive, ok
    custom: [Cat, User],         // both custom, ok
    mixed: [User, Number] as {new(): User | Number}[] // requires annotation
  }
})

In general, you should avoid mixing primitive type and object type.

Final words

TypeScript has been constantly evolving since its birth. And finally its expressiveness enable us to type Vue’s cannonical API!

Thank you, TypeScript team, for bring us these awesome features!
Thank you, Vue team, for embracing new advance in type system!

TL;DR:

1. Compiler does not understand control flow in closure / callback function.
2. In flow based type analysis, copmiler will be either optimistic or pessimistic. TypeScript is optimistic.
3. You can usually workaround (3) by using const or readonly

This is a long due introduction for TypeScript’s flow sensitive typing (also known as control flow based type analysis) since its 2.0 release. It is so unfortunate that no official documentation is in TypeScript’s website for it (while both flow and kotlin have!). But if you dig the issue list earnestly enough, you will always find some hidden gems there!

To put it short, a variable’s type in a flow sensitive type system can change according to the control flow like if or while. For example, you can dynamically check the truthiness of a nullable variable and if it isn’t null, compiler will automatically cast the variable type to non-null. Sweet?

What can be bitter here? TypeScript is an imperative language like JavaScript. The side-effectful nature prevents compiler from inferring control flow when function call kicks in. Let’s see an example.

1
2
3
4
5
6
7
8
9
10

let a: number | null = 42

makeSideEffect()

a // is a still a number?

function makeSideEffect() {
  // omitted...
}

Without knowing what makeSideEffect is, we cannot guarantee variable a is still number. Side effect can be as innocuous and innocent as console.log('the number of life', 42), or as evil as a billion dollar mistake like a = null, or even a control-flow entangler: throw new Error('code unreachale').

One might ask compiler to infer what makeSideEffect does since we can provide the source of the function.
However this is not practically feasible because of ambient function and (possibly polymorphic) recursion. Compiler will be trapped in infinite loops if we instruct it to infer arbitrary deep functions, as halting problem per se.

So a realistic compiler must guess what a function does by a consistent strategy. Naturally we have two alternatives:

1. Assume every function does not have relevant side effect: e.g. assignment like a = null. We call this optimistic.
2. Assume every function does have side effect. We call this strategy pessimistic.

Spoiler: TypeScript uses optimistic strategy.

We will walk through these two strategies and see how they work in practice.
But before that let’s see some common gotchas in flow sensitive typing.

Closure / Callback

Flow sensitive typing does not play well with callback functions or closures. This is explicitly mentioned in Kotlin’s document.

var local variables - if the variable is not modified between the check and the usage and is not captured in a lambda that modifies it;

Consider the following example.

1
2
3
4
5

var a: string | number = 42 // smart cast to number
setTimeout(() => {
  console.log(typeof a) // what should be print?
}, 100)
a = 'string'

As a developer, you can easily figure out that string will be output to console because setTimeout will call its function argument asynchronously, after assigning string to a. Unfortunately, this knowledge is not accessible to compiler. No keyword will tell compiler whether callback function will be called immediately, nor static analysis will tell the behavior of a function: setTimeout and forEach is the same in the view of compiler.

So the following example will not compile.

1
2
3
4

var a: string | number = 42 // smart cast to number
someArray.forEach(() => {
  a.toFixed() // error, string | number does not have method `toFixed`
})

Note: compiler will still inline control flow analysis for IIFE(Immediately Invoked Function Expression).

1
2
3
4
5
6

let x: string | number = "OK";
(() => {
  x = 10;
})();
if (x === 10) { // OK, assignment in IIFE
}

In the future, we might have a keyword like immediate to help compiler reasoning more about control flow. But that’s a different story.

Now let’s review the strategies for function call.

Optimistic Flow Sensitive Typing

Optimistic flow typing assume a function without side-effect that changes a variable’s type. TypeScript chooses this strategy in its implementation.

1
2
3
4

var a: number | null
a = 42 // assign, now a is narrowed to type `number`
sideEffect() // assume nothing happens here
a.toFixed() // a still has `number` type

This assumption usually works well if code observes immutable rule. On the other hand, a stateful program will be tolled with the tax of explicit casting. One typical example is scanner in a compiler. (Both Angular template compiler and TypeScript itself are victims).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// suppose we are tokenizing an HTML like language
enum Token { LeftBracket, WhiteSpace, Letter ... }
let token = Token.WhiteSpace;
function nextToken() {
    token = readInput(); // return a Token
}

function scan() {
  // token here is WhiteSpace
  while (token === Token.WhiteSpace) {
    // skip white space, a common scenario
    nextToken()
  }
  if (token === Token.LeftBracket) { // error here
    // compiler thinks token is still WhiteSpace, optimistically but wrongly
  }
}

Such bad behavior also occurs on fields.

1
2
3
4
5
6
7
8
9
10
11
12
13

// A function takes a string and try to parse
// if success, modify the result parameter to pass result to caller
declare function tryParse(x: string, result: { success: boolean; value: number; }): void;

function myFunc(x: string) {
    let result = { success: false, value: 0 };

    trySomething(x, result);
    if (result.success === true) { // error!
        return result.value;
    }
    return -1;
}

An alternative here is returning a new result object so we need no inline mutation. But in some performance sensitive code path might we want parse a string without new object allocation, which reduces garbage collection pressure. After all, mutation is legal in JavaScript code, but TypeScript fails to capture it.

Optimistic flow analysis sometimes is also unsound: a compiler verified program will cause runtime error. We can easily construct a function which reassigns a variable to an object of different type and uses it as of the original type, and thus a runtime error!

1
2
3
4
5
6
7
8
9
10

class A { a: string}
class B { b: string}
let ab: A | B = new A

doEvil()
ab.b.toString() // booooooom

function doEvil() {
  ab = new B
}

The above examples might leave to you a impression that compiler does much bad when doing optimistic control flow inference. In practice, however, a well architected program with disciplined control of side effect will not suffer much from compiler’s naive optimistism. Presumption of immutability innocence will save you a lot type casting or variable rebinding found in pessimistic flow sensitive typing.

Pessimistic Flow Sensitive Typing

A pessimistic flow analysis places burden of typing proof on programmers.
Every function call will invalidate previous control flow based narrowing. (Pessimistic possibly has a negative connotation, conservative may be a better word here). Thus programmers have to re-prove variable types is matching with previous control flow analysis.

Examples in this section are crafted to be runnable under both TS and flow-type checker.
Note, only flow-type checker will produce error because flow is more pessimistic/strict than TypeScript.

1
2
3
4
5
6
7
8
9
10

declare function log(obj: any): void

let a: number | string = 42

log(a) // invalidation!

a.toFixed() // error, a's type is reverted to `number | string`

// To work around it, you have to recheck the type of `a`
typeof a === 'number' && a.toFixed() // works

Alas, pessimistism also breaks fields. Example taken from stackoverflow.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

declare function assert(obj: any): void

class TreeNode<V, E> {
    value: V
    children: Map<E, TreeNode<V,E>> | null

    constructor(value: V) {
        this.value = value
        this.children = null
    }
}

function accessChildren(tree: TreeNode<number, string>): void {
    if (tree.children != null) {
        assert(true) // negate type narrowing
        tree.children.forEach((v,k) => {}) // error!
    }

}

These false alarms root in the same problem as in optimistic strategy: compiler/checker has no knowledge about a function’s side effect. To work with a pessimistic compiler, one has to assert/check repeatedly so to guarantee no runtime error will occur. Indeed, this is a trade-off between runtime safety and code bloat.

Workaround

Sadly, no known panacea for flow sensitive typing. We can mitigate the problem by introducing more immutability.

using const

Because a const identifier will never change its type.

1
2
3
4
5
6

const a: string | number = someAPICall() // smart cast to number
if (typeof a === 'string') {
  setTimeout(() => {
    a.substr(0) // success, `const` identifier will not lose its narrrowed type
  }, 100)
}

And using const will provide you runtime safety or bypass pessimistic checker.

1
2
3
4
5
6
7
8
9
10

function fn(x: string | null) {
  const y = x
  function assert() {
    // ... whatever
  }

  if (y !== null) {
    console.log(y.substr(0)); // no error, no crash
  }
}

The same should apply to readonly, but current TypeScript does not seem to support it.

Conclusion

Flow sensitive typing is an advanced type system feature. Working with control flow analysis smoothly requires programmers to control mutation effectively.

Keeping mutation control in your mind. Flow sensitive typing will not in your way but make a pathway to a safer code base!

With the growing population of SOA (StackOverflow Oriented Architecture), keeping your code copy-paste friendly is becoming more and more important in demonstrating, developing and question answering.

You want your answer to be available instantly, free of fussy debugging. So copy-paste friendliness is a key property of your code robust enough to be ubiquitously runnable and, eventually, help you gain more reputation.

Adopting copy-paste style is amazing. Copy-paste boosts your productivity by freeing you from rumination on architecture complexity. Your boss will be happy with all the SLOC you commit. Your colleagues will respect your absolute ownership of the magnificent, enigmatic, labyrinthine code artifact built by the blob gleaned from everywhere.

1. Always prefer “==” to strict equality.

Implicit conversion grants you additional robustness. Your code will never complain about ill input. Copy-paste without worry. Yeah

1
2
3

var x = '10'
if (x == 10) x += 5
y = x / 5 // wow, it works!

And you can enjoy this artistic equality table when debugging. Cool.

2. Always prefer positive condition.

Never return early on invalid case. You can copy-paste it any where in your control flow. Yeah.

If return is added, you have to remove the unnecessary control flow abrupting keyword when you need to copy paste these code into a deeply nested code block, which is quite common in CPS(copy-paste-style) programming.

1
2
3
4
5
6

function someFunction(someCondition) {
  if (someCondition) {
    // Do whatever you want
  }
}

3. Always prefer tautological check.

Repetition here is for genericity. Every conditional path can be copy-pasted free of reading. They are very safe compile time constructs that preclude the very possibility of runtime undefined panic.

else on individual line is a bonus. You can copy-and paste code without looking at else keyword! What a profit!

1
2
3
4
5
6
7
8
9
10
11
12

if (a && a.b && a.b.c == 1) {
  // never worried about a.b.c is not undefined
}
else
if (a && a.b && a.b.c == 2) {
  // repition is one aesthetic rule
}
else
if (a && a.b && a.b.c == 3) {
  // very safe even programmer inadvertedly paste it to elsewhere

}

4. Always prefer fully qualified name

Never cache common subexpression. Declaring new variable name will force you read code and find declaration before copy-pasting.

1
2
3

topVariable.someProperty.nestedProp.anotherProp = 123
console.log(topVariable.someProperty.nestedProp.anotherProp)
topVariable.someProperty.nestedProp.anotherProp += 1

5. Always prefer inlining statements in function

Never factor out functions. Helpers require nonlocal function name lookup when copy-pasting. Dev experience terminator.

Conclusion

Oh, GG. I forgot GitHub Gists!

Vue has a lot of TypeScript binding libraries. A lot of.

They look similar, with subtle difference. How to choose one?

Here I compare three libraries.

• vue-typescript-component
• vuets
• av-ts

They are representative, and the analysis probably also applies to the official library.

Disclaimer: I’m the author of av-ts. So this article is probably biased. But I tried my best to give an objective review.

0. Overview

Vue-typescript-component and av-ts are decorator based libraries. Vue-typescript-component is more like the popular typescript library, but supports Vue2.0. av-ts is my own library, crated after I read many alternatives.
vuets is quite different. It resembles the original object literal style API from vue.js.
Before I start the review, I want to state a rule of thumb before.