Optimize ast-grep to get 10X faster

In this post I will discuss how to optimize the Rust CLI tool ast-grep to become 10 times faster.

Rust itself usually runs fast enough, but it is not a silver bullet to all performance issues.

In this case, I did not pay enough attention to runtime details or opted for naive implementation for a quick prototype. And these inadvertent mistakes and deliberate slacking off became ast-grep’s bottleneck.

Context

ast-grep is my hobby project to help you search and rewrite code using abstract syntax tree.

Conceptually, ast-grep takes a piece of pattern code (think it like a regular expression but for AST), matches the pattern against your codebase and gives a list of matched AST nodes back to you. See the playground for a live demo.

I designed ast-grpe’s architecture with performance in mind. Here are a few performance related highlights:

  • it is written in Rust, a native language compiled to machine code.
  • it uses the venerable C library tree-sitter to parse code, which is the same library powering GitHub’s codesearch.
  • its command line interface is built upon ignore, the same crates used by the blazing fast ripgrep.

Okay, enough self-promotion _BS_. If it is designed to be fast, how comes this blog? Let’s dive into the performance bottleneck I found in my bad code.

Spoiler. It’s my bad to write slow Rust.

Profiling

The first thing to optimize a program is to profile it. I am lazy this time and just uses the flamegraph tool.

Installing it is simple.

1
cargo install flamegraph

Then run it against ast-grep! No other setup is needed, compared to other profiling tools!

This time I’m using an ast-grep port of es-lint against TypeScript‘s src folder.

This is the profiling command I used.

1
sudo flamegraph -- sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null

The flamegraph looks like this.

Before Optimzation

Optimizing the program is a matter of finding the hotspots in the flamegraph and fix them.

For a more intuitive feeling about performance, I used the old command time to measure the wall time to run the command. The result is not good.

1
2
time sg scan -c eslint/sgconfig.yml TypeScript/src
17.63s user, 0.46s system, 167% cpu, 10.823 total

The time before user is the actual CPU time spent on my program. The time before total represents the wall time. The ratio between them is the CPU utilization. In this case, it is 167%. It means my program is not fully utilizing the CPU.

It only runs six rules against the codebase and it costs about 10 whole seconds!

In contrast, running one ast-grep pattern agasint the TypeScript source only costs 0.5 second and the CPU utilization is decent.

1
2
3
time sg run -p '$A && $A()' TypeScript/src --json > /dev/null

1.96s user, 0.11s system, 329% cpu, 0.628 total

Expensive Regex Cloning

The first thing I noticed is that the regex::Regex type is cloned a lot. I do know it is expensive to compile a regex, but I did not expect cloning one will be the bottleneck.
Much to my limited understanding, dropping Regex is also expensive!

Fortunately the fix is simple: I can use a reference to the regex instead of cloning it.

This optimzation alone shaves about 50% of execution time.

1
2
time sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
13.89s user, 0.74s system, 274% cpu 5.320 total

The new flamegraph looks like this.

Avoid Regex Cloning

Matching Rule can be Avoided

The second thing I noticed is that the match_node function is called a lot. It is the function that matches a pattern against an AST node.
ast-grep can match an AST node by rules, and those rules can be composed together into more complex rules.
For example, the rule any: [rule1, rule2] is a composite rule that consists of two sub-rules and the composite rule matches a node when either one of the sub-rules matches the node.
This can be expensive since multiple rules must be tried for every node to see if they actually make a match.

I have already forsee it so every rule in ast-grep has an optimzation called potential_kinds. AST node in tree-sitter has its own type encoded in a unsigned number called kind.
If a rule can only match nodes with specific kinds, then we can avoid calling match_node for nodes if its kind is not in the potential_kinds set.
I used a BitSet to encode the set of potential kinds. Naturally the potential_kinds of composite rules can be constructed by merging the potential_kinds of its sub-rules, according to their logic nature.
For example, any‘s potential_kinds is the union of its sub-rules’ potential_kinds, and all‘s potential_kinds is the intersection of its sub-rules’ potential_kinds.

Using this optimization, I can avoid calling match_node for nodes that can never match a rule. This optimization shaves another 40% of execution time!

1
2
sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
11.57s user, 0.48s system, 330% cpu, 3.644 total

The new flamegraph.

potential_kinds trick

Duplicate Tree Traversal

Finally, the function call ts_tree_cursor_child_iterator_next caught my eyes. It meant that a lot of time was spent on traversing the AST tree.

Well, I dumbly iterating through all the six rules and matching the whole AST tree for each rule. This is a lot of duplicated work!

So I used a data structure to combine these rules, according to their potential_kinds. When I’m traversing the AST tree, I will first retrieve the rules with potential_kinds containing the kind of the current node. Then I will only run these rules against the node. And nodes without any potential_kinds hit will be naturally skipped during the traversal.

This is a huge optimization! The ending result is less than 1 second! And the CPU utilization is pretty good.

1
2
sg scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null
2.82s user, 0.12s system, 301% cpu, 0.975 total

Conclusion

The final flamegraph looks like this. I’m too lazy to optimize more. I’m happy with the sub-second result for now.

Merging rules

Optimizing ast-grep is a fun journey. I learned a lot about Rust and performance tuning. I hope you enjoyed this post as well.

A quick intro to Angular Ivy

Angular recently announced a new render engine called Ivy.
Ivy is an amazing present from Angular team! It produces hello-world app in mere 3.2KB, on a par with minimal framework like preact. Unfortunately little documentation, if any, exists to explain how Ivy works.

This blog post strives to fill this gap, by translating an amazing answer from zhihu, a superior Q&A website to Quora. I hope this post can communicate Angular’s innovation to broader front-end world by breaking language barrier.

TL;DR: Ivy is an extremely building-tool friendly, web-first render engine that paves Angular’s way to web components.

Opinion is not mine. Words in parentheses are from the original author unless explicitly annotated as from translator. Translator notes are added for audience with no Angular background.

Orignal post:

How to comment on the fact that Angular’s new Ivy render engine produces 3.2KB compiled JavaScript? (traslator note: how to comment is a jargon on zhihu, meaning what's your opinion)

Let’s first answer several frequent questions:

  1. 3.2KB is the result after minification + gzip (a convention to compare framework size), which is the exact payload size we send to browser. (if we use more advanced compression like brotli, the size will be smaller on modern browsers).

  2. Ivy renderer won’t be default in Angular v6. You need to manually enable it by switching on certain compiler option. It might be default in v7 if no further issues are found.

  3. Code completion isn’t proportional to feature completion. 50% code completed doesn’t imply 50% feature completed. For now Ivy has seemingly decent code completion rate, but it still has a long way before ready for production. It can’t even complete a simple app for benchmark.

  4. Compared with previous Angular’s compiled code, Ivy’s result is almost like hand-written. And actually you can write it by hand! (Though in practice you won’t.)

  5. NgModule is challenged, again.

  6. Ivy won’t promise you a plunge in code weight. Your bundle will still be bulky if a lot of CSS is inlined in your components.


Since the question is to comment on 3.2KB compiled file, I will focus on payload size optimization, thus, answering the question “What optimization has Ivy Renderer done?”.

It can never be overstated that the extreme size isn’t achieved by the ADVANCED mode of Closure Compiler. Rollup can achieve the same level optimization (within less than 1KB difference). Therefore Ivy is arguably the true building-tool friendly renderer, not Closure-Compiler-only renderer.

Disclaimer: all the below is based on current (Feb, 2018) implementation, published roadmap. If following version changes leads to the following content outdated or incorrect, please pardon our no updating.

A. Removing dependence on platform-browser

As a platform independent framework, can we run application without platform specific code? The answer is NO, of course. Ivy just inlines DOM Rendere to its core. If you only need run your application on browser (taking no account of WebWorker), you can run Angular without including platform specific code. For example, the code for binding text is:

1
2
3
4
value !== NO_CHANGE &&
((renderer as ProceduralRenderer3).setValue ?
(renderer as ProceduralRenderer3).setValue(existingNode.native, stringify(value)) :
existingNode.native.textContent = stringify(value));

Fallback code is obvious: if no renderer exists (translator note: the condition is better explained as renderer isn’t a ProceduralRenderer3), Ivy will directly modify dom element’s textContent.

Currently no Ivy code relies on platform, thus common problems arise:

  • No support for Event Plugin. e.g. syntactic sugar for keyboard event (like keydown.enter) and events from Hammer, which are all provided by platform-browser.

  • No Sanitizer. Though Angular isn’t string template by itself and is naturally immune to XSS, it still cannot survive abhorrent abuse of innerHTML. Sanitizer, again currently bestowed by platform-browser, comes to rescue by filtering user content. Without platform hardly can we achieve the same level security.

  • No support for Component Style; No View Encapsulation. They are all implemented by different Renderers in platform-browser package. Only inline style is available for component styling (Another better way is independent CSS and dynamic class).

So the current Ivy renderer demoed in hello-world app isn’t a full featured Angular for some users.

B. No NgFactory file anymore

Under the new Ivy mode, compiled template will be stored in the static fields in class, instead of generating new wrapper class (NgFactory). For example:

Component -> ngComponentDef

1
2
3
4
5
6
7
8
@Component({ /*...*/ })
class MyComponent { }

// ->

class MyComponent {
static ngComponentDef = defineComponent({ /*...*/ })
}

Directive -> ngDirectiveDef

1
2
3
4
5
6
7
8
@Directive({ /*...*/ })
class MyDirective { }

// ->

class MyDirective {
static ngDirectiveDef = defineDirective({ /*...*/ })
}

NgModule -> ngInjectorDef

1
2
3
4
5
6
7
8
@NgModule({ /*...*/ })
class MyModule { }

// ->

class MyModule {
static ngInjectorDef = defineInjector({ /*...*/ })
}

WAT? How can Your Majesty NgModule compiled to one single Injector?(

Due to the colocation of compiled template and class, the new compilation can fully enjoy building tool’s normal optimization. (Tree-Shaking for the most part), waiving most special process in build-optimizer.

In fact, this improvment is more significant for building than for size optimization. The new style compiler enables single file compilation, one ts file to one js file. We no longer need to worry about mapping from source file to ngfactory.

Further more, this unifies library consumption in JIT and AOT. Once the compilation is stabilized, all libraries can compile code before publication, rather than at end uses’ bundling phase. This can lead to faster building.

C. Greatly simplify bootstrap code

All Angular applications need to configure bootstrap component (actually it isn’t mandatory, but alternatives are more complex), and initialize one NgModule[Factory]. Something like:

1
2
3
4
5
6
7
8
9
10
import { NgModule } from '@angular/core'
import { platformBrowser } from '@angular/platform-browser'
import { AppComponent } from './app.component'

@NgModule({
bootstrap: [AppComponent]
})
class AppModule { }

platformBrowserDynamic().bootstrapModule(AppModule)

New bootstrapping code is based on component:

1
2
3
4
5
// not available yet
import { renderComponent } from '@angular/core'
import { AppComponent } from './app.component'

renderComponent(AppComponent)

Hey, why we still need AppModule?

Ivy is based on NgModule-less bootstrapping for now. Though not completed yet, compilers can make bootstrapping accept NgModule (if it still exists) to provide Injector. In other word, renderComponent can accept an optional configuration argument.

By the way, this is also friendlier to bootstrapping multiple Angular instances in one page.

D. Redesigned DevMode configuration and checking

In the previous Angular, DevMode is disabled by function call dynamically.

1
2
3
4
5
import { enableProdMode } from '@angular/core'

enableProdMode()

platformFoo().bootstrapBar(baz)

In other word, whether DevMode is on is totally Angular’s internal state. Thus all debug related code is dynamically used and cannot be excluded in compile time. (Even closure compiler is ineffective).

But in Ivy DevMode is purely compile time configuration, all debugging code will be executed after checking the global variable ngDevMode. For example:

1
ngDevMode && assertEqual((state as LView).node, null, 'lView.node');

So building tool can replace the corresponding variable (e.g Webpack’s DefinePlugin), and corresponding debug code can be detected as dead code, and then optimizer (e.g UglifyJS) can remove it.

E. Feature named as Feature

Ivy mode has added a new feature called Feature. Or, Ivy introduces a new concept called Feature. Simply put, Feature is a pre-processor for DirectiveDef, which can be thought as a “Decorator pattern” tailored for DirectiveDef. (Translator note: More simply, you can provide a custom function to Angular, and Angular will apply it against ComponentDef metadata. So you can extend ComponentDef‘s Feature in your own function. Source).

Let’s take an example in Ivy. OnChanges isn’t implemented by Renderer but a predefined Feature. The Feature uses defineProperty to listen on the property decorated by @Input, and automatically stores previousValue in the instance to generate SimpleChanges for change detection, and finally triggers OnChanges by intercepting OnInit and DoCheck without Angular core’s help.

This means OnChanges‘ code is also tree-shakable! If no component ever declares the lifecycle ngOnChanges, no DirectiveDef will import NgOnChangesFeature. So optimizer can reasonably eliminate dead code.

WAT? How about the dirty check promise? From now on we have exception in Angular’s change detection?!

So it is notable that OnChanges in Ivy is not a real lifecycle any more (Translator note: it is not listed in the ComponentDef).
Put differently, users can extend lifecycle themselves, do as they please.

But the direct aim of Feature concept is for the incoming LifeCycle as Observables. Users don’t need to declare methods (ngOnInt) in class, but rather use the lifecycle observables directly. Indeed. by intercepting DirectiveDef, users and third-party library authors can modify lifecycle hooks in runtime. We can even use decorators to declare lifecycles.

In summary, Feature can be thought as a variant of Higher Order Component (compared with class factory): it modifies component type itself based on runtime requirement, instead of changing component’s internal state according to component logic.

F. New Injectable API

Actually this isn’t related to the hello-world demo, nor related to even Ivy (since it can be used in non Ivy mode). However, the new Injectable API has a huge impact on Angular’s size.

Besides the XXXDef properties listed above, we have another one called ngInjectableDef:

1
2
3
4
5
6
7
8
@Injectable()
class MyService { }

// ->

class MyService {
static ngInjectableDef = defineInjectable({ /*...*/ })
}

In classical sense (if anyone ever cared), we can easily find dependency injection and dead code elimination are contradictory in priciple.

  • DI’s nature is side effect: Provider changes execution context via configuration, and Consumer retrieves content from context.

  • DCE’s assumption is side-effect-free: Consumer should import Provider directly, and if Consumer doesn’t exist in application, Provider can be removed at all.

Apparently, all DI based code cannot be effectively DCE optimized.

In current Angular pattern, we will configure Provider in NgModule:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// lib.service.ts
@Injectable()
class LibService { }

// lib.module.ts
import { LibService } from './lib.service'

@NgModule({
providers: [LibService],
})
class LibModule {}

// app.component.ts
import { LibService } from './lib.service'

@Component({ /*...*/ })
class AppComponent {
constructor(libService: LibService) {}
}

// app.module.ts
import { LibModule } from './lib.module'
import { AppComponent } from './app.component'

@NgModule({
declarations: [AppComponent],
imports: [LibModule],
})
class AppModule {}

The dependence flow is: (translator note: in the below items parentheses after is added by translator for Angular newcomer)

  • Library Module imports Library Service (for exposing interface and providing implementation)
  • Application Component imports Library Service (for consuming interface)
  • Application Module imports Application Component
  • Application Module imports Library Module (for configuring which implementation to inject)

Even if service isn’t used in application component, service cannot be removed. This is because we introduce additional dependence during DI configuration. (translator note: consider app component doesn’t import libService. It is desirable libService is excluded from final build. But we cannot eliminate it because libModule imports it, and libModule is further imported by application module)

To solve this problem, Angular allows to invert the dependence of configuration. The new API is like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
// lib.module.ts
@NgModule({ /*...*/ })
class LibModule {}

// lib.service.ts
import { LibModule } from './lib.module'

@Injectable({ scope: LibModule })
class LibService { }

// app.component.ts
import { LibService } from './lib.service'

@Component({ /*...*/ })
class AppComponent {
constructor(libService: LibService) {}
}

// app.module.ts
import { LibModule } from './lib.module'
import { AppComponent } from './app.component'

@NgModule({
declarations: [AppComponent],
import: [LibModule],
})
class AppModule {}

The new dependence flow is:

  • Library Service imports Library Module
  • Application Component imports Library Service
  • Application Module imports Application Component
  • Application Module imports Library Module

Thus, as long as application component is removed, library service is not depended at all and can be safely removed together.

WAT? So why is libModule there? As a cosmetic?

We can optionally choose providers like useClass, useFactory to set implementation to other values instead of current class.
(translator note: they are Angular’s alternative DI functions and are DCE ready)

1
2
3
4
5
6
7
@Injectable({
scope: SomeModule,
useValue: { valueOfLife: 42 },
})
abstract class MyService {
abstract valueOfLife: number
}

Of course, the new Injectable API only handles one ideal situation where dependency declared is dependency used. Nevertheless this is the most common situation. If intermediate Injector nodes overrides Provider, side effect is still ineluctable and thus adds code size. (But the injected dependency is probably used when one does override)

G. New template compilation

Template compilation, at macro level, has only two types:

  • (structural) data
  • (operational) instructions

Take a simple example. If we compile template (or equivalent) to some render method and if:

  • calling render doesn’t make view changed, but its return value is what to be rendered. This is the former type.
  • calling render does update view, and thus no return value is needed. This is the latter type.

The most early Angular, v2 version, compiles template to instructions. One can refer to this zhihu answer for compilation behavior. This style is very similar to Svelte: compile as much detail as possible, in order to reduce common runtime dependency (shared code).

But the problem of this approach is obvious. With more and more template authored, compiled code size will easily exceed shared code (Off-topic: the 0kb-boasting framework Svelte also provides a shared compile option to use library).

Later Angular v4 compiles template to data and introduces View Engine as common dependency. Its compilation is explained here. This style is very close to virtual dom, except that dom-like data is stored per type, not per instance.

Ivy renderer in v6 re-chooses compiling to instruction approach. Different from v2, v6 employs a strategy to minimize compiled code size.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
export class AppComponent  {
static ngComponentDef = defineComponent({
type: AppComponent,
tag: 'my-app',
template: function AppComponent_Template(comp: AppComponent, cm: boolean) {
if (cm) {
E(0, 'div', ['class', 'container'])
E(1, 'div', ['class', 'jumbotron'])
E(2, 'div', ['class', 'row'])
E(3, 'div', ['class', 'col-md-6'])
E(4, 'h1')
T(5, 'Angular v6.x.x (Ivy Renderer)')
e()
e()
E(6, 'div', ['class', 'col-md-6'])
E(7, 'div', ['class', 'col-sm-6 smallpad'])
E(8, 'button', ['type', 'button', 'id', 'run', 'class', 'btn btn-primary btn-block'])
L('click', () => {
comp.run()
detectChanges(comp)
})
T(9, 'Create 1,000 rows')
e()
//...
e()
e()
e()
e()
E(20, 'table', ['class', 'table table-hover table-striped test-data'])
E(21, 'tbody')
C(22, [NgForOf], trTemplate)
e()
e()
E(24, 'span', ['aria-hidden', 'true', 'class', 'preloadicon glyphicon glyphicon-remove'])
e()
e()
}
p(22, 'ngForOf', b(comp.data))
p(22, 'ngForTrackBy', b(comp.itemById))
cR(22)
r(23, 0)
cr()

function trTemplate(row: NgForOfContext<Data>, cm: boolean) {
if (cm) {
E(0, 'tr')
E(1, 'td', ['class', 'col-md-1'])
T(2)
e()
E(3, 'td', ['class', 'col-md-4'])
E(4, 'a', ['href', '#'])
L('click', (e: MouseEvent) => {
comp.select(row.$implicit, e)
detectChanges(comp)
})
T(5)
e()
e()
//...
e()
}
p(0, 'class.danger', b(row.$implicit.id === comp.selected))
t(2, b(row.$implicit.id))
t(5, b(row.$implicit.label))
}
},
factory: () => new AppComponent()
})
}

(The density of information is almost as high as HTML.)

Compiling to instructions has another advantage over compiling to data: common dependency is still DCE friendly.
Used instructions will be directly depended, and non-used instructions will be removed by optimizer.
On the other hand compiling to data requires all operations used by template processor, and thus cannot be optimized statically.

Therefore, Ivy’s new compilation strategy should be the one with minimal code size (not regarding of the cost to implement compiler).

Memory and Speed

  • New view layer employs compact binary frame design (and more bitwise operations).
  • New view layer uses as many sequences (arrays) as possible rather than key-value pairs (objects) to store data.
  • New view layer prefers adding expando properties on exposing types rather than new wrapping type.
  • DI uses bloom filter to speed up Directive searching

(View layer is less friendly to pull request)

Summary

The new Ivy mode push “building-friendliness” to extreme, but you can also say it is not “non-building-friendly”. It won’t be useful for projects built with bare <script> tags.

Ivy’s most crucial target is cooperation with Angular Element. By encapsulating Angular components to custom elements (web components), we can achieve standalone publication, independent importing and independent usage of Angular as widgets. (To some degree this strangulates Svelte?)

The most important part of this strategy is code size. Even wihout common runtime library, Angular should work with minial size.
Components used as Angular Elements will not depended on external packages like forms and router, hence creating great value for size reduction.

Whether should NgModule still exist? It certainly has value on organizing code. That said, Dart version didn’t ever has NgModule. It is sure that application structure can be well formed without NgModule (at least for Googler).
With decreasing usage of NgModule in real world API, it might be optional in future. But I’m sure v6 won’t change NgModule. (I personally favor NgModule, but oppose enforcing it in app).

For applications plumbing many third party libraries, the main size problem might not come from Angular but rather from, for example, incorrect use of RxJS, importing moment.js inadvertently, or writing all styles in “comonent styles”. For specific size problem one needs to analyze it specifically. Don’t pray to Angular’s advanced optimizer.

The worst part of Ivy is OnChanges. It is now implemented by Object.defineProperty! It is no longer right to say Angular is based solely on dirty check. All articles about change detection need updating! And every section needs separate explaination!

Lifecycles might have big change in near future. But it will await Ivy being the default, no earlier than v7. In fact, Angular’s overall extensibility and runtime preprocessing has been greatly improved.

Deep dive into Vue2.5 Typing -- A tour of advanced typing feature

Vue 2.5 improves TypeScript definition! Before that, TS users will have to use class component API to get proper typing, but now canonical API is both precise and concise with few compromises!

For ordinary users, Vue’s official blog and updated documentation will guide you to upgrade or create projects.
But curious audience might wonder how the improvement is done and why TS support isn’t integrated in Vue2.0 at first place.

This blog post will deep dive into the technical details of Vue2.5 typing, which seems daunting at first glance. Don’t worry! We will show how TypeScript’s advanced types can be used in a popular framework.

Note: Reader’s familiarity with Vue and TypeScript is assumed in this post. If you are new to these two, checkout their official website!

TL;DR;

Vue2.5 exploits ThisType, mapped type, generic defaults and a clever trick to cover most APIs.

We will also list some limitations in current typing schema.

this is Vue

Let’s examine a basic Vue usage. We pass an object literal as component option to Vue constructor.

1
2
3
4
5
6
7
8
new Vue({
methods: {
greet() {
this.$el // `this` is Vue
console.log('Hello World!')
}
}
})

this keyword is bound to Vue instance in component option. Prior to Vue2.5, we declare this as a plain Vue type. Here is a simplified ComponentOption type.

1
2
3
4
interface ComponentOption {
methods: { [key: string]: (this: Vue) => any }
// other fields ...
}

However, we cannot access our custom methods/data via the declaration above since this is nothing but Vue. The typing doesn’t capture the fact that the VM injected into methods is instantiated with our custom methods/data/props.

A new type parameter V can allow users to specify their custom properties. So a better solution will be:

1
2
3
interface ComponentOption<V extends Vue> {
methods: { [key: string]: (this: V) => void }
}

And users can use it like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
declare function newVue<V extends Vue>(option: ComponentOption<V>): V

interface MyComponent extends Vue {
greet(str: string): void
hello(): void
}
newVue<MyComponent>({
methods: {
greet(str) {
console.log(str)
},
hello() {
this.greet('hello world')
}
}
})

It works, but also requires one interface declaration and one explicit type annotation.
Can compiler be smarter and infer this for us?

ThisType<Vue>

We can strongly type this by a special marker interface ThisType. It is introduced in TypeScript 2.3, which is the very reason why we didn’t have strong type until Vue2.5.

The original pull request has a detailed introduction and example for ThisType.

The most important rule is quoted here.

(If) the containing object literal has a contextual type that includes a ThisType<T>, this has type T

What does this mean? Let’s break this rule down to several pieces.

object literal means the component option in Vue’s case; contextual type means the component option is passed to a function as argument and the component option is typed via function declaration, without caller’s annotation; and finally ThisType<T> needs to be used in the function parameter declaration. The type parameter T refers to the type of this in the component option. In simple terms, this rule says we can change this keyword’s type according to the component option passed to new Vue or so.

Combining these together, we can write a simple declaration that understands our Vue component option.

Note, you will need noImplicitThis compiler flag to enable this new type checking.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
interface ComponentOption<Method> {
methods: Method
}

declare function newVue<Method>(
option: ComponentOption<Method> & ThisType<Method & Vue>
): Vue&Method

// Method is inferred as
// { greet(str): void, hello(): void }
newVue({
methods: {
greet(str) {
console.log(str)
},
hello() {
// this is typed as Method & Vue
this.greet('hello world') // custom methods works!
this.$el // vue property also works!
}
}
})

This code needs some explanation. First we define an ComponentOption and it takes a type parameter Method, which acts as a “stub” for compiler to infer custom properties on this.
Then in the function we declare a type parameter Method again and pass it to ComponentOption and ThisType.
Finally, ThisType<Method & Vue> means the type of this inside option will be an intersection of Vue and Method.

When we call newVue, compiler will first infer Method from ComponentOption object we pass to the function. Then the Method will flow into this keyword, resulting a type that has both Vue property and our own methods.

Mapping Computed

Typing methods alone is so far so good. However fields like computed have a different story.
The object in methods field has the same shape as part of this type. Say, methods has a hello function property and this also has a function property with the same name (in algebraic terms, endomorphism). But a property in computed is a function that returns a value and this has a namesake property with the same value type. For example.

1
2
3
4
5
6
7
8
9
10
11
newVue({
computed: {
myname: () => 'world' // a function returns string
},
methods: {
greet() {
console.log(this.myname)
// myname is a string, not a function returning string
}
}
})

How can we get a new type from computed definition object?

Here comes the mapped type, a new kind of object type that maps a type representing property names over a property declaration template. In other words, we can create computed type in Vue instance based on that in component option. (algebraically, homomorphism)

In a mapped type, the new type transforms each property in the old type in the same way.

The official documentation is crystal clear. Let’s see how we integrate this awesomeness into Vue.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// we map a plain type to a type of which the property is a function
// e.g. { myname: string } will be mapped to { myname: () => string }
// note this process can also be reversed during type inference
type Accessors<T> = { [K in keyof T]: () => T[K] }

interface ComponentOption<Method, Computed> {
methods: Method
computed: Accessors<Computed>
}

type ThisTypedOption<Method, Computed> =
ComponentOption<Method, Computed> & ThisType<Method & Computed & Vue>

declare function newVue<Method, Computed>(
option: ThisTypedOption<Method, Computed>
): Method & Computed & Vue

Accessors<T> will map the type T to a new type with same property names. But property value type is a function returning the type in the original T. This process is reversed during type inference. When we pass computed field as {myname: () => string} to newVue function, compiler will try to map the type to Accessors<T>, which results in Computed being {myname: string}.

And Computed is mixed into this, so we can access myname as string from this.

We skipped here computed setter style declaration for a more lucid demonstration. Supporting setter in computed is similar.

Prop Types Trick

props has a subtle difference from computed: we define a prop by giving a constructor of that value type.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
type PropDef<T> = { new(...args: any[]): T }
type Props<T> = { [K in keyof T]: PropDef<T[K]> }
interface ComponentOption<Prop> {
props: Props<Prop>
}
type ThisTypedOption<Prop> =
ComponentOption<Prop> & ThisType<Prop & Vue>

declare function newVue<Prop>(option: ThisTypedOption<Prop>): Prop & Vue

class User {}
newVue({
props: {
user: User,
name: String
}
})

One would naturally expect newVue will infer Prop as { user: User, name: string }. Sadly, it is not.

The problem lies in PropDef, which uses constructor type new(): T. Custom constructor is fine. For example User‘s constructor returns User. But primitive value’s constructor doesn’t work because String has the signature new(): String.

Alas! The return value is String, rather than string. Their difference is listed in the first rule of TypeScript Do’s and Don’ts. A string type is what we use and String refers to non-primitive boxed objects that are almost never used.

We can use another signature to type primitive constructor and union custom constructor together. Note every primitive constructor has a call signature, that is, String(value) will return a primitive string value rather than a wrapper object.

1
type PropDef<T> = { (): T } | { new(...args: any[]): T }

It should work, shouldn’t it? Sadly again, NO.

1
2
declare function propTest<T>(t: PropDef<T>): T
propTest(String) // return String, not string!

Because String satisfy both call and constructor signature in PropDef, compiler will prefer returning String.

How can we nudge compiler to prefer primitive type? Here is an undocumented trick.
The main idea is to exploit type inference priority. If a type parameter is single naked, that is, not in intersection type nor in union type, compiler will prefer to infer from that single naked position over intersection/union position. So we can add an intersection to constructor signature and then compiler will first infer call signature. Exactly what we want! To make the signature more self explanatory, we can use the object type to flag constructor type should not return primitive type.

1
2
3
type PropDef<T> = { (): T } | { new(...args: any[]): T & object }
declare function propTest<T>(t: PropDef<T>): T
propTest(String) // return string, yay!

Now we can happily infer props without manual annotation!

Compatibility

For better inference, our new type has many more type parameters than original ComponentOption<V> which only has one parameter. Nevertheless, it will be a catastrophic breaking change if we ship the new type without proper fallback. Generic defaults introduced in TS2.3 gives us a chance to bring about a more smooth upgrade.

1
2
3
4
5
6
7
8
9
10
11
12
13
interface ComponentOption<V extends Vue, Method=any, Data=any, Prop=any, Computed=any> {
// ....
}

interface MyVue extends Vue {
// ...
}

// users can use ComponentOption without changing their code
// parameter with default can be skipped
var option: ComponentOption<MyVue> = {
// ...
}

Happy ending!

Limitation

The “No silver bullet” rule also applies to typing. The more advanced types we use, the more complex error messages will be generated. Hope this blog post will help you to understand the new typing better and help you to debug your own application.

There are also some type system limitations in Vue typing. Let’s see some examples.

  • functions in computed need return type annotation

Return type annotation is required if computed method uses this. It turns out that using mapped type and ThisType at the same time without explicit annotation will cause cyclic inference error in current compiler.

1
2
3
4
5
6
7
8
9
10
new Vue({
computed: {
foo() {
return 123
},
bar(): number { // required
return this.foo
}
}
})

TypeScript has already opened an issue tracking this.

  • Prop types’ union declaration requires manual type cast

Vue accepts an array of constructors in prop’s definition as union type. However, PropDef cannot unify primitive constructors and custom constructors which have two heterogeneous signatures.

1
2
3
4
5
6
7
Vue.component('union-prop', {
props: {
primitive: [String, Number], // both primitive, ok
custom: [Cat, User], // both custom, ok
mixed: [User, Number] as {new(): User | Number}[] // requires annotation
}
})

In general, you should avoid mixing primitive type and object type.

Final words

TypeScript has been constantly evolving since its birth. And finally its expressiveness enable us to type Vue’s cannonical API!

Thank you, TypeScript team, for bring us these awesome features!
Thank you, Vue team, for embracing new advance in type system!

Grok control flow based analysis in TypeScript.

TL;DR:

  1. Compiler does not understand control flow in closure / callback function.
  2. In flow based type analysis, copmiler will be either optimistic or pessimistic. TypeScript is optimistic.
  3. You can usually workaround (3) by using const or readonly

This is a long due introduction for TypeScript’s flow sensitive typing (also known as control flow based type analysis) since its 2.0 release. It is so unfortunate that no official documentation is in TypeScript’s website for it (while both flow and kotlin have!). But if you dig the issue list earnestly enough, you will always find some hidden gems there!

To put it short, a variable’s type in a flow sensitive type system can change according to the control flow like if or while. For example, you can dynamically check the truthiness of a nullable variable and if it isn’t null, compiler will automatically cast the variable type to non-null. Sweet?

What can be bitter here? TypeScript is an imperative language like JavaScript. The side-effectful nature prevents compiler from inferring control flow when function call kicks in. Let’s see an example.

1
2
3
4
5
6
7
8
9
10

let a: number | null = 42

makeSideEffect()

a // is a still a number?

function makeSideEffect() {
// omitted...
}

Without knowing what makeSideEffect is, we cannot guarantee variable a is still number. Side effect can be as innocuous and innocent as console.log('the number of life', 42), or as evil as a billion dollar mistake like a = null, or even a control-flow entangler: throw new Error('code unreachale').

One might ask compiler to infer what makeSideEffect does since we can provide the source of the function.
However this is not practically feasible because of ambient function and (possibly polymorphic) recursion. Compiler will be trapped in infinite loops if we instruct it to infer arbitrary deep functions, as halting problem per se.

So a realistic compiler must guess what a function does by a consistent strategy. Naturally we have two alternatives:

  1. Assume every function does not have relevant side effect: e.g. assignment like a = null. We call this optimistic.
  2. Assume every function does have side effect. We call this strategy pessimistic.

Spoiler: TypeScript uses optimistic strategy.

We will walk through these two strategies and see how they work in practice.
But before that let’s see some common gotchas in flow sensitive typing.

Closure / Callback

Flow sensitive typing does not play well with callback functions or closures. This is explicitly mentioned in Kotlin’s document.

var local variables - if the variable is not modified between the check and the usage and is not captured in a lambda that modifies it;

Consider the following example.

1
2
3
4
5
var a: string | number = 42 // smart cast to number
setTimeout(() => {
console.log(typeof a) // what should be print?
}, 100)
a = 'string'

As a developer, you can easily figure out that string will be output to console because setTimeout will call its function argument asynchronously, after assigning string to a. Unfortunately, this knowledge is not accessible to compiler. No keyword will tell compiler whether callback function will be called immediately, nor static analysis will tell the behavior of a function: setTimeout and forEach is the same in the view of compiler.

So the following example will not compile.

1
2
3
4
var a: string | number = 42 // smart cast to number
someArray.forEach(() => {
a.toFixed() // error, string | number does not have method `toFixed`
})

Note: compiler will still inline control flow analysis for IIFE(Immediately Invoked Function Expression).

1
2
3
4
5
6
let x: string | number = "OK";
(() => {
x = 10;
})();
if (x === 10) { // OK, assignment in IIFE
}

In the future, we might have a keyword like immediate to help compiler reasoning more about control flow. But that’s a different story.

Now let’s review the strategies for function call.

Optimistic Flow Sensitive Typing

Optimistic flow typing assume a function without side-effect that changes a variable’s type. TypeScript chooses this strategy in its implementation.

1
2
3
4
var a: number | null
a = 42 // assign, now a is narrowed to type `number`
sideEffect() // assume nothing happens here
a.toFixed() // a still has `number` type

This assumption usually works well if code observes immutable rule. On the other hand, a stateful program will be tolled with the tax of explicit casting. One typical example is scanner in a compiler. (Both Angular template compiler and TypeScript itself are victims).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// suppose we are tokenizing an HTML like language
enum Token { LeftBracket, WhiteSpace, Letter ... }
let token = Token.WhiteSpace;
function nextToken() {
token = readInput(); // return a Token
}

function scan() {
// token here is WhiteSpace
while (token === Token.WhiteSpace) {
// skip white space, a common scenario
nextToken()
}
if (token === Token.LeftBracket) { // error here
// compiler thinks token is still WhiteSpace, optimistically but wrongly
}
}

Such bad behavior also occurs on fields.

1
2
3
4
5
6
7
8
9
10
11
12
13
// A function takes a string and try to parse
// if success, modify the result parameter to pass result to caller
declare function tryParse(x: string, result: { success: boolean; value: number; }): void;

function myFunc(x: string) {
let result = { success: false, value: 0 };

trySomething(x, result);
if (result.success === true) { // error!
return result.value;
}
return -1;
}

An alternative here is returning a new result object so we need no inline mutation. But in some performance sensitive code path might we want parse a string without new object allocation, which reduces garbage collection pressure. After all, mutation is legal in JavaScript code, but TypeScript fails to capture it.

Optimistic flow analysis sometimes is also unsound: a compiler verified program will cause runtime error. We can easily construct a function which reassigns a variable to an object of different type and uses it as of the original type, and thus a runtime error!

1
2
3
4
5
6
7
8
9
10
class A { a: string}
class B { b: string}
let ab: A | B = new A

doEvil()
ab.b.toString() // booooooom

function doEvil() {
ab = new B
}

The above examples might leave to you a impression that compiler does much bad when doing optimistic control flow inference. In practice, however, a well architected program with disciplined control of side effect will not suffer much from compiler’s naive optimistism. Presumption of immutability innocence will save you a lot type casting or variable rebinding found in pessimistic flow sensitive typing.

Pessimistic Flow Sensitive Typing

A pessimistic flow analysis places burden of typing proof on programmers.
Every function call will invalidate previous control flow based narrowing. (Pessimistic possibly has a negative connotation, conservative may be a better word here). Thus programmers have to re-prove variable types is matching with previous control flow analysis.

Examples in this section are crafted to be runnable under both TS and flow-type checker.
Note, only flow-type checker will produce error because flow is more pessimistic/strict than TypeScript.

1
2
3
4
5
6
7
8
9
10
declare function log(obj: any): void

let a: number | string = 42

log(a) // invalidation!

a.toFixed() // error, a's type is reverted to `number | string`

// To work around it, you have to recheck the type of `a`
typeof a === 'number' && a.toFixed() // works

Alas, pessimistism also breaks fields. Example taken from stackoverflow.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
declare function assert(obj: any): void

class TreeNode<V, E> {
value: V
children: Map<E, TreeNode<V,E>> | null

constructor(value: V) {
this.value = value
this.children = null
}
}

function accessChildren(tree: TreeNode<number, string>): void {
if (tree.children != null) {
assert(true) // negate type narrowing
tree.children.forEach((v,k) => {}) // error!
}

}

These false alarms root in the same problem as in optimistic strategy: compiler/checker has no knowledge about a function’s side effect. To work with a pessimistic compiler, one has to assert/check repeatedly so to guarantee no runtime error will occur. Indeed, this is a trade-off between runtime safety and code bloat.

Workaround

Sadly, no known panacea for flow sensitive typing. We can mitigate the problem by introducing more immutability.

using const

Because a const identifier will never change its type.

1
2
3
4
5
6
const a: string | number = someAPICall() // smart cast to number
if (typeof a === 'string') {
setTimeout(() => {
a.substr(0) // success, `const` identifier will not lose its narrrowed type
}, 100)
}

And using const will provide you runtime safety or bypass pessimistic checker.

1
2
3
4
5
6
7
8
9
10
function fn(x: string | null) {
const y = x
function assert() {
// ... whatever
}

if (y !== null) {
console.log(y.substr(0)); // no error, no crash
}
}

The same should apply to readonly, but current TypeScript does not seem to support it.

Conclusion

Flow sensitive typing is an advanced type system feature. Working with control flow analysis smoothly requires programmers to control mutation effectively.

Keeping mutation control in your mind. Flow sensitive typing will not in your way but make a pathway to a safer code base!

How to write copy-paste friendly code -- An introductory parody

With the growing population of SOA (StackOverflow Oriented Architecture), keeping your code copy-paste friendly is becoming more and more important in demonstrating, developing and question answering.

You want your answer to be available instantly, free of fussy debugging. So copy-paste friendliness is a key property of your code robust enough to be ubiquitously runnable and, eventually, help you gain more reputation.

Adopting copy-paste style is amazing. Copy-paste boosts your productivity by freeing you from rumination on architecture complexity. Your boss will be happy with all the SLOC you commit. Your colleagues will respect your absolute ownership of the magnificent, enigmatic, labyrinthine code artifact built by the blob gleaned from everywhere.

1. Always prefer “==” to strict equality.

Implicit conversion grants you additional robustness. Your code will never complain about ill input. Copy-paste without worry. Yeah

1
2
3
var x = '10'
if (x == 10) x += 5
y = x / 5 // wow, it works!

And you can enjoy this artistic equality table when debugging. Cool.

2. Always prefer positive condition.

Never return early on invalid case. You can copy-paste it any where in your control flow. Yeah.

If return is added, you have to remove the unnecessary control flow abrupting keyword when you need to copy paste these code into a deeply nested code block, which is quite common in CPS(copy-paste-style) programming.

1
2
3
4
5
6

function someFunction(someCondition) {
if (someCondition) {
// Do whatever you want
}
}

3. Always prefer tautological check.

Repetition here is for genericity. Every conditional path can be copy-pasted free of reading. They are very safe compile time constructs that preclude the very possibility of runtime undefined panic.

else on individual line is a bonus. You can copy-and paste code without looking at else keyword! What a profit!

1
2
3
4
5
6
7
8
9
10
11
12
if (a && a.b && a.b.c == 1) {
// never worried about a.b.c is not undefined
}
else
if (a && a.b && a.b.c == 2) {
// repition is one aesthetic rule
}
else
if (a && a.b && a.b.c == 3) {
// very safe even programmer inadvertedly paste it to elsewhere

}

4. Always prefer fully qualified name

Never cache common subexpression. Declaring new variable name will force you read code and find declaration before copy-pasting.

1
2
3
topVariable.someProperty.nestedProp.anotherProp = 123
console.log(topVariable.someProperty.nestedProp.anotherProp)
topVariable.someProperty.nestedProp.anotherProp += 1

5. Always prefer inlining statements in function

Never factor out functions. Helpers require nonlocal function name lookup when copy-pasting. Dev experience terminator.

Conclusion

Oh, GG. I forgot GitHub Gists!

Compare TypeScript libraries for Vue -- a review

Vue has a lot of TypeScript binding libraries. A lot of.

They look similar, with subtle difference. How to choose one?

Here I compare three libraries.

They are representative, and the analysis probably also applies to the official library.

Disclaimer: I’m the author of av-ts. So this article is probably biased. But I tried my best to give an objective review.

0. Overview

Vue-typescript-component and av-ts are decorator based libraries. Vue-typescript-component is more like the popular typescript library, but supports Vue2.0. av-ts is my own library, crated after I read many alternatives.
vuets is quite different. It resembles the original object literal style API from vue.js.
Before I start the review, I want to state a rule of thumb before.

The more flexiblity a library boasts, the harder for a type checker to verify it.

It is quite common in programming world. For example, JavaScript is very flexible and dynamic, but it is usually hard to determine whether a property access is legal. Java is a static language, javac can help you check many pernicious typos in code, but Java cannot easily do reflective programming.

Vue has a very flexible and elegant API, but it comes at the cost of type safety. That’s why all these Vue TypeScript libraries exist. How to balance between expressiveness and type safety is the main concern of such a library.

1. Compatibility

All the three libraries, vue-typescript-component, vuets and av-ts, have good support for Vue2 and TS2. So there is not much to compare here. (vuets has vue1.0 support while the other two do not)

2. Extensibility

I think this is the best part of av-ts. av-ts provides a Component.register for custom decorator. For example, if you want to use vuex in your project but also want to preserve type safety, you can create your own decorator. This is exactly what I do in kilimanjaro, a typed vuex fork.
vue-ts-copmonent does not provide extensibility at all. vue-ts has the same extensibility as original vue because it uses object literal style (that is, the most extensible one among the three). However, vuets also support class component like syntax. Mixing two style is somewhat confusing.

3. Idiom

av-ts and vue-ts-component are both idiomatic TypeScript code. class/property/component just feel like plain-old-typescript. vue-ts chooses object literal style API. While it looks the most similar to original vue, it is not as idiomatic as the other two in TypeScript land. Also, object literal style requires many manual typing annotation, which feels foreign to both JavaScript and TypeScript.

4.Conciseness

vue-ts is the most verbose as mentioned above. Object literal style API is almost impossible to have good type inference for contemporary compilers. You can read more about it here. Basically I don’t think vue-ts is an ideal approach.
av-ts and vue-ts-component are concise. Most methods/properties require only annotating once. av-ts requires more decorators for special methods like render and lifecycle, while it has a more concise and type safe property syntax. So I think this is a tie.

5. Type Safety

av-ts and vue-ts-component can inject Vue’s native types by extends Vue. Neat. vuets users have to re-annotate all the method again, alas. av-ts has more type safety when defining special methods like render, lifecycle and transition. This is powerer by new decorators introduced. Again, these decorators are created by Component.register and the implementation is not hard-coded into one file.

6. Expressiveness

vuets is as expressive as original vue, at the cost of more annotation. vue-ts-component can express most basic API in vue, but there is something it cannot. av-ts can use mixin and can use props in data method. I have thought over these scenarios before hand.

Conclusion

av-ts may not be the best component library for vue, but I believe it strikes the sweet point between conciseness, type safety and expressiveness. I’m happy to see more TS+Vue users try it out, and give me feedback so it can become better.

Vue2 + TypeScript2 -- an introductory guide

Safety check and error detection are important to development experience. Vue has gone to great lengths to provide component information when typos or mistakes happen. But we can do better, error reporting can happen at compile time. This is why we want to use TypeScript with Vue.

final result

Our goal: writing Vue in TypeScript

Goals

In this guide we will finally achieve these goals:

  • writing type safe vue instance declaration
  • writing TypeScript in Vue’s single file component
  • semantic completion in pure TypeScript file
  • type checking all TypeScript, even TypeScript in *.vue‘s script tag.

All code example can be found here, with separte commits standing for sections in this article.

Prerequisite

You have to know about TypeScript2.0 and Vue2.0. And optionally webpack’s knowledge is very useful. And of course if you’re reading this article you must have known npm.

Basic setup:

First make a new directory for your project. And then init your new project and install dependencies.

1
2
3
4
5
6
mkdir vue-ts-test
cd vue-ts-test
npm init
# pressing `enter`s and done
npm install vue --save
npm install typescript --save

Note both vue and TypeScript need to be version 2. They have both definition files baked in so it’s much easier to start up with them than before. (I still respect definitely typed!)

make html template, say, index.html:

1
2
3
<div id="app"></div>
<script src="node_modules/vue/dist/vue.js"></script>
<script src="app.js"></script>

Then in your app.ts:

1
2
3
4
5
6
7
declare var Vue: any
new Vue({
el: '#app',
render(h) {
return h('h1', 'hello world')
}
})

And compile your code:

1
./node_modules/.bin/tsc app.ts

Then open index.html in your browswer! It should give you a hello world!

Exploring Vue with TypeScript

But there are a lot of feature missing in the above example. For example, you want a build step to combine all your scripts. You want component style co-locates with template and script, yet still sytnax highlighting every thing.

Let’s use Vue’s genius single file component. It requires webpack and vue-loader

1
2
3
npm install webpack --save-dev
npm install vue-loader --save-dev
npm install css-loader --save-dev

Or you can use vue-cli

1
vue init webpack-simple vue-ts-test

You need to make vue-loader to transpile your TypeScript code. vue-ts-loader can work with vue-loader.

1
npm install vue-ts-loader --save-dev

And in webpack.config.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
module.exports = {
entry: { app: './app.ts', },
output: { filename: 'app.js' },

// resolve TypeScript and Vue file
resolve: {
extensions: ['', '.ts', '.vue', '.js']
},

module: {
loaders: [
{ test: /\.vue$/, loader: 'vue' },
{ test: /\.ts$/, loader: 'vue-ts' }
],
},
vue: {
// instruct vue-loader to load TypeScript
loaders: { js: 'vue-ts-loader', },
// make TS' generated code cooperate with vue-loader
esModule: true
},
}

Now you can add write Vue code in single file component!

Suppose in app.vue

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<template>
<h1 @click="hello">hello world</h1>
</template>

<script>
export default {
methods: {
// type annotation!
hello(): void {
alert('hello world')
}
}
}
</script>

And now you can change your app.ts into an entry file.

1
2
3
4
5
6
7
8
9
10
declare var require: any

import Vue = require('vue')
var App = require('./app.vue').default

new Vue({
el: '#app',
components: { App },
render: h => h('app')
})

Now webpack your in your application directory. You can still see the hello world? Click it should give you an alert.

Cool? And here is the source.

Toward a more type safe API

Vue has a typechecker hostile API.
We can use helper library to get a better type checking. av-ts, Awesome Vue for TypeScript, is a good start point to use.

Install it by npm, again.

1
npm install --save av-ts

Then you can change your app.vue to this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<template>
<h1 @click="hello">hello {{name}}</h1>
</template>

<script>
// import dependency
import {Vue, Component} from 'av-ts'

// decorat vue class
@Component
export default class App extends Vue {
// undecorated property will be packed in `data` option
name = 'world'
// undecorated method is just method
hello() {
alert('hello ' + this.name)
}
}
</script>

webpack will give you an error. Because decorator is still considered experimental. You need to switch it on explicitly.

Add a tsconfig.json in your directory.

1
2
3
4
5
6
7
{
"compilerOptions": {
"module": "commonjs",
"target": "es5",
"experimentalDecorators": true
}
}

Now it compiles! Check it out in browser!

av-ts has several features that isn’t present in other libraries.
First, prop is defined in class, so you can access these as properties in other methods without declaring in class again. Second, data, render and lifecycle hooks are declared by decorator, which is a mark to tell you these methods are different. Plus, their names and signatures can be checked by decorators. Third, extension is easy in av-ts. You can register new decorator, and still keep type safety.

For source as a whole, you can take a look here.

Even more typesafety

Vuex is another popular state management library for vue.

Kilimanjaro is a type safe fork for Vuex.

To create a store is familiar with its original API.

1
2
3
4
5
6
7
8
9
10
11
import { create } from 'kilimanjaro'

var store =
create({count: 0}) // create a state with initial state
.getter('count', s => s.count) // a getter
.mutation('increment', s => () => s.count++) // define a mutation
.mutation('decrement', s => () => s.count--) // payload is in the second parameter parenthesis
.done()

store.commit('increment') // make a state change
// store.commit('ncrement') // typo won't compile! magic!

By heavy use of string literal type and overloading, kilimanjaro provides a similiar type safe API to vuex.
If you mistakenly commit a wrong mutation name, or the mutation payload does not have correct type, kilimanjaro will make TypeScript compiler report that for you.

There is also component helper in kilimanjaro. Check the example below

1
2
3
4
5
6
7
8
9
10
11
import {Vuex, getHelper} from 'kilimanjaro'

const { getters, commit } = getHelper(store)

@Componet
class App extends Vue {
@Vuex count: number = getters('count')
@Vuex add: Function = commit('increment')
// @Vuex typo: Function = commit('ncrement') // won't compile
// @Vuex wrong: string = getters('count') // type incompatible
}

You can get component helpers by calling getHelper. Helpers are function that take a getter/mutation/action name and return corresponding result. Given a mutation name, commit helper will return a function that, once called, changes store state. The similiar for dispatch helper.

getters is more magical. Given a getter name, it will return a symbol with correct result type. But at runtime, it will be changed to a computed accessor in the VM. The magic is done by the Vuex decorator, which is an extension of av-ts. Like mentioned above, av-ts is a very extensible library.

For a working example, you can take a look here. And Live demo is here.

Vue-router is type safe enough for pragmatic usage. It is really a well written and designed library. Designing a library that provides 100% type safety is just an overkill bringing marginal benefits at huge cost of debugging compiling error.
If vue-router cannot cater for a type paranoid, this sketch will be interesting.

Limitation

You might want even more features from TypeScript, say

  • semantic completion in single file component
  • type checking template code

go home, you're drunk

They are way way hard to implement. Semantic completion in vue file requires editor support and compiler support. None of them is easy to implement.

You can write your vm code in a separate file and import into your vue template. But that is opposite to the design philosophy of single file component. In reality, keeping vue small and clean is better choice. With small number of symbols in source file, generic completion should also work well. (at least as well as plain JavaScript)

Template code is an even more heculean task. Vue compiles its template into with(this) { ... } format (doc). with statements are not checked in TypeScript, nor flow-type. Plus, vue compiler does analyze no expression, making it impossible to do checking without reimplementing a compiler. However, again, it is no worse than plain JavaScript.

Conclusion

Vue is an elegant and lightweight library. With the integration of TypeScript, Vue can be more safe and productive!

Compared to other frameworks, you can instantly get precompiled template code, small code footprint, flat learning curve.
But when it comes to scale bigger, you might want additional help.

There are overhead to integrate TypeScript with Vue, indeed. But the error checking and refactoring help still worth your try.

Tools we used to achieve type-safety

  • TypeScript2 and Vue2, for their definition files
  • vue-ts-loader: load TypeScript in vue’s single file component
  • av-ts: wrap vue’s API in type safe way

Optionally

  • kilimanjaro: type safe vuex, requires deeper understanding of TypeScript’s advanced features.

Type Safe Routing for Vue.js

Vue router is elegant and type safe enough for most use cases. But I want to experiment how far typescript can help us.

Typesafe routing isn’t easy to do.

The easiest way to declare multiple class and instantatiate their paramater type. For example, typed-url. But this is too verbose.

Another approach is to use functional combinator. To put it simple, combinator is high order function that can abstract various operation. Routing combinators usually are a bunch of functions that can accept strings as static url segment or a function as dynamic url parameter. Both purescript and swift. But monad is too monad-y. My head just explodes.

One unique way to provide type routing is using reified generic! A demo video has illustrted how to implment it.(Spoiler: for a function with type A => Response, one can access the class by A.type and cast value by guard let param: A = .... in swift. Whoa, reification is powerful). Github repo is here: https://github.com/NougatFramework/Switchboard

Compile time reflection is ideal for routing thing. Yesod uses template haskell to do this, Example. Macro paradise!

Scala has yet another unique construct called pattern match. Tumblr’s colossus is a great example to use pattern match for type safe routing.

And of course, haskell has many type safe routing library. Check out the review for more info.

JavaScript does not have powerful constructs like macro/pattern match. Combinator is the only way to achieve type safety but for client side component based routing, declaring more functions solely for routing doesn’t feel natural. And specifically TypeScript is still too feeble to describe routing. However, by combining tagged template, function overloading (or fun-dep), and intersection type (or row polymorphism), we can still do some interesting thing. If this were written in flow-type, more interesting thing could happen.

Frankly type safety in router does not grant you much: it cannot check tempalte code, it can only help you to double check the shape of parameters in $route. It can help you to type router instance better, but requires all routes to have a name field.

This is only a sketch for type safe routing design. Useful? No. Concise? Partly. Safety outweighs ease of use? No. Maybe it’s only suitable for type safe paranoid.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
interface Route<T> {
<K extends string, C>(c: StaticRouteConfig<K, C>): Routes<StaticRouteAct<K> & C & T>
<K extends string, R, P extends R, C>(c: RouteConfig<K, R, P, C>): Routes<RouteAct<K, P> & C & T>
}

interface StaticRouteAct<K> {
(location: {name: K}): void
}

interface RouteAct<K, T> {
(location: {name: K, params: T}): void
}

interface Router<T> {
push: T
replace: T
}

interface Routes<T> {
route: Route<T>
done: () => Router<T>
}

interface StaticRouteConfig<K extends string, C> {
name: K
path: StaticPath,
children?: (p: OriginPath) => Routes<C>
component?: {$routes?: {params?: {}}}
}

interface RouteConfig<K extends string, R, P extends R, C>{
name: K
path: Path<P>,
children?: (p: PathMaker<P>) => Routes<C>
component?: {$routes?: {params?: R}}
}

interface Path<T> {
__pathBrand: T
}

interface StaticPath extends Path<{}> {
__staticPathBrand: never
}

interface PathMaker<Parent> {
<A, B>(t: TemplateStringsArray, n: A, n2: B): Path<A & B & Parent>
<A>(t: TemplateStringsArray, n: A): Path<A & Parent>
(t: TemplateStringsArray): Path<Parent>
}

interface OriginPath {
__originPathBrand: never
<A, B>(t: TemplateStringsArray, n: A, n2: B): Path<A & B>
<A>(t: TemplateStringsArray, n: A): Path<A>
(t: TemplateStringsArray): StaticPath
}

var UserComponent = {$routes: {params: { uid: '333'}}}
const unimplement: never = null as never
const route: Route<never> = unimplement
const p: OriginPath = unimplement

const rs =
route({
path: p`user/${{uid: ''}}/profile/${{pageType: ''}}`,
name: 'user',
component: UserComponent,
children: p => route({
name: 'tab',
path: p`tab/${{tab: ''}}`,
component: {$routes: {params: {uid: '123', tab: '333'}}},
})
})
.route({
path: p`post`,
name: 'post'
})
.done()

rs.push({name: 'post'})

Type Safety in Vue.js

Static typing has become a hot word in frontend land: hundreds of tweets and blogs appears on social network and XXXX weekly, rivaling type checkers compete their features with each other.

Correspondingly new frameworks have a consciousness of type safety in their API design.
Angular2 has partial type safety in ViewModel code notwithstanding template code. (There has been some efforts to pursue more type safety, though)

React has full, and strict, when checked with flow, type safety by embedding templates in JavaScript(X).

But stakeholders of Vue.js, the thirdwhee.. another popular MVVM framework , might be disappointed by Vue’s type-checker hostility…

Type Checker Hostile API

Vue provides a set of simple and elegant API via heavy use of reflection that extinguishes compiler’s type inference.

It’s not Vue’s fault. Up to now static type checkers in JavaScript land have several limitations:

  1. They cannot understand modification to objects’ type or perform key-wise type inference. (more elaboration later)
  2. Some cannot annotate function’s this type. (not in flowtype 0.30, supported in TypeScript)
  3. Some cannot annotate Type Property Type. (not in TypeScript until #10425 is merged, flowtype has undocumented $Magic type like $Keys, $Record)

For point1, an example will elucidate itself.

Suppose we are going to provide a type definition file for Vue’s config option.

1
2
3
4
5
6
7
8
interface VueConfig<D, P, PD, C, M, W> {
data?: D
props?: P
propData?: PD
computed?: C & {[k: string]: (this:D & P & C & M ) => {}}
methods?: M & {[k: string]: (this:D & P & C & M) => {}}
watch?: W & {[k: string]: (this:D & P & C & M) => {}}
}

This is not very precise but does highlight some basic idea of Vue’s API. computed is a field of which the value is a function with this pointed to the object that has mixed in data, props and method. this in Vue’s option is an object made out of reflection.

Then we will write some function like this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
function getVue<D, P, PD, C, M, W>(opt: VueConfig<D, P, PD, C, M, W>): D & P & C & M
{
return null // placeholder
}

let a = getVueConfig({
data: {
a: 123,
},
watch: {
a: function() {
console.log(this.a) // oops
}
}
})

Compiler will complain about this.a in the watch function. Why? this cannot be inferred. To infer this, compiler will have to first infer D, P, C, M respectively. To infer D & P & C & M, compiler will have to first infer the whole expression for resolving all the type arguments. But to infer the whole expression we need first infer watch, where we need to infer this. So comes a recursion. Compiler cannot be too eager to infer type otherwise it will jump into a recursion trap. Sloth is a virtue here, even Betelgeuse cannot blame.

Alternative API

Vue’s original API is doomed to be hard to infer. However, we can build a thin layer of wrapper to leverage type checkers.

I have two alternatives to present here. One is chaining DSL, a novel approach to induct type checking and inference into Vue. The other alternative is more established and angular like: class decorator.

Chaining DSL

We can work around the recursion problem by nudging compiler to do more diligent work. Because every method/function call return a new type symbol, we can use it to escape from recursion trap:

Definition:

1
2
3
4
5
6
declare class VueTyped<T> {
data<D>(d: D): VueTyped<T & D>
method<M extends {[k:string]: (this: T) => any}>(m: M): VueTyped<T & M>
get(): T
static new(): VueTyped<{}>
}

Usage:

1
2
3
4
5
6
VueTyped.new()
.data({msg: 'hehehe'})
// return a new type symbol with field `msg`
.method({method() {return 'hello: ' + this.msg}})
// create a new type symbol with `msg` and `method`
.get() // type as {msg: string, method(): string}

Quick explanation. data has a signature like data<D>(d: D): VueTyped<D & T>. The intersection type in return position mocks mixin behavior. method<M extends {[k:string]: (this: T) => any}>(m: M): VueTyped<T & M> is more complicated. Parameter M is required for compiler to garner properties of option passed to method call. M is bounded by a constraint that every function in option must have this typed as the object we defined previously and that only defined property can be accessed via this. The final returning intersection type acts the same as data.

Note: method does not work for current TypeScript. Probably it is a bug

But step-wise inference still cannot resolve watch and computed property. $Keys magic type or keysof type does not exist in TS yet! Meanwhile, flowtype does not support this-typed function.

computed option is even harder to handle. There is no way to define this type in a getter/setter method. If we do not pass a getter/setter but a plain function as value, we cannot merge the computed properties into the resulting object.

A verbose workaround is forward reference in type annotation:

1
2
3
4
5
6
7
8
var a = VueTyped.new()
.data({ msg: 'hehe' })
.computed({
get computed(this: typeof a) { // forward reference
return this.msg + ' from WET computed!'
}
})
.get()

It’s not DRY.

Furthermore, this approach does not support language service feature like “looking for definition” or “finding all usage” because intersection type needs casting in implementation to work.

Irreparable! Irredeemable! Irremediable!

However, this approach has some benefits. First, it is easier to extend its functionality. If one would like to add vuex field in option, it just requires defining a new method. It also prevents cyclic dependency because you cannot use fields before declaring. The API itself is akin to its original version, and thusly the implementation is very thin.

Class Decorator

This approach is much more conventional, and is discussed broadly in Vue’s issue.

The basic idea is to define as many methods as possible in a class and to decorate fields to add Vue specific logic.

One exceptional API is provided by itsFrank’s vue-typescript

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import * as Vue from 'vue'
import { VueComponent, Prop } from 'vue-typescript'

@VueComponent
class MyComponent {
@Prop someProp:string;

@Prop({
type: String
})
someDefaultProp:string = 'some default value';

@Prop someObjProp:{some_default:string} = {some_default: 'value'}; //vue-typescript makes sure to deep clone default values for array and object types

@Prop someFuncProp(){ //defined functions decorated with prop are treated as the default value
console.log('logged from default function!');
}

someVar:string = 'Hello!';

doStuff() {
console.log('I did stuff');
}
}

This TSish approach enables more capability of compiler tooling such as usage finding and definition lookup. Class decorator also guarantees every method’s this correctly points to class instance, which cannot be achieved in Chaining DSL approach.

With higher abstraction comes more confusion. Indeed, class decorator smooths out the discrepancy between Vue and type checker. But syntactically its API is much further from Vue’s original one. Adding new API is also harder because every decorator is hard coded in VueComponent decorator’s code. For example, adding @vuex is almost impossible without rolling out a new VuexComponent. It also cannot transform all Vue’s API, such as watch and computed: { cache: false }, into idiomatic TypeScript, leaving some orifices in type safety.

I have an alternative API bike shedding but not ready to present. Maybe I will try it later.

Conclusion

This article presents type-safety problem in Vue and two ways to mitigate the problem. Rewritten in ES015 and type-checked by one of the most advanced type checkers, Vue is designed in ES5 era and, satirically, is still designed for ES5 code.

Vue doesn’t come with type safety in mind. But this is might be a mirroring of some part in the community where some developers have almost kind of Stockholm Syndrome: they encounter so many type unsafe ordeals that they are very happy and proud with their lavish use of reflection which backfires to themselves.

Yet one should always keep a leery eye a Static Typist‘s maniacal malarkey. Static typing system works the same way as BDSM: the more constraints, the more pleasure. Once having tasted the relish of bondage, a bottom will avariciously demand more complex tricks and more powerful constraints from typing system. That urge is so strong that the bottom loses incentives to lumber out of the fifty shades of types.

A collection of nodejs CDN for Celestial Imperial

Living inside of the Great Fire Wall is a so onerous ordeal that survival guides can be compiled into an anthology.

Here is how to configure various popular open source project to use their CDN in Ch**na.

Plenty of nodejs project provide environment variable to configure mirror/CDN to download source code or binary.

nvm / nodejs

Ref

1
NVM_NODEJS_ORG_MIRROR=https://npm.taobao.org/mirrors/node nvm install 4

npm

Ref

1
npm --registry=https://registry.npm.taobao.org install koa

Kudos to fengmk2

PhantomJS

Ref

1
PHANTOMJS_CDNURL=https://npm.taobao.org/dist/phantomjs npm install phantomjs --registry=https://registry.npm.taobao.org --no-proxy

Kudos to fengmk2, again!

node-sass

Ref

1
SASS_BINARY_SITE=https://registry.npmjs.org/node-sass npm install node-sass

Kudos, again and again, to fengmk2

Docker

Ref

Install Docker

1
curl -sSL http://acs-public-mirror.oss-cn-hangzhou.aliyuncs.com/docker-engine/internet | sh -

Config accelerator

You need to register an account in daocloud.io.

1
2
echo "DOCKER_OPTS=\"--registry-mirror=http://xxxxxx.m.daocloud.io\"" | sudo tee -a /etc/default/docker
sudo service docker restart

Kudos to daocloud.

Pypi

Ref

1
2
3
4
cat <~/.pip/pip.conf <<EOF
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/
EOF

Kudos to aliyun

sbt

Ref

1
2
3
4
5
6
7
8
9
cat > ~/.sbt/repositories <<EOF
[repositories]
local
osc: http://maven.oschina.net/content/groups/public/
typesafe: http://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext], bootOnly
sonatype-oss-releases
maven-central
sonatype-oss-snapshots
EOF

Kudos to freewind

And these links is useful:
加速 SBT 下载依赖库的速度
how to make sbt jump over GFW

Gopm

Ref

It is quite clear in the doc and help command.

1
2
3
4
5
6
7

gopm get <import path>@[<tag|commit|branch>:<value>]
gopm get <package name>@[<tag|commit|branch>:<value>]

useful options:
--update, -u update package(s) and dependencies if any
--gopath, -g download all packages to GOPATH

Conclusion

Hail Aliyun! Hail fengmk2!
For the unfortunate who live in a crappy ignorant country.