My wishlist for JavaScript transpiler

stereobooster - Dec 10 '18 - - Dev Community

I thought about this article a long time ago but wasn't sure about form. So instead I wrote What is the next big thing in frontend development?. Recent Dan Abramov's post inspired me for this form. (alternative name: steal this idea...)

Preface

ECMAScript

When we work with modern JS (or ECMAScript to be precise) we have to deal with different versions of the language, like ES5, ES6 (aka ES2015), ES2016, ES2017, ES2018. Those versions can define

  • very big language changes, like new operators (... spread operator, for example), which would require transpilation
  • and pretty small changes, like the introduction of a new function (Array.prototype.includes for example), which can be easily polyfilled

Web APIs

ECMAScript is not the only one standard which defines what you have to deal with when you have to do web development. There is a big set of Web APIs, like DOM, Service Workers API, Fetch API, IndexedDB etc. They can be described with WebIDL.

Not so standard

Standards sound good, but the problem is that every browser implementation slightly differs. Some doesn't implement some API at all (because they didn't have time for it or don't want to) or they implement only part of API or there are bugs in the implementation. As a result, when you develop for the web you can't write code and test it one browser, you need to test it in all browsers. This is a long-standing problem people tried to solve it over and over again, for example:

  • developers only tested in one browser and added disclaimer this website is best viewed in internet explorer, or later in google chrome
  • same browsers (by the same vendor) doesn't work the same way on different machines, so people introduced virtual machines with browsers for testing and tools like browserstack appeared
  • people wrote some libraries which worked around browser differences and bugs, one of the biggest examples is jQuery, but modern libraries like React still do this
  • people come with ideas of polyfills to make browser behavior more consistent and later created polyfill.io
  • there is also autoprefixer which will add browser-specific prefixes to CSS properties
  • there is babeljs which can transpile ES6 or later down to ES5 (which is widely supported)

So yes this is a really big problem, a lot of people worked on it, a lot of time were spent, businesses were built around.

Caniuse

Caniuse is a crowdsourced database of what browser (vendor, version) supports which ECMAScript and/or WebAPI. It is available as an online website or standalone database.

Browserslist

Browserslist is a DSL in which you can specify what kind of browsers you want to support. Which vendors, down to which version etc.

Web bundlers

The most used bundler out there is WebPack, there are also Parcel and broccoli and browserify.

Wish 1: painless polyfills

  1. Decide which browsers we want to support and declare it with Browserslist. There is defaults so you don't have to choose if you don't want
  2. Write some code as if all the latest features are available
  3. The compiler analyzes the code and decides which polyfills your code needs (but only stable standards)
  4. By default, the compiler uses latest stable features like babel-preset-latest

The compiler takes into account what your code needs (3) and what you target (1) and creates buckets of browsers which share the same features. So there will be browsers which support all (or almost all features) and code can be delivered without transpilation, for example, ES6 with dynamic import and fetch unpolyfilled. How cool is that?

Transpiler will need to analyze all dependencies too, because they may require additional polyfills, which means that dependencies should be distributed as ES6 modules (unpolyfilled). And some libraries are implemented in different languages, like TypeScript and they need to be transpiled down to ES6 modules. Also, it means that we will need ES6 minifier.

Aside from browser specific bundles, the compiler will generate configuration for asset router e.g. some config which will be used by server or Cloudflare-Worker or AWS Lambda@Edge to decide which assets to serve to which browser.

A potential problem of "bucketing" is that we can end up with a big number of buckets, so there should be a way to limit the number of buckets and some heuristic to decide optimal scheme.

Inspiration:

Wish 2: installable components

Again long-standing problem and a lot of people tried to solve it and we are almost there.

It began at the time of DHTML and jQueryUI when people started to build rich UIs in JS. There were components, like date picker or combobox. To install one you would download a zip file with all files, like JS, CSS, images etc. You would need to make sure that versions of dependencies match manually, you need to change paths of assets manually, you need to handle caches somehow and you don't update it. Mileage may vary, but I guess this is pretty close to the overall experience

People tried to solve it with:

  • CommonJS, AMD, UMD, ES6 modules and npm, Bower - to be able to distribute components as packages
  • Grunt, Gulp - as asset pipeline, to configure relative paths, minify and create unique names (for caches)
  • Browserify, WebPack, Parcel - as asset pipeline, but also with import(asset) and ability to understand different module formats, like CommonJS and ES6

We almost there. We can

  • ✓ install package from npm
  • ✓ use it by simply import from or import()
  • ✓ in case of React (or Vue etc) component, with CSS-in-JS we don't need to worry about including a CSS file
  • ✓ in case of React (or Vue etc) component, we can use inline SVGs (as components) for icons and images
  • ❌but we still can't distribute packages with asset dependencies, like images or fonts, via npm because using import for assets is invalid JS

This is a wishlist, so I don't have a solution. I just want to install package, require component and it suppose to work without the need to configure paths for assets, without the need to configure minification, unique names for assets, without the need to configure bundler or transpiler etc.

CSS-in-JS and inline SVG components make us closer to the target, but CSS-in-JS has it's drawbacks like it doesn't work without JS (there are some projects which tries to solve it) also it "vendor-locks" to some JS specific implementation. There is no single solution for CSS-in-JS, so we can end up with more than one CSS-in-JS solution inside one project.

On the other side, we have pretty tolerable middle ground between CSS-in-JS and pure CSS - CSS Modules. To use it in npm packages we need to allow bundler specific "asset-import", but we kind of already allow some bundler specific code like, process.env.NODE_ENV === "development". And also we need to distinguish CSS Modules for general CSS - for this, we have an established convention, use .module.css for CSS Modules. So maybe we can pull this off.

Wish 3: a fast feedback loop

People worked on this for quite some time, including:

Current implementation already pretty good, but there is also TypeScript, which can be pretty slow. The solution would be to separate compilation and type checking, for example, compile with Babeljs and in separate process run type checker.

Babel and Webpack and Parcel are pretty good, but not as good as OCaml for example. I wonder if it can be done better?.. With some high performant Rust code compiled down to WASM 🤔

Inspiration:

Wish 4: zero-configuration

When people talk about zero-configuration they refer to those:

  • convention over configuration, authors of a tool took time to decide what will work best, what is a safe bet. A good example here is, Create-React-App it guards you against foot guns, it provides all most-required polyfills, it supports a lot of things out of the box
  • seamless integration between tools, authors of tools took time to make sure they work together, without you need to worry about. A good example here is Parcel, you just use what packages you need and they simply work

Zero-configuration allows to remove incidental complexity of tools. You can start to code immediately without the need to worry about configuration.
Zero-configuration allows to upgrade tooling easily.

We have some outstanding examples of zero-configuration, but I still have to face with a lot of configuration or worry about integration between tools.

Photo by Louie Martinez on Unsplash

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .