Anyone who hasn't heard of WebAssembly?

Great.

Anyone here who has had a reason to use WebAssembly in a production app?

All right.

So we are all about on the same page.

I would like to invite you back into journey of time to the year 2017.

Like most of you, I heard about WebAssembly but beyond surface level stuff I didn't know

much about it.

I read this headline and thought huh, I guess if WebAssembly really does kill JavaScript

I will figure that stuff out but it seems like people have wanted JavaScript to die

for a while and we are still here.

What does a typical web dev need Wasm for?

I work for a consulting firm based in Seattle called GEN/UI.

I think our dev manager described it best when he said our web expertise is in doing

something weird.

We built a screen recording and editing app for Chrome OS that runs entirely client-side.

Our requirements were as follows: The app was meant to look and feel like it was native

to Chrome OS and act as a competitor to others out there.

The client requested the video provided video editing capability and needed to run locally

offline.

There are a couple ways to develop for Chrome OS and we decided to use Chrome package OS

and they are essentially web apps given additional hardware APIs.

As we started to investigate how we wanted to implement the project, it was a surprise

to discover how much it is possible to do with media in native JavaScript.

For instance, if the user consents, you can capture their whole screen or Webcam and use

the native recorder API to record the screen, create a video and allow the user to download

it to their machine.

That is possible in any modern web browser.

There is nothing special about Chromebook that lets you do it.

If all we needed was a screen recording app we would be set with those tools but the client

had requested some video editing capabilities.

As a result, we started to get into a realm of things that are technically possible in

JavaScript that is just not a great idea for a variety of reasons.

For instance, if you wanted to create a water mark you can create a media stream that collects

data from a canvas element.

You can then draw your watermark on to every frame of that canvas element and merge it

into the stream from your screen recording but unsurprisingly it is too memory intensive.

The resulting videos were kind of like watching a very badly dubbed foreign film.

You have got all the pieces there but they don't line up.

Another video editing feature we wanted to implement was trimming or shortening the video.

It turns out you can actually do that in plain JavaScript so long as you only want to trim

the end of the video.

If you want to trim the beginning, you are out of luck, at least in Chrome.

That brings us to functionality that simply isn't possible in native JavaScript.

Once you create that WebM file you are done.

There is no way to add a watermark or filter after the fact and it isn't possible to re-encode

the video.

The media recorder API gives you a WebM so if you want a gif or MP4 you can't do that

with native functionality.

At this point, we went back to the client and recommended a server-side only solution.

There are two big reasons for this.

Video editing is a memory intensive process and Chromebooks are meant to do most work

from the Cloud by design.

And client side video editing pushes the limits on what is possible today.

We had two people working on this without an infinite budget.

The client asked us to move ahead with a client-side only implementation eventually.

This set us off on the journey of determining what is possibly limiting.

We determined video editing isn't possible in native JavaScript we needed to find a way

to run a code that is not native JavaScript.

Running non-JS code in a browser has a long history.

Anyone remember this thing called Flash?

Anyone?

It is kind of obscure.

Any ActionScript devs out there?

Any OUBT about Silver Light?

You may have heard WebAssembly isn't just another way to write plugins and that is true

but without understanding more about plugins in general it is hard to know why the comparison

is made.

The '90s are in again.

Let's go back to 1995.

We are all wearing JNCOs and the Slater guy is this guy.

Your browser of choice is Netscape and they just released the Netscape plugin API or NP

API which was the first way to develop plugins or browser extension and handle media from

within the browser.

What happened was the browser would come across content it couldn't handle and detect the

file type and load the appropriate plugins that will run in place on your web page.

As of today, NP API is deprecated in every browser.

That is because the Advent of HTML5 means the browser can handle a lot of things natively

that would otherwise have been handled by plugins.

Plugins didn't actually fully die with NP API.

Google released the pepper API which is a build chain that produces binary file types

native client or pepper native.

It was meant to provide a more secure, less complex alternative to NP API.

You can recompile libraries written in C or C++ and that could be run within the browser

and accessed in JavaScript.

For the WebAssembly fans in the crowd this may sound familiar.

Why aren't we talking about pepper and how it harkens the death of the most widely used

program language?

An open standard for pepper was not created and they were only supported on Chrome, Chromium

and blink-base browsers.

Portable native client is deprecated for every platform except Chromebook but we were building

for Chromebooks so let's put a pen in that.

WebAssembly, like portable native client, takes advantage of common hardware capabilities

to deliver native or near native performance and allows libraries written in C, C++ and

rest into a Wasm format.

Developers can write directly in WebAssembly for the super nerds out there.

Unlike pepper WebAssembly is an open standard and has been adopted by every modern browser.

It is really something that you can use and deliver to the vast majority of the web.

That doesn't actually say so much about what is great about WebAssembly in the first place,

or why we would want to use C or C++.

There are a couple reasons why you might use WebAssembly in your application.

The first of these is performance.

Performance on WebAssembly is faster than native JavaScript and that is mostly due to

a lack of dynamic typing in WebAssembly.

You might look into WebAssembly if you have a particularly heavy operation you could perform

in JavaScript but prefer not to.

Every single page web application framework has a DOM diffing operation.

It is a heavy thing to do and it can have negative performance implications for updating

your page particularly in component-heavy scenario.

Glimmer, which is the DOM rendering engine by the Ember team has an existing build that

leverages WebAssembly for this purpose and that is a methodology that should result in

a much faster rendering engine.

I am definitely not saying that because of the primary architecture of Glimmer is speaking

later today.

There are folks looking into this as well for React.

Performance aside, the second reason you may want to use Wasm speaks to video editing.

You cannot edit video in JavaScript but you can edit video in C++.

You can do several things in C++ or Rest like video editing, platform immolation and games.

Wasm was built for our use case and we needed to choose a video editing library we can use.

It was important to find a library because there were only two of us and we had neither

the budget or expertise to build something from scratch.

It turns out there are not tons of video editing options in C++ or Rust and that may be due

to FFmpeg which is a command line tool that supports adding watermark or tripping or cropping

video or anything you might want to do with it.

FFmpeg could be used through portable native client or WebAssembly and we were unable to

find another good candidate.

The first thing we did was go to the mpm registry and there is an FFmpeg.

I am grateful to the author of this package and the person who forked it and added extra

documentation.

Without these folks, we would have been lost but these were enough bread crumbs for us

to figure out how to do this.

Getting back to my MPM install.

I saw a couple concerning things off the bat.

First is the size.

The entirety of FFmpeg is about 15 bytes and that is a lot to load and a lot to ask of

users to download.

The default build includes tons of functionality we didn't plan on using.

It also includes the H2-64 kodak which was still pat patented.

This is an old version, also.

We came to the conclusion there is an existing MPM install for this library, but we needed

to compile our own Wasm file.

This is the point at which we started to get into technologies that JS devs don't tend

to use on a day to day basis and the first is called Emscripten.

For those that haven't done a lot of this thing it is vaguely analogous to a tool like

Babel.

It takes uncompiled C++ library and builds it in a way that a Wasm is produced.

You can pass in various build options.

If you want to build your Wasm binary in a way it is compatible with web workers you

will do that as part of the overall Emscripten and build command.

That is a lot of high level information.

Let's bring it down.

The following example is a little hello, world written in C. Like most of you I don't program

in C# but the main method is what runs when the app loads.

We imported the standard input and output library.

In a C# application this prints to the console and in our app prints to the JavaScript console

but we will display it in the DOM also.

Now we need to compile our hello world to a Wasm file.

You can do that in Emscripten with the following command.

You can see we do need to specify Wasm and that is because Emscripten can compile to

other formats too.

And dash 0 output to HTML lets us know I want Emscripten to build the page rather than just

compile a binary.

Let's look at the web page.

There is a lot more going on than just LEL hello world.

The default gives you a page with a console you can use to see the Wasm output and it

has JavaScript glue code.

If this was a Thanksgiving turkey drawing tutorial we are at this strep drawing an outline

of the hand.

You know what comes next?

We don't need hello, world.

We need FFmpeg and there are a few steps in between.

It is worth just trying here to convert FFmpeg and see what happens next.

Make is a Unix tool.

And clang is a front end compilation for interface and gcc is the GNU/Unix.

This is the root folder of the FFmpeg library.

We need to use the project's make file and substitute cling and gcc with the own Emscripten

build tool.

Piece of cake.

In case you thought it was cake, let me assure you it isn't.

Emscripten built their own tool to help you substitute for clang and gcc.

It is called M con emconfigure.

We will run emmake make and that should build the library into the format and bit code.

Assuming everything is well, we can run a command similar to the hello, world command

from earlier but of course, everything hasn't gone well.

After about 20 minutes of compilation, emmake make has an error generating deep from within

the heart of this massive library.

Anyone who has struggled to get builds to translate knows the pain that can accompany

production builds particularly as they become more complex.

Building a project in C# isn't different and compiling tat project into a format it was

never intended for adds a greater degree of complexity.

Looking at the hello, world.

C from earlier it seems reasonable to debug the process even though we are not C# developers.

It didn't see reasonable to debug this giant make file and dive in the heart of this legacy

framework.

Emconfigure makes assumptions about your C# base build environment that may not be true.

It was necessary to go in and replace every reference to the existing build change.

Emscripten can't use anything using ASM and that is used in any C# libraries to provide

compiler instructions to gcc.

Since we are not building with gcc we need to pass the option to build without this.

And finally, Emscripten will not convert every possible application.

In the end we prevailed.

Here we are looking at a video that has been created in native JavaScript through a screen

recording of a YouTube clip.

Our media recorder returned a blob we can convert to a file and save to the user's local