Subtitles section Play video
JavaScript: Off the Grid
Sophia Shoemaker KATIE: Hello. Welcome back to the SitePen
track. We have Sophia Shoemaker. And you don't want to live in San Francisco with her because
her grandfather was born right before the 1906 earthquake and she was in the 1989 earthquake.
So, earthquakes follow her family around. If you live in San Francisco, you might want
to move. Or maybe get her to move. Anyways. I am super excited about Sophia's talk because
she's going to be telling us a story about the real world where some of like the hot
topics, progressive web apps, what everybody is talking about, where the rubber hits and
the road and these really cool technologies that we geek out about can actually bring
a positive impact to communities that we want to serve. So, I think this is gonna be a really
meaningful and awesome talk and I hope you're excited for it. So, let's give it up for Sophia.
[ Applause ] SOPHIA: Come on, computer. Okay. I think we're
good. There we go. Okay. Okay. Hi, everybody. I'm Sophia Shoemaker. I work as a freelance
developer of shoes. Just kidding. [ Laughter ]
I'm actually a web developer, a freelance web developer. But I really did make she's
shoes. They're little baby shoes that I made. One of the companies that I work for is called
Fullstack.IO. We write tons of tech books. We have a book about Angular, a couple books
about React. We have a Vue book. The latest book that we're hoping to publish in the next
couple of months is called JavaScript Algorithms and Data Structures. If you would like a beta
version of the book, you can come find me after my talk. Yeah. Okay. So, before I get
started with like the technical side of my talk, I want to give a little bit of background.
So, for various political, social and economic reasons, there are large numbers of orphanages
in many regions of Africa. For example, in Uganda there were 2500 orphanages in 1992.
In 2013, there's 50,000. So, governments have begun to crack down on these orphanages because
in a lot of cases they're not actually helping these children. And in a lot of cases there's
abuse happening so they shut down and suspended or banned international adoptions. And for
the past couple of years I have been working on a project for a nonprofit organization
called Kaeme. They are working with orphanages and the government in Ghana to get these children
out of these orphanages and back with their families. In a lot of cases these children
often have family members alive. Sometimes their parents are still alive. So, they try
to get them back with people that can take care of them or get them adopted. And the
application that I built allows them to collect data and generate reports based on the data.
So, I actually inherited this project from a group of grad students at Santa Clara University.
The first version of the application was built with PHP. Anybody do PHP out here?
[ Applause ] Which is great. PHP is great. But it needed
a little bit of reworking to get it in a better shape so I re architected it to use Laravel.
The Laravel framework. Which was fine. It totally worked great. But the team as the
team started to use it more and more, it was getting pretty frustrating for them because
their Internet connection in Ghana is limited. So, every page load required a round trip
to the server which when your Internet is slow is not a great experience. So, in addition
to speeding up the application, Kaeme also wanted the ability to use the application
offline so that when they went out to the orphanages where there was no Internet, they
wanted to be able to enter the data into the application directly rather than filling out
a paper form and taking the paper form back to the office where there was an Internet
connection and there's the chance of data not getting transferred properly or missing
data. So, in re architecting this a second time, I opted to leave the database structure
alone because the database structure was fine. So, it's I used a MySQL database and left
that alone and focused on the backend code and the frontend code. So, for the backend,
I opted to use AWS Lambda. And I used Lambda for two reasons. One, I am not a DevOps expert.
And Lambda makes it really, really easy to write server side code without having to maintain
a server. And since I was the only person working on this application, not using my
headspace to maintain a server was really valuable. And then the other reason was switching
from an EC2 instance that was running PHP to AWS Lambda significantly reduced the costs
on AWS. So, right now Kaeme is essentially paying for the database because Lambda is
so cheap. And then finally, I used React for the frontend because I am a React fan girl.
And I also needed a single page application. Like I mentioned earlier, making a round trip
to the server every time a page needed to get loaded was getting frustrating. And so,
making a single page application was a really great way to reduce the amount of data and
pages that were getting loaded or the amount of times that I needed to make get stuff
from the server. So, okay. So, what is a progressive web app? I'm sure a lot of you are familiar
with this term. But I just to want run through a couple of like features of a progressive
web app because this is relevant to what I built for Kaeme. And this isn't a comprehensive
list of features. But these are the ones that are relevant to the application. So, obviously
it's progressive. But what does that really mean? It means it works for every user, regardless
of browser choice. Whether somebody's on a mobile phone that has a really slow connection
or an awesome Macbook Pro. It works on all different types of devices. Oops. It's responsive.
Which means it fits any form factor, desktop, mobile, tablet. It's connectivity independent.
Which means it is enhanced with service workers which I'll get to in a minute to work offline
around low quality networks. And then it's safe. It's HTTPS to ensure content is served
securely. So, one of the things you can do to improve your web application is to use
a tool called Lighthouse. It's from Google. And if you're not familiar with this tool,
it allows you to do an audit on your web application to see where you can improve in performance,
accessibility, progressive web apps, all those features that I just talked about. And it
can give you some great insight into where you can improve your application. So, I tried
to run Lighthouse on the PHP version of my application and the audit didn't even finish
running. It stalled out. I don't know if my PHP application was just really unprogressive
or I don't know what. Or maybe Lighthouse isn't meant to run on PHP apps. It just didn't
work. I don't know if you can see, but the text says, as page load time increases from
1 second to 7 seconds, the probability of a mobile site visitor bouncing increases 113%.
So, yeah. it just was not great. But, so, I ran it again
on the new version and this is just a, you know, a screenshot of all the information
you can get from Lighthouse. It's a pretty extensive report. It's a great way to just
get a gut check of what how you can improve your application. Here's just another screenshot
of performance. Just some various things that you can do to improve your application. And
then here's another this section is specifically about progressive web app features that you
can improve on. Okay. So, service workers. So, in order to make any application accessible
offline, you have to make use of service workers. Service workers are JavaScript files that
allow you to cache other resources such as other JavaScript files, CSS, images. Then
they can then be available offline. Service workers only run over HTTPS and make use of
promises. So, let's take a look at how service worker works. So, there are three stages in
the life cycle of a service worker. The registration phase, the installation phase and the activation
phase. To register, a service worker will need to add code like this. The honor block
detects whether or not service workers are supported before trying to register it. Then
the register function registers a service worker which is just a JavaScript file in
our application. And then we have a then and a catch to handle what happens when the promise
is either resolved or rejected. So, this is what our service worker file actually looked
like. It handles the install event and caches the context. We add the install event and
call the wait until method on the event. This holds the service worker in the installing
phase until everything is ready is good to go. Inside the wait until sorry. Inside
the wait until the cache is dot open method is called and that creates a new cache called
v1 which will be v1 of the site's resources. And that can be any name that you want. It
doesn't have to be v1. So, if your service worker has been previously installed and there's
a new version of your worker available on refresh or page load. For example, you've
updated your application. You have a new JavaScript file or new CSS or it's been edited. The new
version is installed, but it's not activated. So, in this code you'll notice that it's the
same install code that I showed you in the previous slide. We've renamed the cache and
we have new assets to install. It's now called v2. So, the new version of your service worker
is only activated when there are no other pages loaded that are still using the old
service worker. And as soon as there's no more pages loaded, then the new service worker
gets activated. So, if you're a React fan like me, create React app does this all for
free so you don't have to write this stuff. So, when you create a new React application,
there's an option to add service workers and cache assets offline. And this is an example
of what your index.JS file looks like in React. So, if you switch the last line, service worker.unregister
to service worker.register, that will allow you to make use of service workers. Another
aspect of this project that was pretty critical, I don't know if this was obvious or not, but
I needed to know when the user was online or offline. The navigator online property
returns the status with true, meaning online, and false meaning offline. So, in the application
I start nod notice a lot of code that looked like this. A bunch of if elses all over the
place. And to make my code more maintainable, I created a function that took two callbacks
as parameters. An online callback and an offline callback. So, using this function, I could
customize what happens when a user is offline versus online. So, when the user is offline,
how do we store data that's getting created and updated? There's a few options for storing
data offline. But first I want to give a few guidelines on storing data offline. Number
one, like, the limit of you have a limit of 50% of the total hard drive storage. Like,
for example, if you have a 500 gigabyte hard drive, 250 gigabytes total allowed for the
browser. And then there's a limit of 10 megabytes to 2 gigabytes per group. A group is an origin
like Google.com. Okay. So, in researching the technologies I needed to use for this
project, I found that there was a few different options out there for storing data offline.
There are a lot out there. But these are the ones that I became familiar with. Familiar
with for this project. One is PouchDB in connection with CouchDB. PouchDB is a library that syncs
with CouchDB, a NoSQL database. It makes going offline. If you have a user that's editing
data offline and a different use offline and there are merge conflicts, PouchDB allows
you to gracefully handle those issues. So, the con in using this for my project was that
I would have to migrate my MySQL database structure to a NoSQL format which is something
I'm not familiar with. And I have been burned in the past biting off more than I can chew
with refactors. I wanted to leave the database along so I opted not to use PouchDB with CouchDB.
And on top of it, the application the data was really meant to be relational database.
It didn't really fit the NoSQL format of things. Okay. So, another option is IndexedDB, a low
level way to store significant amounts of data including files and blobs. And this is
from the MDN docs. IndexedDB API is powerful but may seem too
complicated for simple case. If you'd prefer a simple API, try indications such as that
huge long list that make IndexedDB a little more programmer friendly. I read this and
said, hm, I'm not going down the path. I'm the only developer on this project. This looks
a little more complicated and I don't have seven, eight, nine, ten months to work on
this. So, I opted not to. But I think if I went back and re architected the application
I would go down this path of using the IndexedDB because it fit better with the data that I
was saving offline. But ultimately I ended up using localStorage. So, localStorage is
a property, and the stored data is saved across browser sessions. It's a key value store that
has a set, get and remove item methods on it. So, only strings can be stored in localStorage.
So, if you want to store anything other than a string, you have to turn your data into
a string. One caveat is that while storage limits for localStorage are larger than using
cookies, you can't store like gigabytes for the user unless the user allows for it for
the domain. So, typically localStorage is limited to 5 10 megabytes. Here is an example
of how I used localStorage in the Kaeme application. I was using JSON, user the JSON.stringify,
and then I would pull them out of local storage and turn them back into JSON objects. So,
in order to properly save a child's record online and offline, I used my online offline
function I showed you a couple slides back and saved it via API to the database or updated
the Q value in local storage. And then any time the child's record is saved offline,
I set a flag in the local storage record indicating the record had an optimistic update. You can
see right there. Okay. hopefully this demo goes okay. Let me go back. This is what the
Kaeme application looks like. Let me get some test data here. You can seeing right here
this first record here. Actually, let me go back a little bit. So, this black button right
here allows me to download a particular record and save it offline. It might take a little
bit of time. We'll see how long it takes. See if the Wi Fi works okay. Come on. Great.
Okay. So, now it's gray, that means that the that record particular record is saved
offline. Right now I'm online. so, if I go in and edit, I can just change hello, hello.
And then I'll save it. And then if I go back up here. And if I go up here and then turn
off Wi Fi. You can see now that the only records that are showing up are the ones that I've
downloaded to my application. And so, then I'll go in here and edit this one again. Earth
are not great. Okay. So, then if I go back. So, I have these two records here that are
highlighted red which means that there's been data that's been saved offline. So, then if
I go back online. Get connected again. And then we go back. If guy in here, it says this
child has updates that were done offline. So, riff the option to upload the offline
data. Or I can opt to not upload that data if I wanted to. So, yeah. Let me just go back
to here. Okay. So, yeah. That's how it works. [ Applause ]
Okay. So, in this process there were quite a few challenges and frustrations that came
with this project. Dealing with offline applications is not an easy task. At one point the team
in Ghana was having some issues with the application and I couldn't reproduce the issue on my end.
It always works on your own computer, right? So, we had a Skype call with them and they
were literally holding a laptop up with the, you know, the application running with the
computer that was offline. Because you can't do a screen share with the computer that's
offline. So, I had them opening like Chrome Dev tools and inspecting the elements and
clicking inside, you know, trying to figure out what the problem was. So, like the time
difference, the language barrier and the technical capabilities of the team made for some interesting
challenges and required a lot of patience on both ends. And we're still working out
some bugs. But in general it's a much better experience. So, this is the Kaeme team in
Ghana. So, in thinking about how to build this application, I knew that I could impose
a few constraints on how the offline interactions work. The Kaeme team in Ghana is very small.
There's just four people working on it to gather information about the children. So,
it was easier to manage how the users would be allowed to use the application. I knew
that there would not be a risk of a user potentially editing a child's record online and then somebody
else editing the record offline because they all work together. Obviously, if the team
scaled up in any way, I would have to rethink my approach. And there would be different
constraints. For example, as a number of users scaled up, maybe I would have a child worker
assigned to a particular user to ensure that the data would not get overwritten or maybe
I would consider using PouchDB with CouchDB so that I could handle the conflicts there.
And then finally one last thing, I just wanted to share something that I have seen floating
around the Internet in a few places. It's this Japanese concept of ikigai. If you happen
to speak Japanese, you can come up to me later and correct my pronunciation. Everyone according
to Japanese culture has one. And discovering it brings satisfaction and meaning to life.
When you combine these four things, what you love doing, what the world needs, what you
can get paid to do and what you are good at, you can find your sense of purpose, your Ikigai.
For me, this project has been an Ikigai experience. I loved stretching my engineering skills and
feeling like this project has provided more value to the world than some of the other
projects that I worked on. So, I hope you can all kind oh. There's the Ikigai. It's
in the center of the circle there. I hope you can use JavaScript in helpful and useful
ways that go beyond the typical ways that we think of creating web applications. I hope
you can all kind your Ikigai experience with JavaScript. And that's it. Thanks.
[ Applause ] KATIE: Hello, wow. That was so good. How many
of you feel like your current job is your Ikigai? Yeah. I can I can't see much. I
don't know how many people actually raised their hands. I'm sorry. That didn't work as
well as I'd hoped. Anyway. Oh, my goodness. It's gonna be a great day today, folks. I
hope you're ready. So, coming up next here in the SitePen track we have Lara Schenck
is going to talk about CSS algorithms which might seem like a contradiction in terms,
but actually is not. So, we'll all meet up back here at 11:15. And I hope to see you
then. Take care. [ Applause ]