Subtitles section Play video Print subtitles PAUL: Welcome to this session on Performance Culture. I am Paul, on the left there. I work on Google's Developer Relations team where I spend my time looking at performance, design, and UX and normally where the three of those meet. LARA: And I am Lara Swanson. I work at Etsy and I'm the Engineering Manager for the Performance team. We help all of the feature teams who are building products or features or experiments, make sure that whatever they're shipping is as fast as humanly possible. PAUL: Alright so, I guess the opening question is, why are we even talking about performance cultures? Why does this session even exist? And I think to answer that we need to understand a little bit about where our industry is and what we are seeing and where we think it's going to go. And it's really about this. The multi-device web. We've seen more devices than ever before come online and I think really the one that stands out for me the most is this one. It's the smartphone. It represents the most performance constrained device that we have. Whether that is in terms of the screen size, or the GPU, the CPU, or its connection to the network. And the thing is more and more people are using smartphones. In fact, one study suggests that just over a third of American adults use their smartphone as the primary means of accessing the internet. Which means for those people their first experience of you, your site, your brand is going to be through their smartphone. And that is something that we need to be thinking through. In fact, it's not just in the States it's also globally. So, here we have the percentage of total global internet traffic according to Stat Counter and you can see that the trend is pretty clear upwards. And it looks I think sort of the middle of next year, if the trend line is correct, that well cross that 50% marker. So the thing about this is-- LARA: Mobile networks can add a tremendous amount of latency. So, before a mobile device can transmit or receive data it has to establish a radio channel with the network. And this can take several seconds over a 3G connection. After the device talks to a radio tower to negotiate when it can transmit data, the network carrier then must transmit that data to its internal network, and then to the public internet. So, the combination of these steps can add up to tens of thousands of milliseconds of extra latency. So, the most important question I hear when I say this is, well what about 4G? 4G is awesome right? Our networks are improving. But you have to remember the new device and fast network with an excellent connection that you have in your pocket right now is not a good representation of what your end users are really experiencing. So, while it's true that networks are getting better you need to remember that your users are on a variety of connections, with various levels of connectivity, and latency. It's also important to remember that if you're a site is slow people will go elsewhere. In fact, one study suggests that as many as 40% of people will leave a site when it takes longer than three seconds to load. So, we have some options. In the face of the stuff that's a little bit scary what can we do about it? The first option is to ignore it. You can certainly say, you know what we just saw those awesome stats. We say that we're actually-- we're trying, but we don't think it's that important. Maybe mobile isn't the future, maybe that trend line is going to lie to us, maybe networks will be better immediately, I don't think that this is true. PAUL: No, I tend to think-- no. LARA: I think that we can't just ignore it. PAUL: I'd like to. LARA: Yeah, it'd be really cool, but I think the other option is to assign performance cops. Right? Who in here would identified as a performance cop or janitor within their organization? PAUL: Anybody ever been one of those? LARA: Yeah, somebody who comes in afterwards cleans up. PAUL: Look at these hands. Look at this. [DEEP VOICE] Yes that's me, I don't know. LARA: So, performance cops, right? We come in and we say, hey designer, hey developer here's some better ways you can do this. You come in cleaning up after people, you try to make your site better, but the responsibility solely rests on you. And this can often lead to a tremendous amount of burnout and frankly, it's not sustainable. There's always going to be new people joining your organization, your site will continue to get slower, it'll continue to be iterated upon, the hardware age, being a performance cop is not a sustainable thing, so that's why we're here to talk about building performance cultures. PAUL: Wow, I wonder which one we'll choose from that list? [LAUGH] PAUL: So, we're all set to build a performance culture, super. But we have to ask the question what do we even mean by a culture. Well I guess you could come up with your own definitions goodness knows there's probably a bunch of them that you could think up, but for us today, at least in this conversation, here's the things that we actually have in mind. Firstly, it's a way of saying, I belong to this group, I get them, hopefully they get me, I understand this particular group. So, for example being British, I would identify with British culture. LARA: You're British? I know right it's a relief for anybody in the room going, where is he from? His American accent is really strange. LARA: [LAUGH] Yeah, I know. So, it's that first and foremost is just saying, yep this is my kind of people. It's also about how you think and how you reason and rationalize the world around you and the kind of social cues that you look for or the slightly strange obsession with green liquids you may or may not have, but it's that sense of the world around you. It's also how you do things. Whether that's the side of the road you drive on maybe the wrong side, the words that you use for things or perhaps in our case how we go about crafting the code. And then lastly, it's how you celebrate things, the things that are important to you, the things that you celebrate and how you go about celebrating it is unique to your culture. And we see cultures at the highest level internationally, nationally, all the way down to our homes and so forth, and actually of course, our workplaces, which brings us back to that whole performance culture thing. LARA: Yeah, we were joking when we were talking about earlier that performance cultures are kind of a team sport. It's important to remember that everybody has to play. And as you say about sports there is no "I" in performance. PAUL: That is true, but if you're willing to look hard enough you'll find there is "prance-for-me". You can stop the slide now. LARA: Alright, alright. So, thank you for that. So, culture change is scary and it's hard and I can understand especially for those of us in this room, we don't know necessarily how to approach it. How do we start to create or enact performance culture within our environments? I'm going to go through some real things that have been said to me as I've gone on this journey within the organizations I've worked in. Lara, "I don't want to think about mobile." Right. PAUL: So whiney! LARA: Yeah. "Lara, I'm a project manager or a product manager and I've look at the deadlines, I've got some resources and frankly building for both desktop and mobile is going to take twice as long." [WHISPER] It's a lie. "Lara, you at the performance Manager, don't you know responsive web design is inherently bad for performance?" Or my personal favorite, "everything changes too fast!" So, here's the thing we've been through this before and when it comes to web development we've gone through these kinds of change the past. Who here has built sites with tables for layout? Yes. PAUL: Yes! Look at all these guilty hands. [DEEP VOICE] I don't do it any more. LARA: It was a pain. So, at probably every job I worked at between 2006 and 2010 I remember these arguments, do we have to move to a CSS based layout? In fact, we did. We did it pretty successfully. This is more like what we would use today, as is actually said to change again with web components. Who has heard this from a very important person, excuse me, the important part is not above the fold. When the EP's would have to scroll to see content. Yeah. This is again what we're seeing today. We used to worry about 800 by 600 pixel monitors, now we're worrying about the spectrum of screen sizes. This graphic represents all the devices that come to ETSY.com that have more than 1,000,000 visits per month in traffic. We're no longer seeing just handsets and tablets and desktops we're seeing a spectrum of screen sizes. You may also think about how we've changed developing for different browsers over time. I remember back in the day we would talk about cross browser compatibility or making things look the same across browsers. Browser hacks, prefixes, et cetera and this is again what we're seeing today. We have to start thinking about mobile web browsers in addition to just our desktop clients. But this is all to say that change is scary, but it's also constant in our industry and it's good. This is a good part of our jobs. We've lost crazy table layouts and now we have semantic markup and more people than ever before can use our sights on our apps. This is good for us as developers and it's really good for users. PAUL: So, we need to embrace change, good, fine. So, where do we start when we want to create one of these performance cultures? Well, the first step for us is to actually gather some data because essentially you don't know where you're going if you don't know where you are and you need to understand how real people use your site, like what devices are they on? Where are they in the world? What journey did they take through your site? Which pages did they arrive on and which ones did they leave on? You need to understand your own situation and if you don't know today you can't start telling that story to the people around you and get them on board to build this culture with you. So, the immediate thing that springs to mind is just going and have a look at your analytics. Go and look at Google Analytics. Figure out what it's telling you. Spend a bit of time, make reports, and start to get that sense of where you're at. The next thing you want to do is actually look at you load stats. So, how long does it take to load it? How long does it feel like it takes to load your site? Again this is coming down to the data that you can share, but it's also in this case coming down to a little bit of empathy as well. How does it actually feel to be one of those people that's using your site on a day to day basis from somewhere else in the world. And for that we can turn to Web Page Test and you can ask it to load your site from a range of locations, a bunch of devices, and different connection types. And it is going to give you a load of really useful, really amazing data. And one feature I particularly like is the ability to compare pages, so you could take for example your own site and oh, I don't know, the site of your co-presenter and see who's fastest. LARA: Not cool bro. PAUL: Oh yeah. I actually did that, as you can see and depressingly it's a draw. So, yeah. Could have been bad, but it wasn't. I'm sure you can start to see how you could start figuring out the story here. Like how does it feel to be one of our users when you compare it to somebody else in the world or-- da-duh, da-duh, da-duh you get this idea. OK. So, you start to get in this insight and you don't even have to leave your desk. That's the best bit. But we can get even more data at Web Page Test and we can share that with our team. Like how long does it take before our server sends some data back or how long before our pages start rendering? But one number I want to call out is this, which is the speed index. It's a bit weird the way it's calculated. You can click on there there's a link. And you can click on it and it will tell you, in all the gory detail, how it figures it out and it's pretty cool. LARA: It's magic. PAUL: This is magic. But it is a really good number for figuring out in milliseconds how long it feels like it takes for your page to load. So, that's a really good measurement, better than say, the onload event. But speaking of perceived performance, we don't just want to stop caring when people-- when they've landed on the site and we go oh, what have we done. We need to care about scrolling and animations and interactions and they all need to be this silky smooth 60 frames a second because people care about that. They want that feel of responsiveness and smoothness. Again you go to like Dev Tools to look at your pages for data and figure out what your frames per second is like. So we get frames per second, we can dive into the frames a little bit there and find out where we've got issues and then start fixing them. Now if all that's new to you it's time for a shameless plug, which is to say I've got a session tomorrow morning that will tell you exactly what you need to know about these tools, "The Tools You Need to Succeed." It rhymes, it's true. So, come to that if you're interested. Anyway, hopefully at the end of this situation we have a load of information, we have a lot of data, and we're starting to hopefully build this story in our heads of what it feels like to be one of our users. LARA: So, you've got this data and you're excited about it. What do you do with all of that excitement and all of that data? You should go talk to some very important people. You want to enlist the help of the very important people at your company and get them to understand what your users are feeling and experiencing. There are few things more powerful than a very important person saying, this is important. Performance is important. It's crucial to getting others at your company invested in performance. So, what can you do to make this happen? Here's an idea that we came up with, you should send your very important person one stat a day for two weeks and see how long it takes before they cave or just get really annoyed with you. One that we often like to cite is this Google study that found an additional 200 milliseconds of delay triggered a 0.3% loss of engagement. You may say to yourself, 0.3% is not a lot, but Google is pretty popular. So any kind of loss of engagement is going to have a huge effect. Or remember that old stat that we just had where 40% of people will leave a site that takes longer than 3 seconds to load, tell your very important person that stat and say, by the way our takes 5 seconds to load on a 3G connection. So, after the stat thing has run its course consider running some experiments. Measure the business impact of these experiments. Know what kinds of metrics your very important person cares about. Maybe it's balance rate, maybe it's conversion rate, maybe it's something else. Go ahead and run performance experiments and see what the effects are. On Etsy, we ran an experiment in which we added 160 kilobytes of page weight to a page on a mobile device and we saw that it increased mobile bounce rates by 12%. This is the kind of thing that we can share with the folks at Etsy and say, hey, this stuff is important look, at the effects that it has. However, this is kind of a negative stat and I kind of prefer to make a positive. So, what we found when we eliminated jank, that's that scrolling issue, when we eliminated jank on Etsy's activity feed we saw people "favorite" more often, and "favorite" more items. And this is another kind of experiment that you could run, make something better and measure the effects before and after and pick and choose the engagement metrics that you know your very important person is going to care about. You could also use that web page test film strip, thanks for that-- PAUL: Your welcome. LARA: --and take the video output and compare that before and the after the experiment, make an improvement, show how slow it was before and show how long-- how short it took afterwards. And get that person to care and feel about how this works. Or compare your site on desktop to how it loads on mobile or compare you're site in the United States to how it loads globally. Get your very important person to see and to feel these things so that they actually care about this stuff too. You want to get them to feel it. Once they understand you can help them empower others. By changing how you're very important people think about performance it will help everybody who's surrounds you. PM's, designers, developers care and work on performance. PAUL: Speaking about those other people, so, we've got data, we've got our VIP's to say, OK performance is important, we see that it's got real value. How do we then make it so that the people around us can actually-- start actually taking this and using it. Well, the easiest way is to educate them through something like a brown bag, lunch and learn session on all the tools, all the data, all the things that you've figured out about how you're actually doing. And it's not-- as we said it's not just developers, it's designers, is UX, it's project managers, it's anyone who's involved in what you do because it's a team sport. And we want to make sure that the people that we're working with know the impact of the decisions that they take, so it's not the case of, well the developers have done all they can, but actually the problem is originated from another place. You know if you include everybody you've got a much better chance of success. The next thing that's well worth doing is to start documenting your best practices and start writing things up. Because what gives you is an easily copy, pasteable set up that you have. It makes it more obvious about how to do the right things as far as you're concerned. Anybody who joins your company is going to start figuring out, oh this is how I'm supposed to do things, rather than just having to guess and muddle their way through. And you can use it to show what worked and what didn't work and if nothing else it kind of gives everybody a focal point and say, right this is-- because you will find there will be trade offs in everything you do. This thing and doing it this way works because of this. So it's a really good place to sort of-- it's really good to start writing these things down. Another thing that is definitely worth doing is seeing what you're uses are seeing, which is to say invest in a device lab. And I mentioned this to somebody the other day and they went, "psha $10,000, really?" Because that's what they were assuming it was going to cost. Well first of all it doesn't necessarily have to cost that much. Secondarily, the cost of not testing across multiple devices may be way higher and so, that's something that you want to decide for your selves. But it's a sort of-- it's an out of sight out of mind kind of problem, this one. If you put these devices in front of everybody it kind of gives everyone that reminded that there's this whole range of devices they should be testing on and that they are building for the multi-device web and that they need to care about performance on all of them. It's also a social place as well, so people can meet and gather around these devices and start to figure out together how you can solve particular problems and again that will broaden the horizons beyond just the developers. One thing that is pretty cool out in the Design Sandbox, if you haven't seen it we have a Device Lab. And the team there will talk to you about how you can go about making your own device lab. The source apparently is on GitHub because it does some crazy stuff where it which is to all the devices at the same time, which is really cool. So go and see the Design Sandbox people they will help you. LARA: And then also if you go to laraswanson.com/devicelab you can see all the slides that I just gave for a tutorial on how to build your own device lab. How to choose devices, how to troubleshoot power, and all of the other things that we've learned along the way. PAUL: Awesome. Alright, so, you-- it's great to have the device lab, but it's not great if you don't use it. It's a bit like having a gym membership and they never getting on the treadmill. Yeah. So, make time to actually test your stuff for real. It's good you've hopefully got the VIP's to say it's important so you can say actually we need time now to make sure that things are working well. So go and give your devices a big old hug. Let them know you love them. Like we said earlier though educating your team isn't just about developers and performance isn't just about an internal thing some of it-- sorry it's just not an external thing it's also internal, get that the right way around. It's about everyone who builds for the web in your company. And that means in a lot of ways that you want to eliminate duplicate code, you want to basically embrace responsive. Instead of having a mobile team that does the M-dots and you know the other team that does all the other stuff. M-dot. What do you do for TV? Do you do TV-dot? LARA: What do you do for refrigerators? PAUL: F-dot? LARA: R-dot? PAUL: R-dot. LARA: I don't know. Don't do that. PAUL: No, don't do that. That seems like it's not going to scale well. LARA: No. PAUL: OK, you said earlier they do taxies. LARA: Oh yeah, taxis in New York have the little-- you can access the internet. Mind, [SHOOTING SOUND] blown. Alright, point here is your going to eliminate duplicate code, you going to make it easier for your team to iterate more quickly, hopefully, and start to break down the barriers between you, the designers, the developers, and it just gives you a much greater chance of success. So, in summary, when you've got the green light from your VIP start teaching your team about performance and the impact of their work, it's going to empower them. Sorted. LARA: So, you need to start surfacing performance within People's daily lives. We can talk about performance all day about how important it is and when you ship something new you can see the end of the day how much performance was impacted negatively or positively. But this isn't help people as they're designing or developing. So, the first thing you can do is add performance to your build tools. Automate image compression. Get a command line tool running like Grunt or Gulp that runs performance tests and compares your new work to your performance budget. You could use something like Travis to do the in a continuous integration environment. And these tools can even report back to dashboards, which is my next recommendation. So, at Etsy we look at something like this every time we deploy code, which happens upwards of 50 times a day. When someone deploys something new they'll go over to a dashboard and say, how did this impact performance? It's really important that as people are continually pushing out new work they can see immediately how their stuff has affected it. But also at Etsy we have this tool bar that sits at the top of every page of etsy.com if you logged in as an Etsy employee. And in this we've got information about visitor traffic and experiments, but we also have performance data. So, we surface the timing's for people as they're developing in developing, staging, and production environments. And we also have a clear baseline for what is acceptable. We'll let you know if you're violating one of our performance service level agreements. By surfacing your team's performance data during the workflow it'll help improve their work, it'll become baked into their workflow. They'll start to care about it because they can see how their work directly impacts it as they're designing and developing. And this brings me to our last point. This stuff is going to make people so excited and people are going to ship things that are improvements to your site. You should be celebrating everything. Again, I'm all about the excitement. I'm all about getting people pumped up about performance. PAUL: Prepping the talk with Lara was like an object lesson in excitement. LARA: Nice. So, for me, this is actually the most important part. Back in the day in August of 2011 the man behind the sunglasses, Seth Walker, published Etsy's first performance report. He published it on Etsy's news blog, so it's super public. He chose a bunch of baseline business metrics or page load time metrics, rather and he showed how long are our major pages were taking to load and you can see there's a huge outlier here, the homepage. It was embarrassing, but he published it anyway because this stuff isn't a secret. What's amazing about this is that developers teamed up to fix this. As soon as we publish it, as soon as Seth put that into the world people got excited about it and they wanted to help and they want to fix it. And by the next time we published reports, which was in November of 2011 there was a huge improvement. Not only did the home page significantly improve in page load time. It was the fastest one on this list. So, you should be sharing your findings. Make sure that you're celebrating externally the stuff that you're doing to help improve things along the way, but also as Paul mentioned it's important to do this internally as well. This is an example dashboard that we have, the Performance Hero a dashboard, in which we celebrate engineers and designers who make huge improvements to Etsy's page load time and they're not on the performance team. In this case Chris Fairbanks, who's on the checkout team, eliminated a ton of duplicate code and had what we call a Baumgartner. Remember that guy who jumped out of the hot air balloon and landed on earth? Yeah so every time we have a graph like this-- PAUL: It wasn't a dance move, right? LARA: It wasn't-- no, it wasn't a dance move it was a plummet to earth. Every time we have one of these, a Baumgartner, we celebrate this internally. We put this up on a dashboard and we have that little printout sign, and we hang out with them and we're like, thanks. Thanks buddy. And we want to celebrate the things internally lots of high fives, lots of doughnuts, lots of cake. You should be rewarding your innovators. It's not just about coming down with a hammer. It's not about being a cop or a janitor. This is really about celebrating improvements and getting people excited and invested in their performance. Celebrating wins will motivate your team to be performance champions. So, to summarize gather the data. You want to make sure that you have a baseline of understanding how your users feel. Again, this is all about understanding the overall user experience. It's the aesthetics and it's page load time. You then want to share all that stuff with your VIP's. Get them to understand what your users are really experiencing. Get them to understand what it's like on mobile and globally. Get them to understand also how this stuff impacts the business metrics, run some experiments and share that data with them. Once you get a very important person to say, this is important or just a few find that it's awesome enough to say this is important, educate your team. Do those brown bags and lunch and learns. Make sure that your surfacing that data in their daily work flows. Make sure also-- your celebrating things. Again this is a journey right? You may be at any step in this process and you might have to start again gathering more data, getting more VIP buy in, educating the team again. New designers and developers will join and you'll have to do more brown bag lunch and learns. New technologies will emerge and you'll want to share that data and that new information with them. You'll find new wins and you want to celebrate that stuff. This is a continual journey and continually positive process. PAUL: OK. So, as we said at the start we're in this period of dramatic change where we're seeing mobile traffic go, whoosh straight up, and we're facing this world where essentially we need to deal with these performance constrained devices like building for this tomorrow that we're looking at involves facing that fight and saying you know what most of the devices that people are going to be using are not desktop, they are not powerful devices. The thing is it's not really tomorrow it's kind of now. And that's the thing about dealing with performance is it really starts and it can and should start today, but the only way you're going to get to that point is if absolutely everybody in your organization from the top to the bottom cares deeply about performance and its impact. And the only way you're going to get there is if you start building performance cultures. Thank you. We'd love your feedback. Please do snap the code or take down the URL. And we'll be outside if you want to ask any questions and say, hi. LARA: Come hang out. PAUL: Yeah. LARA: Thanks everybody. PAUL: Thank you.
A2 lara performance paul etsy data load Google I/O 2014 - Perf culture 69 13 Hhart Budha posted on 2014/06/16 More Share Save Report Video vocabulary