Placeholder Image

Subtitles section Play video

  • (audience applause)

  • - Hi everyone.

  • So I know this is probably a huge cliche since probably

  • every single speaker says this, but it's a huge honor

  • to be here.

  • I was sitting in your seats about 10 years ago which seems

  • like not that long and yet quite a long time.

  • My class, the class of '06 just celebrated our 10 year

  • reunion here like a couple months which was a lot of fun.

  • So it's really really great to be here.

  • I remember when I was sitting in your seats and you know

  • I had just come from Texas, that's where I was raised

  • and I had done some web design in the past,

  • but I didn't really know anything about Silicon Valley,

  • didn't know anything about this kind of tech hub

  • and what it means to have a career here and startups

  • and all of that prior to coming to Stanford.

  • And I remember sitting in the audience and listening

  • to the stories of people who had come before me

  • and in particular Marissa Moyer and Jeff Fitzpatrick.

  • They'd stand up here and they would take about their

  • adventures after leaving Stanford and all of the great

  • things they got to go and build and work on.

  • And I was just like wow, this is super super exciting.

  • And you know, now I'm excited to be here talking to all

  • of you guys and hopefully sharing a couple of my stories

  • along the way.

  • So as you guys heard, I was here, I did my computer science

  • degree, my bachelors and my masters and I was also lucky

  • enough to be part of the Mayfield Fellows Program.

  • So how many of you guys know about

  • the Mayfield Fellows Program?

  • Okay so for those of you who don't know, this is an awesome

  • program that is really about diving into entrepreneurship

  • and so one part of the program is that between your junior

  • and your senior year, you go and get an internship

  • with a startup somewhere in the Valley.

  • And along the way as we're interning at our respective

  • companies we're learning about entrepreneurship, we're

  • taking classes we're doing case studies.

  • And it's fabulous, it's awesome, it's taught by Tina.

  • I had a lot of fun doing it.

  • So at the time the start up that I chose was actually

  • Facebook and you know this is a little bit of a little

  • cheating because in 2006 Facebook was already

  • about 100 people, so it was a startup but on the larger end

  • of the startup.

  • But it was something I was really excited to work on.

  • It was a product that I and all of my classmates at the time

  • used very religiously.

  • So I went there and I remember my first day at Facebook

  • in fact, I was signed up to be an engineer and work on

  • some of their photos features and I had a mentor

  • and her name was Rudgy and on the first day she said,

  • okay there's been a change of plans I'm switching jobs,

  • I'm not going to be an engineer anymore, I'm going

  • to be a PM, I don't really know what that means for you,

  • but let me introduce you to this pod of people over here.

  • This is our design team, why don't you sit with them

  • and you know, chat with them.

  • I've got to go run.

  • And so she left me in this pod with this group of designers

  • and before that, I'm not a designer by trade, I didn't

  • really know anything about design, but I got to talking

  • to these people and I sat at that pod.

  • And low and behold, fast forward 10 years

  • and now my career's in design and when I look back on it

  • it sort of seems really quite random how it happened.

  • But for me what was so exciting about design was the chance

  • to work at the forefront and thinking about what are

  • the people who are going through the experience

  • that we're building, what are they feeling,

  • what are they thinking, how are they able to understand

  • the products that we've built.

  • And so to me that's what was super super fascinating

  • about design and I also wasn't that crazy because

  • at the time the designers that we hired were also

  • our front end engineering team, so it was a little bit

  • of both designing and a little bit of coding.

  • This is back in the world where we were still operating

  • with just websites, so you just had to do some CSS

  • in JavaScript and front end PHP and it wasn't nearly

  • as complicated of an ecosystem as it is today.

  • But we got to building and one of the first things I worked

  • on was photos and photo products, I also remember one

  • of my first big launches a couple months later,

  • which was Newsfeed back in September of 2006.

  • Over the years I've worked on things like the Facebook

  • platform and ushering a bunch of applications built

  • on top of Facebook including a lot of games,

  • back in the era of 2008 and 2009.

  • I worked on Profile and Timeline and I worked on Newsfeed.

  • And today the team that I lead, we work on design

  • for all of the core features of the Facebook application.

  • So when you go on Facebook and you want to catch up

  • with your friends, you want to share, you want to watch

  • videos or join a group, those are a lot of the things

  • that my team works on.

  • So all that said, that's a little bit of an intro

  • into kind of how I got here.

  • The thing that I think has been the most fascinating for me

  • as I look back on all the things I've worked on

  • is the moment when you're starting a project and you're

  • thinking about here's this awesome new idea that I have

  • and I can imagine how it's going to work and how people

  • are going to use it and I want to build it right now

  • and you rally a bunch of other people together with you

  • and you guys are all excited about this idea.

  • That's how everything happens.

  • That's how any idea ever sees the light of day.

  • But the question that I found super fascinating, is how

  • do we know at that point in time whether this is an idea

  • that's going to be successful?

  • How do we know after it's all said and done and we've

  • worked our nights and weekends and we've gotten it out

  • to the world, will it actually be something that people

  • find valuable and that they'll find easy to use

  • and that they'll find well-crafted?

  • And being at Facebook, we've had our share of features,

  • some of them that have gone on to be great success,

  • and some of them that haven't.

  • And a lot of times at the beginning, you know, it feels

  • like there's really no way to tell.

  • It feels like you're kind of rolling the dice a little bit.

  • And this is the thing that I wanted to really study

  • and reflect on over all of the different products

  • that we've built.

  • Is what were the patterns for the things

  • that were successful, are there ways that we can tell

  • as we are building whether this is something that's going

  • to work or not.

  • And what came out of that, and a lot of discussions

  • and a lot of postmortems and a lot of just looking at

  • the things that we had built, was a framework of just

  • three simple questions at Facebook that we now use

  • to ask ourselves whether the things that we're building

  • feel like that they're on track.

  • And they're three very simple questions because you know

  • it wasn't going to be a manual that everybody was going

  • to memorize and understand, but it was three questions

  • that we wanted everyone at Facebook, no matter what their

  • role was or what they worked on, whether they were engineers

  • or designers or product managers, to keep in mind

  • when they're having a team meeting or whether we're

  • reviewing the product or whether they're actually just

  • talking with a colleague about an idea to just think

  • of these three questions and to ask them.

  • And to make sure that we have really really good answers

  • to these questions.

  • So that's why I'm going to talk with you guys about today.

  • So the first question is the most basic and the question is,

  • what people problem are we trying to solve?

  • And the key word here is really the word people.

  • Because of course whenever we build anything we're trying

  • to solve the problem.

  • But what tends to happen is that you start to think

  • in the mentality of your team or your company and you start

  • to say things like, the problem we need to solve is that

  • we need to optimize the click through rate of our page.

  • And you know, you'll hear things like this all the time,

  • boil down in small ways and in large ways.

  • And that's not a people problem.

  • A people problem as we define it, is if you go out and you

  • talk to someone on the street and they were to articulate

  • a problem that they were having, that's how they would

  • say it, that is the people problem statement.

  • So there's a couple things that we look at to make sure

  • that this is a valid people problem statement.

  • The first is that it needs to be human and straightforward.

  • So we're not using words like CTR, we're not using words

  • like optimize or integrate, like these are not words

  • that people on the street would use, these are not words

  • that people who are outside of the tech community

  • are going to use to talk about their problems.

  • The second thing is we want to make sure

  • that it's solutions-agnostic.

  • A lot of times we start problems by saying I am going

  • to build an app that blank, or I'm going to design

  • a website that blank.

  • And already in that statement is an inkling of what

  • the solution is, but what if you know the app's not

  • the right way or what if it's not supposed to be a website?

  • So a good people problem statement gets away from trying

  • to already constrain it into a particular solution.

  • The third thing is that it shouldn't be about you know

  • Facebook or your company or your team or whatever winning.

  • And I think a classic example is if you say, our service

  • is going to be the best at blah blah blah.

  • A person on the street doesn't care if your service

  • is the one that is the best at that, they just want to know

  • hey for this problem that I have, what is the best solution?

  • So stating any problem as we, our team, our company,

  • whatever wants to win, is not getting close to the things

  • that people actually want.

  • The fourth thing is that it gets at the why, so sometimes

  • you might have a problem like people aren't discovering

  • this page and that is a problem that we need to solve.

  • But it doesn't go one layer deeper, it doesn't get at well

  • why, what's actually the root cause of why people aren't

  • discovering this page.

  • Is it too hidden for them, is it too confusing for them

  • to find, it's gotta get to not just what is happening,

  • but what is the reason why it's happening.

  • And finally a good people problem statement can not just

  • solve functional problems like a confusing flow,

  • but could also get at emotional or social problems.

  • Sometimes people just want to feel like they belong,

  • sometimes people just want to feel like they're validated.

  • Those are things that also constitute something that

  • when you ask people they might say.

  • So let's look at some examples.

  • So earlier this year I had the privilege of attending

  • the Mom 2.0 Summit and this is a gathering of a lot

  • of influential bloggers and journalists who are mothers

  • and who talk about parenthood and those topics.

  • And so I got a chance to meet a lot of the attendees

  • and be on a panel with some of them.

  • And one of the things that I heard over and over again,

  • was basically this, and this is an example to me

  • of a people problem statement.

  • I want to talk about an interest with other people who are

  • also interested, but I don't know where to find them.

  • And this is especially relevant at this conference

  • because a lot of times what would happen is you're a new

  • mom and a baby, in your frame group, nobody else is having

  • babies around the same time as you.

  • But at the same time there's so many things that goes

  • through your mind you want to know is this normal,

  • my baby just did this, like should I be worried,

  • what are some resources?

  • You want to have a community of other new moms to be able

  • to talk to and so a lot of times women will not know

  • exactly where to find that community and if they're lucky

  • friends or other people will say hey there's a local

  • group here or there's a neighborhood mom's group,

  • but this is something that a lot of people in the audience

  • were talking about, is something that was a problem to them.

  • So this is an example of a people problem statement.

  • Here's another example.

  • This one we've heard over and over again, ever since

  • we introduced the Like button on Facebook.

  • Everybody would ask why don't you guys have a Dislike button

  • and that would be one of the topmost requests that we've

  • had for years and years.

  • But you know that's not really a people problem statement,

  • that's sort of just a suggestion.

  • If we dig deeper, the reason why people wanted a Dislike

  • button is frankly because not everything in Newsfeed

  • is likable.

  • People write about hard times that they're going through,

  • tragedies that are happening in their lives.

  • Recently we just went through this election which was very

  • very charged and people would read things or say things

  • that made them feel a lot of different emotions,

  • not just happy ones.

  • And so that is basically the people problem statement.

  • Not everything in Feed is likable and I want to be able

  • to easily express other things.

  • And finally one more example, so sometimes I call these

  • people problem statements, sometimes they're not just

  • problems in the way of like hey is someone going to bring

  • this up as like a hardship in their life that they want

  • solved, sometimes it's just more of an opportunity

  • or a thing that if you ask people if they want it,

  • they would say yes.

  • An example here is the desire to share spontaneously

  • and authentically.

  • A lot of times we're going through a pretty cool experience

  • and we might capture it at that moment and then an hour

  • later go and upload it on Facebook but how cool it would be

  • to actually have that experience live with our friends

  • and our family.

  • So that is an example of another people problem statement.

  • So those are basically the first thing that we always ask,

  • is okay do we have a statement that is really about what

  • problem that we're trying to solve for people.

  • That's the first question.

  • So now the second question is how do we know

  • this is a real problem?

  • And by know, I mean what evidence do we have?

  • Is there a qualitative evidence that it is,

  • is there quantitative evidence that we can look at?

  • And I highlight the word real, not because you know I think

  • that there are fake problems, but just because I think

  • the question to ask, is is this a problem

  • that's worth solving?

  • All of us have limited time, energy, resources, money,

  • whatever to be able to devote our being to trying to solve

  • problems, what makes this one the one that we should pick

  • out of the thousands or millions of problems

  • that are out there.

  • And so this question's really about just making sure

  • that the opportunity is something that is worth tackling

  • and that we aren't just solving problems for ourselves

  • individually but that we actually are very aware

  • of the problems that the audience that we're building

  • for is facing.

  • So for the example of groups, I want to talk about,

  • I want to find other people who are interested in discussing

  • the same things as I am.

  • The way that we try to validate whether or not this was

  • a big enough use case was of course we went and we talked

  • to a lot of people but we also looked at some of the data

  • that we had.

  • So when we designed the group's product to start with

  • we imagined that most of the time you would get invited

  • to the group.

  • So I might start a book club and I'm going to invite

  • some of my friends who like reading the same books as me

  • and that's how we're all going to know about the group.

  • So in our initial formulation of groups it was really all

  • about being invited by somebody else who was already a part

  • of this group.

  • But one of the things actually that some of the engineers

  • on the group's team had hacked over the years was this

  • little unit called groups you should join.

  • And so some of you guys might have seen it

  • sometimes you are scrolling through your feed

  • and notice a little story will pop up.

  • And based on what your friends have joined and groups

  • in your community and groups that are similar to things

  • we think you might be interested in, you'll see

  • some recommendations and you can go and explore them

  • and decide to join them if you want.

  • And we thought that this was just a small feature,

  • but really most of the inviting was going to happen through

  • the normal invite process.

  • But when we took a look at the data what we found was that

  • actually one third of group memberships were starting

  • to happen via this Groups You Should Join unit.

  • Which meant that there actually was a need and a desire

  • already using the tools that we currently had that

  • people wanted to find things that they weren't

  • explicitly being invited to.

  • That there were examples of all sorts of things that

  • they wanted to be able to participate in and see

  • that they just weren't getting connected to any other way.

  • So this was I think good evidence for us that if we

  • wanted to devote something, a larger solution to helping

  • people discover groups, that that was a worthwhile

  • endeavor.

  • We also talked to a lot of people in a lot of different

  • groups, one of the folks we talked to referenced this

  • group called the Physician Moms group, this is actually

  • one of the most active groups on Facebook.

  • And I think something like one out of four doctors

  • who are mothers in America are part of this group.

  • And one of my friends actually happens to be a doctor

  • who just had a baby and so she was lucky enough

  • to have one of her colleagues add her to this group.

  • But she talked about how meaningful it was to her and she

  • would spend hours on this group every single week

  • because these doctor moms would be sharing tips on how

  • to balance their work and really tackle a lot of the issues

  • that come from being in that profession.

  • And this is an example of a group that everyone who's a part

  • of it finds it hugely meaningful, but not everyone knows

  • about it because how would you know that this thing even

  • existed out in the world unless somebody told you.

  • Another example is you know I've recently attended our

  • 10 year reunion and at the reunion, a lot of people

  • were using groups to coordinate events and meetups

  • and the biggest barrier is that half of the people

  • I talked to didn't know that these groups existed

  • that there was a class of 2006 group and that there

  • was a reunion group and that there were groups

  • for different dorms who were planning their

  • get-togethers.

  • And finally there are a lot of groups, I'm a designer

  • and one of the top groups that a lot of designers

  • in the community join is this group called Designer's Guild

  • and a lot of designers share tools and articles that

  • they've read and other resources.

  • And every time I happen to add someone 'cause I thought

  • about them, that they might enjoy it, they're always

  • like wow I didn't know about this, that's the first

  • thing that everyone always says, I didn't know

  • about this.

  • So all of this when we did our focus groups in our research

  • that was more evidence that in fact we should be doing

  • more to help people discover groups.

  • So what we ended up doing was actually building a pretty

  • simple discover feature.

  • If you go to your Groups tab right next to it, there will

  • be a tab called Discover and we did our research on what

  • were the topics that people were most interested in.

  • So you can scroll through.

  • You'll see suggestions based on things that you are already

  • a part of but then you'll also be able to go and browse

  • by category and this is where you can find parenting

  • groups, if you're a photographer, people to go on photo

  • trips with on the weekends or play sports with,

  • or joining a writing group.

  • And these are all groups that are public and that actually

  • want to welcome new members.

  • For the people problem of not everything that I see

  • in Newsfeed is likable, and I want to be able to express

  • other emotions.

  • We wanted to make sure that this was actually a problem

  • that many people faced and that the solution that we

  • had would directly address it.

  • So we talked to a lot of users, we had them go through

  • their feeds and describe to us for each story what

  • was their reaction, what were they feeling, sort of

  • like free flow tell us what's going through their minds

  • as they're going through their Feeds.

  • And you know a lot of times people would say, well

  • there should be more ways for me to just say something

  • because what I like about the Like button is that

  • it's so simple.

  • I don't have to go and comment and the keyboard comes up

  • and I have to two handedly type something I like the fact

  • that I can just in one gesture scroll through things

  • and say that I like it, but the only thing I can do

  • is Like.

  • And there's gotta be other ways for me to express

  • other emotions.

  • We also looked at how are people expressing that they

  • don't like something today and we looked at the stickers

  • that people were using, leaving us comments as well as

  • the emoji and what were kind of the most popular ones,

  • how frequently were people just leaving a simple sticker

  • or a simple emoji.

  • And finally we looked at also short comments.

  • So we looked at how many comments were just one or two words

  • and all they expressed was like awesome or that sucks

  • or a very very short phrase that we felt we could

  • encapsulate.

  • So with all that, we decided we'd looked at all that data

  • we'd looked at what were the most common emotions that

  • people wanted to express and we designed something we felt

  • fulfilled that criteria of being really really lightweight

  • so you can still do it with one hand and one gesture,

  • it's not multiple taps and we also, taking all the data

  • we had gotten from what were the most common things,

  • the common reactions that they had.

  • We built the reactions product and so you just scrub

  • through and we took the top most what we wanted

  • to be universal reactions and that's the options that you

  • see in that tray.

  • And finally for the people problem of I want to share

  • spontaneously and authentically in the moment, this was

  • interesting to us because we had the Facebook Live

  • product out for celebrities and we built it

  • for celebrities and public figures first because when we

  • talked to them that was something that they told us

  • very directly that they wanted.

  • And it was something that they were used to because they

  • lived their lives in the spotlight and they're very

  • used to going on the red carpet and doing interviews

  • and being broadcast live.

  • And it was something they felt they could do to connect

  • with their fans a little bit more.

  • So this was something that we had out that was only

  • available to public figures.

  • But the interesting insight for us was well is this

  • something that people want who aren't public figures?

  • Is this something that people in the audience, you guys,

  • me, all of us, would this be something that non public

  • figures would use and find valuable?

  • And that is the question that we set out to uncover

  • and prove.

  • And actually one of the easiest ways for us to do that

  • was to just build it for ourselves, we already had

  • the code that ran for public figures and we just turned

  • it on and see if anybody at the office uses it

  • and what their reactions were.

  • And we saw actually a lot of really creative ways

  • in which people used it, some people used it to broadcast

  • their team meetings for people who were working from home

  • a lot of times when RVC programs weren't working

  • super well.

  • The other benefit is that you record the video and then

  • it's available afterwards.

  • So for people who couldn't tune in live, they still had

  • a chance to watch the contents of the meeting.

  • We also saw people just take it out for a weekend

  • so they'd go to brunch on Saturday with their family

  • and then they would go live and people would chime in

  • and there would be this conversation and the people

  • who did this were like yeah it was awesome,

  • I had a great time.

  • And then we also saw people taking it for a run.

  • So a team at Facebook was doing this relay race

  • and they wanted other folks to cheer them on and give

  • them support so they went live and this was also

  • a super fun thing for them and for all of the people

  • who participated.

  • So that was some inkling that hey this could actually

  • be something that other people wanted.

  • And so what we did was we launched it to a small

  • percentage of our users as a test to see if there was

  • interest in the market.

  • And that was the Facebook Live product on Android,

  • it launched on Android first.

  • And right away we did see a lot of really really creative

  • uses, one woman who's a tattoo artist started to broadcast

  • live as she was in the process of tattooing somebody.

  • We also saw a woman who broadcasted her wedding live

  • because she had family members who couldn't make it

  • and who couldn't travel the long distance.

  • And it was really really cool to be able to see

  • how she could engage and interact with those audience

  • members and almost have it feel like they were a part

  • of that experience.

  • I had a lot of fun taking it out, I managed to get

  • reservations to one of the best restaurants

  • in San Francisco, Lazy Bear and it's a really cool

  • experience 'cause the chef stands up there and he

  • explains every single dish and he talks about all

  • of the love and care that went into how this dish

  • came together and I just decided to go live and a ton

  • of my friends tuned in and everybody was like I really

  • want to go to this restaurant too, so that was really fun.

  • Okay so that brings us to the last question, so let's say

  • we built this, we validate it's something that people want

  • or we think there's enough reason to believe that

  • it's something that is a real problem and that's something

  • that is worth solving so now we build it and we put

  • it out there.

  • But even actually before the thing is out in the world

  • in fact even before maybe we have a full prototype

  • the question for us to ask is how will we know if we

  • solved this problem?

  • What would be different in the world?

  • If we fast forward and now this thing that we've done

  • is out there, how would we know if we should be happy

  • if we should be excited, if we think it didn't live up

  • to our expectations, what exactly

  • should our expectations be?

  • And I think this is a really important question

  • to ask up front, too often what ends up happening

  • is we have this idea and then we build it and we launch it

  • into the world and then results come in.

  • We're looking at the dashboards, we're looking at how many

  • people are downloading it and what they're saying

  • and there's tons of data points that we're trying

  • to interpret and put together.

  • But it's hard at that point in time to be very objective

  • about did we solve the problem because a lot of our

  • natural inclination is to read into the good things that

  • people are saying and to consider all of that effort to have

  • been worth it.

  • And that there's biases that come from when you're already

  • looking at data and you're trying to interpret whether

  • or not it's good or bad.

  • It's much better before you launch to figure out what

  • constitutes success for you so that you can go

  • into the launch with that understanding and as results come

  • in you can map it to your previous, this was my criteria

  • for whether or not we solved the problem or whether

  • this was successful.

  • And so what we do here is we want to make sure we set

  • measurable goals and metrics and measurable is really

  • key, measurable doesn't always mean like numbers or data.

  • But it does mean there's a criteria where if I did this

  • thing and I got this result I know what to make of that

  • result.

  • So for the example of groups, we wanted to help people

  • solve the problem of helping them find other people

  • to talk about their interests with.

  • We ended up building a Groups Discover dashboard.

  • What does success look like for us?

  • So before we launched we determined that if we were

  • successful at actually helping people solve this problem

  • we would see that more people are then discovering

  • groups that they're interested in and joining them.

  • But not just joining because I could make a giant button

  • flash and make it red and I'm sure more people would

  • click on it and more people would join, that doesn't

  • really count right.

  • What really matters is that they join these groups

  • and these groups are actually meaningful to them.

  • Which means that if we fast forward a couple months,

  • three months let's say and then we look back, these

  • people are still using those groups and they're actively

  • engaging, meaning they're talking with other people,

  • they're reading the content, they're sharing content,

  • they're liking, they're commenting.

  • We want to know that people are joining groups that

  • are actually valuable for them and that they're spending

  • time on.

  • So that was actually the more important metric, was not

  • just joins but meaningful joins.

  • For the example of I want to be able to quickly express

  • my feelings about a story but not everything is likable,

  • we launched Reactions.

  • And prior to launching Reactions, we wanted to make sure

  • we measured a couple of things.

  • If we were successful here then what we should see

  • is well people are using these reactions and not just

  • using them, it's not good enough if all the people that were

  • previously using a sticker or an emoji or a short comment,

  • then converted to using Reactions, 'cause then it's sort of

  • a one for one trade.

  • If Reactions was actually more lightweight, we should see

  • more people using that then the previous set of people

  • who were using all of these other tools, so that was

  • really important to us.

  • And so that was the first thing that we looked at.

  • The second thing we looked at was we wanted to make sure

  • every reaction did get a fair amount of usage across

  • different markets.

  • And this was important to us because in the beginning

  • even when we started to design Reactions, we had considered

  • lots of different options.

  • We considered maybe it would be funny if you were

  • an awesome button or there's tons of different things

  • that we could've put in order to make it more expressive.

  • And the reason why we distilled it back down to six

  • is because we wanted to make sure that it was simple

  • to use and if you were going through and choosing

  • from a grid of 20, suddenly it doesn't feel that

  • lightweight anymore.

  • It feels like you're pulling up an interface

  • and you're choosing something and it's going to

  • take more than a couple of taps to get you what you want.

  • So it was really important that we nailed and got

  • to a really small set, but that we pick the right set.

  • So we wanted to make sure that these were universal and it

  • wasn't like people in certain countries were using certain

  • ones more or people in certain demographics were favoring

  • others because it was a more niche thing.

  • We wanted to make sure that these were actually things

  • that expressed emotions people wanted to express in all

  • of our different countries and all demographics.

  • And then finally we wanted to make sure that the experience

  • was good for the receiver.

  • So if you posted something and then you got a bunch

  • of reactions, did that make you feel bad did that make

  • you feel confused, did you not know what was going on,

  • or did you just feel like it was a negative experience

  • and it made you want to share less in the future,

  • that was also really important and that was actually one

  • of the reasons why we never straight up put a dislike

  • button because dislike in and of itself can be very

  • ambiguous.

  • Am I disliking the content that you posted, am I disliking

  • you as a person, am I just not agreeing with your point

  • of view and so in all of our research and understanding

  • of dislike we knew that it was something that people

  • were going to get confused by and it wasn't going to be

  • a great experience.

  • So when we launched Reactions we similarly wanted to make

  • sure that that wasn't the case that the emotions that

  • were being expressed felt like they were adding

  • to the poster's understanding of what their friends

  • and audience had to say.

  • And finally for the example I want to spontaneously

  • and authentically share in the moment, what we decided

  • to measure and look at was that of course people were

  • broadcasting but again it's not just did they broadcast

  • once because that's something you can make happen by

  • doing a ton of promotion and putting it front and center.

  • People will try a lot of things once because they don't know

  • what it is or they think it sounds exciting.

  • The true test for us was when they went live and they had

  • that experience did they like it enough to go and do it

  • again, now that is the key that tells us we built

  • something that is valuable and that's worthwhile because

  • people will come back and do it again and do it again

  • and they'll incorporate it and make it part of their

  • lives.

  • The second thing we looked at is we wanted to make sure

  • that it'd be fine if broadcasters loved it, everybody

  • wants to go live all the time but is it really a good

  • product if nobody wants to watch what people

  • are broadcasting live and then it's one-sided

  • and over time broadcasters are not going to want to do

  • 'cause nobody ever watches anything.

  • We wanted to make sure that the content was actually

  • interesting and the way that we can measure that

  • is the time spent watching live videos going up

  • proportionally.

  • And when we run surveys and other things are people saying

  • that this is a valuable thing that they want to watch.

  • So to recap I went over the three questions that we always

  • ask in product reviews and you know anytime we're talking

  • about products.

  • We want to make sure what people problem we're trying

  • to solve that it's very clearly defined, that we can picture

  • the audience or a particular person in mind as we're

  • designing the product and that we're not solving problems

  • for ourselves or our company or our team but that we're

  • actually focused on an audience and a problem that

  • matters for them.

  • The second thing is just making sure we know that it's

  • a real problem through looking at whatever quantitative

  • data we have through doing focus groups and talking

  • to different users, understanding that this is something

  • that is of good use of time for us to work on

  • and to solve.

  • And finally being very rigorous about what does success

  • look like and how will we know if we put something

  • out there and it did what we wanted it to do.

  • And if not, then we go back to the drawing board,

  • we learn what didn't work and we keep iterating.

  • But these three questions have helped keep our teams

  • focused on what matters which is ultimately

  • the people that we design for.

  • Thank you guys, thank you very much.

  • (audience applause)

(audience applause)

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it