FACEBOOK BANNED SEVERAL HIGH

PROFILE ACCOUNTS THAT IT SAYS

ENGAGE IN VIOLENCE AND HATE,

INCLUDING WHITE SUPREMACISTS,

NATION OF ISLAM LEADER LOUIS

FARRAKHAN, AND A NUMBER OF

RIGHT-WING COMMENTATORS

INCLUDING ALEX JONES.

IN THE WAKE OF LIVE-STREAMED

ATTACKS AND PUBLISHED THREATS,

THERE ARE NEW CALLS FOR SOCIAL

MEDIA COMPANIES TO DO MORE TO

BLOCK EXTREMISM AND HATE.

AT FACEBOOK, THOUSANDS OF REAL

PEOPLE KNOWN AS MODERATORS ARE

THE FRONTLINES OF THIS BATTLE,

LOOKING AND LISTENING TO SOME

HORRIFIC MATERIAL.

I RECENTLY SPOKE WITH CASEY

NEWTON, SILICON VALLEY EDITOR

FOR "THE VERGE," WHO REPORTED ON

THE TOLL THIS CONTENT IS TAKING

ON FACEBOOK MODERATORS.

THIS IS NOT JUST IN THE CONTEXT

OF THE NEW ZEALAND ATTACKS.

THIS IS SOMETHING THAT IS

STRUCTURALLY BUILT IN SO THAT

BASICALLY THERE ARE PEOPLE WHO

SEE THINGS ON FACEBOOK SO THAT

THE REST OF US DON'T HAVE TO,

RIGHT?

>> ANYTIME YOU SEE SOMETHING ON

FACEBOOK THAT YOU DON'T LIKE,

YOU CAN CLICK A BUTTON TO REPORT

IT.

AND IF YOU DO THAT, A HUMAN

BEING WILL LOOK AT IT AND

DETERMINE WHETHER IT VIOLATES

FACEBOOK'S COMMUNITY STANDARDS

OR NOT.

A LOT OF THE STUFF THEY SEE IS

BENIGN, BUT A LOT OF IT IS

REALLY SHOCKING AND DISTURBING,

AND SOME OF THAT CAN HAVE A

LONG-TERM TOLL ON THEIR LIVES.

>> Sreenivasan: SO, WHAT KIND OF

THINGS... WITHOUT GETTING TOO

GRAPHIC, WHAT KIND OF THINGS ARE

FACEBOOK MODERATORS LOOKING AT

PRETTY MUCH ALL DAY?

IT'S SHIFT WORK.

>> THAT'S RIGHT.

I MEAN, SOME OF THE THINGS THAT

FOLKS HAVE DESCRIBED TO ME WOULD

INCLUDE VIOLENCE FROM DRUG

CARTELS, MIGHT INCLUDE TERRORIST

CONTENT FROM ISIS, CHILD

EXPLOITATION, AND, YOU KNOW,

SORT OF EVERY FLAVOR OF VIOLENCE

THAT YOU CAN IMAGINE.

>> Sreenivasan: AND SO, THEY ARE

EXPECTED TO DO WHAT?

AND AT WHAT PACE AND WHAT SPEED

DO THEY LOOK AT THIS STUFF AND

DECIDE WHAT TO DO WITH IT?

>> YEAH.

SO, THEIR GOAL FOR IMAGES IS TO

REMOVE THOSE WITHIN ABOUT 15

SECONDS.

AND WITH VIDEOS, IT'S ABOUT 30

SECONDS.

AND OVER THE COURSE OF THE DAY,

THEY MIGHT BE ASKED TO MAKE 300

DECISIONS, SOMETIMES EVEN 400

DECISIONS.

AND ONE OF THE REASONS THAT'S

VERY DIFFICULT IS BECAUSE THE

POLICY CHANGES ALMOST EVERY DAY

AS MODERATORS GET NEW GUIDANCE

ABOUT WHICH POSTS WILL AND WILL

NOT BE ALLOWED TO STAY UP.

WE SHOULD ALSO SAY THAT THEY ARE

EVALUATED BASED ON THEIR

ACCURACY.

SO, IF THEY TAKE SOMETHING DOWN

AND THEY WEREN'T SUPPOSED TO,

THAT'S A BLACK MARK ON THEIR

RECORD.

AND IF THEY HAVE JUST A FEW OF

THOSE OVER A WEEK-LONG PERIOD,

THEY CAN BE FIRED.

>> Sreenivasan: SO, IS THERE A

FINE LINE ON WHAT IS AND WHAT IS

NOT ALLOWED ON FACEBOOK AS A

PLATFORM?

>> ABSOLUTELY.

IF YOU JUST READ THROUGH THE

GUIDELINES ON NUDITY, FOR

EXAMPLE-- SOMETHING THAT YOU

MIGHT THINK WOULD BE RELATIVELY

EASY TO DEFINE WHAT WOULD AND

WOULD NOT BE ALLOWED-- IT'S

ACTUALLY INCREDIBLY DETAILED.

IT'S A VERY LONG SECTION OF THE

GUIDELINES, AND IT GOES INTO

BREATHTAKING DETAIL ABOUT

EXACTLY WHICH BODY PARTS AND HOW

MUCH OF WHICH BODY PARTS CAN BE

DEPICTED IN A PHOTO OR VIDEO.

AND THEN, OF COURSE, THERE ARE

DOZENS OF OTHER CATEGORIES THAT

MODERATORS NEED TO FAMILIARIZE

THEMSELVES WITH AND THEN NEVER

MAKE A MISTAKE.

>> Sreenivasan: IN THAT CASE,

YOU CAN TECHNICALLY SEE WHAT'S

HAPPENING.

SOMETIMES IT'S AUDIO, AS WELL.

I MEAN, IS THERE... IS THERE A

LINE WHAT YOU CAN HEAR, WHAT YOU

CAN SEE, WHAT'S IMPLIED, AND

THEN WHAT'S FREEDOM OF SPEECH?

>> EXACTLY.

WELL, YOU KNOW, SO, WHAT YOU'RE

GETTING AT IS THAT WE'RE ASKING

THESE FOLKS TO MAKE JUDGMENT

CALLS, RIGHT?

IF YOU ACCEPT THAT MUCH OF OUR

POLITICAL DISCUSSIONS NOW TAKE

PLACE ON SOCIAL NETWORKS, THEN

THE FOLKS WHO ARE MODERATING

THIS CONTENT ARE ESSENTIALLY

FIRST RESPONDERS.

THEY'RE ON THE FRONT LINES, AND

WE'VE ENTRUSTED THEM WITH THESE

REALLY FUNDAMENTAL QUESTIONS OF

SAFETY AND SECURITY.

AND WHILE WE'VE GIVEN THEM

VOLUMINOUS GUIDELINES FROM WHICH

TO APPLY THE POLICY, THE TRUTH

IS THAT SOME OF THESE ARE ALWAYS

GOING TO BE JUDGMENT CALLS.

I HAD MODERATORS TELL ME ABOUT

CASES WHERE THEY WOULD BE TOLD

TO TAKE A POST DOWN, AND THEY

WOULD DO THAT AND THEN LATER

THEY WOULD BE TOLD, "OH, NO, NO,

LEAVE IT BACK UP."

AND THEN, HOURS LATER, TAKE IT

BACK DOWN AGAIN.

IT'S THAT KIND OF INTRICATE OF A

DECISION-MAKING PROCESS.

IT'S VERY FAST, IT'S VERY FLUID,

AND FOLKS JUST CHANGE THEIR

MINDS.

SO, IT'S INCREDIBLY DIFFICULT

WORK THAT, OF COURSE, IS ALSO

VERY LOW PAID.

>> Sreenivasan: NOW, THESE ARE

JUST A FEW THOUSAND PEOPLE IN

ONE LOCATION OR ANOTHER, BUT

THEY'RE NOT ACTUALLY GOING OUT

AND SEEING ALL THE PAGES THAT

EXIST ON FACEBOOK.

THAT WOULD BE IMPOSSIBLE.

SO, THERE'S, WHAT, AN ALGORITHM

THAT'S KIND OF SORTING IT

THROUGH AND THEN KIND OF PUTTING

IT UP ON A SILVER PLATTER TO

THEM?

>> THAT'S RIGHT.

SO, IF YOU SEE SOMETHING YOU

DON'T LIKE ON FACEBOOK, YOU CAN

CLICK A BUTTON TO REPORT IT, AND

ALL OF THAT GOES INTO A BIG

QUEUE.

AND THEN, THAT GETS SERVED UP TO

THESE MODERATORS, OFTEN IN NO

PARTICULAR ORDER.

SO, THEY SIT DOWN AT THEIR DESK,

THEY CLICK A BUTTON THAT SAYS

"RESUME REVIEWING" INSIDE A

CUSTOM PIECE OF SOFTWARE THAT

FACEBOOK HAS, AND THEN THEY JUST

START SEEING THIS MASS OF POSTS.

AND SOME OF THEM WILL BE VERY

SIMPLE BULLYING, SOME OF THEM

WILL BE VERY BENIGN, AND THEN

SOME OF THEM WILL BE INCREDIBLY

DISTURBING.

>> Sreenivasan: TELL ME A LITTLE

BIT ABOUT THE KIND OF RIPPLE

EFFECTS AND THE CONSEQUENCES

OF... I MEAN, YOU TALKED TO A

LOT OF DIFFERENT PEOPLE FOR THIS

STORY, SOME WITH THE BLESSING

OF THE EMPLOYER AND THEN QUITE A

FEW WHO SPOKE TO YOU ON KIND OF

THE CONDITION OF ANONYMITY.

>> THAT'S RIGHT.

AND THE REASON FOR THAT IS THAT

FACEBOOK REQUIRES ALL OF THESE

FOLKS TO SIGN A NONDISCLOSURE

AGREEMENT.

THAT HAS SOME BENEFITS.

IT DISCOURAGES THEM FROM TALKING

ABOUT PERSONAL INFORMATION OF

FACEBOOK USERS, AND IT ALSO

PROTECTS THEIR SAFETY.

FACEBOOK, YOU KNOW, IF FOLKS

KNEW THAT THEY WERE MODERATING

CONTENT FOR FACEBOOK-- MAYBE

FACEBOOK REMOVED ONE OF THEIR

POSTS-- YOU CAN IMAGINE THAT

THAT COULD LEAD TO SOME... SOME

PRETTY UGLY CONFRONTATION.

SO, THOSE N.D.A.s HAVE A

POSITIVE SIDE TO THEM, BUT THEY

ALSO HAVE A NEGATIVE SIDE WHICH

IS THAT A LOT OF THE FOLKS THAT

I SPOKE WITH SAID THEY DIDN'T

EVEN FEEL COMFORTABLE TALKING

ABOUT THEIR WORK WITH THEIR...

THEIR PARTNERS, THEIR SPOUSES,

THEIR FAMILY MEMBERS, THEIR

CLOSE FRIENDS BECAUSE THEY WERE

WORRIED THAT THEY WOULD BE HELD

LEGALLY LIABLE FOR VIOLATING

THIS N.D.A.

AND WHEN YOU CONSIDER THE KIND

OF CONTENT THAT THEY'RE LOOKING

AT AND HOW DEPRESSING AND

UPSETTING SOME OF THAT CONTENT

IS, THAT CAN PUT SOME OF THEM

INTO A VERY DARK PLACE, RIGHT,

WHERE THEY FEEL LIKE THEY

CAN'T... THEY DON'T HAVE ANYONE

THAT THEY CAN TALK TO.

SO, I TALKED TO A NUMBER OF

FOLKS WHO TOLD ME THAT THEY HAD

DEVELOPED SYMPTOMS THAT CLOSELY

RESEMBLE POST-TRAUMATIC STRESS

DISORDER.

AND THEN, IN ANOTHER SORT OF

STRANGE TWIST, A LOT OF THE

MODERATORS I SPOKE WITH SAID

THAT THE MORE THAT THEY REVIEWED

CONTENT ABOUT SORT OF FRINGE

CONSPIRACY THEORIES, THE MORE

THEY THEMSELVES CAME TO BELIEVE

IT.

AND SO, A LOT OF THE MODERATORS

I'VE SPOKEN WITH NOW BELIEVE IN

SOME OF THE CONSPIRACY THEORIES

THAT THEY'D BEEN ASKED TO

MODERATE.

>> Sreenivasan: YOU KNOW, THIS

IS YOU... THERE'S A PARAGRAPH

FROM YOUR STORY I WANT TO QUOTE:

"THE MODERATORS TOLD ME IT'S A

PLACE WHERE THE CONSPIRACY

VIDEOS AND MEMES THEY SEE EACH

DAY GRADUALLY LEAD THEM TO

EMBRACE FRINGE VIEWS.

ONE AUDITOR WALKS THE FLOOR

PROMOTING THE IDEA THAT THE

EARTH IS FLAT.

A FORMER EMPLOYEE TOLD ME HE HAS

BEGUN TO QUESTION CERTAIN

ASPECTS OF THE HOLOCAUST.

ANOTHER FORMER EMPLOYEE WHO TOLD

ME HE HAS MAPPED EVERY ESCAPE

ROUTE OUT OF HIS HOUSE AND

SLEEPS WITH A GUN AT HIS SIDE,

SAID, 'I NO LONGER BELIEVE 9/11

WAS A TERRORIST ATTACK,' AND

THIS IS A RESULT OF THE JOB THAT

THEY'RE DOING EVERY DAY."

>> YEAH.

YEAH, THAT'S EXACTLY RIGHT.

I MEAN, AND CERTAINLY FOR ME AS

A REPORTER, YOU KNOW, AND WE

SHOULD SAY THERE'S BEEN SOME

GREAT REPORTING ABOUT CONTENT

MODERATORS IN THE PAST AND A

VARIETY OF OTHER OUTLETS.

A LOT OF IT IS LINKED IN MY

PIECE.

BUT THAT PASSAGE THAT YOU READ

CONTAINED SOME OF THE STUFF THAT

SURPRISED ME THE MOST IN MY

REPORTING.

IT WAS SOME STUFF THAT I DON'T

THINK HAS REALLY BEEN EXPLORED

BEFORE.

AND THAT'S THE LONG-TERM EFFECT