Writing Tests For CSS Is Possible! Don’t Believe The Rumors - Gil Tayar | CSSconf EU 2019

Subtitles section Play video

Hi, everyone.
Wow.
Um, thank you for having me.
I have to say I'm not Ah, I'm not a front and developer.
I've been switching back and forth between front and back end.
Currently, I'm in my back and face and the presentation is not loading.
Wait.
Turned life out.
There we go.
Hopefully, uh, but one thing I didn't notice both in Jing's presentation and in Jason's is this Did everybody knows that they were both showing us how responsive the things are.
And I was thinking to myself, Oh, wow.
And they were like, This is simple.
True.
But what if I change this?
Yes.
Says that that's problematic because I have to check it every time, time and time again.
Hi.
I'm guilty.
Are, uh, testing is is what is has been my passion since about 2000.
I'm a developer, advocate and a software developer, mostly software developer at Apple Tools.
And what we do is we specialize in visual testing tools.
Uh, and I want to talk about fear.
Um, definitely fear.
And we'll talk about fear of what in a sec.
But first, I want to talk about web developers first.
Let's start with the back and developers back.
And developers have been there since before the Web and I'ma back in developer.
Right now, they use words like, you know, thread pools and message cues and scholars and you know they're back there.
And without them, come on way wouldn't be able to do our friend so round of applause.
There you go.
What about front and front and is divided front.
And let's talk about front and JavaScript with developers.
They're wizards.
They deal with things like a JavaScript and angular and react and States management and Darman and all that kind of stuff.
And they're really wizards.
And the way they handled things about front and CSS developers, front entrances.
Developers are beautiful.
That's where you come on a round of applause for you.
There we go.
Not only are you all beautiful, you create beautiful things using CSS, and I'm constantly amazed by these things and how you managed to work with them s So you deal with responsive design, media queries and flex box, and something has wonderful is happening in the Web development world.
A really wonderful thing.
And what is it?
Developers are starting to write tests the test their own code.
This has been true for something like five years or so.
Obviously enough people been testing before that, but it's become slightly mainstream of the last five or seven years.
But why?
Why are developers writing tests?
The word I use eyes, eh?
Go program a phobia.
Anybody know what a go program a phobia is probably not given that I've just invented it.
It's fear of one's own code.
Okay, it's, uh and I will explain that you know the manager, he says.
You just add that feature and the way they think about it is just, you know, adding that Lego to the existing code and and we're done.
The truth, as we all know, is something like this.
That's fear.
The fear of doing that kind of thing doesn't always happen, usually doesn't.
But when it does out removing CO, we want to re factor.
This is especially true in CSS.
We think about it.
It's just, you know, removing that one single line or in CSS just I don't need that rule.
That rule shouldn't be there.
Let's just take it out.
But we all know what happens in the real world.
When I take out that CSS somewhere, some page is gonna break.
Yeah, re factoring I really want to re factor.
I really want a reflector, that JavaScript But I'm afraid this will happen to me.
And it doesn't matter whether our code is a horribly written pile of spaghetti or a wonderfully beautiful, crafted set of loosely coupled modules.
We really fear our own call, which we fear, changing it because changing code may mean breaking the APP and Web developers, as all developers have have discovered, that developer testing or testing, in short removes the need for this manual testing after co changes.
And developer testing is writing code that tests your own code as part of writing the code itself.
So it's not que es automation engineers doing that for you.
It is you doing that for yourselves, and testing abolishes that fear.
It makes that fear mostly go away because you code and then you write the tests for that code and doesn't really matter whether you're writing it before you write the co TT.
These dollar after I don't really care and you know you run those tests and they break and you fix the bogs, et cetera, and it works.
And once you change your code, you just run the tests again and you're done.
Testing abolishes fear that is the most important thing.
Testing abolishes the fear of one's own code.
Who's writing tests?
Who's abolishing that fear?
Well back and developers have been writing tests for quite some time.
They've had that methodology even before the Web.
The methodologies and tooling are mostly established.
There will be working on it for quite a while.
Front and JavaScript developers.
Well, the Web is young in terms of methodologies.
It's not totally there yet.
It's slightly confusing.
Nobody really knows exactly what it's, but the tooling is almost there.
It's happening.
It's happening.
It's happening for quite sometimes, a lot of mentors and teachers can see Daws just in serials.
Kevin Lamping, Gleb Okamoto, Shy Resnick And there's hopefully including me were slowly building for JavaScript developers, a methodology based on unit testing component and multi component testing.
Using James Dom and things like Cyprus and selenium and Web driver Io.
What about you?
You're sad.
No tests for you, you don't know and why?
Why is that?
Well, I'll explain a second front.
Let's talk about front and JavaScript tests.
They're functional.
Let's have a function.
You put an input.
It does its thing.
You grab the output, you check that.
The output is correct.
You're done.
You've written your unit tests.
What about the You know you eye tests?
Not a problem.
You opened the browser using also automation tools.
You're right.
You do the click click clicks and the type type types.
And then you check that this field is the correct value in that field Is the correct value in this field is the great value.
Same stuff that nothing changed.
But how do I functionally test something like this?
I could check the wits of the columns when I you know, when I do the responsible design, it's possible.
But it's a huge amount of code and you are not testing everything.
And you're saying, quite rightly, we can't do functional testing.
And you're right.
You can't.
I tried a few years ago at Wickes.
We did.
It was my front end phase and I tried writing functional tests.
Yeah.
Whoo!
It worked.
No problem.
The tools and doing as I said is there.
And then I tried to do visual testing couldn't succeed.
Testing CSS and testing visuals is ah, hard problem.
You're not.
But why?
Because it's visual, not functional.
Anything to do with vision is problematic.
But don't believe the rumors writing tests versus is possible.
And the point is not that writing testes for assistance is possible is that you should be starting to write the tests for a CSS.
It is time for that.
How?
I've bean doing a lot of talks on this issue and talking to a lot of people.
People think the developers think it goes like this.
You navigate the page using web automation tools like selenium or Cyprus take a screenshot, and then you tell the tool whatever library using for testing.
Does this look good?
Right?
They really think that Or maybe Does this conform to a certain design system?
Okay, this is my design system.
Does this you?
I conform to it.
And the answer is no.
Unfortunately, we're not.
There are machine learning algorithms and r a I.
And we need a eye for this type of stuff.
Is just not that good yet, but right now.
So how does it work?
We're not actually doing visual testing.
We're doing visual, Rick.
Ration testing.
Where change.
We're checking that what has been still is okay, that nothing has regressed or changed, and this is a typical visual test.
It's that simple.
I'm using Cyprus.
I love Cypress.
It's my go to automation tool, but there are tons of other selenium Web driver Io Test Cafe.
All of them support visual testing, tow you on a point or another.
First of all, we set the View board because setting the View port, as we all learned and we all know in responsive design is crazy important.
Then we use Cyprus to visit, and then it opens that browser to whatever site we want.
Then we do the typing and clicking to get to whatever page in whatever state of the why we want, and we call the tool visual testing tool.
Just call it and it does its magic.
This is where the magic happens.
And what is that magic.
The tool, first of all, takes a screenshot of the act.
It compares the screen shot with a baseline that was previously given, and I'll explain why when it was given it checks for differences.
If there are no differences.
Good your test path.
If there are differences, it doesn't know whether that difference is a bug, which it usually is, or a feature.
I've just added a button.
I've just changed the styling, so you need to check manually, accept or reject that difference.
And where does that baseline come from?
That bass line comes from a previous test the first time this tool runs it.
Except all the screen shots doesn't ask anything.
That's the baseline.
When you accept a difference because it's a feature, not about that becomes the new baseline and so on and so on.
This is visual testing.
In a nutshell.
That's how you just your CSS thank you very much questions now just joke.
I wish there's an elephant in the room because it's visual.
It's hard.
It's not that easy.
Four elephants in the room.
Let's talk about else.
Went number one.
I'm good.
We've seen dogs.
Christine Katz.
Now we have elephants.
First of all, taking a screenshot is hard.
It's actually the smallest elephant, but it's still there.
Are you using Cyprus?
That's easy.
Taking a screenshot in Cyprus is great.
Cyprus runs under chrome, chrome has great screenshot taking facilities.
You want only the view port?
Only the window.
Fine.
You want the whole full page?
Fine.
You want a specific region or a specific element?
We're good.
We don't always need the full page.
Sometimes.
Want to take a specific segment of that page?
Otherwise, if you're using, if your job script developers are using selenium or Web driver I or others, that's a problem.
Ouch.
You can only do window.
How do you take a full page when you can only take a screenshot of the window?
So what if I Java script if our JavaScript developers used to Lenny and we just, you know, I won't do visual testing?
I'll just do this manually all day.
Um, and the answer is no.
As I said, don't believe the rumors taking a screen shot is possible.
If you're using Cyprus, you're done.
If you're using selenium or others want.
There are open source and commercial tools that know how to do it in weird and wonderful ways.
Basically, take a screenshot view port of the View port, then ceases transformed the body up, take another screen shots.
Yes, just transformed the body up etcetera, etcetera, until we stitched all the screen shots together and we're done.
So Tooling knows how to do that.
Just look for it.
That's the easy part.
Elephant number two.
I think this is the biggest one, but comparing the screen shots is hard, so you have your baseline screen shot and you have your current screenshot.
How do you compare them most?
Open source tools and others do pixel by pixel.
Just check pixel by pixel to see whether anything has changed.
That's easy to do.
But look at this.
These air to screenshots of a certain page from 2004.
I think this is the My Windows machine, and this is why I use Mac.
But this is what the top one is.
My machine and the bottom is my friend's machine.
They look exactly the same, but notice they're not.
Why is that?
Why is the same machine same chrome or Firefox or whatever Same Windows operating system giving us different results?
And the answer, incredibly enough, is graphics drivers, graphics, drivers, graphics, ah, car graphic cards, et cetera, the difference on a re slight.
But they are there.
Let's do another one.
This is a picture from Chrome.
We found it out during the move from Chrome 67 68.
It blew my mind.
This is chrome 67.
This is Chrome 68.
Same J pick.
Look exactly the same, right?
No, this is the difference in the pixels.
Yeah, Most of the pixels are different.
Basically, it's probable.
They probably change the library that they used to render the J.
P or something.
I don't know, but there is a difference.
Remember, the testing bookstore will see a demo shortly.
This one really?
On the same.
That's what I was doing while while preparing the talk.
I found that out.
Using a pixel by pixel library.
The same one same machine, five minute difference between the two.
These are the two screenshots you can see zero difference, right?
No, really, really slight.
I have no idea why.
Really?
I was actually surprised.
That's somebody who deals with these pixel differences a lot.
I have no idea why there was a difference there, but there was like in one of 10 times I ran the test, which is why all the tools have this.
It's a slider which says, Do you want I do on false positives or false negatives.
It's It's basically it's basically saying how much percentage of air is okay.
If you say no percentage of ares, you'll get a lot of false negatives.
If you move that slider to the right, you'll get a lot of false positives or the other way around.
I can never figure it out, but you have to tweak it constantly and per page.
As I said, That's it.
That's the biggest problem because we don't want to constantly manage our screen shots.
No, it's not the same because a lot of these tools employ advanced screenshot comparison algorithms.
Aye aye, machine learning just really, really complex algorithms.
What they do is they don't look at the pixels.
They try and find the elements in the page and look at the elements and compare them.
If there are slight difference, is there, they just ignore them.
They're actually seeing as a human would see, or at least trying to do that.
So no false positives, no false negatives.
If you're using advanced comparison algorithms, it is possible.
And that is the biggest advance in visual testing that we've seen in the last few years.
And all of these, if they're commercial tools, they all have free and O.
S s plans.
So go ahead and use them.
Elephant number three Managing those comparisons is hard.
Okay, so we've got a difference and it's like we're doing 100 screenshots and a lot of them.
Let's change the header.
All of them have changes there.
That's the part where you accept or reject the difference.
You have to do that manually.
And, as I said, hundreds of comparisons.
Is that a problem?
Let's see.
One way to deal with it is we're doing Delta's small deltas.
So a lot of times it's the same difference in all the screen shots.
So I just use the tool and say, Yeah, fine update, update the baseline.
They're all fine, because if it's about, there's no problem.
I don't need to do anything.
But if it is ah feature, if there's an addition, I just accept it and go away.
The other way is taking like dragging from the folder that has the current screenshots, dragging it to the baseline screen shots and dealing with that every way.
Looking at the day of saying OK, this is about this is the future.
This is about the future and dragging whichever way you want.
Other tools, mostly commercial.
Deal with it using a dashboard where you can accept or reject the differences as you wish.
It's basically a salt problem.
H tool in its own way, but you have to think you need to manage those screens on.
This comes to this one the big one, testing all those responsive wits and browsers We're talking 10 24 retina iPad.
We've seen that, and it's getting more and more interesting because more and more sites are becoming responsive.
Not only that, it's not just responsiveness.
It's chrome Firefox, safari, etcetera, etcetera, different pixel densities.
We're talking different pixel density is also do we abandon ship because, oh my God, how how do we do that?
One solution is just run the test multiple times.
I have a four each on the test on all the sizes.
I want iPhone, iPad, et cetera.
On each, I run the test multiple times and change the view port accordingly and take the screen shot and we're done definitely doable.
The other solution.
Well, that's a solution, but it takes a lot of time.
If I have 100 tests now, I have 500 tests.
The other solution is do the same, but paralyzed the testing.
That takes a lot of infrastructure, but it's definitely do doable in A lot of companies do that.
Solution number three is Let a cloud provider do that for you.
Send over not the screenshot, but sent over a snapshot of your Dom in your CSS et cetera and then run all those tests all those screen shots on a lot of browsers.
In the end, it really is as easy as this.
I want to show a demo because I do have some more time.
Oh, no, let's see a demo.
Okay, let's let's start with the application under test.
This is the application under test.
I can filter right here test and it filters it.
Obviously, it's all responsive, but you've seen that already.
Now let's look at the test.
Let's look at the regular O.
S s tool, Asai said.
It's just as it were.
View port, visit, get type, type the test thing and we're done.
I will be using capitals eyes where I have to open the test and close the test, but it's basically very simple.
Simple.
I run the actions and then do a check window.
Very similar.
Same tooling as I said, they are all the same.
Let's let's run the test.
We can run it non interactively.
But Cyprus is really cool.
Um so let's do that.
There we go.
The test is running.
Hopefully.
Are we good?
Yes, WiFi is working.
At least the Ethernet is working.
It's running.
It did the check window, but now it's waiting for the answer from from Capitals.
From Eyes on, let's go to the dashboard.
And there it is.
We've just passed notice that it doesn't give one screenshot.
All that we do did give one screenshot.
It's running four tests and chrome and one testing.
11.
This is actually an iPhone X emulation.
So you get the responsive design.
As I said, we're we're letting the cloud do our work for us.
Okay, now let's break something.
Sorry about the artificiality of it.
This is No, that's not the extent of my sisters.
I know it better, but Okay, so let's change the title to read.
I will close it.
There we go and we'll run it again.
So it's running now.
You can.
You can see that the testing bookstore is now red and it's running, throwing over those dumb snapshots to the cloud and waiting for the results of those tests.
Let's go to wherever I need to go and we have an error in all the pages.
Okay, As you can see, it's this error.
I'll show it like this.
Okay, uh, now we need to say whether this is a bugger, our feature.
Let's say that it's a future We wanted to change it to read.
Sorry, my design abilities are really low.
What I can do is I can group all of them on Dhe Approve the two that it grouped on and save the changes.
I won't save the changes.
So maybe if I give the talk another time, I can do that again.
So basically, this is the level of its basically that easy.
Remember, just write the code that uses the tool.
Does the visits does the clicks?
Wherever you want to take a screenshot, you use the tool to take that screen shot, and then you do have the managing of the screen shots.
So, as I said it really is as easy as that.
As long as you take care of those four elephants.
That's important.
I'm in summary.
So do we believe the rumors?
No writing tests for CSS is possible.
And so join the back end in the front and JavaScript developers start testing your CSS.
You can cure your CSS ago program.
A phobia.
It's a hard problem.
Don't don't worry.
It is a hard problem because it's a visual one and visual stuff is always hard Taking the screenshots managing those screenshots.
Comparing the screen shot and take testing across various browser whips and heights and platforms is really, really difficult.
But it is solvable with open source and commercial tools.
So who knows what this is?
Come on, raise your hands.
Anybody read?
Do and thank you By Frank Herbert.
Uh, we're gonna get a movie soon.
I must not fear.
We're talking about fear.
Fear is the mind killer.
Fear is the little death that brings total obliteration.
I will test my c assess.
I will permit it to pass over and through me.
And when it has gone past, I will turn in her eye to see its path where their fear has gone where the fear is gone There will be nothing on Lee.
My tests will remain.
Thank you very much.