Google I/O 2014 – Perf culture


PAUL: Welcome to this session
on Performance Culture. I am Paul, on the left there. I work on Google’s
Developer Relations team where I spend my time looking
at performance, design, and UX and normally where the
three of those meet. LARA: And I am Lara Swanson. I work at Etsy and I’m
the Engineering Manager for the Performance team. We help all of the
feature teams who are building products or
features or experiments, make sure that whatever
they’re shipping is as fast as humanly possible. PAUL: Alright so, I guess the
opening question is, why are we even talking about
performance cultures? Why does this
session even exist? And I think to
answer that we need to understand a little bit
about where our industry is and what we are seeing and
where we think it’s going to go. And it’s really about this. The multi-device web. We’ve seen more devices
than ever before come online and I think really the
one that stands out for me the most is this one. It’s the smartphone. It represents the most
performance constrained device that we have. Whether that is in terms
of the screen size, or the GPU, the CPU, or its
connection to the network. And the thing is more and more
people are using smartphones. In fact, one study suggests
that just over a third of American adults
use their smartphone as the primary means of
accessing the internet. Which means for those people
their first experience of you, your site,
your brand is going to be through their smartphone. And that is something that we
need to be thinking through. In fact, it’s not just in the
States it’s also globally. So, here we have the percentage
of total global internet traffic according
to Stat Counter and you can see that the
trend is pretty clear upwards. And it looks I think sort
of the middle of next year, if the trend line is
correct, that well cross that 50% marker. So the thing about this is– LARA: Mobile networks can add
a tremendous amount of latency. So, before a mobile device
can transmit or receive data it has to establish a radio
channel with the network. And this can take several
seconds over a 3G connection. After the device
talks to a radio tower to negotiate when it
can transmit data, the network carrier then
must transmit that data to its internal network, and
then to the public internet. So, the combination
of these steps can add up to tens of
thousands of milliseconds of extra latency. So, the most
important question I hear when I say this
is, well what about 4G? 4G is awesome right? Our networks are improving. But you have to remember the
new device and fast network with an excellent connection
that you have in your pocket right now is not a good
representation of what your end users are really experiencing. So, while it’s true that
networks are getting better you need to remember
that your users are on a variety of connections,
with various levels of connectivity, and latency. It’s also important to remember
that if you’re a site is slow people will go elsewhere. In fact, one study suggests
that as many as 40% of people will leave a site when it
takes longer than three seconds to load. So, we have some options. In the face of the stuff
that’s a little bit scary what can we do about it? The first option
is to ignore it. You can certainly say,
you know what we just saw those awesome stats. We say that we’re
actually– we’re trying, but we don’t think
it’s that important. Maybe mobile isn’t the future,
maybe that trend line is going to lie to us, maybe networks
will be better immediately, I don’t think that this is true. PAUL: No, I tend to think– no. LARA: I think that we
can’t just ignore it. PAUL: I’d like to. LARA: Yeah, it’d be
really cool, but I think the other option is
to assign performance cops. Right? Who in here would identified
as a performance cop or janitor within their organization? PAUL: Anybody ever
been one of those? LARA: Yeah, somebody who
comes in afterwards cleans up. PAUL: Look at these hands. Look at this. [DEEP VOICE]
Yes that’s me, I don’t know. LARA: So, performance
cops, right? We come in and we say, hey
designer, hey developer here’s some better ways
you can do this. You come in cleaning
up after people, you try to make
your site better, but the responsibility
solely rests on you. And this can often lead to a
tremendous amount of burnout and frankly, it’s
not sustainable. There’s always going to
be new people joining your organization, your site
will continue to get slower, it’ll continue to
be iterated upon, the hardware age,
being a performance cop is not a sustainable
thing, so that’s why we’re here to talk about
building performance cultures. PAUL: Wow, I wonder which one
we’ll choose from that list? [LAUGH] PAUL: So, we’re all set to build
a performance culture, super. But we have to ask
the question what do we even mean by a culture. Well I guess you could come
up with your own definitions goodness knows there’s
probably a bunch of them that you could think
up, but for us today, at least in this conversation,
here’s the things that we actually have in mind. Firstly, it’s a way of saying,
I belong to this group, I get them, hopefully
they get me, I understand this
particular group. So, for example being
British, I would identify with British culture. LARA: You’re British? I know right it’s a relief
for anybody in the room going, where is he from? His American accent
is really strange. LARA: [LAUGH] Yeah, I know. So, it’s that first and
foremost is just saying, yep this is my kind of people. It’s also about how
you think and how you reason and rationalize
the world around you and the kind of
social cues that you look for or the slightly strange
obsession with green liquids you may or may
not have, but it’s that sense of the
world around you. It’s also how you do things. Whether that’s the
side of the road you drive on maybe the
wrong side, the words that you use for
things or perhaps in our case how we go
about crafting the code. And then lastly, it’s
how you celebrate things, the things that are
important to you, the things that you
celebrate and how you go about celebrating it
is unique to your culture. And we see cultures
at the highest level internationally, nationally,
all the way down to our homes and so forth, and actually
of course, our workplaces, which brings us back to that
whole performance culture thing. LARA: Yeah, we
were joking when we were talking about earlier
that performance cultures are kind of a team sport. It’s important to remember
that everybody has to play. And as you say about sports
there is no “I” in performance. PAUL: That is true, but if
you’re willing to look hard enough you’ll find there
is “prance-for-me”. You can stop the slide now. LARA: Alright, alright. So, thank you for that. So, culture change is
scary and it’s hard and I can understand especially
for those of us in this room, we don’t know necessarily
how to approach it. How do we start to create
or enact performance culture within our environments? I’m going to go through
some real things that have been said to
me as I’ve gone on this journey within the
organizations I’ve worked in. Lara, “I don’t want to
think about mobile.” Right. PAUL: So whiney! LARA: Yeah. “Lara, I’m a project
manager or a product manager and I’ve look at the deadlines,
I’ve got some resources and frankly building for both
desktop and mobile is going to take twice as long.”
[WHISPER] It’s a lie. “Lara, you at the
performance Manager, don’t you know
responsive web design is inherently bad for performance?” Or my personal favorite,
“everything changes too fast!” So, here’s the thing we’ve
been through this before and when it comes
to web development we’ve gone through these
kinds of change the past. Who here has built sites
with tables for layout? Yes. PAUL: Yes! Look at all these guilty hands. [DEEP VOICE] I don’t
do it any more. LARA: It was a pain. So, at probably every job
I worked at between 2006 and 2010 I remember
these arguments, do we have to move to
a CSS based layout? In fact, we did. We did it pretty successfully. This is more like what
we would use today, as is actually said to change
again with web components. Who has heard this from a very
important person, excuse me, the important part is
not above the fold. When the EP’s would have
to scroll to see content. Yeah. This is again what
we’re seeing today. We used to worry about
800 by 600 pixel monitors, now we’re worrying about the
spectrum of screen sizes. This graphic represents
all the devices that come to ETSY.com that
have more than 1,000,000 visits per month in traffic. We’re no longer seeing
just handsets and tablets and desktops we’re seeing
a spectrum of screen sizes. You may also think
about how we’ve changed developing for
different browsers over time. I remember back in the day we
would talk about cross browser compatibility or making things
look the same across browsers. Browser hacks, prefixes,
et cetera and this is again what
we’re seeing today. We have to start thinking
about mobile web browsers in addition to just
our desktop clients. But this is all to say
that change is scary, but it’s also constant in
our industry and it’s good. This is a good part of our jobs. We’ve lost crazy table
layouts and now we have semantic markup and
more people than ever before can use our
sights on our apps. This is good for
us as developers and it’s really good for users. PAUL: So, we need to
embrace change, good, fine. So, where do we
start when we want to create one of these
performance cultures? Well, the first step for us is
to actually gather some data because essentially you
don’t know where you’re going if you don’t
know where you are and you need to understand
how real people use your site, like what
devices are they on? Where are they in the world? What journey did they
take through your site? Which pages did they
arrive on and which ones did they leave on? You need to understand
your own situation and if you don’t
know today you can’t start telling that story
to the people around you and get them on board to
build this culture with you. So, the immediate thing
that springs to mind is just going and have a
look at your analytics. Go and look at Google Analytics. Figure out what
it’s telling you. Spend a bit of
time, make reports, and start to get that
sense of where you’re at. The next thing you
want to do is actually look at you load stats. So, how long does
it take to load it? How long does it feel like
it takes to load your site? Again this is coming down to
the data that you can share, but it’s also in
this case coming down to a little bit of
empathy as well. How does it actually feel to
be one of those people that’s using your site on
a day to day basis from somewhere
else in the world. And for that we can
turn to Web Page Test and you can ask it to load your
site from a range of locations, a bunch of devices, and
different connection types. And it is going to give you a
load of really useful, really amazing data. And one feature I
particularly like is the ability to compare pages,
so you could take for example your own site and
oh, I don’t know, the site of your co-presenter
and see who’s fastest. LARA: Not cool bro. PAUL: Oh yeah. I actually did
that, as you can see and depressingly it’s a draw. So, yeah. Could have been
bad, but it wasn’t. I’m sure you can
start to see how you could start figuring
out the story here. Like how does it feel
to be one of our users when you compare it to somebody
else in the world or– da-duh, da-duh, da-duh
you get this idea. OK. So, you start to
get in this insight and you don’t even have
to leave your desk. That’s the best bit. But we can get even more
data at Web Page Test and we can share
that with our team. Like how long does it take
before our server sends some data back or how
long before our pages start rendering? But one number I
want to call out is this, which is
the speed index. It’s a bit weird the
way it’s calculated. You can click on
there there’s a link. And you can click on it
and it will tell you, in all the gory detail,
how it figures it out and it’s pretty cool. LARA: It’s magic. PAUL: This is magic. But it is a really good
number for figuring out in milliseconds
how long it feels like it takes for
your page to load. So, that’s a really
good measurement, better than say,
the onload event. But speaking of
perceived performance, we don’t just want
to stop caring when people– when
they’ve landed on the site and we go oh, what have we done. We need to care about scrolling
and animations and interactions and they all need to be
this silky smooth 60 frames a second because
people care about that. They want that feel of
responsiveness and smoothness. Again you go to like Dev Tools
to look at your pages for data and figure out what your
frames per second is like. So we get frames
per second, we can dive into the
frames a little bit there and find out
where we’ve got issues and then start fixing them. Now if all that’s
new to you it’s time for a shameless
plug, which is to say I’ve got a session
tomorrow morning that will tell you exactly what you need
to know about these tools, “The Tools You Need to Succeed.” It rhymes, it’s true. So, come to that if
you’re interested. Anyway, hopefully at the
end of this situation we have a load of information,
we have a lot of data, and we’re starting to hopefully
build this story in our heads of what it feels like
to be one of our users. LARA: So, you’ve got this data
and you’re excited about it. What do you do with
all of that excitement and all of that data? You should go talk to some
very important people. You want to enlist the help
of the very important people at your company and get them to
understand what your users are feeling and experiencing. There are few
things more powerful than a very important person
saying, this is important. Performance is important. It’s crucial to getting others
at your company invested in performance. So, what can you do
to make this happen? Here’s an idea that
we came up with, you should send your very
important person one stat a day for two weeks and see how
long it takes before they cave or just get really
annoyed with you. One that we often like to
cite is this Google study that found an additional
200 milliseconds of delay triggered a 0.3%
loss of engagement. You may say to yourself,
0.3% is not a lot, but Google is pretty popular. So any kind of
loss of engagement is going to have a huge effect. Or remember that old stat that
we just had where 40% of people will leave a site that takes
longer than 3 seconds to load, tell your very important person
that stat and say, by the way our takes 5 seconds to
load on a 3G connection. So, after the stat
thing has run its course consider running
some experiments. Measure the business impact
of these experiments. Know what kinds of metrics
your very important person cares about. Maybe it’s balance rate,
maybe it’s conversion rate, maybe it’s something else. Go ahead and run
performance experiments and see what the effects are. On Etsy, we ran an
experiment in which we added 160 kilobytes
of page weight to a page on a
mobile device and we saw that it increased
mobile bounce rates by 12%. This is the kind
of thing that we can share with the folks
at Etsy and say, hey, this stuff is important look,
at the effects that it has. However, this is kind
of a negative stat and I kind of prefer
to make a positive. So, what we found when
we eliminated jank, that’s that
scrolling issue, when we eliminated jank
on Etsy’s activity feed we saw people
“favorite” more often, and “favorite” more items. And this is another
kind of experiment that you could run, make
something better and measure the effects before and
after and pick and choose the engagement
metrics that you know your very important person
is going to care about. You could also use that web
page test film strip, thanks for that– PAUL: Your welcome. LARA: –and take
the video output and compare that before and
the after the experiment, make an improvement, show
how slow it was before and show how long– how
short it took afterwards. And get that person to care
and feel about how this works. Or compare your site on desktop
to how it loads on mobile or compare you’re site
in the United States to how it loads globally. Get your very important person
to see and to feel these things so that they actually
care about this stuff too. You want to get them to feel it. Once they understand you can
help them empower others. By changing how you’re
very important people think about performance it will help
everybody who’s surrounds you. PM’s, designers, developers
care and work on performance. PAUL: Speaking about those other
people, so, we’ve got data, we’ve got our VIP’s to say,
OK performance is important, we see that it’s got real value. How do we then make it so
that the people around us can actually– start actually
taking this and using it. Well, the easiest way is to
educate them through something like a brown bag,
lunch and learn session on all the tools, all
the data, all the things that you’ve figured out about
how you’re actually doing. And it’s not– as we
said it’s not just developers, it’s
designers, is UX, it’s project
managers, it’s anyone who’s involved in what you
do because it’s a team sport. And we want to make sure
that the people that we’re working with know the impact of
the decisions that they take, so it’s not the case
of, well the developers have done all they can,
but actually the problem is originated from
another place. You know if you
include everybody you’ve got a much better
chance of success. The next thing that’s
well worth doing is to start documenting
your best practices and start writing things up. Because what gives
you is an easily copy, pasteable set up that you have. It makes it more
obvious about how to do the right things as
far as you’re concerned. Anybody who joins your company
is going to start figuring out, oh this is how I’m
supposed to do things, rather than just having to guess
and muddle their way through. And you can use it to show what
worked and what didn’t work and if nothing else it kind of
gives everybody a focal point and say, right this is– because
you will find there will be trade offs in everything you do. This thing and doing it this
way works because of this. So it’s a really
good place to sort of– it’s really good to start
writing these things down. Another thing that is
definitely worth doing is seeing what you’re
uses are seeing, which is to say invest
in a device lab. And I mentioned this to
somebody the other day and they went, “psha
$10,000, really?” Because that’s what they were
assuming it was going to cost. Well first of all it
doesn’t necessarily have to cost that much. Secondarily, the cost of not
testing across multiple devices may be way higher and
so, that’s something that you want to
decide for your selves. But it’s a sort of–
it’s an out of sight out of mind kind of
problem, this one. If you put these devices
in front of everybody it kind of gives
everyone that reminded that there’s this whole range of
devices they should be testing on and that they are building
for the multi-device web and that they need to care about
performance on all of them. It’s also a social
place as well, so people can meet and
gather around these devices and start to figure out
together how you can solve particular problems and
again that will broaden the horizons beyond
just the developers. One thing that is pretty cool
out in the Design Sandbox, if you haven’t seen it
we have a Device Lab. And the team there
will talk to you about how you can go about
making your own device lab. The source apparently
is on GitHub because it does some
crazy stuff where it which is to all the devices
at the same time, which is really cool. So go and see the Design Sandbox
people they will help you. LARA: And then also if you go
to laraswanson.com/devicelab you can see all the slides that I
just gave for a tutorial on how to build your own device lab. How to choose devices, how to
troubleshoot power, and all of the other things that
we’ve learned along the way. PAUL: Awesome. Alright, so, you– it’s
great to have the device lab, but it’s not great
if you don’t use it. It’s a bit like having a gym
membership and they never getting on the treadmill. Yeah. So, make time to actually
test your stuff for real. It’s good you’ve
hopefully got the VIP’s to say it’s important so you can
say actually we need time now to make sure that
things are working well. So go and give your
devices a big old hug. Let them know you love them. Like we said earlier
though educating your team isn’t just about
developers and performance isn’t just about an
internal thing some of it– sorry it’s just not an external
thing it’s also internal, get that the right way around. It’s about everyone who builds
for the web in your company. And that means in a
lot of ways that you want to eliminate
duplicate code, you want to basically
embrace responsive. Instead of having a mobile
team that does the M-dots and you know the other team
that does all the other stuff. M-dot. What do you do for TV? Do you do TV-dot? LARA: What do you do
for refrigerators? PAUL: F-dot? LARA: R-dot? PAUL: R-dot. LARA: I don’t know. Don’t do that. PAUL: No, don’t do that. That seems like it’s
not going to scale well. LARA: No. PAUL: OK, you said
earlier they do taxies. LARA: Oh yeah, taxis in
New York have the little– you can access the internet. Mind, [SHOOTING SOUND] blown. Alright, point
here is your going to eliminate duplicate
code, you going to make it easier for
your team to iterate more quickly, hopefully,
and start to break down the barriers between you, the
designers, the developers, and it just gives you a much
greater chance of success. So, in summary, when you’ve got
the green light from your VIP start teaching your
team about performance and the impact of their work,
it’s going to empower them. Sorted. LARA: So, you need to
start surfacing performance within People’s daily lives. We can talk about
performance all day about how important it is and
when you ship something new you can see the end of the day how
much performance was impacted negatively or positively. But this isn’t help
people as they’re designing or developing. So, the first thing you can
do is add performance to your build tools. Automate image compression. Get a command line tool
running like Grunt or Gulp that runs performance
tests and compares your new work to your
performance budget. You could use
something like Travis to do the in a continuous
integration environment. And these tools can
even report back to dashboards, which is
my next recommendation. So, at Etsy we look at something
like this every time we deploy code, which happens
upwards of 50 times a day. When someone deploys
something new they’ll go over to
a dashboard and say, how did this impact performance? It’s really important that
as people are continually pushing out new work
they can see immediately how their stuff has affected it. But also at Etsy we
have this tool bar that sits at the top of
every page of etsy.com if you logged in as
an Etsy employee. And in this we’ve
got information about visitor traffic
and experiments, but we also have
performance data. So, we surface the
timing’s for people as they’re developing
in developing, staging, and production environments. And we also have
a clear baseline for what is acceptable. We’ll let you know
if you’re violating one of our performance
service level agreements. By surfacing your
team’s performance data during the workflow it’ll
help improve their work, it’ll become baked
into their workflow. They’ll start to care
about it because they can see how their work
directly impacts it as they’re designing
and developing. And this brings me
to our last point. This stuff is going to
make people so excited and people are
going to ship things that are improvements
to your site. You should be
celebrating everything. Again, I’m all about
the excitement. I’m all about getting people
pumped up about performance. PAUL: Prepping
the talk with Lara was like an object
lesson in excitement. LARA: Nice. So, for me, this is actually
the most important part. Back in the day
in August of 2011 the man behind the
sunglasses, Seth Walker, published Etsy’s first
performance report. He published it on Etsy’s news
blog, so it’s super public. He chose a bunch of baseline
business metrics or page load time metrics,
rather and he showed how long are our major
pages were taking to load and you can see there’s a huge
outlier here, the homepage. It was embarrassing, but
he published it anyway because this stuff
isn’t a secret. What’s amazing about this
is that developers teamed up to fix this. As soon as we
publish it, as soon as Seth put that into the world
people got excited about it and they wanted to help
and they want to fix it. And by the next time we
published reports, which was in November of 2011
there was a huge improvement. Not only did the home
page significantly improve in page load time. It was the fastest
one on this list. So, you should be
sharing your findings. Make sure that
you’re celebrating externally the stuff that you’re
doing to help improve things along the way, but
also as Paul mentioned it’s important to do
this internally as well. This is an example
dashboard that we have, the Performance
Hero a dashboard, in which we celebrate
engineers and designers who make huge improvements
to Etsy’s page load time and they’re not on
the performance team. In this case Chris Fairbanks,
who’s on the checkout team, eliminated a ton
of duplicate code and had what we
call a Baumgartner. Remember that guy who jumped
out of the hot air balloon and landed on earth? Yeah so every time we
have a graph like this– PAUL: It wasn’t a
dance move, right? LARA: It wasn’t– no, it wasn’t
a dance move it was a plummet to earth. Every time we have one
of these, a Baumgartner, we celebrate this internally. We put this up on
a dashboard and we have that little printout
sign, and we hang out with them and we’re like, thanks. Thanks buddy. And we want to celebrate
the things internally lots of high fives, lots
of doughnuts, lots of cake. You should be rewarding
your innovators. It’s not just about
coming down with a hammer. It’s not about being
a cop or a janitor. This is really about celebrating
improvements and getting people excited and invested
in their performance. Celebrating wins will
motivate your team to be performance champions. So, to summarize
gather the data. You want to make
sure that you have a baseline of understanding
how your users feel. Again, this is all about
understanding the overall user experience. It’s the aesthetics and
it’s page load time. You then want to share all
that stuff with your VIP’s. Get them to understand what your
users are really experiencing. Get them to understand what it’s
like on mobile and globally. Get them to understand also how
this stuff impacts the business metrics, run some experiments
and share that data with them. Once you get a very
important person to say, this is important or just a few
find that it’s awesome enough to say this is important,
educate your team. Do those brown bags
and lunch and learns. Make sure that your surfacing
that data in their daily work flows. Make sure also– your
celebrating things. Again this is a journey right? You may be at any
step in this process and you might have to start
again gathering more data, getting more VIP buy in,
educating the team again. New designers and
developers will join and you’ll have to do more
brown bag lunch and learns. New technologies will
emerge and you’ll want to share that data and
that new information with them. You’ll find new wins and you
want to celebrate that stuff. This is a continual journey and
continually positive process. PAUL: OK. So, as we said at
the start we’re in this period of
dramatic change where we’re seeing mobile
traffic go, whoosh straight up, and we’re facing this
world where essentially we need to deal with these
performance constrained devices like building for this
tomorrow that we’re looking at involves facing
that fight and saying you know what most of the
devices that people are going to be using are not desktop,
they are not powerful devices. The thing is it’s not really
tomorrow it’s kind of now. And that’s the thing about
dealing with performance is it really starts and it
can and should start today, but the only way you’re
going to get to that point is if absolutely everybody in
your organization from the top to the bottom cares deeply about
performance and its impact. And the only way
you’re going to get there is if you start
building performance cultures. Thank you. We’d love your feedback. Please do snap the code
or take down the URL. And we’ll be outside if you
want to ask any questions and say, hi. LARA: Come hang out. PAUL: Yeah. LARA: Thanks everybody. PAUL: Thank you.

Leave a Reply

Your email address will not be published. Required fields are marked *