OpenStack block storage 101 final - VoiceTube: Learn English through videos!

Subtitles section Play video

Hey there. My name is John Griffith. I am an open source engineer at SolidFire out in
Boulder, Colorado. We're building a clustered storage device that's designed specifically
for the cloud. Today I wanted to do a quick introduction
into OpenStack, give you a 101 as far as what OpenStack is, how it came to be, what problem
it's trying to solve, and maybe a little bit about what it's not, as well. I'd also like
to talk a little bit about the block storage project, specifically, and give you some more
information about what that is, what you can do with it, what some of the features are,
and a little bit about the design. Anyway, this is definitely a 101-type introduction.
Hopefully, this will get you started and get you enough information. We can continue and
move on, and we can dig deeper after this. All of this pretty much started with the hypervisor.
The hypervisor gave us the ability to run virtual machines all on a single box, and
really start to utilize our hardware better, and save us from things like multiple boxes
under the desk. If you're like me, you can remember the day when you had four or five
workstations sitting under your desk, each one having a different OS, or maybe all even
the same OS just different configurations. You had to do compile and testing for your
code and everything on each one individually. It was somewhat of a pain and somewhat expensive,
as well. Thanks to the hypervisor, we started to have
the ability to do things like have one single workstation and be able to run four or five
different VMs inside of it, each with our configuration. You could also go ahead and
save those configurations and save those setups, and so you could bring them back up later
as a VM without having a server or a workstation sitting around doing nothing.
Over the years, as virtualization technology continued to improve and we continue to make
advancements in compute [00:02:00] resources, what happened was is we started to move more
and more things into the hypervisor, so we started to do more things there, whether it
be running ecommerce apps, web apps, web servers, DNS servers, so on and so forth. That was
all great, and we continue to scale that, but what we started to run into then was it
started to become more and more difficult to do things like deploy a VM. You may not
have direct access. You would have to go to an admin, an IT admin, and request a VM be
deployed, not to mention the difficulty added with trying to deploy more hypervisors.
All of these things started to get more and more complex. Then you started to see things
like storage performance degradations, trouble setting up networks and getting everything
to talk. We started to really get into some management complexity that caused some difficulties,
and started to lose some of the efficiencies that we've gained. Now, it's great. We're
utilizing our hardware much, much better, but on the other hand, things are getting
increasingly complicated and difficult to manage.
That's where the whole concept of cloud computing starts to come in. It's no longer just about
a collection of virtual machines. It's now an orchestration or a management layer of
hypervisors in virtualization. In addition to that, there's some key concepts and key
things that I look for in terms of what is a cloud and what is cloud computing. The whole
concept to keep in mind is that it's dynamic. It's elastic resources that are on demand,
so it's not a "file a ticket and next week, or hopefully in 72 hours, or whatever, I'll
get a VM." It's, "I need it now. I get it now."
There's some key aspects. Like I said, the on demand. One of the other ones is it's self-service
for the end user. This means that, typically, you go to a webpage or internal network portal,
whatever it might be, and you actually request your resources from there and set them up
yourself [00:04:00], get them right away, use them, and then return them when you're
done. Some of the key things that you look for as well are the ability to scale dynamically.
By scale, I mean horizontally scale. It's not, "take down the server and put in some
more RAM, or put in another drive," whatever it might be. It's actually, "scale horizontally,
throw in another node, and have that consumed and utilized as a resource."
In addition to all those key concepts, the other idea is to be able to take more and
more IT infrastructure and move it into that utility consumption model. Now we talk about
taking things like the networking stack and things like that, and moving it in as a resource
that you can actually check out and utilize on demand and pay for what you use, as opposed
to having to put three nicks in a machine, or whatever.
All of these things are great, and this all sounds fantastic, until you try and actually
build it, and then try and manage it and get it all to work together. That's where OpenStack
came in. Back in 2010, NASA and Rackspace were both facing these same sorts of challenges.
They got together and they created an open source project called OpenStack. The basic
premise of OpenStack is to deliver a platform for you to build and manage massively scalable,
super elastic, and feature-rich clouds with those characteristics that I talked about
in the last slide. That's the key. OpenStack provides a number of data center
resources, including compute, storage, and networking. There's also a significant number
of other projects that are starting up and services that are more specific. The key is,
is they all use the same building blocks, and they all use the same management-type
interface that OpenStack provides. That sounds great, and that gives you a basic
idea of what the marketing slide would say. What's the real deal here? What's OpenStack
really trying to do? On top of giving you the ability to actually build a cloud, which
is huge, the biggest things are is it's abstracting out physical [00:06:00] and virtualized resources
and giving you a single common management layer to work with them. We do do things more
than just virtual machines. We do do bare metal, as well. We do storage. We do networking,
so on and so forth. One of the keys about OpenStack is, is it
provides an easy-to-use REST API. Again, it's self-service, like we talked about. It's fully
dynamic. It's designed to scale. When I say scale, I mean massively scaled to the point
of tens of thousands of VMs running hundreds of nodes. That's going to continue to grow,
and we continue to push that limit. I've heard different talks on what some of the goals
are. I think it's basically, as long as somebody wants to grow, keep growing and fix what needs
to be fixed to make it do that. The good thing is, is that the architecture, as it stands
right now, will allow that, and it will let us continue to do that.
Some of the value propositions that are associated with OpenStack. One of the biggest things
about OpenStack is you have choice. You have the ability to configure things the way you
want, and use the things that you want. In some respects, this makes things a little
bit more complex and a little more difficult for people to grasp and figure out how to
set this up. On the other hand, it's fantastic, because it gives you the ability to utilize
what you want. Also, it gives you the ability to mix and match those things.
Hypervisors is a great example. We have support for VMware, KVM, Hyper-V, Xen, and some others.
The key is you can pick which one of those you want to use. Not only that, not only can
you pick which one, you can also have a single OpenStack install that utilizes all of them.
You can have Xen on one compute node. You could have KVM on another. You could have
VMware on another. The idea is all of those are abstracted out into the management layer
so that, from an end user's perspective, they just have to request, "Give me a VM." Then
depending on some of the characteristics that they might ask for in addition to just a basic
VM, that may determine which one of those [00:08:00] hypervisors it goes to.
The other thing about OpenStack is is that it supports the whole Infrastructure as a
Service stack, not just the compute side. We're talking storage, networking, security,
so on and so forth. We drive significantly greater resource utilization across the entire
infrastructure. The other thing is, is we fix some of the scaling issues that you have
with just compute virtualization or just the hypervisor. Those are some really big things.
The other thing that is key, that a lot of people are focusing in on and really want
to drive inside of OpenStack, is trying to deliver compatibility between private and
public clouds. Everybody knows, from the public cloud space, one of the big players that everybody
always thinks of is Amazon. The idea here is, with Amazon, you deploy some things in
Amazon, and that's great. Everything works fine, but you can't bring all of that same
stuff back into your in-house cloud quite so easily, because of the differences in compatibility,
and so on and so forth. Now, however, if you go to a service provider
that utilizes OpenStack and does a public cloud that way, you do have the ability to
do things like have overlap between the two. You can have a private cloud running OpenStack
and a public cloud that you go to a provider for running OpenStack. You can actually have
things cross and meld in between, which is a huge advantage. The key thing about OpenStack
is you're really bringing the whole concept of software-defined infrastructure to life.
You're really making that happen. Really similar to what we talked about with
the hypervisor, a lot of the early-use cases for OpenStack were test and development. Over
time, though, just like we saw with the hypervisor, we started to see more and more enterprise
IT, DevOps, continuous integration, things like that, all starting to move into OpenStack
as well and starting to move into the cloud. We're starting to see more and more things
like databases of service [00:10:00], whether that be MySQL, or MongoDB, or other NoSQL
type databases. More and more of those things are moving into OpenStack, as well. We're
also seeing it utilized heavily for internal web services as well as external web services.
We're seeing things like eCommerce, is a huge consumer of OpenStack. You can look at folks
like WebEx utilizing it. There's also situations starting with DVI. People are using it to
deploy DVI implementations. Being an open source project, there's always
some good information about the community to look at, and to see how healthy it is and
how things are going. OpenStack has a community of almost 10,000 people across 87 countries
right now, which is just massive. There's greater than 45 companies right now that contributed
to the last release of OpenStack. There's more than 40 listed case studies of public
companies willing to talk about what they've done with OpenStack, how they've used it,
and how it's worked out for them. There's over 200 vendors involved in the ecosystem.
By vendors, I mean companies like SolidFire that I work for that actually develop a product,
SolidFire, NetApp, Red Hat, Canonical, so on and so forth. The ecosystem is extremely
large and really, really diverse. Let's delve a little bit into storage specifically.
One of the first questions that I typically get from people is, "What's the difference
between Cinder and Swift?" Swift is object storage, while Cinder is traditional block
storage. We put together this little slide. You can take a look. It gives you a decoder
of what some of the different objectives are between the two, in terms of some of the different
use cases and workloads that you might target one over the other.
The block storage is definitely the more traditional, the high-performance-type workload, so on
and so forth ... high-change content, smaller, random read/write, [00:12:00] bursty IO, so
on and so forth ... while the object storage is more of, "Hey, I've got this object," whether
it be an image, or a file, or whatever, a video clip, so on and so forth, full object,
"that I'm actually moving from one place to another." That's the big thing.
That's one of the first questions that come up, so I wanted to talk on that. I'm not going
to go too much into the Swift or object storage. I do want to talk a little bit more about
the block storage project. OpenStack's block storage project is codenamed Cinder. It's
architected to provide traditional block-level storage and abstract those block storage resources
out. It presents persistent block storage in the form of volumes that can be used and
consumed by Nova. It can be either attached to a compute instance,
or it can actually be used as the storage location for a compute instance. When I say
instance, an instance in OpenStack is basically a VM. We call it an instance. Cinder manages
the creation, detaching, and deleting of these volumes across multiple storage back-ends.
Really similar to what we talked about in terms of the hypervisor choices. You can use
KVM, you can use Xen, you can use Hyper-V, so on and so forth.
We also do the same sort of model inside of the block storage project. You can have different
vendors' back-ends. You can have a NetApp back-end. You can have a SolidFire back-end,
an IBM back-end, whatever it might be. You can have those different back-ends if their
drivers are included in the project and supported. You can have those, and you can also still
also have multiples. You can have multiple nodes with multiple back-ends, or even a single
node with multiple back-ends attached to it that you can use. I'm not going to go into
too much detail about that. I'll touch on it a little bit more later. Then there is
also a built-in block storage back-end that is implemented using LVM that comes with OpenStack,
as well. Some of the basic features that you get with
the block storage [inaudible 00:13:59] with Cinder [00:14:00]. Of course, obviously, we
need create and delete of volumes. The thing that's nice is it's create and delete, again,
on demand. You need a volume, you go and you create it. The call is sent down to the back-end,
and the back-end is creating that resource on the fly dynamically as you need it, and
returning the information that you need to connect it and use it. Then when you're done
with it, you can delete it and have it just go away.
We also have the ability to do things like specify custom types, and then also add things
that we call extra-specs. An admin can go in and set up types and extra-specs for the
volume service. These will do things like specify, say, I want certain QoS levels or
I want a certain back-end to be targeted for this type, so on and so forth.
In addition, we also do things like cloning. You can take an existing volume, and you can
actually create a clone of it. You can copy an image out of the Glance, which is the image
repository. It's where all the images are stored to create instances. That would be
somewhat like an equivalent to your ISO or CD-ROM file for your Ubuntu VM, for example.
You can copy that image down to a volume, or you can take in a volume that you have
that's bootable, and actually send that back up to Glance, as well. What that does is that
allows you to do the whole boot-from-volume thing.
You can also do point-in-time copies, so snapshots. You can also turn around and create a volume
from those snapshots. The snapshots really, for the most part, are, like I said, they're
a point-in-time copy. They're more used as a vehicle for a lot of the underlying things
that the block storage service does, in my opinion. It's a great way to get that quick,
fast, instantaneous point-in-time copy of what's going on, of the system and stuff,
and then do things with it afterwards, like a clone, or make a new volume, or so on and
so forth, to actually replicate it. We also recently [00:16:00] have added things
like backup of volumes. You can back up to object storage, including Swift, and now Ceph
we recently added, as well. You can transfer the ownership of a volume. The whole concept
in OpenStack is you have what's called tenants. Tenants are individual users or projects inside
of OpenStack. They don't have visibility to each other's resources.
The idea is if you have a volume for your dev group A, and it's got some cool stuff
on it, or it's got a certain development environment, or something that you want to share with somebody,
or whatever, you can actually go ahead and transfer ownership of that volume over to
one of your other tenants in the cloud and give them access to it, or basically just
give them the volume. In addition, some of the really cool things
that we introduced the last release include scheduling. It used to be we just had simple,
basic scheduling, and it was really, really basic. What we have now is we have the ability
to set up scheduling filters. You can do things like, I mentioned before, use volume types,
for example, to specify what back-end you go to or a volume is deployed on.
You can also do more complex things. You can do things like capacity filters. You can specify
things to run in a manner where the volume that is requested is being deployed on the
next back-end with the most capacity. You can do things like volume counts, so on and
so forth. There's a number of options. Then in addition, you can also write your own custom
filters, which is a huge benefit. In addition to all these things, as with pretty
much everything inside of OpenStack, there's also per-tenant usage quotas that you can
set up. You can set up quotas for different tenants. You can say this tenant is only allowed
to set up this many volumes, or only allowed to consume this amount of storage, or they're
only allowed to create these types of volumes. There's a number of things that you can do
there. Again, there's a lot of [00:18:00] configuration, a lot of options, a lot of
choices that you can set up. One of the things that people ask about ... because
of the fact that the whole point of the block storage service is to abstract those resources
and treat them as pools, and stuff like that ... is, well, then, what's the advantage of
using, say, for example, a SolidFire device over the LVM, or whatever? There's, of course,
all the basic things that come with it, like performance, and efficiencies, power management,
all those sorts of things. On top of that, you still can expose your
unique features. You can do this either through custom types ... the volume types and the
extra-specs stuff that I talked about ... or OpenStack also provides this mechanism called
extensions. Extensions can be written and put in to do things above and beyond what
the API does on its own, to expose some of those extra features.
Again, like I said, different back-ends have different use cases. There's definitely clear
choices between them, just based on the characteristics of the device itself. There is also things
that you can do in terms of features that you can expose those things. You get the best
of both worlds without having things get too out of control in terms of that management
interface and that API. This slide shows a little depiction of how
things tie together in a really high-level architectural view of the block storage service.
You can see up there you have the user. That user is sending in a rest command to, whether
it be Nova, whether it be Cinder, whatever, into the service of interest. One of the things
that this doesn't show is that user or that interface, that could be a client, a command-line
client. It could be just a cURL call. It could be the OpenStack dashboard project horizon,
which is the web UI. It could be any one of those. They all work basically the same thing.
They'll send a request down to the service. For example, they'll send it to Cinder via
the volume API. We have a set API set up, and so the user [00:20:00] knows what to expect
and what's available. That API will then go ahead and deploy that out to the scheduler.
The scheduler will figure out what back-end service to use, so on and so forth, and actually
deploy the call down to the back-end. In the case of SolidFire, you can see we basically
just plug into the volume manager from a SolidFire driver. That driver then just sends JSON-RPC
commands down to the SolidFire cluster to do what we need.
I touched on earlier about all the different vendors involved in OpenStack. We wanted to
put together a little slide here to show how diverse this group is and how large it is.
You can see we've broken it up back to ... based on their major areas of interest, in terms
of where they contribute the most and where they play the most. You can see there's a
really diverse group. On the compute side alone, we've got HP, Dell, IBM, all the players.
Everybody's there. The same on storage. On the storage side, SolidFire, EMC, NetApp,
HP. The point about this is what you have is you
have all of the major vendors, all of the major players, all obviously agreeing and
believing that OpenStack is a good thing and a good direction forward. They're all contributing
and investing. What that also means for you is, for an end user or for an implementer,
is that you're not only getting what, for example, SolidFire thinks is good, or what
SolidFire thinks is the best way to do it. You're getting a collaboration among all the
top key players. That's the whole point of open-sourcing the project to begin with.
The whole idea is to take all of the best ideas and the best people from each of the
different vendors and each of the groups, putting them all together, and having them
all work together to build the best project possible. That's what I really think is happening
with OpenStack. I think the model is working extremely well. I think that the uptake [00:22:00]
that we have with all of the vendors and stuff is what makes it great and what makes it what
it is. One of the key things with an open source
project and with OpenStack, one of the key things that you need to do is you need to
be able to have some case studies and share those things. OpenStack's doing a really good
job of that, I think. There's a lot of really big household names that are coming out as
early adopters that are sharing their story, people like Best Buy, PayPal, Comcast, Bloomberg.
You look at all of those, and you get some information about what they're doing and stuff.
It brings all of this to life. It shows that it really is succeeding. It really is working,
and it's doing what it says it does. The thing about all of these early adopters
and these case studies, and so on and so forth, the key is, is all of these folks are deploying
OpenStack. They're all configuring it and using it differently. They're using all of
those customization features and things like that that, sure, adds complexity and makes
it a little more difficult, but at the same time, it also enables them to do what they
want. That's key. This is really cool to see these. You can check these out at the openstack.org
website and keep an eye on these. It's cool to see it grow, and see more people come in,
and see what they're doing, and learn more about it.
That was a really quick rundown on the basic concepts and the ideas. SolidFire is committed
to OpenStack. We believe in OpenStack. I am a dedicated resource. This is actually my
job at SolidFire, is to contribute to OpenStack. The key is not only to contribute to the benefit
for our product and making our device fit as good as possible. Of course, I think we
have a great device, and I think it's one of the best designs for cloud infrastructure.
But, we're really focused on the success and advancement of OpenStack in general for everybody.
I think that's key [00:24:00]. That's a little bit more about us and our
philosophy towards OpenStack. We're also really interested in partnering with other people
that have that same vision. That's why we've partnered with people like Rackspace and Nebula,
folks that have a global community view of OpenStack and advancing OpenStack as a whole.
There's a number of related resources you can check out, not only on the OpenStack side.
You can go to the SolidFire webpage. We actually have an OpenStack solutions page. You can
go there and get more information about OpenStack, specifically, what SolidFire's doing with
OpenStack via reference architectures, configuration guides for SolidFire and OpenStack.
You can also get some information about just basic features and things that are new inside
of the block storage project, inside of Cinder. We do a screencast. Every month or so, we
try and put up a new screencast about a new feature, how it works, how you can use it,
and just do a screencast demo on it. It's a real quick, easy-to-follow type of thing.
We're also got some blogs on there. We do recaps of each summit.
Every six months, OpenStack has a summit that everybody gets together. One of the key purposes
is all the engineers, all the contributors get together, and we hash out all the ideas
for the next cycle. We do six-month release cycles. We'll get together, and we'll spend
a week in some location and go through just figuring out, okay, what are the top features
we want to work on, and brainstorm. It's our chance to get together. For the other six
months, we just work remotely. We connect via IRC and things like that. It's a great
opportunity for us to all to get together and get in the same room. It's a pretty cool
event. If you're looking at this and you're thinking,
"Hey, I've got some ideas. I want to get involved. How do I contribute to OpenStack?", it's a
lot easier than you might think. The first place I tell people to start is if they want
to contribute [00:26:00], go to the wiki.openstack.org webpage. There's a "how to contribute" section.
It'll walk you through some of those things and get you started.
If you want to just play with OpenStack and get started with OpenStack ... I don't have
it on this slide. I should, but ... go to devstack.org. You can download and deploy
DevStack, which is a full-running OpenStack install and setup. You can run all of that
and do it in a VM. You can deploy the VM on your laptop, whatever, run DevStack on it,
and have a running and working OpenStack setup that you can play with, and get a better idea,
and get a feel, and start experimenting. Then from there, you can do things like dive in
and start figuring out how to do things, like add different options, and so on and so forth.
Then if you want, play with the code. If you're looking at this and you still have
some questions, you want to talk more or learn more, we're always happy to help. At SolidFire,
you can contact me with technical ideas or block-storage-specific things. My email there,
john.griffith@solidfire. If you're interested in partnerships ... like I mentioned, we like
to do partnerships with other people that are interested in OpenStack and advancing
the OpenStack project ... you can get a hold of McClain Buggell at solidfire.com. Then
of course, if you just want to learn more about SolidFire and you're interested in SolidFire,
check out sales@solidfire.com. They'll get you squared away.
Anyway, I hope that was a decent introduction for you and gave you a quick, high-level view
of what OpenStack is and where it's going. Feel free to send me any questions that you
might have. Also, keep an eye, like I said, on that OpenStack solutions page. We have
some webcasts and screencasts coming up. We'll continue to add things there, so there's more
to come. Thanks a lot.