One of the OpenStack projects that seem to get less attention than others such as Nova and Neutron is the Cinder block storage service project.  I though it may be helpful to others if I wrote up some things I’ve learned about Cinder.  I’ll start, in this post, by walking through the basics of Cinder; then following on my recent series on VMware vSphere with OpenStack Nova Compute, I wrote a post on how vSphere uniquely integrates with this block storage service.  First, let’s put Cinder in proper context by taking a look at the available storage options in OpenStack.

Storage Options in OpenStack

There are currently three storage options available with OpenStack – ephemeral, block (Cinder), and object (Swift) storage.  The table below from the OpenStack Operations Guide helpfully outlines the differences between the three options:Cloud instances are typically created with at least one ephemeral disk which is used to run the VM guest operating system and boot partition; an ephemeral disk is purged and deleted when an instance is terminated.  Swift was included as one of original OpenStack projects to provide durable, scale-out object storage.  Initially, block services was created as a component of Nova Compute, called Nova Volumes; that has since been broken out as its own project called Cinder.

So what are the characteristics of Cinder and what are some its benefits?  Cinder provides instances with block storage volumes that persist even when the instances they are attached to are terminated.  A single block volume can only be attached to a single instance at any one time but provides the flexibility to be detached from one instance and attached to another instance.  The best analogy to this is of a USB drive which can be attached to one machine and moved to a second machine with any data on that drive staying intact across multiple machines.  An instance can attach multiple block volumes.

In a Cloud platform such as OpenStack, persistent block storage has several potential use cases:

• If for some reason you need to terminate and relaunch an instance, you can keep any “non-disposable” data on Cinder volumes and re-attached them to the new instance.
• If an instance misbehaves or “crashes” unexpectedly, you can launch a new instance and attach Cinder volumes to that new instance with data intact.
• If a compute node crashes, you can launch new instances on surviving compute nodes and attach Cinder volumes to those new instances with data intact.
• Using a dedicated storage node or storage subsystem to host Cinder volumes, capacity can be provided that is greater than what is available via the direct-attached storage in the compute nodes (Note that is also true if using NFS with shared storage but without data persistence).
• A Cinder volume can be used as the boot disk for a Cloud instance; in that scenario, an ephemeral disk is not required.

The Cinder Architecture

As seen in the diagram above from Avishay traeger of IBM, the Cinder block service is delivered using the following daemons:

• cinder-api – A WSGI app that authenticates and routes requests throughout the Block Storage service.  It supports the OpenStack APIs only, although there is a translation that can be done through Nova’s EC2 interface, which calls in to the cinder client.
• cinder-scheduler – Schedules and routes requests to the appropriate volume service.  Depending upon your configuration, this may be simple round-robin scheduling to the running volume services, or it can be more sophisticated through the use of the Filter Scheduler.
• cinder-volume – Manages Block Storage devices, specifically the back-end devices themselves.
• cinder-backup – Provides a means to back up a Cinder Volume to various backup targets.

Cinder Workflow Using Commodity Storage

As originally conceived, all cinder daemons can be hosted on a single Cinder node or alternatively, all cinder daemons, expect cinder-volume can be hosted on the Cloud Controller nodes with the cinder volume daemon installed on one or more cinder-volume nodes that would function as virtual storage arrays (VSA).  These VSAs are typically Linux servers with commodity storage; volumes are created and presented to compute nodes using Logical Volume Manager (LVM).

The following high-level procedure, outlined in the OpenStack Cloud Administrator Guide and visualized in a diagram from Avishay traeger of IBM, demonstrates how a Cinder volume is attached to a Cloud instance when invoked via the command-line, the Horizon dashboard, or the OpenStack API:

Cinder Workflow Using Enterprise Storage Solutions

Cinder also supports drivers that allow Cinder volumes to be created and presented using storage solutions from vendors such as EMC, NetApp, Ceph, GlusterFS, etc..  These solutions can be used in place of Linux servers with commodity storage leveraging LVM.

Each vendor’s implementation is slightly different so I would advise readers to look at the OpenStack Configuration Reference Guide to understand how a specific storage vendor is implementing Cinder with their solution.  As an example, James Ruddy, from the Office of the CTO at EMC, has a recent blog post walking through installing Cinder with ViPR.  Note that the architecture and workflow is similar across all vendors except that the cinder-volume node communicates with the storage solution to manage volumes but does not function as a VSA to create and present Cinder volumes using storage attached to the cinder-volume node.

I am intentionally keeping to the basics in this post, leaving out such Cinder features such as snapshots and boot from volume.  For more information on other facets of Cinder, I would encourage readers to read through the OpenStack Configuration Reference Guide and the OpenStack Cloud Administrator Guide.  There, you can also get walk-throughs on how to set up Cinder using commodity hardware and enterprise storage solutions.  In part 2, I detail what VMware has done in Havana to integrate Cinder with vSphere.

Related articles

• Laying Cinder Block (Volumes) In OpenStack, Part 1: The Basics (cloudarchitectmusings.com)