Are You Ready for Extreme Scale?

Just as server virtualization and automation was the key to the IaaS market, software-defined storage on the right hardware is what takes storage to hyper-scale

Storage at cloud scale isn’t just about dealing with the ever-growing amount of data produced; it’s about meeting performance, availability and resiliency needs for a wide range of audiences and workloads at competitive cost, with demanding SLAs. Cloud storage providers, hosters and large enterprises consolidating into mega-scale data centers all need storage systems designed to deliver both performance-focused and capacity-focused storage efficiently and flexibly, with low management overhead and in a way that scales up and down efficiently with demand.

Fast or far away

Traditional storage architectures centralize I/O but monolithic storage arrays can’t match the performance of directly-attached disks; the disk controllers and storage network interfaces throttle I/O throughput. A transactional database can run out of CPU capacity on a server with a large number of cores long before it hits the I/O limits of SSD, but a storage array could be a bottleneck that means the application slows down before it saturates the CPU.

But putting SSDs and, especially, expensive PCIe flash in servers can lead to poor resource utilization because that performance and capacity is only available to applications on the specific server. Juggling demanding applications onto specific hardware for storage performance loses all the benefits of virtualization.

Not only do SANs and virtual storage appliances not scale IOPS as well, they can also be less flexible to scale and reconfigure, especially with a separate storage network (and the data center infrastructure required for that). For extreme storage scale, you need to be able to grow storage in a modular, cost-effective way that offers storage elasticity and fast configuration. It’s far easier to scale capacity and performance with a distributed storage system.

In converged and hyper-converged storage systems, every node in the cluster serves I/O, which means that IOPS scales well as you add nodes to a cluster. Using a combination of SSDs, PCIe flash and low-cost, commodity hard disk drives and industry standard servers to create a SAN from local application server storage with software-defined block storage tools like ScaleIO that can provision and remove storage on the fly gives you massive scale, but also the flexibility to handle the changing priorities of different customers and workloads.

ScaleIO converts the direct-attached storage in servers into shared block storage that applications can treat like a SAN (via the ScaleIO Data Client device driver on each node) even though it’s abstracted, pooled and automated. Each node also runs the ScaleIO Data Server which makes the local storage part of the distributed virtual SAN.

Data volumes look unified to applications, but they’re split into chunks of data distributed across all the nodes in a cluster, and mirrored across all the disks in that cluster, making the storage both highly available and very high performance.

Scaling performance and reliability

Because every server and local storage device in a cluster is used in parallel to both protect data and process IO, the read and write bandwidth both scale close to linearly as new servers and storage are added.

Clusters don’t have to be made up of uniform nodes; they can have different compute and storage resources, and you don’t have to scale compute and storage together. You can upgrade capacity or add faster storage to existing servers or add more servers independently. ScaleIO starts at 3 modes but can go up to 1,000.

There’s no downtime as components are added or removed – or if they fail; data is automatically rebuilt and redistributed across storage devices and nodes. Distributing and mirroring the chunks of the volume across the nodes also speeds up rebuilding and rebalancing data because small pieces of data are coming from multiple nodes in parallel.

Extreme scale means running a wide variety of workloads that can come and go at any time, at competitive costs and meeting SLAs without overprovisioning. Scaling down can matter as much as scaling up. ScaleIO lets you set different performance tiers giving individual applications as much performance or capacity as they need- very much like allocating compute and memory resources to a virtual machine. Quality of Service guarantees storage performance, throttling back ‘noisy neighbors’ to support large-scale multi-tenant environments. Protection domains, storage pools and data encryption at rest keep data in multi-tenant environments segregated.

You can also set QoS for each volume, limiting the IOPS or bandwidth for each application or use volume migration to move lower priority data to lower cost data.

Hardware on demand

One big advantage of ScaleIO is that you can quickly deploy it alongside existing data center infrastructure. There are multiple configurations; a storage-only setup that creates an external storage system with server nodes or a hyperconverged option that runs on the same servers that host applications and storage. It works with a wide range of hypervisors and server operating systems, including vSphere, Windows Server, Hyper-V, Linux and OpenStack.

Hardware choices are also flexible − you can maximize your infrastructure investment by running it on existing server hardware, or you can buy pre-configured ScaleIO Ready Nodesfrom Dell EMC. These come set up in known good configurations to maximize uptime, while saving you time. That means as you grow your infrastructure for extreme scale, you can automate the physical scale out of storage and compute resources by ordering hardware that’s ready to plug in, tested and validated for reliability, and with a single vendor for support.

@ScaleIO Ready Nodes are based on @Dell EMC PowerEdge 14th generation servers with the latest @Intel Xeon processors. The higher core count, increased memory channels and faster memory speed up applications and data and better drive cooling improves reliability. The PowerEdge R640 is a 1U, two-socket platform but for maximum storage performance and sensity the 2U, two-socket R740XD supports up to 24 NVMe cards for an all-flash system. If you want to take advantage of the flexibility to mix and match node configurations, you can fit both 2.5” and 3.5” drives in the same chassis. Hard drives, fans and the redundant PSU are all hot-swappable to minimize downtime.

Extreme scale needs a combination of software and hardware, plus automation to get the maximum value out of your infrastructure, so you can deliver the low cost and high performance and service that businesses expect from cloud-scale storage. You need agility and flexibility, as well as scale to adapt to fast-moving business needs. Hyperconverged software-defined storage can give you performance, reliability and elastic scaling, but it’s also less demanding to manage with the skills you bring to managing the rest of the data centers, making storage just another service rather than a special and demanding snowflake

http://www.datacenterknowledge.com/industry-perspectives/are-you-ready-extreme-scale