Programmatic software-defined storage is taking over monolithic appliances

Software is eating the world, Marc Andressen famously said. Well not surprisingly, enterprise infrastructure is also being caught in this tornado, thanks primarily to two phenomena.

Container technologies and the micro-services philosophy allow for more agility when it comes to developing and deploying applications. In addition to such revolutionary technologies are software-defined infrastructure solutions, from networking to storage, which have been in used in development environments for years but are now about to disrupt enterprise IT.

These two trends have been obvious to developers and operators for quite some time already, in particular following the exceptional growth of Docker. But it happens that enterprises are also embracing these radical changes as Keith Fulton, CTO of ADP, made clear in his talk at DockerCon'16.

Keith Fulton talk during DockerCon'16 (from 1:06:00)

Enterprises feel that it is about time, having learned the hard way that their storage infrastructure is stuck in another age. Expensive appliances, non-scalable protocols, monolithic setups and non-programmable components are just too far off to keep pace with ever-changing business requirements.

As a result, many enterprises are no longer afraid to consider drastically modernizing their infrastructure. This process starts by replacing their hardware storage setup with a naturally distributed storage platform. By decoupling the hardware from the software, one can expect to reduce costs and greatly gain in flexibility.

This trend is already made clear both through market research analyses and the lost of market shares from all the major hardware vendors: EMC, NetApp, HP, Dell etc.

Software-defined storage is making its way into production environments.

Whether you like it or not, software-defined storage is making its way into production environments because of compelling advantages over existing hardware-based solutions:

  1. Scalability: Software-defined solutions can scale up to several petabytes (even to exabytes if well-architectured) whereas appliance-based solutions are inherently limited in terms of storage capacity. In addition, software-based solutions can very quickly scale out to thousands of computer nodes, in particular if benefiting from a decentralized architecture (i.e no metadata server): Infinit, Cassandra, Tahoe-LAFS etc.

  2. Cost: While enterprises have been locked in with existing vendors for decades, software-defined solutions allow them to finally decide which hardware to use: re-use legacy appliances, buy custom or commodity hardware. Enterprises can expect a 70+% reduction in total cost of ownership. In addition, because different servers can compose the software-defined infrastructure, it is much more cost effective than traditional solutions to keep pace and adapt to the exponentially growing amount of data to store by integrating modern hardware.

  3. Security: While the firmware of hardware-based solutions is rarely updated, leading to critical security holes that are never fixed, software-based solutions can be updated on a regular basis to benefit from security patches and performance optimizations.

  4. Flexibility: Software-based solutions can, by definition, run anywhere (or they shouldn't call themselves software-defined): bare metal, virtual machines or containers. Given this flexibility, such solutions are ideal for more complex environments: multi-site, hybrid cloud etc. In addition, some software-defined solutions (Infinit and Hedvig primarily) go further by allowing multiple storage infrastructure to evolve in parallel, each with its own set characteristics: topology, composition etc. This capability allows enterprises to choose which data goes where: to the cloud for rarely accessed data for instance, or on premise for benefiting from a faster access or to keep business critical information under control.

  5. Programmatic: Even more important to modern flows is the possibility to control the behavior of a storage infrastructure through a set of properties: redundancy, security, data placement, versioning and more. This allows developers and enterprises to set up different infrastructure with different optimizations that will suit the needs of the overlying application/user.

For the past decade, several software-defined solutions have paved the way to what is now accepted as a viable solution to production challenges, most notably RedHat's Ceph and GlusterFS but also Nexenta.

Even though such storage solutions are not necessarily playing very well with the cloud, it is worth noting that software-defined storage, by definition, can be complementary by allowing for hybrid cloud deployments.

Also, thanks to the advent of containers, enterprises could also create their own cloud by relying on cloud-based bare metal for instance, and further reduce their costs: Packets, Scalway, etc.

A new and more modern wave of software-defined storage products like Infinit, Hedvig, Portworx and more) is however making its way to the market, in particular because well adapted to more dynamic environments such as large-scale containerized applications.

Given the diversity of workflows and setups, it is clear that this is not a winner-takes-all market, as the history has shown with numerous big players. However, the advent of container technologies and software-defined technologies are reshuffling the cards...

Get started with Infinit in less than 10 minutes! Test it!