Software-Defined Storage Simplifies the Management Process

Powered by:

StorageeBook

StorageInsider

Software-defined Storage simplifies the management processManaging all storage resources from one central location and automatically – irrespective of manufacturer

Published by

2 Storage-Insider.de | Software-defined Storage: Management

3 Software-defined storage: The devil is in the detail

SDS is the framework of the future

7 Software maintains storage Imagine a world where software is the key to centralised, automated

management of all your storage assets.

Content

DataCore Software GmbHBahnhofstr. 18, 85774 UnterföhringPhone +49 89 4613570-0E-mail [email protected] www.datacore.com/de

Vogel IT-Medien GmbHAugust-Wessels-Str. 27, 86156 Augsburg, GermanyPhone +49 (0) 821/2177-0E-mail [email protected] www.Storage-Insider.deGeneral manager: Werner NieberleEditor in chief: Rainer Graefen, responsible as per press laws, [email protected] date: September 2014

Title image: vege - Fotolia.com Liability: Should any articles or information be inaccurate, the publisher will only be liable in the event of proven gross negligence. Where articles identify the author by name, the author himself will be responsible.Copyright: Vogel IT-Medien GmbH. All rights reserved. Reprints, digital use of any kind and/or duplication hereof are only permitted with the written consent of the editorial staff.Reprints and electronic use: If you would like to use any articles from this eBook for your own publications, such as special prints, websites, other electronic media or customer newsletters, you can obtain the necessary information and the required licences online at www.mycontentfactory.de, phone

+49 (0) 931/418-2786.

Powered by:

http://www.Storage-Insider.de

http://www.Storage-Insider.de/facebook

http://www.Storage-Insider.de/twitter

http://www.Storage-Insider.de/xing

https://www.facebook.com/DataCoreSoftware

https://twitter.com/datacore

https://www.youtube.com/user/DataCoreVideos

http://www.linkedin.com/company/datacore-software


IDC, Forrester or Gartner: regardless of analyst house, market experts universally agree that software-defined storage will become the de-facto platform for storage provision in the future. The main reason for this, they say, is that businesses of any size will always need more capacity to store their data. Moreover, the requirements of performance and availability tend to increase in importance, depending on the applications in use. Yet companies only have limited funds available to invest in storage. Based on the most recent research by leading analyst company,

Ask any market analyst and he will agree - software-defined storage is the platform of the future.

The term „software-defined“ is now regularly used to describe storage.

You would be hard-pressed to find any major supplier who today does

not use this keyword to describe their products. Yet, what exactly does

this concept imply, what are the advantages for businesses when

they implement it and at what point is it worthwhile considering

software-defined storage?

451 Research, the percentage of funds invested in storage compared to the overall IT budget has, in fact, decreased over the last two years.To satisfy this purse tightening, there is demand for solutions that can be scaled to fit exact needs; that can offer businesses a higher degree of flexibility and that promise to save costs. This is where software-defined storage (SDS) comes in.

SDS: simply a marketing claim?

What exactly is SDS? The explanation provided by IDC analysts can be used as an initial reference: they interpret the term software-defined storage as “a storage software stack installed on shared resources (x86 hardware, hypervisors or in the cloud) or on commercially available computer hardware”. This provides the basis for “allowing the bundling of existing storage resources, the improvement of their utilization and the capability to structure a service-based infrastructure”.By contrast, manufacturers are still having a hard time finding a generally applicable definition or even a willingness to agree on standards. This is understandable because storage hardware suppliers are, of course, primarily interested in continuing to sell their own systems successfully. In

Software-defined storage: The devil is in the detail


© v

ege

- Fo

tolia

.com


the storage component. The result is that storage is no longer defined by physical limits, but instead can be distributed more flexibly, thus becoming logically accessible.

A division between the physical and logical brings several advantages: Existing resources can be used more efficiently, expansions are easier to implement, data can be migrated without interruption, management can be centralised and new functions can be introduced at all levels.

In the solution, the choice between the numerous technical options primarily depends on which direction each of the manufacturers has decided to follow. As examples, in SAN, virtualization can

either take place by means of an in-band, out-band or split-path process, either in the host or in the storage controller of the storage system. Generally, with technology inherently tied to specific devices or models, we must accept that they only work properly with the systems offered by their particular manufacturer.Thus, for a long time now, one tried tested and therefore effective solution has been to revert to software-based solutions. They can bundle every resource at a software level that is valid for all devices. The less products that are bound to specific platforms and/or manufacturers, the better. The result is that all performance criteria can be made available at all levels, irrespective of the existing hardware; access to the storage systems can be controlled at a central level and the entire storage infrastructure can be uniformly managed from a single console.

Approaches that focus on hardware suffer from limitations

There is much to be said about integrating “cookie-cutter” functions and management solutions at software level and replacing classic hardware-focused architectures with non-proprietary virtual and

the meantime, however they continue to deliver models carrying an “SDS” label.More often than not deployment does not bring about any change, because the required functions are still tied to their specific storage platforms, typically proprietary software. Thus the system‘s

own set of features can neither interact with new components nor with other manufacturers‘ systems. Needless to say, this contradicts the principles of SDS, were the software determines the functions of the storage and does so entirely independently from the underlying devices or selected topologies.

Storage virtualization serves as an SDS vehicle

Generally, manufacturers revert to storage virtualization techniques as a means to an

end, typically integrating an abstraction level between the application server and


SDS solutions bundle all resources into a software layer used by all devices. This solution allows all performance criteria to be made generally available to all devices, irrespective of the existing hardware. (Image: DataCore)


software-defined approaches. There are quite a few reasons for doing so. Firstly, data volumes will continue to increase, making it difficult to determine just how much storage space must be reserved in the medium term. Applications - sophisticated tier 1 applications in particular - and the requirements of storage infrastructure become more demanding with an increase in work load.Yet classic systems are not designed for this and are not flexible enough to adjust to these changing conditions. What makes things even more difficult is the limited useful life of the hardware, which for storage arrays is on average around five or at most seven years. Frequently businesses purchase oversized storage space so that they are equipped for any scenario during

this period. This approach does not allow the appropriate flexibility to react to new requirements.However, if capacity and performance are not sufficient in the day-to-day work environment, expansions will be required, combined with the need to procure additional devices that more often than not have to be managed separately or at worst, require a complete change of architecture.This, in turn, creates even more problems. The result is a highly complicated jumbled mess of storage environments, requiring a great deal of effort to operate and manage. Additional hardware takes up much more space, expenses for power and cooling increase and, in equal measure, an increase in maintenance expenses.

SDS frees businesses from technical constraints

Both from a technical as well as an economic perspective, old-fashioned hardware-based storage architectures will eventually reach their limits in the short or long term. With this in mind, software-defined storage represents a future-orientated conceptual approach that may be interesting for both small and medium-sized businesses alike. It is worthwhile to put some detailed thought into SDS, especially when it becomes necessary to purchase new storage hardware, or when the use of flash/SSDs or server and desktop virtualization

projects is on the agenda. This is just as important when business continuity is a key topic of discussion, requiring a fail-safe, high performance and high availability IT infrastructure as the basis for running business processes without interruptions.

No matter which of these scenarios applies: by separating the storage services and functions from devices, businesses are given the freedom to make use of standard software, irrespective of its type, for their


The future of SDS from the point of view of analysts

IDCBased on a survey conducted by IDC, a majority of European businesses do in fact deal with SDS as a topic, yet by now only eight percent of them have implemented relevant solutions. Despite this, software-defined storage represents an attractive approach - 42 percent of the IT decision makers questioned in the survey consider software to be a key engine for innovation in the field of storage.

GartnerMarket researcher Gartner considers SDS to be a concept still in the making, but one that businesses should already be discussing now. From an analyst’s point of view, one of the greatest benefits of SDS is the integration of hardware infrastructures that are not manufacturer dependent; that are operated based on SLAs and that can provide solutions to problems that once posed challenges to conventional data storage. Based on estimates by Gartner analysts, however, it will take at least another ten years before SDS becomes prevalent on a large scale.

ForresterAccording to Forrester, storage budgets can no longer keep up with the demand for storage. As a result, IT administrators are being challenged and are seeking solutions that will allow them to make storage capacities and performance available as needed, preferably automatically. The analysts do not think that integrating additional platforms would be the best response available today, because in their opinion this would increase the silo mentality even more, thus making the storage environment even more complicated. Instead, they are convinced that the weak points of this conventional approach only serve to accelerate the introduction of SDS.


own purposes and to manage all their storage needs with software. In this way, existing traditional hard drive storage can be combined with flash media and hybrid systems in storage architectures tailored to their own individual requirements.This is the key to replacing existing island solutions and to finally being able to say goodbye to parallel, block-orientated SAN, file-based NAS and separate backup

and disaster recovery systems, various hypervisors or flash solutions.

SDS: Single source cross-platform and cross-system management

Managing heterogeneous storage infrastructures is not only complex and time-consuming, it is also costly. IT employees have to learn how to use a variety of different tools, with significant training and education implications. Centralisation, on the other hand, reverses the need for multiple knowledge sets across many storage infrastructures. Bundling a wide range of different hardware from any random number of manufacturers into one resource that can be shared now opens up the capability of

SDS makes it possible to manage data from all storage resources centrally and automatically - irrespective of the manufacturer.

© v

ege

- Fo

tolia

.com


managing them uniformly from a single management interface. Another key word in addition to centralisation is automation, which saves administrators from having to do all kinds of routine tasks. From a cost perspective, SDS is therefore a superior model compared to traditional storage management approaches, providing IT employees with all the necessary tools to control the storage environment across all devices.This eguide provides a detailed overview of the positive effects that can be achieved resulting from central management alongside and an explanation of the tools and functions that can be used to simplify data management. Tina Billo


Block- and file-based services can be used simultaneously with SDS. (Image: DataCore)

Storage environments in companies can often be compared to a patchwork quilt, knitted together, over the years, creating storage islands of various technologies

and devices sadly all incapable of working together. In such a legacy environment, individual systems usually need to be managed separately from each other, meaning that infrastructure management can become complex and time consuming. Software-defined storage promises just the opposite, ushering in the transition to a service-orientated storage environment. The new approach is structured according to storage virtualization principles, but at

the same time takes users one step further by adding a software layer between the application server and the storage hardware. This allows all devices to be combined and interwoven to form a resource that can be shared by all applications and to move the storage functions that in the past have been inherently tied to physical components into a centrally manageable virtualization layer.

Software-defined storage is compatible with block and file

Consequently, SDS offers companies the chance to free themselves from the restrictions of the hardware they are using. Systems that in the past were run separately for a range of different purposes can now be effectively combined.

Managing from a central location and automatically

Software-defined storage (SDS) is initiating a paradigm shift in

the field of computer storage: We are moving away from providing

hardware-based storage functions and services towards a

software-orientated alternative. As a result, systems management

can be automated and centralised, therefore its functions can be

shared on all systems throughout the infrastructure, irrespective

of the specific device or manufacturer. Below is an overview of the

added value that can be achieved with regard to data management

by switching to SDS and its invaluable functions and tools.

Software maintains storage


Common block- and file-based services controlled by software can be offered simultaneously and automatically with an intermediate interface and consequently used to manage all functions for storage management and storage monitoring.

One console is all you need

Bundling devices into a single unit offers other combined benefits with shifting intelligence from hardware to software. The entire storage infrastructure can be monitored from a single

console and can be managed centrally using universal management tools. The functions required to do so are made available as a service to all the devices installed in the widespread storage pools, irrespective of their model and brand.As described in the “2014 European Storage Survey” by IDC, IT managers see this as one of the SDS qualities that stands out most. Almost 50 percent of the IT managers questioned as part of the study claimed that they are primarily looking forward to more simplified data management and easier administration options by implementing an SDS solution. Likewise, around 35 percent consider the capability of being able to automate critical storage tasks using a rule-based management option to be another strong point of SDS.

Automating time-consuming storage tasks makes data management much easier

Centralising administrative tasks on its own is simply not enough; instead it must go hand-in-hand with the ability to automate repetitive routine tasks. Time-consuming repetitive processes

For example, it is not unusual for separate solutions to be used for the efficient storage and management of both structured and unstructured data. Block-based SAN storage options are available to meet the requirements of applications that not only take up a great deal of bandwidth, but are also transaction- and flow-intensive, such as databases, data warehousing and ERP solutions, for example.Because of the simplified expansion options and the shared access to a wide variety of files, Network Attached Storage (NAS) is by comparison much more suitable for storing Office documents, PDFs, e-mails, instant messages as well as audio, video and image files. An SDS solution consolidates these resources into a storage pool that can be used collectively. Any incompatibilities between devices are no longer an issue; the devices can communicate with each other and work together.The central, global storage service platform in between makes this block-based storage transparently available to the application servers via standard protocols such as Fibre Channel or iSCSI and NAS resources by way of CIFS of NFS authorisation.


It could not be easier: The entire storage pool can be managed using a single management console. (Image: DataCore)


that once needed to be done manually are therefore prone to errors, can be significantly streamlined with this solution. Automation saves time, with readily available storage resources. With SDS, previously autonomously run systems are combined into storage pools that can be shared, forming the basis for centralised and automated management of the entire storage infrastructure.They can be made up of a wide range of different solutions, from internal hard drives to classic external or directly connected

SAN arrays through to Solid State Disks (SSDs), Flash or even Cloud resources and broken down into different storage classes (“tiers”) based on both their cost and their performance capabilities.Building on this, virtual drives of various sizes can be generated from the physical pool of resources based an capacity, performance and availability requirements, seamlessly creating storage profiles for VMs.

Access to pre-defined templates makes this task much easier. Here, one can choose between performance parameters such as “critical”, “high”, “normal”, “low” and “archive”, as an example. Based on the rules set up for each of the virtual machines, a sync process is performed whilst the data being written to verifies which of the available storage tiers is the best match for the storage profile. Hard drive blocks that are accessed often are automatically assigned to a storage class made up of fast SSDs or flash drives. Less active data, on the other hand, is placed on a level where media are run at lower speeds, such as SAS or SATA disks. Thus existing resources can be used extremely efficiently and defined Service Level Agreements (SLAs) can be met but at a much lower administrative cost. If even better results are needed in terms of drive utilisation, thin provisioning is another option that can be set up on the virtual drives.

Thin provisioning: The key to optimum storage utilisation

The ability to determine exact storage needs in the future, speculating years in advance on exponential increases in data volume is not a task for the faint hearted. In the past, most experts considered it far better to purchase more storage than actually necessary and to upgrade when there was a need. However, this is proven a very expensive option. It was just as difficult having to make a decision about how much capacity should be assigned to individual applications and workloads.Capacity assignments were often simply based on estimates, whereas resources were then permanently committed to individual applications, with the result that they could not be distributed as needed. Because of the rigidity of the specifications, any attempt to correct imbalances was very


With SDS, previously autonomously run systems are combined into storage pools that can be shared, forming the basis for centralised and automated management of the entire storage infrastructure. (Image: DataCore)


cumbersome. If an application required additional capacity, the rule was first to copy all data to another medium, to delete the Logical Unit Number (LUN) and then to create a new one.To prevent this kind of laborious administrative work, administrators were

more inclined to be more generous when allocating capacity. Thin provisioning, which for software-defined storage is an integral element of the scope of functions, puts an end to storage oversizing.Essentially, the technology relies on provisioning storage from the storage pool in virtual form, irrespective of the available

physical capacity. In the process, application servers are “fooled into believing” that they have more space than is in fact reserved for them. However, capacity is not allocated until the data is actually written. In consequence, capacity no longer needs to be made available continuously, but only the hard drive blocks needed at that point. Once the physical limits of storage consumption have been reached or the threshold has been exceeded, the administrator is alerted and also notified of additional status information. If additional hard drives exist, they can be added to the storage pool during an ongoing operation without any interruptions and its capacity can be dynamically expanded in this manner. Similarly, there are also functions that allow the storage space to be reclaimed.The automatic, need-based and lean allocation of storage space on a “just-in-time” basis results in utilisation that is up to three times more effective. As a result, the number of SSDs, flash media or conventional hard drives required can be reduced considerably, thereby saving space. Another benefit are the positive effects relating to a reduced energy balance and the lower power consumption and ventilation requirements. Add these factors, together along with a reduced hardware investment and regular operating expenses, and the savings can be significant.

Analysis and reporting tools allow for more confident decisions

To be prepared for all eventualities, a number of analysis and reporting tools are provided as an integral part of SDS solutions. Using them, the status of the entire infrastructure can be monitored and analysed in real time. In this, a so-called “heat map” illustrates how individual disks or entire pools are utilised. I/O bottlenecks


Thin provisioning puts an end to oversizing storage. (Image: DataCore)


can be detected immediately, automated storage tiering can be adjusted accordingly and refined for prioritised workloads. It is also possible to determine whether additional resources are needed.In addition, administrators are given insight into performance values, the real size of virtual drives, the status of connections and much more. This allows them to respond immediately to changing conditions. Based on recorded performance data and other metrics, trend analyses can be performed and their results can be shown in diagrams or documented in Excel spreadsheets or CVS fileswhich comes in handy when forecasting future system utilisation. In the same way, any load peaks or potential “bottleneck” situations that might occur can be flagged at an early stage. This makes it much easier for IT managers to plan and manage overall capacities.An overview of the performance history gives IT managers the opportunity to analyse weak points and confirm adherence with internal service levels.

Summary

Implementing SDS offers the opportunity to overcome a wide range of challenges

in connection with storage management. IT administrators benefit from the large number of integrated tools. They can now set up virtual pools and disks, allocate, monitor and manage far beyond the limitations of hardware in a single console.Resources can be made available automatically, dynamically and as needed to match the application requirements and their utilisation can be optimised. As a result, in certain scenarios, four times the capacity utilisation can be achieved and more than 50 percent of storage space can

be released.IT managers are boosted and empowered with monitoring, analysis and reporting tools to map processes, affording them valuable insight into the entire infrastructure, including availabilities, I/O performance or storage usage. Imminent bottlenecks can be promptly detected, thus preventing failures. Critically, software-defined storage also plays a major role in the uninterruptible operation of storage environments that can be seen in day-to-day practice. In a recent survey conducted by TechValidate Research, businesses that switched to SDS were able to reduce downtimes by up to 100 percent.Centralising and automating storage management also releases administrators from the countless time-consuming routine tasks which in the past took up a majority of their work hours. Their workload can be reduced significantly by using SDS and they can be reassigned to other tasks.This reassignment of resources is definitely worthwhile - survey respondees reveal that in total, this amounted to three quarters of their time. Moreover, all the resources required for administration are available


With SDS, IT managers are offered quite a few monitoring, analysis and reporting tools. (Image: DataCore)

http://www.techvalidate.com/product-research/datacore-sansymphony-v

http://www.techvalidate.com/product-research/datacore-sansymphony-v


from a single source, irrespective of the storage component used. IT employees no longer need to be trained in the use of the various manufacturer-specific tools.To summarise, with software-defined storage we have a more effective and efficient management tool that plays

a significant role in allowing storage resources to be operated and managed much more economically, despite the steadily mounting data volumes. However, if you think that SDS is only an option for large corporations, think again. By introducing a purely software-based virtual SAN platform, even small and medium-sized companies are given access to the benefits of centralised storage management, including all the enterprise functions that come with it. To do so, they do not need to invest in any complex SAN infrastructure, because the SDS software

can be installed on existing application servers, thus merging directly connected hard drive and flash storage media into a shared high-performance storage pool.SDS is therefore a superior model compared to traditional storage management approaches, and is a worthwhile investment for any company. Tina Billo

Strongly frequented blocks benefited greatly from the performance of faster disks in the disk pool as a result of hot-spot detection in the virtual disks. (Image: GepaNet)


Technology

Software-Defined Storage Simplifies the Management Process