For IT departments, evaluating storage products is a critical process that can determine an organization’s entire digital infrastructure.
A poorly designed storage solution can significantly impact a service’s performance and lead to major outages or, in the worst case, permanent data loss. Conversely, by making an intelligent decision based on the right factors, your business can benefit from a scalable shared storage solution.
And capable of meeting the service level objectives in terms of performance and reliability for the proposed system design.
On a large scale, IT architecture maintenance can be like maintaining an old car. If you don’t have the time or money to find a better alternative, this process can be costly and require considerable resources.
IT system administrators working with outdated and inefficient hardware can struggle to catch up and support data transformation initiatives.
Choosing an Appropriate Storage Solution
When designing a greenfield solution, it is important to first understand the general architecture and system design of the proposed solution, as well as potential resource bottlenecks across the entire stack.
This will enable application and storage architects to choose and design the appropriate storage solution.
We highlight some key questions storage architects should ask to help them make an informed decision:
- What is the storage solution intended for?
- Does it need access to block, file, or object storage?
- What is the typical workload?
- What are the IOPS, throughput and latency requirements?
- What is the required availability? (99.9%, 99.99%, 99.999%?)
- Does the data need to be backed up? How often ?
- Does the data need to be replicated?
- What are the disaster recovery requirements in terms of recovery time objective (RTO) and recovery point objective (RPO)?
- What are the data retention requirements?
- How much does the data change daily, weekly, monthly, yearly?
- What is the planned capacity growth per year?
Business data requirements
Understand block, file, and object requirements
When integrating new applications, it is important to understand the type of data stored in order to make an informed decision on whether to use block, file, or object storage.
Block storage is the most common use case for DAS and SAN environments. In the case of a DAS, a RAID volume or entire physical drive is presented to the operating system as a raw, unformatted volume.
In the case of SAN environments, the entire LUN (compromise of multiple physical drives) presented by the storage array is presented to the operating system over a high-speed network and appears as an unformatted raw volume.
The underlying layers of the raw volume are made up of smaller extents or sectors that the operating system processes, and then the underlying storage subsystem can map these logical blocks to specific physical blocks on the specific drive(s).
Block-level storage is fast, reliable, and ideal for data that continually changes, such as relational databases, online transaction processing (OLTP) databases, email servers, or data processing infrastructure. virtual desktop, where high transaction throughput and low latency are required.
Object storage stores data (and the metadata associated with it) in containers with unique identifiers, without folders or subdirectories like those associated with file storage. They use the concept of key-value stores, where each key points to a specific “value” or piece of data, and is retrieved via APIs.
It is mainly used to process large amounts of unstructured data, such as emails, backup images, CCTV footage or, in IoT, data management for machine learning and analytics. data.
Object storage is suitable for processing very large amounts of data and can scale as quickly as the application requires, but it is slow to retrieve data, making it ineffective for databases or high-performance computing.
Examples of object storage: Amazon S3, Google Cloud or Azure Blob.
File storage stores data in files, organized into folders and subdirectories, and is shared over a network using SMB (Windows) or NFS (Linux).
It is a suitable solution for centralizing storage files such as videos, images or documents, but its scalability is limited when the amount of data continues to grow.
It is not the best application for processing very large amounts of unstructured data or constantly changing data such as OLTP databases.
Successful companies are therefore concerned with building high-performance computing (HPC) systems.
They leverage local databases and data services to perform transactional calculations, then enable native integration with cloud object stores to store large amounts of unstructured data.
Thus, high-throughput transactions and IOPS can be performed in the fast block and file storage of local data centers and in slower cloud object storage to store a large amount of unstructured data.
Processing data at scale requires a data storage solution based on the type of data your business needs to analyze.
For example, to process and analyze unstructured data on-premises or in the cloud, businesses need a file data platform for hybrid storage infrastructure that can provide real-time analytics and insights.
Storage performance testing
Testing and validation of storage products constitutes a central pillar of their evaluation.
The benefits of testing are numerous. Improving application performance, optimizing storage costs and reducing risk are all outcomes that can be tested with the right tools.
That said, small or underfunded IT departments may struggle to do this, as DIY or shareware tools often prevent the rigorous variety of testing needed to replicate a company’s real-world production environment.
Tests can be used to answer any or all of the following questions:
- How much can I improve my application performance by implementing new storage technologies/products?
- Can I afford to improve performance?
- Will new techniques reduce the cost per gigabyte without affecting performance too much?
- How can I choose the best technology/product/configuration for my application workloads?
- Which workloads will benefit most from the new architectures/products?
- What are the performance limits of potential new configurations?
- How will storage media behave when reaching performance limits?
If you’re choosing a scalable enterprise data storage solution, it’s essential to pay attention to how your chosen storage works with data and applications.
Support for storage products
A great product can unfortunately be undermined by the lack of a support team to help the company deal with any issues they encounter while using it.
Conversely, a good product can be elevated to new heights through the exceptional efforts of its technical support team.
When making a decision about a potential change to your storage solution, it may be helpful to consider your existing professional relationship with your company’s existing storage provider.
Additionally, any service level agreements (SLAs), such as meeting key performance indicators (KPIs) like latency, throughput, or IOPS during specific workloads, should impact your choice.
If your chosen vendor has a strong reputation in the industry (for example, it typically exceeds industry standard benchmarks), you can take them at their word when they announce features like a high number IOPS and high throughput for acceptable latency for each platform.
Another thing to keep in mind is the cost of the storage products you are considering purchasing. Not only the acquisition cost must be taken into account, but also the maintenance cost and the total cost of ownership (TCO).
