Data Storage Infrastructure 2022 Predictions

My team put on their thinking caps and contributed towards a list of data storage-centric predictions for the upcoming year. Admittedly some are based on, or projected from, analyst insights that we are especially close to because of our day-to-day work. But some are more qualitative and were the product of own observations and related expectations. Enjoy, and let me know what you think (we’ll have to check back at the end of the year to see how we did).

HPE Storage Solutions for Microsoft workloads team conjuring 2022 predictions
  1. IT equipment supply issues continue deep into 2022 – Major IT equipment manufacturers work within an especially global value chain, with the products we design then getting produced in conjunction with developers, component suppliers and manufacturing partners on the other side of the world. And just based on our known logistics and component (such as processors) issues, we expect supply challenges to continue well into the new year. In addition, some major disruption in southeast Asia, whether political (e.g. China south sea), a pandemic resurgence, or extreme weather condition (e.g. flooding) could quickly impact a majority of the world’s top 10 busiest ports, including our supply lines, inventory… as well as sadly putting an end to your dream of getting that cool Apple iPhone 13 anytime soon as it would probably come via a Foxconn facility in Zhengzhou and the port of Shanghai. Luckily the broadening investment in real-time logistics info-tech may help supply chain players better see, optimize, and work around problems. Absent a major crisis, we should have IT trade back to ‘normal’ by the end of the year.
  2. Companies become more concerned with managing Data than Storage –  As more customers evolve to a ‘service-oriented’ model, such as via an HPE GreenLake based solution, they are becoming less concerned about the specifics of what infrastructure is being used to store the data. Whether the deployment is on something like HPE dHCI, software-defined x86 scale-out or traditional arrays, the trend is for more focus on the desired business outcomes around the data and ensuring requirements are met vs. discussions on storage deployment details.
  3. NVMe a part of every infrastructureIDC already predicts NVMe storage will be used by 91% of companies within the next 2 years, and we are seeing NVMe drives as a key part of our storage solutions. Using a set of NVMe drives is a natural pairing for software-defined infrastructures and intelligent applications that maintain a software-based cache – pin the cache to these speedy drives and you ensure low latency and fantastic workload performance. On a more strategic level, we are also seeing where NVMe as part of more distributed architectures (e.g. NVMe-oF using RDMA, FC, TCP) is going to realize more adoption due to the consolidation of standards at the system level. Manufacturers are continuing to drop GenZ related development and are standardizing around CXL (Compute Express Link). CXL is a new open interconnect standard to reduce the latency of data sharing between CPU and memory within a system node. This system design consolidation for higher-performance within a host and surrounding devices is expected to have a follow-on effect of allowing more innovation in the surrounding fabric, and this is expected to further spur the use of NVMe drives and related media within the system node, across the rack, the aisle, and the datacenter.
  4. Cyber-crime will continue despite government action – Cyber-crime continues to be an endemic problem requiring governmental response. Yet there appears to be a disconnect between the growth in the occurrence of ransomware, trojans, and live criminal actions against enterprise servers and storage, while legislative actions seem more focused on rules to protect data, mostly from a physical perspective, and requirements to purge storage media and servers before they’re decommissioned or disposed of. Whereas the requirement to get certificates of destruction may be a good opportunity for professional services firms to generate a new source of income, it does little to thwart the serious threat of cyber-attacks by criminals, overseas adversaries, and terrorists.
  5. Points of Data Integration will grow – Despite the increasing threat to our data outlined in the previous prediction, we expect to see continued growth of integrations between companies, partners, customers and systems. Past IDC reports and more recent predictions have detailed how organizations are having to manage more APIs as part of doing business, and that “…mastery of APIs… [is] a price of admission to competing on the digital business battlefield.” Look for new data storage specific integrations becoming available especially between hyperscaler clouds, popular IT dashboards, and enterprise data storage platforms.  
  6. Software defined storage will continue to grow – Though there’s still an important place for traditional block storage arrays, the compelling economics and hybrid cloud features of the newest software-defined storage products will continue to gain adoption and expanded use. Products such as Azure Stack HCI especially when combined with hardware components such as NVMe storage, PMEM and GPU will increase utilization as infrastructure for VDI, enterprise virtualization, and big data.
  7. Container storage adoption will continue slowly – This whole area of containers and Kubernetes is one that I’m sure a lot of us have been lured into, from the inherent techno-coolness and social media-fueled enthusiasm. However, more and more is being written about the slow adoption of these technologies. The uptake of Containers and related container-based data storage is lagging – no doubt a function of the technical complexity outweighing what benefits are being realized from the license cost savings of using free opensource software. But just as in the Monty Python sketch, Container-based storage is not dead yet, and still something to continue to watch out for.
  8. The Battle of the Edge will intensify – The idea of Edge Computing is still fairly new, coming into common usage maybe 5-6 years ago, and tightly linked with mobile computing and Internet of Things, specific to wearables, home automation systems, sensors, RFID tags and the like. Within this limited context, the entire market opportunity for the Edge was expected by leading analysts to be less than $3 billion in 2022. Chump change vs. the forecast of all IT spending to top $4 trillion that same year. And though a growing host of companies continue to jockey for this market sliver, a few seem oriented towards a compressed world view where the Edge is almost every compute resource outside the hyperscaler cloud. Through that lens, ‘Storage at the edge’ becomes way more than just things like NAS-attached video cameras, but also includes data storage within remote offices and even enterprise storage arrays within datacenters. Expect the battle for the edge to not just be about products but also philosophies.
  9. Data Management still required despite the cloud – We’re seeing the line between on-prem and cloud blur, with more deployments being at least hybrid cloud, and most new applications starting life native to a hyperscaler environment. It’s being written that “Cloud computing has won”, and while this may have relieved IT from operational tasks centric to that application, it hasn’t eliminated responsibilities around ensuring the availability, protection and access to the data. We expect that through 2022, though IT is still on the hook for data security and locality concerns, teams will continue to lack easy-to-use tools to manage data across clouds, and that a new product market will take shape around enterprise data management, operations and mobility.
  10. The Year of Hybrid Cloud – We’ve got to add this one with a big wink, because this has been a prediction out there for at least the last decade (see Wired article from 2012). Of course, ten years ago the perspective was more theoretical and centric to mitigating periodic ‘inbound spillover’ of excessive application demand – what we’ve since taken to calling ‘cloud bursting’. Hybrid cloud has since continued growing in popularity especially over the past five years. Today we’re seeing adoption of hybrid cloud products that enable the actual mixed usage of on-prem and hyperscaler-based services together and managed within a single pane of glass (think: Azure Stack HCI managed through Windows Admin center). So maybe 2022 IS finally the year of Hybrid Cloud?

Welcome the all-new Azure Stack HCI

It’s a big day for hyperconverged infrastructure (HCI).  The world’s most cloud-connected HCI just got an overhaul, and Microsoft is (re)launching their Azure Stack HCI today <link to Microsoft announcement blog> .

Microsoft’s Azure Stack HCI will be a big factor in the white-hot HCI space. It’s positioned to serve Hyper-V and Azure cloud centric customers who would otherwise have to make do with Nutanix or vSAN. Azure Stack HCI more optimally delivers modern infrastructure that simplifies the management of on-prem resources while seamlessly connecting them to the cloud.

And let’s be clear – this isn’t marketing speak. Azure Stack HCI is now a new and separate O/S, designed for real Hybrid management. Customers will use the Azure Portal as well as Windows Admin Center (WAC) for resource visibility, management, billing, troubleshooting… everything.  

Azure Stack HCI Infrastructure as a Service topology

Goodbye Azure Stack HCI, Hello Azure Stack HCI!

The all-new Azure Stack HCI, available either as a download or pre-installed on hardware, replaces the current Azure Stack HCI offering which was built around Windows Server 2019 features. And with the new software comes new functionality including stretch clustering for DR – built-in async or sync replication that you can use for local high-availability or across metro distances, with optional encryption. That’s powerful HA for free, that you may not get with other HCI products.

Other advantages of the new Azure Stack HCI include a set-up wizard within WAC, and integrated billing with Azure cloud.  Did I mention the licensing is per core/per month, and gets bundled into your existing Azure account?  No need to track separate licenses, CALs, etc.

HPE and Azure Stack HCI

This is an important time for our Azure Stack HCI on Apollo 4200 solution in that it’s gotten better than ever – thanks to newly qualified drives, it now accommodates over twice the data capacity! The Azure Stack HCI on HPE Apollo solution was already the best solution for ‘Data centric’ workloads – where you need high capacity per node to run workloads like SQL Server, Exchange, Analytics, secondary storage and the like. But now with the support for even larger capacity media, it can accommodate 829 Terabytes of data within its highly space-efficient 2U footprint, and almost 17PBs in a single rack.

If you need more Performance or flexibility in your Azure Stack HCI architecture, then look no further than a solution built on HPE ProLiant servers.  Or if you prefer the entire offering – software and hardware – all as-a-Service, then HPE has that coming soon in a GreenLake offering. 

With HPE you’ll find the same broad portfolio of solutions around this HCI product, including our own Azure Stack HCI on the HPE Apollo 4200 solution. Microsoft just updated the new program catalog so you’ll find just a few of our HPE validated solutions listed as of today, but that list will re-expand over the coming days.  

Next steps

So if you’re with an enterprise looking for a 60 node deployment in your datacenter, or an organization who just wants to try things out with a 2 node cluster, you can start by taking a look at our current portfolio of all HPE solutions for Azure Stack HCI online here.  Info specifically on the Azure Stack HCI on HPE Apollo solution is here.  And I ask you to engage with us as we develop and evolve our portfolio based on customer needs and requests – post a related question or comment in the space below.

Insights from Deploying Microsoft Exchange at Scale on Azure Stack HCI

Microsoft Azure Stack HCI has established itself as a solid hyperconverged infrastructure offering, based on the leading operating system, Microsoft Windows Server 2019. IT staff are able to efficiently consolidate traditional workloads on this familiar platform, thanks to multiple technological features including both compute virtualization with Hyper-V as well as data storage virtualization with Storage Spaces Direct. There’s also support for the use of non-volatile memory express (NVMe) SSDs and persistent memory for caching in order to speed system performance.

However, with such dynamic technology in play at the OS layer, things get interesting when you add a sophisticated workload that also has its own intelligent performance enhancing features including storage tiering, a metacache database (MCDB), and dynamic cache. In this case we’re talking about Microsoft Exchange email, which recently introduced the new Microsoft Exchange Server 2019.

One Wall Street firm was a power user of Microsoft Exchange – with over 200,000 users, many having massive mailboxes of dozens up to 100 or more GBs in size. As part of their infrastructure planning, the customer wanted to compare the performance and cost of continuing to run Exchange on physical servers with external attached storage (JBOD), versus evolving to an Azure Stack HCI infrastructure. 

The combination of these products and technologies required complex testing and sizing that pushed the bounds of available knowledge at the time, generating learning useful for other companies who are also early in adopting various combinations of demanding enterprise workloads on top of Azure Stack HCI.

Field experts share their insight

“This customer had an interest in deploying truly enterprise-scale Exchange, and eventually the latest server version, using their HCI infrastructure,” began Gary Ketchum, Sr. System Engineer in the Storage Technology Center at HPE.  “Like vSAN or any other software-defined datacenter product, choosing the hardware is very important in order to consistently achieve your technical objectives.”

This observation especially holds true when implementing Storage Spaces Direct solutions. As stated in the Microsoft Storage Spaces direct Hardware requirements page, “Systems, components, devices, and drivers must be Windows Server Certified per the Windows Server Catalog. In addition, we recommend that servers, drives, host bus adapters, and network adapters have the Software-Defined Data Center (SDDC) Standard and/or Software-Defined Data Center (SDDC) Premium additional qualifications (AQs). There are over 1,000 components with the SDDC AQs.”

A key challenge of the implementation was in how to realize the targeted levels of improved flexibility, performance, and availability, within a much more complex stack of technologies, multiple virtualization layers, including potentially competing caching mechanisms.

Anthony Ciampa, Hybrid IT Solution Architect from HPE explains key functionality of the solution. “Storage Spaces Direct allows organizing physical disks into storage pools. The pool can easily be expanded by adding disks. The Virtual Machine VHDx volumes are created from the pool capacity providing fault tolerance, scalability, and performance. The resiliency enables continuous availability protecting against hardware problems. The types of resiliency are dependent on the number of nodes in the cluster.  The solution testing used a two-node cluster with two-way mirroring. With three or more servers it is recommended to use three-way mirroring for higher fault tolerance and increased performance.” HPE has published a technical whitepaper on Exchange Server 2019 on HPE Apollo Gen10 available today online.

Microsoft Azure Stack HCI on HPE Apollo 4200 Gen10 solution

At Microsoft Ignite 2019, HPE launched its solution for the new Microsoft HCI product, Windows Azure Stack HCI with HPE Apollo 4200 Gen10. This new software-defined hyperconverged offering, built on the high capacity yet dense Apollo storage server, delivered a new way to meet the needs of the emerging ‘Big Data HCI’ customer. A new deployment guide details solution components, installation, management and related best practices.

Exchange on Azure Stack HCI Solution Stack

The new Azure Stack HCI on HPE Apollo 4200 solution combines Microsoft Windows Server 2019 hyper-converged technology with the leading storage capacity/density data platform in its class. It serves a growing class of customers who want the benefits of a simpler on-premises infrastructure while still able to run the most demanding Windows analytics and data-centric workloads.

Findings from the field

Notes from the deployment team captured some of the top findings of this Exchange on Windows HCI testing, that will help others avoid problems as well as confidently speed these complex implementations.

  1. More memory not required – The stated guidance for Azure Stack HCI requires additional memory, specifically an SSD NVMe (cache tier) beyond JBOD physical deployment. However HPE’s Jetstress testing showed that similar performance was also possible from just JBOD. Thus the server hardware requirements are similar between Azure Stack HCI and JBOD, and even if the customer plans to deploy JBOD MCDB tier with Exchange 2019, the hardware requirements are still very similar. Note, there could be other cost factors to consider such as the cost of overhead for additional Compute and RAM within the Azure Stack HCI, as well as any other additional software licensing cost for running Azure Stack HCI.
  • Size cache ahead of data growth – The cache should be sized to accommodate the working set (the data being actively read or written at any given time) of your applications and workloads. If the active working set exceeds the size of the cache, or if the active working set drifts too quickly, read cache misses will increase and writes will need to be de-staged more aggressively, hurting overall performance.
  • More volumes the better – Volumes in Storage Spaces Direct provide resiliency to protect against hardware problems. Microsoft recommends the number of volumes is a multiple of the number of servers in your cluster. For example, if you have 4 servers, you will experience more consistent performance with 4 total volumes than with 3 or 5. However, testing showed that Jetstress provided better performance with 8 volumes per server compared to 1 or 2 volumes per server.

Where to get more info

Microsoft Azure Stack HCI on HPE Apollo 4200 Gen10 server is a new solution that addresses the growing needs of the Big Data HCI customer – those who are looking for an easy-to-deploy and affordable IT infrastructure with the right balance of capacity, density, performance, and security.  Early work with this solution, especially where it’s being combined with demanding and data intensive workloads, can create non-intuitive configuration requirements, so IT teams should seek out experienced vendors and service partners.  

A new deployment guide details solution components, installation, management and related best practices. Information in that document, along with this blog, and future sizing tools expected out from HPE, will continue to provide guidance for enterprise deployments of this new HCI offering.

The deployment guide is available online today at this link: <link to Deployment Guide>

HPE Brings Big Data to Hyperconverged Infrastructure with New Apollo Solution

If you were at Microsoft Ignite last month you may still have missed the launch of HPE’s latest hyperconverged infrastructure (HCI) solution: Microsoft Azure Stack HCI on HPE Apollo 4200 storage. It would be understandable, as Ignite was a major industry event packed with technology news, especially with lots of HPE show activity, including prominent HPE mainstage appearances for both Azure Stack and the new Azure Arc.
But among the new and enhanced solutions we demonstrated at the show, our presentations about Azure Stack HCI on HPE Apollo storage were well-received and timely given the growing emphasis on HCI, hybrid cloud and all things software-defined. The key message for this solution was that it is pioneering a new area in software-defined HCI for Windows Big Data workloads. The solution uniquely delivers the convenience of hyperconverged Infrastructure on a high-capacity platform for the most data-intensive applications.

The emergence of Big Data HCI
We’ve all heard about the explosive growth of data, and that we’re in an age of zettabytes. IDC made a specific prediction, that by 2024, just data created from AI, IoT and smart devices will exceed 110 zettabytes (source: IDC FutureScape: Worldwide Cloud Predictions 2020).
At the same time, organizations are trying to simplify their IT infrastructures to reduce cost, complexity and the need for specialized expertise. The conflict is that the applications required to harvest this explosion of data can be the most demanding in terms of performance and management. I’m seeing companies – even the largest most capable enterprises – are recognizing the value of easy-to-use hyperconverged infrastructure to alleviate some of the strain of delivering these demanding, data-centric workloads.
Azure Stack HCI on HPE Apollo 4200 storage is a new solution that addresses the needs of the growing “Big Data HCI” customer. Azure Stack HCI on HPE Apollo is built on the highest capacity Azure Stack HCI qualified 2U server, bringing an unmatched ability to serve big data workloads on a compact Windows software-defined HCI appliance.

HPE Apollo HCI solution key components
Azure Stack HCI is Microsoft’s software-defined HCI solution that pairs Windows Server 2019, Hyper-V, Storage Spaces Direct, and Windows Admin Center management, along with partner x86 hardware. It is used to run Windows and Linux VMs on-premises and at the edge with existing IT skills and tools.
Azure Stack HCI is a convenient way to realize benefits of Hybrid IT, because it makes it easy to leverage the cloud-based capabilities of the Microsoft Azure cloud. These cloud-based data services include: Azure Site Recovery, Azure Monitor, Cloud Witness, Azure Backup, Azure Update Management, Azure Network Adapter, and Azure Security Center to name a few.
The Azure Stack HCI solution program includes Microsoft-led validation for hardware, which ensures optimal performance and reliability for the solution. This testing extends to technologies such as NVMe drives, persistent memory, and remote-direct memory access (RDMA) networking. Customers are directed to use only Microsoft-validated hardware systems when deploying their Azure Stack HCI production environments.

HPE Apollo 4200 Gen 10 – largest capacity 2U Azure Stack HCI system

HPE Apollo 4200 Gen10 Server – leading capacity/throughput for Windows HCI
The HPE Apollo 4200 Gen10 server delivers leading scale and throughput for Azure Stack HCI. The HPE Apollo 4200 storage system can accommodate 392 TBs of data capacity within just a 2U form-factor. This leads all other Azure Stack HCI validated 2U solutions as seen in the Microsoft Azure Stack HCI catalog (Microsoft.com/HCI). In addition, the HPE Apollo storage system is a leader in bandwidth, supporting 100Gb Ethernet and 200Gb Infiniband options. Customers are already running large scale, data-centric applications such as Microsoft Exchange on HPE Apollo systems, and can now add Azure Stack HCI as a means to simplify the infrastructure stack, while preserving performance and the space-efficient 2U footprint.
The HPE Apollo Gen10 system is future-proofed with Intel Cascade lake processors for more cores and faster processing, along with memory enhancements and support for NVMe storage. The HPE Apollo systems leverage a big data and high performance computing heritage, and have an established Global 500 customer track record.

Azure Stack HCI on HPE Apollo solution – more than just hardware
The HPE Apollo 4200 system is at the core of this Microsoft software-defined HCI solution, but there’s much more to the solution. HPE solution engineering teams perform testing on all solution designs, and publish technical whitepapers to provide guidance on implementation, administration, and performance optimization, for example the recent Microsoft Windows Server 2019 on HPE Apollo 4200 implementation guide. HPE also trains authorized reseller partners to help assure fast, successful deployments and fast time-to-solution for customers.
Windows Admin Center (WAC) is becoming the new standard interface for Windows system management. HPE is developing Extensions for WAC that will make it easier to manage HPE Apollo systems within Windows Server 2019 environments as well as specifically within Azure Stack HCI clusters.
As an HPE Storage solution, customers also enjoy high availability through HPE InfoSight predictive analytics that deliver the uptime benefits of AI to the datacenter.

Get started with HPE Apollo HCI
The Azure Stack HCI on HPE Apollo solution is available today. It’s the largest capacity 2U Azure Stack HCI validated solution available, and has been officially qualified for All-Flash, Hybrid SAS SSD, and NVMe providing options for affordable and high-performance data storage.
The Azure Stack HCI on HPE Apollo solution is the go-to choice for analytics and data-centric Windows workloads. Get easy to manage infrastructure with native Microsoft Windows administration. Available with the solution are published technical guidance including whitepapers and related resources, with WAC extensions on the way.
The launch webinar was recorded and is available on demand – watch it to learn more:
https://www.brighttalk.com/webcast/16289/374384/simplify-your-big-data-infrastructure-with-azure-stack-hci-on-hpe-apollo