When people think of Kubernetes (K8s), the terms they often think of are related to large-scale environments, such as "cloud scale", "infinite scalability", or even "huge". However, the reality is that a large part of the IT world needs to start development and production from a smaller K8s environment. In order to provide cloud-level convenience, flexibility and scalability, and to achieve cost advantages for local and hybrid cloud infrastructure, Supermicro has launched a turnkey solution that uses its industry-leading hardware and top-of-the-line Line software integration. In this article, we will learn about the components of the Supermicro Rack plug-and-play solution, how it works, and the economics of running an internal K8s platform and a cloud-based platform.

When people think of Kubernetes (K8s), the terms they often think of are related to large-scale environments, such as "cloud scale", "infinite scalability", or even "huge". However, the reality is that a large part of the IT world needs to start development and production from a smaller K8s environment. In order to provide cloud-level convenience, flexibility and scalability, and to achieve cost advantages for local and hybrid cloud infrastructure, Supermicro has launched a turnkey solution that uses its industry-leading hardware and top-of-the-line Line software integration. In this article, we will learn about the components of the Supermicro Rack plug-and-play solution, how it works, and the economics of running an internal K8s platform and a cloud-based platform.

The four components that make up any K8s installation are computing, storage, networking, and software. In order to attract as many customers as possible, Supermicro designed different systems and rack-level configurations based on this program. The hardware of this solution utilizes Intel's proven third-generation Xeon CPU and Optane Persistent Memory (PMem), which we will introduce later. The architecture of their solution consists of different computing node classifications following industry standard classifications:

Supermicro built a compact 3-node cluster based on their X12 BigTwin, and includes the X12 Ultra system to provide additional storage capacity for the JumpStart program, which we will introduce in this article. For entry-level clusters, Ceph storage is not required.

X12 BigTwin is a 2U 4-node system with dual Intel Xeon Platinum 6338N processors. Each node contains 72 Intel computing cores, 4TB Intel Optane PMem, 512GB DDR4 memory and 184TB NVMe-supported storage. In terms of connectivity, it has a 100Gb interface for NVMe-oF, supports persistent storage across 3 nodes and 25Gb to connect to object storage clusters. Three of the X12 Ultra servers are storage nodes, each with up to 1.1 PB of object storage and up to 80 computing cores. One of the verticals that are particularly suitable for this solution is media and entertainment for video streaming, content delivery and analysis purposes.

As mentioned above, the BigTwin system uses Intel Optane PMem Series 200 and traditional DDR4 memory. By doing this, Supermicro can increase the memory capacity as economically as possible, because the price of Optane PMem is about half of DDR4 (per GB).

It should be noted that X12 BigTwin uses a shared power supply and heat dissipation design to reduce OPEX. Supermicro shared with us a specific use case about their clients in the financial sector. Compared with the X12 1U system, they found that the X12 BigTwin system saves more than 20% in power, because the BigTwin system only consumes 675W of power when running the same workload, while the X12 1U system consumes 980W.

Large-scale systems are not for everyone. Supermicro's compact clusters are ideal for organizations looking for an entry-level, ready-to-deploy DevOps environment. Supermicro provides a free remote access program called Rack Plug and Play JumpStart for compact clusters for IT and developers to test their workflow and evaluate usability and performance. X12 BigTwin can be customized to use SYS-620BT-DNC8R scale-out object storage. SYS-620BT-DNC8R is a 2U 2-node SKU and supports 3.5-inch SAS/SATA drives. With this configuration, a 100Gb interface may not be needed to optimize performance per dollar. Compact clusters are also available as a total rack solution with pre-installed Red Hat OpenShift, which we will introduce in detail in this article.

If the demand becomes smaller or larger, Supermicro has a reference architecture for edge, regional, and core data center solutions that can be easily expanded to the required capacity.

Supermicro also designed four other rack plug-and-play configurations-Edge, Performance, High Density and Extreme (in the order of Intel computing cores from least to most).

In order to better understand the hardware attached to each configuration, we looked at Edge's BOM (SKU: SRS-OPNSHFT-3N), which consists of three master nodes, three application nodes, and three ODF nodes located in the same location. . In order to minimize the number of nodes, infrastructure nodes are omitted, and their responsibilities are distributed among other nodes.

The solution has 72 Intel computing cores, 768GB of RAM and 138TB of storage space (46TB is NVMe storage space). For redundant interconnection, it has a dual 10Gb network (four in total) for management and data exchange through dual SX350X-12 switches. All of these take up only 6U of rack space and can be powered by two standard 120v circuits, which can usually be found in offices or homes.

Supermicro envisions its edge configuration use case as running AI/ML at the edge and targeting target markets such as retail, healthcare, manufacturing, and energy. We also believe that this will provide a good low-cost solution for small development teams and distributed content delivery networks.

The BOM of Performance SKU (SRS-OPNSHFT-10) consists of three main nodes, three Infra nodes at the same location, three application nodes, and three ODF nodes. The solution has 288 Intel computing cores, 3TB RAM and 138TB NVMe storage. For redundant interconnection, it has a 1Gb network for management (four in total) and a four-way 25Gb network for data exchange through dual SX350X-12 switches, and management traffic is passed through SSE-G3648BR switches. All of these take up 42U of rack space.

The high-density SKU (SRS-OPNSHFT-20) has 336 Intel computing cores, 3TB RAM and 18 nodes. This high-density SuperRack solution is a scale-out version of a compact cluster, but uses two SYS-220BT-HNTR (2U 4-node system) as the controller and infrastructure nodes, and SYS-620BT-DNC8R (2U 2-node system) As an application node.

Furthermore, the Extreme SKU (SRS-OPNSHFT-30) has 640 Intel computing cores, 8TB RAM and 22 nodes. Below are the complete specifications of all four Supermicro SKUs.

The software pre-installed for all SKUs is Red Hat’s very popular OpenShift, an enterprise-level K8s platform with full-stack automation for managing K8 deployments. Red Hat is a strong supporter and participant of the K8s community, and OpenShift is well received by the community. OpenShift not only allows system-level management, but also allows self-service for development teams. One of the advantages of OpenShift is that if you do decide that you need to use public cloud resources, you can use the same interface workflow. This is huge because it reduces the learning curve and prevents errors due to changes in workflow.

To briefly summarize, OpenShift is a very mature product because it was originally developed more than ten years ago, during which time it has been Red Hat’s Platform as a Service (PaaS) and is completely open source. Its software components are based on a highly curated stack using the best and most popular components. For orchestration and scheduling, it uses K8s and Docker as the container runtime, and of course Red Hat for the operating system.

A key component in the software stack is the OpenShift Container Platform (OCP) and Red Hat Enterprise Linux CoreOS (RHCOS), both of which are provided with CoreOS (a company acquired by Red Hat in 2018).

RHCOS is a streamlined version of Red Hat Enterprise Linux, designed for container use. OCP is a platform as a service (PaaS) built for Linux containers, orchestrated and managed by K8s. Both have been thoroughly tested by Red Hat and have been widely used in this field. These are the key factors to reduce the bare metal cost of the solution.

In order to speed up development and deployment, OpenShift includes the Operator framework. Operator in K8s is software that encapsulates the human knowledge required to deploy and manage applications on K8s. Operator Framework is a set of tools and K8s components that facilitate Operator development and centralized management on multi-tenant clusters. OpenShift has operators suitable for popular applications such as Redis and Cassandra. By using operators, regardless of the skill level of the deployer, you can save a lot of time and frustration when deploying applications.

One of Supermicro's hidden gems is SuperCloud Composer (SCC), which is a single glass panel that allows you to monitor and manage servers and deploy operating systems on your computer. It has an API that allows others to use its Redfish compatible integration, and its use ranges from power management to asset management. This is part of the secret weapon to complete the solution stack by providing hardware-level support.

The prerequisites for Supermicro compact clusters are similar to Red Hat's standard OpenShift installation. These prerequisites include but are not limited to the following:

If all the prerequisites are met, the compact cluster can be configured directly. In the future, Supermicro plans to release Ansible Playbook to help orchestrate the setup process, including HW RAID 1 configuration for NVMe M.2 boot drives and OpenShift cluster installation. Essentially, this will help to achieve a zero-touch configuration, which we can usually only see from ISVs with OEM equipment.

Node D is physically located in a 2U cabinet, but has been isolated to a separate BMC network for use by Supermicro administrators who actively support the JumpStart program. This node has a 1Gb NIC for "configuration bridging" of the SCC virtual machine. Based on the network topology, the JumpStart program allows remote users to securely access the SCC virtual machine and OpenShift cluster to explore the functions of the cluster, including video-on-demand demonstrations running on the Pod. The demo workflow is shown in the figure below.

Investing in new solutions is always accompanied by uncertainty about workload and workflow compatibility. To alleviate these concerns, Supermicro has established a strong Jumpstart program to give potential customers the opportunity to try out the solution before purchasing.

The public cloud is booming because it is generally believed to be cheaper than on-premises deployment. However, in many cases, owning hardware is more attractive when comparing its stable pricing plan with the ongoing costs associated with cloud solutions. Yes, cloud providers do have the advantage of scale, and they can spread the cost of operating personnel across thousands of nodes. However, Supermicro alleviated this situation by using OpenShift, where Red Hat took on the burden of testing and maintaining the solution. This means that Supermicro customers do not have to verify updates and patches to their systems, which can be a time-consuming and costly process.

Pricing stability is also a factor affecting Supermicro. There must be more than one horror story of public cloud consumers overspending their budgets due to excessive consumption of resources. The reality is that for a highly automated solution like K8s, a simple misconfiguration will produce a large number of applications that consume public cloud resources. In contrast, with an on-premise solution, this is impossible.

This solution is not only cost-competitive with public clouds, but Supermicro also offers other cost-reducing suggestions, such as running a bare metal subscription for Red Hat Open Data Foundation (ODF) instead of running it on a hypervisor, because it is usually more Cheap and avoid administrative procedure tax.

We asked Supermicro customers how to purchase OpenShift (the software component in their solution), and they said they could include it in their single invoice, or the customer could purchase it directly from Red Hat.

Supermicro kindly calculated for us some figures about the cost difference when deploying OpenShift on bare metal or using hypervisors. We were surprised to find how much money can be saved by running it on bare metal and running it on a hypervisor. Since K8s applications require storage, they also include a file system that is estimated to have only 4 cores, and block storage for object storage.

The numbers show that depending on how much computing and storage resources each cluster requires, running OpenShift on bare metal is 3 to 9 times more cost-effective than running OpenShift on a hypervisor. Two options were analyzed to estimate cost savings. At the lower end of the spectrum, 16 OCP subscriptions (32 cores) and 2 ODF (4 cores) are considered. At the high end of the spectrum, all 64 cores of each node are considered for calculation and storage. By default, each node may support 250 Pods. Supermicro recommends careful planning of OpenShift deployment. This OpenShift 4.8 planning document is a good starting point.

Data locality is a tricky subject. Many companies and governments have very strict regulations on where data must reside. With an on-premise solution, you and any auditors can rest assured, even the location of the data. Physical access to storage in the public cloud is impossible-time, the end of the story.

Supermicro identified a market and created a reliable and affordable solution to fill it, as it has often done in the past. In this case, the market is suitable for on-premises K8s clusters using Intel’s proven third-generation Xeon processors to improve reliability, and combined with Intel Optane PMem to not affect performance Control costs and Red Hat’s proven K8s software. These predefined solutions allow customers to quickly deploy K8S clusters because these solutions come from Supermicro's pre-architectural, verification and testing.

Their solutions can be deployed within a few days, instead of deploying their own solutions, which usually takes several weeks, and since Supermicro's solutions have undergone extensive compatibility testing, you will not experience any time or cost damage The problem occurs during the deployment of the new technology. When it comes to cost, Supermicro's solution pricing model prevents the sticker shock that may occur when using public clouds.

No matter what your needs are, Supermicro has developed solutions to meet your needs; from Edge SKUs designed for edge AI/ML work or small development teams, to production workloads designed for regional and core data centers Extreme SKU can be used to replace public cloud deployments.

To get more information about Supermicro's K8s solution, you can visit their portal here.

Check out the Jumpstart plan to experience this solution for yourself.

Newsletter | Youku | Podcast iTunes/Spotify | Instagram | Twitter | Facebook | RSS Subscription

Tom Fenton has accumulated a wealth of IT practical experience in various technologies in the past 27 years, and has focused on virtualization and storage for the past 20 years. He previously worked as a senior course developer, solution engineer, and competitive marketing team at VMware. He also served as a senior verification engineer at The Taneja Group, led the verification services laboratory, and played an important role in launching his vSphere Virtual Volumes practice. He is on Twitter @vDoppler

Subscribe to the StorageReview newsletter for the latest news and reviews. We guarantee no spam!

StorageReview.com is the world's leading independent storage authority, providing in-depth news reports, detailed reviews, SMB/SME consulting, and laboratory services on storage arrays, hard drives, SSDs, and related hardware and software that make these storage solutions work. . Our focus is on storage solutions for the mid-market and enterprises, and the coverage of core brands that provide customer storage solutions is limited.

Copyright © 1998-2021 Flying Pig Ventures, LLC Cincinnati, Ohio. all rights reserved.