Who doesn’t love an inspirational quote?! The idea of tying one of my favorite quotes into a tech post kinda excites me. So, here goes.
This quote pops into my head on the regular. Mostly, it’s when a project’s to-do list is way too long and I feel overwhelmed. I think of this quote and proceeding thoughts are usually slow down, focus on what really needs to be done and forgot about the rest (for now). Complexity can send you in a million different direction and distracts you from executing on tasks that need to be done. Complexity is the anti-thesis of simplicity.
Simplicity was a theme from VAST Data’s messaging at Storage Field Day 18. While VAST joked that they were trying to take over the world, the fact is that they aren’t. They are focusing on key tasks that need to happen one step at a time.
Now, that I’ve shared my inspirational quote. Let’s explore who is VAST Data, what is their value proposition is, and how removing complexity has aided their ability to execute.
VAST Data’s answers to the “Who, What, and Why Now” questions
After about 3 years in stealth mode, VAST Data officially debuted in February 2019. Storage Field Day 18 was their first big public appearance. Renen Hallak, VAST CEO and Founder, was one of three presenters. Renan, hold 17+ patents and was a developer and architect of XtremIO.
VAST claims that their target customers need hyperscaler agility and scalability. However, for reasons like cost or data gravity, they aren’t able to fully get this in a public cloud. Their customers’ application-centric workloads require the large global datasets and the fast performance for machine learning and artificial intelligence.
VAST’s was conceived in era where major innovations like NVME-oF and Storage Class Memory (SCM) were just on the horizon. If you pair these disruptive advancements with really fast network speeds like 400 Gigabit Ethernet, it’s a recipe “disaggregated share everything architecture”. This architecture removes a lot of complexity for which HCI and traditional direct attached share nothing storage have had to contend for resiliency and scalability purposes.
VAST’s “disaggregated share everything storage architecture” allows compute and storage to be scaled independently. It also addresses the scalability and resiliency complexity for which traditional scale-out storage vendors have had to contend.
Currently, a typical VAST deployment is an appliance with a minimum of 1 petabyte from a NVMe JBOF (Just a Bunch of Flash) arrays and 4 containers in which the storage software lives. Supported protocols are NFS and S3. More protocols and deployment models are expected further down the road. This is another case where VAST is focusing on the protocols that their customers have requested.
Where VAST scrapped some of the complexity:
- Getting to the why first. VAST gathered customer pain points for 6-months before they started created requirements and designing their solution This allowed them to design storage to meet these needs rather than hobbling together solutions and looking for markets. Anyone who has ever done software development will tell you that a thorough requirements gathering process at the beginning saves you time, money and scope in the end.
- Only NVMe will do. NVMe commandset (only 13 required commands) that was designed specifically for non-volatile memory and flash. By forgetting about HDDs, VAST can skip the SCSI and the complexity that comes with using commands that are for a slew of devices and not only storage. NVMe-oF also lets disks connect to the fabric without storage controllers. In addition, there is no need to tier between performant and capacity disks. It’s all fast storage.
- Caching of Metadata is unnecessary. Metadata structures are stored on storage class memory. Self-describing data structures remove the need for caching and any subsequent cache consistency mechanisms. These self-describing file attributes allow VAST to handle file locking at the file attribute level not in a traditional metadata database that would have to replicated between multiple nodes. Note: Any atomic operations happen in RAM
- Keepin’ it stateless. Software logic lives in stateless containers and state lives in JBOFs. You can lose all containers and not lose data. You don’t have to manage availability failover/giveback/quorum that traditional scale-out storage clusters handle. This supports disaggregation and moves bottleneck to CPU allowing you to scale compute and storage independently. Also, it enables global algorithms for all devices in system.
No matter where you’ve worked in IT, you probably have a story about how avoiding complexity has helped you execute. I really enjoyed watching VAST weave this theme throughout their presentation.
By avoiding complexity where it’s not required, VAST can execute on their plan to create scalable and resilient storage solution to rival hyperscalers, but for the masses (or those without the means to roll their own storage architecture).
Btw, if you haven’t watched VAST’s Storage Field Day 18 presentations, it’s not too late. Even if storage isn’t your jam, VAST’s videos are still worth watching to see how a great technical presentation should look. Simple slides without too much content, including relevant historical context, and answering all of the questions that one would need to understand who VAST data is and what their key differentiators are makes a great story.
Disclaimer: As Storage Field Day 18 delegate, GestaltIT paid for my hotel, airfare, and meals. With this invitation came no obligation to blog about or endorse any the presenting vendors or GestaltIT. Any content written or opinions expressed are solely mine.