LUSTRE HPC File System Deployment Method

With the mass adoption of clusters and explosive growth of data storage needs, I/O bandwidth challenges are becoming common in a variety of public and private sector environments. Lustre file system, an open source shared file system, is a natural fit for places where traditional shared file systems, such as NFS, do not scale to the required aggregate throughput requirements of these clusters.

The ability of Lustre file system to scale capacity and performance for any need reduces the need to deploy many separate file systems such as one for each compute cluster. Lustre aggregates and scales the storage capacity and I/O throughput of many servers and can be easily increased by adding servers dynamically. Lustre can be scaled up or down with respect to the number of client nodes, disk storage and bandwidth. It can be deployed in wide variety of configuration. Installation and configuration of Lustre file system with a number of Lustre data nodes and Lustre Clients is not just executing RPMs but a complex and time consuming task.

In this Knowledge Sharing article, Upanshu Singhal and Tarun Mathur explore installation and configuration of Lustre File System with a step-by-step approach. This article provides:

• Introduction of Lustre File System

• Describing the Lustre File System components

• System requirement for Lustre installation

• Building Lustre components from the sources

• Installing Lustre components on Linux hosts

• Configuring Lustre File System in an enterprise environment

• Introduction to basic tool set and command lines

• Introduction to troubleshooting

• Introduction to generating I/O load

• Introduction to performance measurement using various tools Benefits provided by this article:

• Helps understand HPC File System, its components, and usage

• Helps Professional Services to install and configure Lustre File System in EMC provided environment

• Single point of reference to Lustre File System and its installation and configuration

• Self-learning, with very minimal effort and time required from others

Read the full article.