Managing Lustre for the Cray Linux Environment (CLE)

S-0010-4201 - Jul 2013

Provides instructions for configuring and managing Lustre file system(s) on Cray systems running the Cray Linux Environment (CLE) operating system software. This book documents the differences in Cray's Lustre implementation and utilities specific to CLE. The Lustre Operations Manual from Oracle/Whamcloud is available online and is the primary source for information about Lustre file systems.


Available formats

Adobe Acrobat Reader S-0010-4201.pdf
Valid HTML 4.01 html-S-0010-4201

Table of contents

Introduction to Lustre on a Cray System
    1.1  Lustre File System Documentation
    1.2  Lustre Software Components
    1.3  Lustre Framework
Lustre File System Configuration
    2.1  Configuring Lustre File Systems on a Cray System
    2.2  File System Definition Parameters
    2.2.1  Required File System Definitions  Device Configuration  Mount Path Patterns
    2.2.2  Optional File System Definitions
    2.3  Creating Lustre File Systems
    2.4  Mounting Lustre Clients
    2.5  Mounting and Unmounting Lustre Clients Manually
    2.6  Exporting Lustre with NFSv3
    2.7  Lustre Option for Panic on LBUG for CNL and Service Nodes
    2.8  Updating the Bootimage
Lustre File System Management
    3.1  Storage and Network Information
    3.1.1  Storage Devices for Lustre
    3.1.2  Lustre Networking (LNET)
    3.2  Configuring Striping on Lustre File Systems
    3.2.1  Configuration and Performance Trade-off for Striping
    3.2.2  Overriding File System Striping Defaults
    3.3  Lustre System Administration
    3.3.1  Lustre Commands for System Administrators
    3.3.2  Identifying Metadata Server (MDS) and Object Storage Targets (OST)
    3.3.3  Starting and Stopping Lustre
    3.3.4  Adding an OST
    3.3.5  Recovering From a Failed OST
    3.3.6  OSS Read Cache and Writethrough Cache
    3.3.7  Lustre 2.x Performance and max_rpcs_in_flight
    3.3.8  Checking Lustre Disk Usage
    3.3.9  Lustre User and Group Quotas
    3.3.10  Checking the Lustre File System
    3.4  Ensuring a File System Definition is Consistent with Cray System Configurations
    3.4.1  Verifying Lustre File System Configuration
    3.4.2  Updating the File System Definition File for CLE 4.2
    3.5  Troubleshooting
    3.5.1  Dumping Lustre Log Files
    3.5.2  Lustre Users Report ENOSPC Errors
    3.5.3  File System Error Messages
Lustre Failover
    4.1  Lustre Failover on Cray Systems
    4.1.1  Configuration Types for Failover
    4.2  Lustre Manual Failover
    4.2.1  Configuring Manual Lustre Failover
    4.2.2  Performing Manual Failover
    4.2.3  Monitoring Recovery Status
    4.3  Lustre Automatic Failover
    4.3.1  Lustre Automatic Failover Database Tables  The filesystem Database Table  The lustre_service Database Table  The lustre_failover Database Table
    4.3.2  Backing Up SDB Table Content
    4.3.3  Using the xtlusfoadmin Command
    4.3.4  System Startup and Shutdown when Using Automatic Lustre Failover
    4.3.5  Configuring Lustre Failover for Multiple File Systems
    4.3.6  Imperative Recovery
Lustre Failback
    5.1  Lustre Failback
    5.1.1  Failback in Manual and Automatic Failover

Software Releases this book supports

Product Version Sub Product Release Date
Cray Linux Environment (CLE) 4.2.UP01 Jul 2013
Cray Linux Environment (CLE) 4.2.UP02 Oct 2013

Other versions of this book

Publication Number Release Date Supported Software Releases
S-0010-5203 Apr 2015 Cray Linux Environment (CLE) 5.2.UP03, Cray Linux Environment (CLE) 5.2.UP04
S-0010-52 Mar 2014 Cray Linux Environment (CLE) 5.2.UP00, Cray Linux Environment (CLE) 5.2.UP02, Cray Linux Environment (CLE) 5.2.UP01
S-0010-42 Apr 2013 Cray Linux Environment (CLE) 4.2
S-0010-4101 Dec 2012 Cray Linux Environment (CLE) 4.1.UP01
S-0010-5001 Nov 2012 Cray Linux Environment (CLE) 5.0.UP02, Cray Linux Environment (CLE) 5.0.UP03, Cray Linux Environment (CLE) 5.1.UP00, Cray Linux Environment (CLE) 5.1.UP01
S-0010-4002 Dec 2011 Cray Linux Environment (CLE) 4.0.UP02, Cray Linux Environment (CLE) 4.0.UP03
S-0010-4001 Sep 2011 Cray Linux Environment (CLE) 4.0.UP01
S-0010-40 Jun 2011 Cray Linux Environment (CLE) 4.0
S-0010-31 Jun 2010 Cray Linux Environment (CLE) 3.1, Cray Linux Environment (CLE) 3.1.UP02
S-0010-30 Mar 2010 Cray Linux Environment (CLE) 3.0
S-0010-22 Jul 2009 Cray Linux Environment (CLE) 2.2
S-0010-21 Nov 2008 Cray Linux Environment (CLE) 2.1
S-0010-10 Sep 2008 Knowledge Base 1.0