Cray Linux Environment (CLE) Software Release Overview

S-2425-31 - Jun 2010

Provides an overview of the Cray Linux Environment (CLE) 3.1 software release, including software enhancements, differences from previous CLE releases, documentation, contents of the release package and software services provided with the release. Because CLE 3.0 was released for a single hardware platform, this book includes enhancements and differences which were introduced with that release, in addition to those introduced with CLE 3.1.


Available formats

Adobe Acrobat Reader S-2425-31.pdf
Valid HTML 4.01 html-S-2425-31

Table of contents

    1.1  Emphasis for the CLE 3.1 Release
    1.2  Supported System Configurations
    1.3  Description of the CLE 3.1 Release Software
    1.4  CLE 3.1 Support Policy
Software Enhancements
    2.1  Cray XE Systems Supported
    2.1.1  Gemini Based System Interconnection Network
    2.1.2  XIO Service Blades for Cray XE Systems
    2.1.3  Software for the Cray Gemini ASIC
    2.1.4  Performance Analysis for Cray XE Systems
    2.2  Cray X6 Compute Blades Supported
    2.3  SUSE Linux Enterprise Server (SLES) 11 Upgrade
    2.3.1  OpenFabrics InfiniBand Upgrade
    2.3.2  SLES 11 Documentation
    2.3.3  Differences and Limitations Introduced with SLES 11
    2.4  Configuration and Packaging Enhancements
    2.5  Lustre 1.8 File System Upgrade
    2.5.1  Adaptive Time-outs
    2.5.2  Version-based Recovery (VBR)
    2.5.3  OST Pools
    2.5.4  OSS Read Cache
    2.6  Lustre Imperative Recovery
    2.7  Dynamic Shared Objects and Libraries (DSL)
    2.8  Cluster Compatibility Mode (CCM)
    2.8.1  Limitations for Using CCM
    2.9  Core Specialization
    2.9.1  Limitations for Using Core Specialization
    2.10  Node Health Checker (NHC) Enhancements
    2.10.1  Suspect Mode Returns Healthy Nodes to Resource Pool Faster
    2.10.2  NHC Verifies ALPS Acknowledgement of a Node's State Change
    2.10.3  Automatic Recovery If a Login Node Crashes While xtcheckhealth Monitoring Nodes
    2.10.4  xtcheckhealth Logs Dumped by xtdumpsys
    2.10.5  Enhancements to NHC Tests
    2.10.6  New and Changed NHC Variables
    2.11  Cray Data Virtualization Service (Cray DVS) Enhancements
    2.11.1  Stripe Parallel Mode for Cray DVS
    2.11.2  Failover and Failback for Cray DVS Stripe Parallel and Cluster Parallel Modes
    2.11.3  Collecting Statistics for Cray DVS
    2.12  Application Level Placement Scheduler (ALPS) Enhancements
    2.12.1  ALPS Exclusive Access
    2.12.2  Batch Application Scheduler Interface Layer (BASIL) Protocol Updated
    2.12.3  Comprehensive System Accounting and ALPS Interface
    2.12.4  ALPS Interface to Cray Management Services (CMS)
    2.12.5  New aprun -B Option
    2.13  Out of Memory (OOM) Killer
    2.14  XPMEM Kernel Module
    2.15  System Resiliency Enhancements
    2.15.1  Support for Multipathing I/O
    2.15.2  SDB Node Failover
    2.15.3  Compute Node Failover Manager
    2.15.4  Cray DVS Failover
    2.16  System Administration Enhancements
    2.16.1  New CLEinstall Options and Configuration Parameters  CLEinstall Support for New Features  NFS Tuning Support in CLEinstall.conf
    2.16.2  New Node Allocation Mode of other
    2.16.3  Call of rc.local Script During Boot
    2.16.4  New clerelease File Indicates Installed Release Level
    2.16.5  ldump Enhancements  New -q Option Turns Off Printing of a Dump's Progress  For Cray XE Systems: New Default Access Method Command-line String Name, xt-bhs  For Cray XE Systems: ldump Uses New NIC Mapping
    2.17  Bugs Addressed Since the Last Release
Compatibilities and Differences
    3.1  Binary Compatibility
    3.2  PCI-X Network Interface Cards are Not Supported
    3.3  Changing the Default syslog-ng Configuration
    3.4  Commands Removed from the Release
    3.5  Deprecated Commands
    3.6  Requirements for Cray Performance Analysis Tools
    3.7  make Utility Enhancements and Differences
    3.8  Software Packages/Releases That Must be Reinstalled
    3.9  Changes in Setting System-wide Default Modulefiles
    3.10  ALPS Reservations File Format Changed
    3.11  Removed Obsolete Routing Entry in xtnodestat Legend
    3.12  Node Health Checker (NHC) Changed Functionality
    3.12.1  NHC Release Upgrade Compatibility
    3.12.2  NHC Files, Binary, and Script Moved
    3.12.3  xtcleanup_after Script Changes
    3.12.4  xtcheckhealth Binary Interface Changes
    3.12.5  CLE 3.0–CLE 3.1: Additional NHC Changes
    3.13  Installation and Configuration Changed Functionality for System Administrators
    3.13.1  Supported Upgrade Path
    3.13.2  System Management Workstation (SMW) Upgrade Requirements
    3.13.3  Installation Time Required
    3.13.4  XTinstall Utilities Renamed CLEinstall
    3.13.5  /dev/sd* Device Ordering is Not Guaranteed
    3.13.6  Changes to the CLEinstall.conf Installation Configuration File
    3.13.7 Script Differences
    3.13.8  shell_* Installation Scripts on the Boot Node Have Moved
    3.13.9  Local Changes to *rc.local Scripts are Maintained After a CLE Upgrade
    3.14  General System Administration Differences
    3.14.1  For Cray XE Systems: Gemini Component Naming and Numbering
    3.14.2  Configuration and Packaging Changes
    3.14.3  Quotes are Allowed in the System Configuration File
    3.14.4  IP Forwarding No Longer Required on the Boot Node
    3.14.5  Virtual IP Addresses for Boot and SDB Nodes
    3.14.6  Pluggable Authentication Module (PAM) Now Controls /etc/nologin
    3.14.7  ALPS Adds Additional Job Information to Syslog Messages
    3.14.8  Lustre 1.8 File System Compatibility
    3.14.9  Lustre File System Configuration Requires Persistent Device Names
    3.14.10  scsidev is Obsolete
    3.14.11  cnos Specialization Class for Compute Node Shared Root
    3.14.12  xtpackage and xtclone Must be Invoked as root
    3.14.13  xtrelswitch No Longer Supports Switching the SDB
    3.14.14  xtprocadmin Command Differences
    3.14.15  ldump Command Differences
    3.14.16  xtcdr2proc Supported Only on the Boot Node
    3.15  Documentation Differences
    4.1  Cray Documentation Website
    4.2  CrayPort Website
    4.3  CrayDoc Documentation Delivery System
    4.4  Accessing Product Documentation
    4.5  Cray-developed Books Provided with This Release
    4.5.1  Additional Cray-developed Release Documents
    4.5.2  Cray-developed Books No Longer Provided with this Release
    4.6  Third-party Books Provided with This Release
    4.7  Changes to Man Pages
    4.7.1  New Cray Man Pages
    4.7.2  Removed Cray Man Pages
    4.7.3  Changed Cray Man Pages
    4.8  Other Related Documents Available
    4.9  Additional Documentation Resources
    4.10  Ordering Documentation
    4.11  Cray Glossary
Release Contents
    5.1  Hardware Requirements
    5.2  Software Requirements
    5.2.1  Release Level Requirements for Other Cray Software Products
    5.2.2  Third-party Software Requirements
    5.3  Supported Upgrade Path
    5.4  Contents of the Release Package
    5.4.1  CLE 3.1 Software Components
    5.5  Licensing
    5.6  Ordering Software
Customer Services
    6.1  Technical Assistance with Software Problems
    6.2  CrayPort
    6.3  Training
    6.4  Cray Public Website
Configuration and Packaging Changes
Differences Between CLE 3.0 and CLE 3.1

Software Releases this book supports

Product Version Sub Product Release Date
Cray Linux Environment (CLE) 3.1.UP02 Jan 2011
Cray Linux Environment (CLE) 3.1.UP03 Mar 2011
Cray Linux Environment (CLE) 3.1 Jun 2010

Other versions of this book

Publication Number Release Date Supported Software Releases
S-2425-52xx Apr 2014 Knowledge Base 1.0
S-2425-52 Mar 2014 Cray Linux Environment (CLE) 5.2.UP00, Cray Linux Environment (CLE) 5.2.UP03, Cray Linux Environment (CLE) 5.2.UP02, Cray Linux Environment (CLE) 5.2.UP04, Cray Linux Environment (CLE) 5.2.UP01
S-2425-51 Sep 2013 Cray Linux Environment (CLE) 5.1.UP00, Cray Linux Environment (CLE) 5.1.UP01
S-2425-42 Apr 2013 Cray Linux Environment (CLE) 4.2, Cray Linux Environment (CLE) 4.2.UP01, Cray Linux Environment (CLE) 4.2.UP02
S-2425-5002 Mar 2013 Cray Linux Environment (CLE) 5.0.UP02, Cray Linux Environment (CLE) 5.0.UP03
S-2425-4101 Dec 2012 Cray Linux Environment (CLE) 4.1.UP01
S-2425-40 Jun 2011 Cray Linux Environment (CLE) 4.0.UP02, Cray Linux Environment (CLE) 4.0.UP01, Cray Linux Environment (CLE) 4.0, Cray Linux Environment (CLE) 4.0.UP03
S-2425-30 Mar 2010 Cray Linux Environment (CLE) 3.0
S-2425-22 Jul 2009 Cray Linux Environment (CLE) 2.2
S-2425-21 Nov 2008 Cray Linux Environment (CLE) 2.1
S-2425-20 Oct 2007 Cray XT Programming Environment 2.0, System Management Workstation (SMW) 3.0.1, UNICOS/lc 2.0
S-2425-15 Nov 2006 Cray XT3 Programming Environment 1.5, System Management Workstation (SMW) 1.5, UNICOS/lc 1.5
S-2425-14 May 2006 Cray XT3 Programming Environment 1.4, System Management Workstation (SMW) 1.4, UNICOS/lc 1.4
S-2425-13 Nov 2005 Cray XT3 Programming Environment 1.3, System Management Workstation (SMW) 1.3, UNICOS/lc 1.3
S-2425-12 Aug 2005 Cray XT3 Programming Environment 1.2, System Management Workstation (SMW) 1.2, UNICOS/lc 1.2
S-2425-11 Jun 2005 Cray XT3 Programming Environment 1.1, System Management Workstation (SMW) 1.1, UNICOS/lc 1.1