Chapter 11. Using FFIO

11.1.1. Reading and writing text tiles

Most human-readable files are in text format; this format contains records comprised of ASCII characters with each record terminated by an ASCII line-feed character, which is the newline character in UNIX terminology. The FFIO specification that selects this file structure is assign -F text.

The FFIO package is seldom required to handle text files. In the following types of cases, however, using FFIO may be necessary:

• Optimizing text file access to reduce I/O wait time

• Handling multiple EOF records in text files

• Converting data files to and from other formats

I/O speed is important when optimizing text file access. Using assign -F text is expensive in terms of CPU time, but it lets you use memory-resident and SDS files, which can reduce or eliminate I/O wait time.

The FFIO system also can process text files that have embedded EOF records. The ~e string alone in a text record is used as an EOF record. Editors such as sed(1) or other standard utilities can process these files, but it is sometimes easier with the FFIO system.

On UNICOS and UNICOS/mk systems, the text layer is also useful in conjunction with the fdcp(1) command. The text layer provides a standard output format. Many forms of data that are not considered foreign are sometimes encountered in a heterogeneous computing environment. If a record format can be described with an FFIO specification, it can usually be converted to text format by using the following script:

OTHERSPEC=$1 INFILE=$2 OUTFILE=$3 assign -F ${OTHERSPEC} ${INFILE} assign -F text ${OUTFILE} fdcp ${INFILE} ${OUTFILE}

Use the fdcp command to copy files while converting record blocking.

11.1.2. Reading and writing unblocked files

The simplest form of data file format is the simple binary stream or unblocked data. It contains no record marks, file marks, or control words. This is usually the fastest way to move large amounts of data, because it involves a minimal amount of CPU and system overhead.

The FFIO package provides several layers designed specifically to handle this binary stream of data. These layers are syscall, sds, and mr. These layers behave the same from the user's perspective; they only use different system resources. The unblocked binary stream is usually used for unformatted data transfer. It is not usually useful for text files or when record boundaries or backspace operations are required. The complete burden is placed on the application to know the format of the file and the structure and type of the data contained in it.

This lack of structure also allows flexibility; for example, a file declared with one of these layers can be manipulated as a direct-access file with any desired record length.

In this context, fdcp can be called to do the equivalent of the cp(1) command only if the input file is a binary stream and to remove blocking information only if the output file is a binary stream.

11.1.3. Reading and writing fixed-length records

The most common use for fixed-length record files is for Fortran direct access. Both unformatted and formatted direct-access files use a form of fixed-length records. The simplest way to handle these files with the FFIO system is with binary stream layers, such as system, syscall, cache, cachea, sds, and mr. These layers allow any requested pattern of access and also work with direct-access files. The syscall and system layers, however, are unbuffered and do not give optimal performance for small records.

The FFIO system also directly supports some fixed-length record formats.

11.1.4. Reading and writing COS blocked files

The COS blocking format is the default file structure for all Fortran sequential unformatted files on UNICOS and UNICOS/mk systems, except tape files. The cos layer is provided to handle these files. It provides for COS blocked files on disk and on magnetic tape and it supports multifile COS blocked datasets.

The cos layer must be specified for COS blocked files. If COS is not the default file structure, or if you specify another layer, such as sds, you may have to specify a cos layer to get COS blocking.