Chapter 6. Optimizing Processor-bound Code

6.1.1. Vectorization Guidelines

An explicit or implied loop must be present for vectorization to occur. In determining what loops to vectorize, the compiler considers a number of factors. There are no simple rules that will predict the compiler's behavior in all cases.

These general guidelines apply to vectorization:

• Vectorize loops whenever possible, even if the application is not multistreamed. Vectorization is the optimization technique that usually provides the greatest performance increase. The vector processor gains its performance advantage over the superscalar processor in two ways. First, the vector processor runs at twice the clock speed of the superscalar processor. Second, the vector processor chains all like operations and performs them in order, then moves on to the next set of like operations, and so on until the entire task is completed. Vector processing is usually faster than scalar processing by 20 times or more. More typically, vectorized loops can run up to 40 times faster than their scalar counterparts.

• Make vectorized loops as "fat" as possible; that is, put as much work as possible inside the loop. See Section 6.2.1 for a definition of work. The reason this is desirable is data reuse; that is, data can be used multiple times after it is loaded from memory. The greatest performance bottleneck once a loop is vectorized is low bandwidth because cache and memory are much slower than the processor.

• Use stride-1 loops (loops that increment or decrement by 1) whenever possible. Stride-1 reads and writes provide maximum bandwidth because they access data most likely to be in cache.

• For nested loops, vectorize the loop with the highest trip count if possible. This is not an absolute. Under certain conditions, the compiler may determine that vectorizing a loop with less than the maximum trip count will produce the greatest net benefit. There is setup time spent in vectorizing a loop, so the trip count should be 4 or higher.

• Use dense calculation loops; that is, use loops with a high ratio of floating point operations to memory references. (This flops-to-load ratio is also referred to as floating-point intensity.) For example, the statement:

  c(i) = a(i) * b(i) + d

  has a flops-to-load ratio of 1:1 (2 floating point operations to 2 vector loads).

6.1.2. Fully Vectorized Loops (V)

In analyzing source code, if there are no compiler options or directives prohibiting vectorization, the compiler checks all explicit and implied loops to determine whether they can be vectorized.

A fully vectorized loop is indicated in loopmark listings with V and should not be altered unless the code is producing incorrect results or you would prefer to vectorize a different loop in the loop nest.

6.1.3. Less Than Fully Vectorized Loops (Vp)

When the compiler cannot generate a fully vectorized loop, it tries to:

• Conditionally vectorize the loop, creating a scalar version and a vector version. A runtime conditional expression chooses the scalar loop when needed to avoid a recurrence; otherwise, the vector loop is chosen.

• Partially vectorize the loop. A loop is partially vectorized if it is split into multiple loops, at least one of which is scalar. This generally indicates that the loop contains some cross-iteration dependency that prevents full vectorization.

• Generate a vector loop with a computed-safe vector length (VL). The safe VL is generated at runtime. If the computed-safe VL creates a fully vectorized loop (that is, it is greater than or equal to the machine-maximum VL), it is still moderately slower than its fully vectorized counterpart because of the overhead involved in the safe VL computation.

• Generate a vector update loop. A vector update loop performs arithmetic on existing elements of an array and stores the results back into the same array. This type of loop requires both a gather operation from and a scatter operation to the same memory location and is not as fast as a fully vectorized loop.

Less than fully vectorized loops are indicated by Vp in the loopmark listing. Vp loops may present opportunities for programmer-assisted vectorization, as described in Section 6.1.4.

If a Vp type loop is in a critical portion of the code, it may be worth recompiling with the decompile option (-rd) and then examining the resulting .opt file to determine what was vectorized, what was done as scalar code, and where the limiting factor or dependency resides.

The decompilation listing shows the code generated to test for the conditions -2 + NX >= 0 and S1 == 0.

6.1.4. Programmer-assisted Vectorization

The Cray Fortran, C, and C++ compilers automatically produce highly optimized code by default. However, there are a number of factors that can inhibit automatic vectorization. Loops with the following attributes cannot be vectorized without programmer assistance:

• Loops with ambiguous data dependencies

• Loops containing non-inlined calls to subroutines and functions

• Loops containing I/O statements

• Statement labels with references from outside the loop

• References to Fortran character variables

• Intrinsic functions that do not vectorize

• Fortran RETURN, STOP, or PAUSE statements

• Loops in which C++ exceptions are thrown or caught

• C++ typeid and runtime dynamic_cast operators

You can avoid many of these inhibitors by using compiler options or directives or by modifying the source code. Before you use any of the techniques described in the following sections, use CrayPat profile reports to focus on the functions that consume the most time. See Section 2.2.5 for details.

It is important to note that the superscalar processor performs the steps of a loop in a different order than the vector processor. For example, consider the following loop:

for(i=0; i<=2; i++) { s[i] = t[i] + r[i]; x[i] = y[i] + z[i]; }

The superscalar processor performs operations in this order:

s[0] = t[0] + r[0] x[0] = y[0] + z[0] s[1] = t[1] + r[1] x[1] = y[1] + z[1] s[2] = t[2] + r[2] x[2] = y[2] + z[2]

In contrast, the vector processor performs operations in this order:

s[0] = t[0] + r[0] s[1] = t[1] + r[1] s[2] = t[2] + r[2] x[0] = y[0] + z[0] x[1] = y[1] + z[1] x[2] = y[2] + z[2]

When you vectorize a loop, be sure to compare the vector and scalar results. Run your program with the vectorized loop, check results, then insert the Fortran !dir$ nextscalar directive or C and C++ #pragma _CRI novector directive immediately before the loop, run the program again, and compare results. Apart from small numerical differences due to reductions, if the results differ, incorrect vectorization was performed.

Check loopmark listings (see Section 2.3.1) to identify loops that the compiler did not vectorize or were less than fully vectorized. Then, use one or more of the approaches listed below:

• Use compiler options. Compiler options apply to all loops in the compilation unit.

• Use directives. Directives apply to specific blocks within a program.

  You can apply vectorization to all but selected loops by:

  • Using the Fortran !dir$ nextscalar directive in conjunction with the -O vector2 or -O vector3 option. The !dir$ nextscalar directive applies to only one loop, the first loop that appears after the directive within the same program unit.

  • Using the C and C++ #pragma _CRI novector directive in conjunction with the -h vector2 or -h vector3 option. The #pragma _CRI novector directive applies only to the code block immediately following it.

• Rewrite critical sections of source code.

When you use these techniques, remember that source files can be optimized separately and that there is no one-size-fits-all solution. Check the loopmark listing again to see if your changes causes a loop to be vectorized. Then, compare the results to the scalar version to make sure that the loops yield the correct results.

The following subsections describe situations where the compiler does not vectorize a loop and the techniques you can use to enable vectorization.

The Cray Fortran Compiler options and directives that affect optimization are documented in the Cray Fortran Compiler Commands and Directives Reference Manual. The Cray C and C++ Compiler options and directives that affect optimization are documented in the Cray C and C++ Reference Manual.

Caution:

Use care when experimenting with compiler vectorization options. Illegal vectorizations can introduce errors and/or degrade performance.