Lines Matching full:atomic

10 /// This pass optimizes atomic operations by using a single lane of a wavefront
11 /// to perform the atomic operation, thus reducing contention on that memory
13 /// Atomic optimizer uses following strategies to compute scan and reduced
36 #define DEBUG_TYPE "amdgpu-atomic-optimizer"
196   // Early exit for unhandled address space atomic instructions.
226   // Only 32 and 64 bit floating point atomic ops are supported.
235   // If the pointer operand is divergent, then each lane is doing an atomic
244   // value to the atomic calculation. We can only optimize divergent values if
245   // we have DPP available on our subtarget (for DPP strategy), and the atomic
255   // If we get here, we can optimize the atomic using a single wavefront-wide
256   // atomic operation to do the calculation for the entire wavefront, so
330   // value to the atomic calculation. We can only optimize divergent values if
331   // we have DPP available on our subtarget (for DPP strategy), and the atomic
349   // If we get here, we can optimize the atomic using a single wavefront-wide
350   // atomic operation to do the calculation for the entire wavefront, so
357 // Use the builder to create the non-atomic counterpart of the specified
365     llvm_unreachable("Unhandled atomic op");
589   // Get the value required for atomic operation
629     llvm_unreachable("Unhandled atomic op");
678   // If we're optimizing an atomic within a pixel shader, we need to wrap the
679   // entire atomic operation in a helper-lane check. We do not want any helper
703   // This is the value in the atomic operation we need to combine in order to
704   // reduce the number of atomic operations.
733   // For atomic sub, perform scan with add operation and allow one lane to
769         // which we will provide to the atomic operation.
783       llvm_unreachable("Atomic Optimzer is disabled for None strategy");
788       llvm_unreachable("Unhandled atomic op");
792       // The new value we will be contributing to the atomic operation is the
815       // These operations with a uniform value are idempotent: doing the atomic
821       // The new value we will be contributing to the atomic operation is the
891   // Clone the original atomic operation into single lane, replacing the
902     // Create a PHI node to get our new atomic result into the exit block.
913     // Now that we have the result of our single atomic operation, we need to
915     // we previously calculated combined with the atomic result value we got
916     // from the first lane, to get our lane's index into the atomic result.
925         llvm_unreachable("Atomic Optimzer is disabled for None strategy");
932         llvm_unreachable("Unhandled atomic op");
981       // Replace the original atomic instruction with the new one.
991                       "AMDGPU atomic optimizations", false, false)
995                     "AMDGPU atomic optimizations", false, false)