如何理解GPU中的"All threads in a warp execute the same instruction at the same time."？
How to understand "All threads in a warp execute the same instruction at the same time." in GPU?
我正在阅读 Professional CUDA C Programming，在 GPU 架构概述 部分：
CUDA employs a Single Instruction Multiple Thread (SIMT) architecture to manage and execute threads in groups of 32 called warps. All threads in a warp execute the same instruction at the same time. Each thread has its own instruction address counter and register state, and carries out the current instruction on its own data. Each SM partitions the thread blocks assigned to it into 32-thread warps that it then schedules for execution on available hardware resources.
The SIMT architecture is similar to the SIMD (Single Instruction, Multiple Data) architecture. Both SIMD and SIMT implement parallelism by broadcasting the same instruction to multiple execution units. A key difference is that SIMD requires that all vector elements in a vector execute together in a unifed synchronous group, whereas SIMT allows multiple threads in the same warp to execute independently. Even though all threads in a warp start together at the same program address, it is possible for individual threads to have different behavior. SIMT enables you to write thread-level parallel code for independent, scalar threads, as well as data-parallel code for coordinated threads. The SIMT model includes three key features that SIMD does not:
➤ Each thread has its own instruction address counter.
➤ Each thread has its own register state.
➤ Each thread can have an independent execution path.
第一段提到“All threads in a warp execute the same instruction at the same time.”，而第二段提到“Even though all threads in a warp start together at the same program address, it is possible for individual threads to have different behavior.”。这让我很困惑，上面的说法似乎自相矛盾。谁能解释一下？
没有矛盾。 warp 中的所有线程始终以锁步方式执行相同的指令。为了支持条件执行和分支，CUDA 在 SIMT 模型中引入了两个概念
- 谓词执行（参见
）
- 说明replay/serialisation（参见
）
谓词执行意味着条件指令的结果可用于屏蔽线程，使其不执行没有分支的后续指令。指令重放是处理经典条件分支的方式。所有线程通过重放指令执行条件执行代码的所有分支。不遵循特定执行路径的线程将被屏蔽并执行与 NOP 等效的操作。这就是CUDA中所谓的branch divergence penalty，因为它对性能影响很大
这就是锁步执行支持分支的方式。
我正在阅读 Professional CUDA C Programming，在 GPU 架构概述 部分：
CUDA employs a Single Instruction Multiple Thread (SIMT) architecture to manage and execute threads in groups of 32 called warps. All threads in a warp execute the same instruction at the same time. Each thread has its own instruction address counter and register state, and carries out the current instruction on its own data. Each SM partitions the thread blocks assigned to it into 32-thread warps that it then schedules for execution on available hardware resources.
The SIMT architecture is similar to the SIMD (Single Instruction, Multiple Data) architecture. Both SIMD and SIMT implement parallelism by broadcasting the same instruction to multiple execution units. A key difference is that SIMD requires that all vector elements in a vector execute together in a unifed synchronous group, whereas SIMT allows multiple threads in the same warp to execute independently. Even though all threads in a warp start together at the same program address, it is possible for individual threads to have different behavior. SIMT enables you to write thread-level parallel code for independent, scalar threads, as well as data-parallel code for coordinated threads. The SIMT model includes three key features that SIMD does not:
➤ Each thread has its own instruction address counter.
➤ Each thread has its own register state.
➤ Each thread can have an independent execution path.
第一段提到“All threads in a warp execute the same instruction at the same time.”，而第二段提到“Even though all threads in a warp start together at the same program address, it is possible for individual threads to have different behavior.”。这让我很困惑，上面的说法似乎自相矛盾。谁能解释一下？
没有矛盾。 warp 中的所有线程始终以锁步方式执行相同的指令。为了支持条件执行和分支，CUDA 在 SIMT 模型中引入了两个概念
- 谓词执行（参见
- 说明replay/serialisation（参见
谓词执行意味着条件指令的结果可用于屏蔽线程，使其不执行没有分支的后续指令。指令重放是处理经典条件分支的方式。所有线程通过重放指令执行条件执行代码的所有分支。不遵循特定执行路径的线程将被屏蔽并执行与 NOP 等效的操作。这就是CUDA中所谓的branch divergence penalty，因为它对性能影响很大
这就是锁步执行支持分支的方式。