Implementing mixed-precision training (D) is the most effective way to address low GPU utilization during backpropagation. Mixed precision uses FP16 alongside FP32, leveraging NVIDIA Tensor Cores to accelerate matrix operations in backpropagation, reducing compute time and memory usage. This keeps GPUs busier by increasing throughput, especially in distributed setups where synchronization waits can exacerbate idling.
* More layers(A) increases compute but may not target backpropagation efficiency and risks overfitting.
* Higher learning rate(B) affects convergence, not utilization directly.
* Data pipeline optimization(C) helps forward passes but not backpropagation compute bottlenecks.
NVIDIA's mixed precision is a proven solution for training efficiency (D).