Kernel and hardware libraries implement the primitive operations that compilers and inference engines schedule onto devices.
Responsibilities
- Matrix multiplication, convolution, attention, normalization, sampling, and vector operations.
- Collective communication, topology-aware transfers, and device synchronization.
- Precision-specific paths, layout requirements, workspace allocation, and autotuning.
- Versioned compatibility with drivers, accelerators, and compiler-generated code.
Operational evidence
Record kernel selection, precision, fallback, launch overhead, device occupancy, memory traffic, and communication time when diagnosing performance.