#

flashinfer

Here is 1 public repository matching this topic...

Bruce-Lee-LY / decoding_attention

Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.

gpu cuda inference nvidia mha multi-head-attention llm large-language-model flash-attention cuda-core decoding-attention flashinfer

Updated Sep 14, 2024
C++

Improve this page

Add a description, image, and links to the flashinfer topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the flashinfer topic, visit your repo's landing page and select "manage topics."