#

efficient-attention

Here are 12 public repositories matching this topic...

thu-ml / SageAttention

Quantized Attention achieves speedup of 2-5x and 3-11x compared to FlashAttention and xformers, without lossing end-to-end metrics across language, image, and video models.

cuda triton attention vit quantization video-generation mlsys inference-acceleration efficient-attention llm llm-infra video-generate

Updated Aug 5, 2025
Cuda

lucidrains / ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

attention-mechanism efficient-attention long-context distributed-attention

Updated May 16, 2025
Python

lucidrains / CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

deep-learning routing artificial-intelligence attention-mechanisms efficient-attention

Updated Sep 6, 2024
Python

jlamprou / Infini-Attention

Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval

transformer infinite attention efficient-attention llm qwen

Updated May 9, 2024
Python

Ascend-Research / CascadedGaze

The official PyTorch implementation for CascadedGaze: Efficiency in Global Context Extraction for Image Restoration, TMLR'24.

efficiency transformer image-restoration deblurring denoising efficient-attention

Updated Feb 13, 2025
Python

davidsvy / cosformer-pytorch

Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".

deep-learning neural-network pytorch artificial-intelligence transformer attention-mechanism iclr efficient-attention iclr2022

Updated Oct 29, 2021
Jupyter Notebook

HolmesShuan / Compact-Global-Descriptor

Pytorch implementation of "Compact Global Descriptor for Neural Networks" (CGD).

efficient pytorch convolutional-neural-networks attention-mechanism attention-model efficient-attention

Updated Jan 9, 2025
Python

robflynnyh / hydra-linear-attention

Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)

machine-learning transformers attention linear-attention efficient-attention

Updated Jan 8, 2023
Python

gmlwns2000 / sea-attention

Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)

attention linear-attention efficient-attention sea-attention

Updated Jun 20, 2025
Python

MAGICS-LAB / NonparametricHopfield

Nonparametric Modern Hopfield Models

efficient-transformers efficient-attention modern-hopfield-networks modern-hopfield-model efficient-hopfield-models efficient-hopfield-networks

Updated Jan 8, 2024
Jupyter Notebook

pszemraj / samba-pytorch

Minimal implementation of Samba by Microsoft in PyTorch

language-model ssm pytorch-implementation efficient-attention llm long-context-modeling mamba-state-space-models

Updated Nov 24, 2024
Python

priyanshujiiii / awesome-Attention

Resources and references on solved and unsolved problems in attention mechanisms.

machine-learning deep-learning transformer attention attention-mechanism ai-research survey-paper solved-problems unsolved-problems efficient-attention llm

Updated Aug 11, 2025

Improve this page

Add a description, image, and links to the efficient-attention topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the efficient-attention topic, visit your repo's landing page and select "manage topics."