A concise but complete full-attention transformer with a set of promising experimental features from various papers

Implementation of ViLLA-X, Enhancing Latent Action Modeling in Vision-Language-Action Models

Implementation of rectified flow and some of its followup research / improvements in Pytorch

Implementation of the model architecture for SRT-H

Implementation of the Large Behavioral Model architecture for dexterous manipulation from Toyota Research Institute

Implementation of HS-TasNet, "Real-time Low-latency Music Source Separation using Hybrid Spectrogram-TasNet"

Implementation of Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation, from Ye et al of UCSD

Implementation of ReWiND, "Language-Guided Rewards Teach Robot Policies without New Demonstrations", from USC / Amazon Robotics

The Gaussian Histogram Loss (HL-Gauss) proposed by Imani et al. with a few convenient wrappers for regression, in Pytorch

Exploration into the proposed architecture from Sapient Intelligence of Singapore 🇸🇬

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Implementation of AlphaGenome, Deepmind's updated genomic attention model

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Pytorch implementation of Evolutionary Policy Optimization, from Wang et al. of the Robotics Institute at Carnegie Mellon University

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

An implementation of PPO in Pytorch

Implementation of a transformer for reinforcement learning using `x-transformers`

Implementation of Amplify, Actionless Motion Priors for Robot Learning from Videos

Explorations into PathNet, "Evolution Channels Gradient Descent in Super Neural Networks" from Deepmind in 2017

Implementation of the dynamic chunking mechanism in H-net by Hwang et al. of Carnegie Mellon

Implementation of π₀, the robotic foundation model architecture proposed by Physical Intelligence

A simple cross attention that updates both the source and target in one step

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

Explorations into NEAT and some of its derivative research

Vector (and Scalar) Quantization, in Pytorch

Implementation and explorations into Blackbox Gradient Sensing (BGS), an evolutionary strategies approach proposed in a Google Deepmind paper for Table Tennis

Implementation of 2-simplicial attention proposed by Clift et al. (2019) and the recent attempt to make practical in Fast and Simplex, Roy et al. (2025)

An implementation of local windowed attention for language modeling