(Pronounced: "AI on EKS")

💡 Optimized Solutions for AI and ML on EKS

⚠️ This repository is under active development as we support the new infrastructure format. Please raise any issues you may encounter

Welcome to AI on EKS, your gateway to scaling AI and ML workloads on Amazon EKS. Unlock the potential of AI with a rich collection of Terraform Blueprints featuring best practices for deploying robust solutions with advanced logging and observability.

Explore practical patterns for running AI/ML workloads on EKS, leveraging the power of the Ray ecosystem for distributed computing. Utilize advanced serving solutions like NVIDIA Triton Server, vLLM for efficient and scalable model inference, and TensorRT-LLM for optimizing deep learning models.

Take advantage of high-performance NVIDIA GPUs for intensive computational tasks and leverage AWS’s specialized hardware, including AWS Trainium for efficient model training and AWS Inferentia for cost-effective model inference at scale.

Note: AIoEKS is in active development. For upcoming features and enhancements, check out the issues section.

In this repository, you'll find a variety of deployment blueprints for creating AI/ML platforms with Amazon EKS clusters. These examples are just a small selection of the available blueprints - visit the AIoEKS website for the complete list of options.

🚀 JARK-Stack on EKS 👈 This blueprint deploys JARK stack for AI workloads with NVIDIA GPUs.

🚀 Generative AI on EKS 👈 Collection of Generative AI Training and Inference LLM deployment patterns

For instructions on how to deploy AI on EKS patterns and run sample tests, visit the AIoEKS website.

Kubernetes is a widely adopted system for orchestrating containerized software at scale. As more users migrate their AI and machine learning workloads to Kubernetes, they often face the complexity of managing the Kubernetes ecosystem and selecting the right tools and configurations for their specific needs.

At AWS, we understand the challenges users encounter when deploying and scaling AI/ML workloads on Kubernetes. To simplify the process and enable users to quickly conduct proof-of-concepts and build production-ready clusters, we have developed AI on EKS (AIoEKS). AIoEKS offers opinionated open-source blueprints that provide end-to-end logging and observability, making it easier for users to deploy and manage Ray, vLLM, Kubeflow, MLFlow, Jupyter and other AI/ML workloads. With AIoEKS, users can confidently leverage the power of Kubernetes for their AI and machine learning needs without getting overwhelmed by its complexity.

AIoEKS is maintained by AWS Solution Architects and is not an AWS service. Support is provided on a best effort basis by the AI on EKS community. If you have feedback, feature ideas, or wish to report bugs, please use the Issues section of this GitHub.

See CONTRIBUTING for more information.

This library is licensed under the Apache 2.0 License.

We welcome all individuals who are enthusiastic about AI on Kubernetes to become a part of this open source community. Your contributions and participation are invaluable to the success of this project.

Built with ❤️ at AWS.