ExLlamaV2

The modern consumer class GPU is an inference library designed to run local large language models. It offers paged attention mechanism with Flash Attention support and has a generator structure that includes dynamic grouping, intelligent prompt caching, and K/V cache deduplication features. The project is currently archived and the development process continues through ExLlamaV3.