All Posts
Filter by keyword, tag, and category.
DeepEP: An Efficient Expert Parallelism GPU Communication Library Open-Sourced by DeepSeek
DeepEP is an efficient communication library open-sourced by DeepSeek, specifically tailored for Mixture-of-Experts (MoE) and Expert Parallelism (EP). It provides high-throughput and low-latency GPU all-to-all kernels, supporting NVLink and pure RDMA forwarding. By significantly reducing communication bottlenecks in distributed training and inference of large models, DeepEP serves as a critical open-source component in the current AI infrastructure landscape.
Thunderbolt: A Cross-Platform Open-Source AI Client Breaking LLM Vendor Lock-in
Thunderbolt is an open-source, cross-platform AI client developed by the Thunderbird team, focusing on "model freedom and data sovereignty." It supports all platforms and is compatible with cutting-edge cloud models and local on-premises deployments. Designed to eliminate vendor lock-in, the project is currently undergoing security audits to prepare for enterprise-grade production environments. It is an ideal choice for enterprises and geeks to control their AI infrastructure.
A Powerful Tool for Unifying LLM API Calls: In-Depth Analysis of the LiteLLM Project
LiteLLM is an open-source Python SDK and AI gateway proxy server that supports calling over 100 Large Language Model APIs using the standard OpenAI format. It features enterprise-grade capabilities such as cost tracking, security guardrails, load balancing, and logging. By significantly simplifying development and operations in multi-model environments, LiteLLM has become an indispensable infrastructure component for modern AI application development.