TensorRT LLM

High-throughput LLM inference on NVIDIA GPUs.

By Orchestra Research · Official Docs · Date unknown

NVIDIA's open-source library for optimizing LLM inference with high performance on NVIDIA GPUs.

Platforms: linux.

hermes skills install official/mlops/tensorrt-llm

• #skills
• #optional
• #mlops
• #inference serving
• #tensorrt-llm
• #nvidia
• #inference optimization