Nvidia relies on TensorRT to enhance its generative AI supremacy
October 17 2023
Nvidia is expanding its artificial intelligence (AI) capabilities by integrating its AI-specific software development kit, known as TensorRT-LLM SDK, into more applications. The company plans to add support for Windows and models such as Stable Diffusion. By speeding up the inference process, Nvidia’s software helps large language models (LLMs) run faster. The goal is to play a more significant role in the generative side of AI, with TensorRT-LLM enhancing the efficiency of Nvidia’s H100 GPUs.
Back to Breaking AI News
What does it mean?
- Nvidia: A multinational technology company known for its advancements in computer graphics, artificial intelligence, and more.
- Artificial Intelligence (AI): A branch of computer science dealing with the simulation of intelligent behavior in computers.
- AI-specific software development kit (SDK): A set of tools used for developing applications that leverage artificial intelligence capabilities.
- TensorRT-LLM SDK: A specific software development kit by Nvidia designed to optimize the operation of Large Language Models (LLMs) via techniques such as layer fusion and precision calibration.
- Windows: A group of several families of proprietary graphical operating systems, developed and sold by Microsoft.
- Stable Diffusion: A type of model or method used in AI. It is not fully explained in the text, so a more detailed explanation may be needed depending on the context.
- Inference process: In AI, this term refers to the process of using a trained machine learning model to make a prediction.
- Large language models (LLMs): A type of AI model that can understand and generate human-like text by learning from large amounts of data.
- Generative side of AI: This refers to AI models and techniques that can generate something new, like creating a new piece of writing or a new piece of music.
- H100 GPUs: Graphics Processing Units produced by Nvidia, used for rendering graphics and also accelerating artificial intelligence and scientific computations.