We’re thrilled to share that Cohere is now a supported Inference Provider on HF Hub! This also marks the first model creator to share and serve their models directly on the Hub. Cohere is committed to building and serving models purpose-built for enterprise use-cases. Their comprehensive suite of secure AI solutions, from cutting-edge Generative AI to powerful Embeddings and Ranking models, are designed to tackle real-world business challenges. Additionally, Cohere Labs, Cohere’s in house research lab, supports fundamental research and […]

“Oh, Gradio? That’s a Python library for building UIs, right?” We hear this a lot, and while Gradio does let you create interactive UIs with minimal Python code, calling Gradio a “UI library” misses the bigger picture! Gradio is    

Handling load from multiple users in parallel is crucial for the performance of LLM applications. In the previous part of our series on LLM performance, we discussed queueing strategies for the prioritization of different users. In this second part, we will now focus on the concurrent processing of requests, and how it impacts relevant metrics such as latency    

At TNG, we created a fine-tune of an Optical Character Recognition model based on olmOCR to help us automate our internal document processing workflows. Recently, the Allen Institute for Artificial Intelligence    

New! (May 23, ’25) If you prefer Python, check out the companion post Tiny Agents in Python. Over the past few weeks, I’ve been diving into MCP (Model Context Protocol) to understand what the hype around it was all about. My TL;DR is that it’s fairly simple, but still quite powerful: MCP is a standard API    

We are excited to open-source PipelineRL, an experimental RL implementation that tackles a fundamental challenge in large-scale Reinforcement Learning with LLMs: the trade-off between inference throughput and on-policy data collection. PipelineRL’s key innovation is inflight weight updates during RL training (see Figure 1 below). This allows PipelineRL to achieve constantly high inference throughput and minimize the lag between the weights used for rollouts and the most recently updated model weights. The result: fast and stable RL training for large language […]

As large language models (LLMs) and vision-language models (VLMs) continue to grow in size and complexity, deploying them efficiently becomes increasingly challenging. Quantization offers a solution by reducing model size and inference latency. Intel’s AutoRound emerges as a cutting-edge quantization tool that balances accuracy, efficiency, and compatibility. AutoRound is a weight-only post-training quantization (PTQ) method developed by Intel. It uses signed gradient descent to jointly optimize weight rounding and clipping ranges, enabling accurate low-bit quantization (e.g., INT2 – INT8) with […]

What a boring Jinja snippet tells us about the new Qwen-3 model. The new Qwen-3 model by Qwen ships with a much more sophisticated chat template than its predecessors Qwen-2.5 and QwQ. By taking a look at the differences in the Jinja template, we can find interesting insights into the new model. Chat Templates

Updated! (September 2025) This post has been updated with the latest Gradio MCP features including Resources, Prompts, enhanced authentication, and many more. Gradio is a Python library used by more than 1 million developers    

TLDR: Going forward, we’re aiming for Transformers to be the pivot across frameworks: if a model architecture is supported by transformers, you can expect it to be supported in the rest of the ecosystem. Transformers was created in 2019, shortly following the release of the BERT Transformer model. Since then, we’ve continuously aimed to add state-of-the-art architectures, initially focused on NLP, then growing to Audio and computer vision. Today, transformers is the default library for LLMs and VLMs in the […]