As Large Language Models (LLMs) grow in size and complexity, finding ways to reduce their computational and energy costs has become a critical challenge. One popular solution is quantization, where the precision of parameters is reduced from the standard 16-bit floating-point (FP16) or 32-bit floating-point (FP32) to lower-bit formats like 8-bit or 4-bit. While this approach significantly cuts down on memory usage and speeds up computation, it often comes at the expense of accuracy. Reducing the precision too much can […]

Deploying Transformers models at the edge or client-side requires careful consideration of performance and compatibility. Python, though powerful, is not always ideal for such deployments, especially in environments dominated by C++. This blog will guide you through optimizing and deploying Hugging Face Transformers models using Optimum-Intel and OpenVINO™ GenAI, ensuring efficient AI inference with minimal dependencies. Table of Contents Why Use OpenVINO™ for Edge Deployment Step 1: Setting Up the Environment Step 2: Exporting Models to OpenVINO IR    

In the fast-paced world of research, staying up-to-date with the latest advancements is crucial. To help developers and researchers keep a pulse on the cutting-edge of AI, Hugging Face introduced the Daily Papers page. Since its launch, Daily Papers has featured high-quality research selected by AK and researchers from the community. Over the past year, more than 3,700 papers have    

Llama 3.2 is out! Today, we welcome the next iteration of the Llama collection to Hugging Face. This time, we’re excited to collaborate with Meta on the release of multimodal and small models. Ten open-weight models (5 multimodal models and 5 text-only ones) are available on the Hub. Llama 3.2 Vision comes in two sizes: 11B for efficient deployment and development on consumer-size GPU, and 90B for large-scale applications. Both versions come in base and instruction-tuned variants. In addition to […]

Convert vertex-colored meshes to UV-mapped, textured meshes. Introduction Vertex colors are a straightforward way to add color information directly to a mesh’s vertices. This is often the way generative 3D models like InstantMesh    

The 🇨🇿 BenCzechMark is the first and most comprehensive evaluation suite for assessing the abilities of Large Language Models (LLMs) in the Czech language. It aims to test how well LLMs can: Reason and perform complex tasks in Czech. Generate and verify grammatically and semantically correct Czech. Extract information and store knowledge by answering questions about Czech culture and Czech-related facts. Do what language models were originally trained for—estimate the probability of Czech texts. To achieve this, we’ve sourced 50 tasks […]

In the early 15th century, Zheng He (also known as Chong Ho), a Chinese mariner and explorer during the early Ming Dynasty, led seven major naval expeditions, known as the “Voyages to the Western Oceans”. His journey traced a path that went through Southeast Asia, the Middle East and then reached out to Africa. It was a bold move by China to    

Finding the best LLM models for finance use cases The growing complexity of financial language models (LLMs) necessitates evaluations that go beyond general NLP benchmarks. While traditional leaderboards focus on broader NLP tasks like translation or summarization, they often fall short in addressing the specific needs of the finance industry. Financial tasks, such as predicting stock movements, assessing credit risks, and extracting information from financial reports, present unique challenges that require models with specialized skills. This is why we decided […]

The Xet team at Hugging Face is working on improving the efficiency of the Hub’s storage architecture to make it easier and quicker for users to store and update data and models. As Hugging Face hosts nearly 11PB of    

The Hugging Face platform has many datasets and pre-trained models that make using and training state-of-the-art machine learning models increasingly accessible. However, it can be hard to scale AI tasks because AI datasets are often large (100s GBs to TBs) and using Hugging Face transformers for model inference can sometimes be computationally expensive. Dask, a Python library for distributed computing, can handle out-of-core computing (processing data that doesn’t fit in memory) by breaking datasets into manageable chunks. This makes it […]