Preparing categorical data correctly is a fundamental step in machine learning, particularly when using linear models. One Hot Encoding stands out as a key technique, enabling the transformation of categorical variables into a machine-understandable format. This post tells you why you cannot use a categorical variable directly and demonstrates the use One Hot Encoding in our search for identifying the most predictive categorical features for linear regression. Let’s get started. One Hot Encoding: Understanding the “Hot” in DataPhoto by sutirta […]

Consistent with the principle of Occam’s razor, starting simple often leads to the most profound insights, especially when piecing together a predictive model. In this post, using the Ames Housing Dataset, we will first pinpoint the key features that shine on their own. Then, step by step, we’ll layer these insights, observing how their combined effect enhances our ability to forecast accurately. As we delve deeper, we will harness the power of the Sequential Feature Selector (SFS) to sift through […]

To understand housing prices better, simplicity and clarity in our models are key. Our aim with this post is to demonstrate how straightforward yet powerful techniques in feature selection and engineering can lead to creating an effective, simple linear regression model. Working with the Ames dataset, we use a Sequential Feature Selector (SFS) to identify the most impactful numeric features and then enhance our model’s accuracy through thoughtful feature engineering. Let’s get started. The Strategic Use of Sequential Feature Selector […]

Image by Author In this tutorial, we will explore Retrieval-Augmented Generation (RAG) and the LlamaIndex AI framework. We will learn how to use LlamaIndex to build a RAG-based application for Q&A over the private documents and enhance the application by incorporating a memory buffer. This will enable the LLM to generate the response using the context from both the document and previous interactions. What is RAG in LLMs? Retrieval-Augmented Generation (RAG) is an advanced methodology designed to enhance the performance […]

Many beginners will initially rely on the train-test method to evaluate their models. This method is straightforward and seems to give a clear indication of how well a model performs on unseen data. However, this approach can often lead to an incomplete understanding of a model’s capabilities. In this blog, we’ll discuss why it’s important to go beyond the basic train-test split and how cross-validation can offer a more thorough evaluation of model performance. Join us as we guide you […]

Statistics and Machine Learning both aim to extract insights from data, though their approaches differ significantly. Traditional statistics primarily concerns itself with inference, using the entire dataset to test hypotheses and estimate probabilities about a larger population. In contrast, machine learning emphasizes prediction and decision-making, typically employing a train-test split methodology where models learn from a portion of the data (the training set) and validate their predictions on unseen data (the testing set). In this post, we will demonstrate how […]

The large Language Model, or LLM, has revolutionized how people work. By helping users generate the answer from a text prompt, LLM can do many things, such as answering questions, summarizing, planning events, and more. However, there are times when the output from LLM is not up to our standard. For example, the text generated could be thoroughly wrong and need further direction. This is where the LLM Output Parser could help. By standardizing the output result with LangChain Output […]

Image by Editor | Midjourney In the past, businesses grouped customers based on simple things like age or gender. Now, machine learning has changed this process. Machine learning algorithms can analyze large amounts of data. In this article, we will explore how machine learning improves customer segmentation. Introduction to Customer Segmentation Customer segmentation divides customers into different groups. These groups are based on similar traits or behaviors. The main goal is to understand each group better. This helps businesses create […]

Image created by Editor using Midjourney Today’s digital landscape has never been so diverse. Every individual and company selects their preferred tools and operating systems, creating a diverse technological system. However, this diversity often leads to compatibility issues, making it hard to ensure application performance across different environments. This is where Docker plays a key role as an indispensable tool for application development and deployment. Docker enables us to package any application within a container, building all its dependencies and […]

Data science uses mathematics to analyze data, distill information, and tell a story. The result of data science may be just to rigorously confirm a hypothesis, or to discover some useful property from the data. There are many tools you can use in data science, from basic statistics to sophisticated machine learning models. Even the most common tool can work wonderfully in a data science project. In this 7-part crash course, you will learn from examples how to perform a […]