Building a content-based recommendation system with the tf-idf algorithm - Part 2

In this article, it is time to precisely outline the scope of our study and introduce key definitions that will be extensively utilized throughout this series.

What do we mean by recommendation system ?

A recommendation system is a software application or algorithm designed to provide personalized suggestions or recommendations to users. These suggestions can range from products, services, or content, such as movies, music, books, or articles, based on the user's preferences, behavior, or historical interactions with the system. Recommendation systems are commonly used in various online platforms to enhance user experience by offering tailored suggestions that align with individual preferences.

This comprehensive definition is applicable to various contexts, encompassing both online platforms and traditional brick-and-mortar stores. The concept is not a novelty that emerged solely with the advent of the internet. In physical retail stores, large outlets traditionally highlight products through displays, yet the same products are often promoted universally. The distinct capability of online platforms to present different products to individual users with each interaction has spurred the growth of specialized companies in this field.

In a traditional retail store, both John and Dave would encounter the same product, regardless of Dave's preferences or needs. The static nature of the front display in physical stores hinders adaptation to each user. In contrast, online platforms have the flexibility to rearrange products dynamically with each request, enabling personalized recommendations tailored to individual users based on their preferences.

Our objective is to unveil the methodologies employed, explore the algorithms in use, and implement them through a tangible example.

I need to implement a recommendation system! Where should I begin ?

There are primarily two distinct methodologies for implementing recommendations: content-based filtering and collaborative filtering. We will provide a brief description of each.

Recommendations with content-based filtering

Content-based filtering is a recommendation system technique that suggests items based on the features or attributes of the items themselves and the user's preferences. Instead of relying on the preferences of other users, it analyzes the content of the items the user has interacted with or liked in the past. The system recommends items with similar characteristics to those the user has shown interest in previously. This approach is particularly useful when there is sufficient information about the items and their attributes.

Content-based filtering proves to be highly effective in the context of online newspapers, where user preferences and interests can be inferred from the content and attributes of the articles.

Recommendations with collaborative filtering

Collaborative filtering is a recommendation system approach that relies on the preferences and behaviors of a group of users to make personalized suggestions. It involves analyzing user interactions and preferences to identify patterns and recommend items that similar users have liked or interacted with.

We will only explore content-based filtering in this series, but those eager to delve deeper into collaborative filtering can refer to this book.

Focus on our roadmap

Now that the underlying concepts have been identified and explained, it is time to delve deeper into the topic. As already mentioned, we will implement content-based filtering and apply it to a series of news articles available here. It is a dataset of some articles proposed on the Huffington Post, and our goal will be to propose 5 recommended articles for each item.

Our dataset contains more than 5000 articles, presented as follows. Notice that some articles are written in Spanish and we will have to take this into account.

Each article has various attributes such as headline, URL, short description, and so on. At first glance, it is not at all evident to manually guess what to recommend just from this file. The next article will explore efficient algorithms to address this.