The Wonderful World of RAG Fusion. Illustration by author.Having explored search technologies for almost a decade, I can honestly say nothing has been as disruptive as the recent rise of Retrieval Augmented Generation (RAG). This system is revolutionising search and information retrieval using vector search with generative AI to produce direct answers based on trusted data. In my search projects,

This article covers the following “hyperparameters” sorted by their relevant stage. In the ingestion stage of a RAG pipeline, you can achieve performance improvements by: Data cleaningChunkingEmbedding modelsMetadataMulti-indexingIndexing algorithmsAnd in the inferencing stage (retrieval and generation), you can tune: Query transformationsRetrieval parametersAdvanced retrieval strategiesRe-ranking

Let’s see a brief description of the columns of our dataset: age (numeric)job : type of job (categorical: “admin.” ,”unknown”,”unemployed”, ”management”, ”housemaid”, ”entrepreneur”, ”student”, “blue-collar”, ”self-employed”, ”retired”, ”technician”, ”services”)marital : marital status (categorical: “married”,”divorced”,”single”; note: “divorced” means divorced or widowed)education (categorical: “

Tuning deep learning pipelines is like finding the right gear combination (Image by Tim Mossholder on Unsplash)Why should you read this post?The training/inference processes of deep learning models are involved lots of steps. The faster each experiment iteration is, the more we can optimize the whole model prediction performance given limited time and resources. I collected and organized several P

It’s been quite a year for Graph ML — thousands of papers, numerous conferences and workshops… How do we catch up with so many cool things happening around? Well, we are puzzled as well and decided to present a structured look at Graph ML highlighting 🔥 trends and major advancements. The image was generated by ruDALL-E with a prompt “graphs floating in space”.Whether you are working on a narrower

Image: ShutterstockThis post was co-authored with Petar Veličković. See also my last year’s prediction, Michael Galkin’s excellent post on the current state of affairs in Graph ML, a deeper dive into subgraph GNNs, techniques inspired by PDEs and differential geometry and algebraic topology, and how the concepts of symmetry and invariance form the picture of modern deep learning. Summing up impres

Image made by the authorAs a data scientist, I daily use Python to build applications that rely on credentials and sensitive settings. Here are some examples of those, off the top of my head: API keys to access third-party services

One of FLAML’s algorithms CFO tuning the # of leaves and the # of trees for XGBoost. The two heatmaps show the loss and cost distribution of all configurations. The black dots are the points evaluated in CFO. Black dots connected by lines are points that yield better loss performance when evaluated (image by authors).Authors: Qingyun Wu, Chi Wang, Antoni Baum, Richard Liaw and Michael Galarnyk FLA

Photo by Circe Denyer on PublicDomainPictures.netUsually, when I see BatchNorm and Dropout layers in a neural network, I don’t pay them much attention. I tend to think of them as simple means to speed up training and improve generalization with no side effects when the network is in inference mode. In this post, I will show why this notion is not always correct, and may cause the neural network to

The principle of SOLID coding is an acronym originated by Robert C. Martin, and it stands for five different conventions of coding. If you follow, these principles you can improve the reliability of your code by working on its structure and its logical consistency.

DeepMind recently released their MuZero algorithm, headlined by superhuman ability in 57 different Atari games. Reinforcement Learning agents that can play Atari games are interesting because, in addition to a visually complex state space, agents playing Atari games don’t have a perfect simulator they can use for…