Low-Rank Adaptation
LoRA (Low-Rank Adaptation) is a technique designed to efficiently fine-tune large language models (LLMs) by adapting only a few key parameters, rather than updating the entire…
Blog
Research
Learning
Word2Vec
Word2Vec is a popular word embedding technique developed by Google that captures the semantic meaning of words in a vector format. It transforms words into continuous-valued…
Transformers
The transformer architecture has transformed the landscape of deep learning, particularly in fields like natural language processing (NLP) and computer vision. Transformers…
Fine-Tuning LLMs
Fine-tuning is a process in machine learning where a pre-trained model is adapted to a new, specific task. Instead of training a model from scratch, which is resource-intensive,…
Proximal Policy Optimization
Proximal Policy Optimization (PPO) is a popular reinforcement learning algorithm, particularly known for its efficiency, robustness, and simplicity in training policies for…
Q-Learning and DQNs
Q-learning is a foundational algorithm in reinforcement learning that allwos an agent to learn an optimal action-value fiunction (Q-function), representing the maximum expected…
Attention
In recent years, self-attention has revolutionized the way we approach natural language processing, computer vision, and other fields of artificial intelligence. This powerful…
Gradient Descent
At its core, gradient descent is an iterative optimization algorithm used to find the minimum of a function. In the context of machine learning, it is often used to minimize the…
Reinforcement Learning Basics
Reinforcement Learining (RL) is a type of machine learning where an agent learning to make a decision by interacting with an environment to achieve a goal.
Gaussian Mixture Models
Since most datasets can be modelled using a gaussian distribution. we can assume that clusters come from different gaussians.
Regularization
Regularization is a technique that penalizes a model for being too complex. This helps prevent overfitting—where the model performs perfectly on the training data but struggles to…
Bias vs Variance
Bias and variance are two key sources of error in machine learning models that affect their accuracy and generalization.
Support Vector Machine (SVM)
SVM (Support Vector Machine) is a classification technique. In SVM, the goal is to find a hyperplane that separates data points from each class. A hyperplane is considered optimal…
Naive Bayes
Naive Bayes is a probabilistic classifier based on using Bayes therom. From Bayes TheoramP(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}, Now if we replace that with class labels(y) and…
Logistic Regression
In logistic regression, we create probabilities by applying the sigmoid function, which converts outputs into a probability distribution between 0 and 1. Like in linear…
Linear Regression
Linear regression is used to understand the pattern in a dataset. Given a set of data points, we try to find a line, y=ax+b, that minimizes the error between the data points and…
K-nearest neighbors (KNN)
K Nearest Neighbours (KNN) is a statisitical technique used to classify a new data point by identifying which group or class it most likely belongs to, based on its proximity to…
Conferences & Travel
My Experience at ICML 2024
ICML was my “frog in a well” moment. Surrounded by researchers whose work I’d only read about, I felt both inspired and a little humbled. Every conversation, every presentation,…