dhruv's space (Posts about summary-notes)

Summary Notes: Bayes' Theorem

Dhruv Thakur — Thu, 14 Feb 2019 16:03:21 GMT

I just started Udacity's Intro to Self Driving Cars Nanodegree and the very first thing the instructors teach in it is Bayes' theorem. I read about it in school, but now's the time for me to really get into it, hence this summary note.

Bayes' theorem gives a mathematical way to correct predictions about probability of an event, and move from prior belief to something more and more probable.

The intuition behind application of Bayes' theorem in probabilistic inference can be put as follows:

Given an initial prediction, if we gather additional related data, data that the initial prediction depends upon, we can improve that prediction.

Word Embeddings and RNNs

Dhruv Thakur — Sat, 20 Oct 2018 06:40:16 GMT

One of the simplest ways to convert words from a natural language into mathematical tensors is to simply represent them as one-hot vectors where the length of these vectors is equal to the size of the vocabulary from where these words are fetched.

For example, if we have a vocabulary of size 8 containing the words:

"a", "apple", "has", "matrix", "pineapple", "python", "the", "you"

the word "matrix" can be represented as: [0,0,0,1,0,0,0,0]

Obviously, this approach will become a pain when we have a huge vocabulary of words (say millions), and have to train models with these representations as inputs. But apart from this issue, another problem with this approach is that there is no built-in mechanism to convey semantic similarity of words. eg. in the above example, apple and pineapple can be considered to be similar (as both are fruits), but their vector representations don't convey that.

Word Embeddings let us represent words or phrases as vectors of real numbers, where these vectors actually retain the semantic relationships between the original words. Instead of representing words as one-hot vectors, word embeddings map words to a continuous vector space with a much lower dimension.

Summary Notes: GRU and LSTMs

Dhruv Thakur — Sun, 14 Oct 2018 09:51:16 GMT

This post is sort-of a continuation to my last post, which was a Summary Note on the workings of basic Recurrent Neural Networks. As I mentioned in that post, I've been learning about the workings of RNNs for the past few days, and how they deal with sequential data, like text. An RNN can be built using either a basic RNN unit (described in the last post), a Gated Recurrent unit, or an LSTM unit. This post will describe how GRUs/LSTMs learn long term dependencies in the data, which is something basic RNN units are not so good at.

Summary Notes: Basic Recurrent Neural Networks

Dhruv Thakur — Tue, 09 Oct 2018 10:34:10 GMT

I've been learning about Recurrent Neural Nets this week, and this post is a "Summary Note" for the same.

A "Summary Note" is just a blog post version of the notes I make for something, primarily for my own reference (if I need to come back to the material in the future). These summary notes won't go into the very foundations of whatever they're about, but rather serve as a quick and practical reference for that particular topic.

RNNs are inherently different from a traditional feed-forward neural nets, in that, they have the capability to make predictions based on past/future data. This ability to sort of "memorise" past/future data is crucial to handling cases which have a temporal aspect to them. Let's take the following conversation between me and Google Assistant on my phone:

Summary Notes: Forward and Back Propagation

Dhruv Thakur — Mon, 10 Sep 2018 10:55:00 GMT

I recently completed the first course offered by deeplearning.ai, and found it incredibly educational. Going forwards, I want to keep a summary of the stuff I learn (for my future reference) in the form of notes like this. This one is for forward and back-prop intuitions.