Breaking into Transformers

BiDirectional Attention Flow Model for Machine Comprehension

Highway Networks Similarity Matrix Context2Query and Query2Context Attention Merge Operation Similarity Matrix Context2Query Attention Query2Context The MegaMerge The Highway LSTMs Results and Conclusion References Keep Learning, Keep Sharing 🤟 Question Answering has been a major area of work in Natural Language Processing. I will be discussing as well as implementing...

Generating Handwritten Sequences Using LSTMs and Mixed Density Networks

Problem Definition Mixed Density Networks How does it look like? Long Short Term Memory Networks Model(LSTM + MDN) Loss Function As everyone comes up with a resolution at the start of the year, I would be trying to be more infrequent in my blog postings😀. As it has been over...

Incremental Learning Without Forgetting

Incremental Learning Limitations of Transfer Learning Revise and Learn A Necessary Naive One 👇🏻 Pseudo-Revision 🙄 Knowledge Distillation-Loss Algorithm One of the major area of concern in deep learning is the generalisation problem. This has been a hot topic for research for the past few years. Generally what happens is...

Reinforcement Learning : Proximal Policy Optimization(PPO)

Trust Region Policy Optimization Gradients Proximal Policy Optimization In this blog, we will be digging into another reinforcement learning algorithm by OpenAI, Trust Region Policy Optimization followed by Proximal Policy Optimization. Before discussing the algorithm directly, let us understand some of the concepts and reasonings for better explanations. ON &...

Page Rank Algorithm

What is Page Rank Algorithm? Initial Rank Rank Updation Disconnected Components and Dangling Nodes I generally love reading famous algorithms and the work done by others as they set the platform for the new ideas. I was recently reading about the page rank algorithm which is the base for the...

Multi-Head Self Attention in NLP

Self Attention Multi-Head Self Attention Positional Embeddings In this blog, we will be discussing a recent research done by the Google Team bringing state of art results in the area of natural language processing. Till now, we have widely been using LSTMs and GRUs for the sequential data as they...

Reinforcement Learning : Actor-Critic Networks

A3C Algorithm In the previous blog, we dived into the basic implementation of a deep Q-Learning Neural Network. It was a Policy-based duel- network which was used to learn the thief-police-gold game. Now, I have all of a sudden introduced two terms here, Policy-Based, Duel-Network. Policy-based methods are those which...