• Extrapolating to longer sequence lengths using ALiBi

    Positional Encoding-based Attention Drawbacks of Positional Encoding Limited Context Fixed Patterns ALiBi: Attention with Linear Biases Mathematical Expression Advantages of ALiBi Conclusion Reference One of the core components of transformer models is the attention mechanism, which allows the model to focus on specific parts of the input sequence while processing...

  • The Power of Prompting - A Journey from Simple Prompts to Tree of Thought Strategies

    Simple Prompting: Chain of Thought Prompting: Self-Consistency with Chain of Thought: Tree of Thought Prompting: Thought Decomposition: Thought Generator: State Evaluator: Search Algorithms: Conclusion Reference In the ever-evolving landscape of natural language processing, language models (LMs) have emerged as incredibly versatile tools. These models, such as GPT-4, have demonstrated their...

  • Working with Generative Language Models

    So, what are LLMs? What makes ChatGPT different from other LLMs like GPT, GPT2, GPT3, Bloom etc? How to use RLHF with PPO for improving an LLM? Proximal Policy Optimization Challenges Instruction Based Fine Tuning Dolly by Databricks Conclusion References This extensive article will discuss the current trending technology in...

  • Decide better, Subconsciously

    Introduction The Three Pillars of Mind Ability of Subconsciousness The Hypothetical Analogy Deciding Better Introduction We always have a desire for the things to happen in the manner we want them to be – be it rational or irrational, ethical or non ethical. Ever wondered about the hidden force or...

  • Courses You Should Take

    CS230 MIT Deep Learning Course 2019 DeepMind’s Advanced Deep Learning and Reinforcement Learning Nvidia’s Webinars on Deep Learning Frameworks Deep Learning at Oxford

  • A Worthy Tycoon

    Business by its definition can be said as a group of activities which ensures the movement of goods, money or services across the market by the term called trade. We generally judge a successful business by the amount of money it circulates or the impact it makes in the market....

  • Wading through Graph Neural Networks

    Topics Graph and its motivation Node features: Edge features: Graph Convolutional Networks Graph Attention Networks Gated Graph Neural Networks Graph AutoEncoders Graph SAGE Some history behind GCNs Further approximation and Renormalization Trick Conclusion References In this blog, I am going to discuss Graph Neural Network and its variants. Let us...

  • Being into Quarantine

    Stay Distant, Stay Clean, Stay Safe When the entire world was shaken by the terror of a pandemic, I got an obvious notification from the CEO of my company stating the mandatory policy of working from home. Now for a guy like me, who just does not know how to...