-
Self Attentions In Transformers.
Self Attention In Transformers Table Of Contents: Motivation To Study Self Attention. (1) Motivation To Study Self Attention. In 2024 we all know that there is a technology called ‘GenAI’ has penetrated into the market. With this technology we can create different new images, videos, texts from scratch automatically. The center of ‘GenAI’ technology is the ‘Transformers’. And the center of the Transformer is the ‘Self Attention’. Hence we need to understand ‘Self Attention’ better to understand others. (2) Problem With Word Embedding. The problem with the word embedding is that it doesn’t capture the contextual meaning of the word.
-
What Is Self Attention ?
What Is Self Attention ? Table Of Contents: What Is The Most Important Thing In NLP Applications? Problem With Word2Vec Model. The Problem Of Average Meaning. What Is Self Attention? (1) What Is The Most Important Thing In NLP Applications? Before understanding the self-attention mechanism, we must understand the most important thing in any NLP application. The answer is how you convert any words into numbers. Our computers don’t understand words they only understand numbers. Hence the researchers first worked in this direction to convert any words into vectors. We got some basic techniques like, One Hot Encoding. Bag Of
-
Introduction To Transformers!
Introduction To Transformers ! Table Of Contents: What Is Transformers? History Of Transformers. Impact Of Transformers In NLP. Democratizing AI. Multimodel Capability Of Transformers. Acceleration Of GenAI. Unification Of Deep Learning. Why Transformers Are Created? Neural Machine Translation Jointly Learning To Align & Translate. Attention Is All You Need. The Time Line Of Transformers. The Advantages Of Transformers. Real World Applications Of Transformers. Disadvantages Of Transformers. The Future Of Transformers. (1) What Is Transformers? Transformers is basically a Neural Network Architecture. In deep learning, we have already studied the ANN, CNN & RNN. ANN works for the cross-sectional data, CNN
-
Luong Attention !
Luong’s Attention ! Table Of Contents: What Is Luong’s Attention? Key Features Of Luong’s Attention Model? Advantages Of Luong’s Attention Model? Architecture Of Luong’s Attention Model. Why do We Take the Current Hidden State Output Of The Decoder In Luong’s Attention Model? Architecture Luong’s Attention Model. Difference In Luong’s Attention & Bahdanau’s Attention (1) What Is Luong’s Attention? Luong’s attention is another type of attention mechanism, introduced in the paper “Effective Approaches to Attention-based Neural Machine Translation” by Minh-Thang Luong, Hieu Pham, and Christopher D. Manning in 2015. Luong’s attention mechanism is also designed for encoder-decoder models, similar to Bahdanau’s
-
Bahdanau Attention Vs Luong Attention !
Bahdanau Attention ! Table Of Contents: What Is Attention Mechanism? What Is Bahdanau’s Attention? Architecture Of Bahdanau’s Attention? (1) What Is Attention Mechanism? An attention mechanism is a neural network component used in various deep learning models, particularly in the field of natural language processing (NLP) and sequence-to-sequence tasks. It was introduced in the paper “Attention is All You Need” by Vaswani et al. in 2017. The attention mechanism allows a model to focus on the most relevant parts of the input when generating an output, rather than treating the entire input sequence equally. This is particularly useful when the
-
What Is Attention Mechanism?
What Is Attention Mechanism? Table Of Contents: Problem With Encoder & Decoder Architecture. Solution For Encoder & Decoder Architecture. Math’s Behind Attention Mechanism. Improvements Due To Attention Mechanism. (1) Problem With Encoder & Decoder Architecture. Problem With Encoder: The main idea behind the Encoder & Decoder architecture is that, the encoder summarize the entire text into one vector format and from that vector we need to convert into different language. Let us consider the below example, Your task is to read the entire sentence first, keep all the words in mind and translate it into Hindi without seeing the sentence
-
Encoder Decoder Architecture
Encoder Decoder Architecture Table Of Contents: Milestones In Deep Learning. What Is Sequence To Sequence Model? (1) Milestones In Deep Learning ? Milestone-1: In the initial days we have only tabular information that will consist of rows and columns, like Excel, and CSV data sets. Example student dataset, employee dataset, product details etc. This data is called the Cross-Sectional dataset. To process these data we are using “Artificial Neural Networks.” Milestone-2: Next comes the Image dataset where the data is represented in 2d vector space. To handle this information we are using “Convolutional Neural Networks.” These networks are specially designed
-
Word2Vec Word Vectorization
Word2Vec Word Vectorization Table Of Contents: What Is Word Embedding? Types Of Word Embedding. What Is Word2Vec? Why Are Word Embedding Needed? How Word2Vec Model Works? Pretrained Word2Vec Model. What Are 300 Dimension Numbers Signifies? Intuition Behind Word2Vec. Assumption Behind Word2Vec Mode. Architecture Of Word2Vec Model. Continuous Bag Of Words(CBOW). Skip-Gram Word2Vec. When To Use CBOW & Skip-Gram? How To Increase The Performance Of The Word2Vec Model? Train Word2Vec Model With Game Of Thrones Dataset. (1) What Is Word Embedding? Word embedding is a fundamental technique in natural language processing (NLP) that represents words as dense, low-dimensional vectors of real
-
TF-IDF Word Vectorization
TF-IDF Word Vectorization Table Of Contents: What Is TF-IDF Word Vectorization? What Is Term Frequency? What Is Inverse Document Frequency? How To Calculate TF-IDF? Steps For TF-IDF Calculation. Example Of TF-IDF Calculation. Pros & Cons Of TF-IDF Technique. Python Example. (1) What Is TF-IDF Word Vectorization? TF-IDF (Term Frequency-Inverse Document Frequency) is a widely used word vectorization technique in natural language processing (NLP). It represents text data as numerical vectors, which can be used as input for various machine learning algorithms. (2) Term Frequency The term frequency is a simple count of how many times a word appears in a
-
N-Grams/Bi-Grams/Tri-Grams
N-Grams/Bi-Grams/Tri-Grams Table Of Contents: What Is N-Gram? Example Of N-Gram Unigram. Bigram. Trigram. Python Example. Pros & Cons Of N-Gram Technique. (1) What Is N-Gram? The n-gram technique is a fundamental concept in natural language processing (NLP) that involves analyzing the relationship between sequences of n consecutive words or characters within a given text. It is widely used in various NLP tasks, such as language modeling, text classification, and information retrieval. (2) Example Of N-Gram “I reside in Bengaluru”. SL.No. Type of n-gram Generated n-grams 1 Unigram [“I”,”reside”,”in”,”Bengaluru”] 2 Bigram [“I reside”,”reside in”,”in Bengaluru”] 3 Trigram [“I reside
