-
Self Attentions In Transformers.
Self Attention In Transformers Table Of Contents: Motivation To Study Self Attention. (1) Motivation To Study Self Attention. In 2024 we all know that there is a technology called ‘GenAI’ has penetrated into the market. With this technology we can create different new images, videos, texts from scratch automatically. The center of ‘GenAI’ technology is the ‘Transformers’. And the center of the Transformer is the ‘Self Attention’. Hence we need to understand ‘Self Attention’ better to understand others. (2) Problem With Word Embedding. The problem with the word embedding is that it doesn’t capture the contextual meaning of the word.
