ManikantaGuptha

  Historical Context: Seq2Seq Paper and NMT by Joint Learning to Align & Translate Paper : * Seq2Seq model introduced in "Sequence to Sequence Learning with Neural Networks" paper by Sutskever et al., revolutionized NLP with end-to-end learning. For example, translating "Bonjour" to "Hello" without handcrafted features. * NMT by "Joint Learning to Align and Translate" (Luong et al.) improved Seq2Seq with attention mechanisms. For instance, aligning "Bonjour" and "Hello" more accurately based on context. Introduction to Transformers (Paper: Attention is all you need) : * Transformers introduced in "Attention is All You Need" paper. * Replaced RNN-based models with attention mechanisms, making them highly parallelizable and efficient. Why transformers : * Transformers capture long-range dependencies effectively. * They use self-attention to process tokens in parallel and capture global context efficiently.   ...

Language Modeling ( LLM ) *   What is Language Modeling :   Language modeling powers modern NLP by predicting words based on context, using statistical analysis of vast text data. It's essential for tasks like word prediction and speech development, driving innovation in understanding human language. Types of language models :  N-gram Models  : These models predict the next word based on the preceding n-1 words, where n is the order of the model. For example, a trigram model (n=3) predicts the next word using the two preceding words. N-gram Language Model Example : Consider a simple corpus of text: I love hiking in the mountains. The mountains are beautiful. Trigram Model: P("in" | "hiking", "love") = Count("hiking love in") / Count("hiking love") P("are" | "mountains", "the") = Count("mountains the are") / Count("mountains the") Neural Language Models: Neural network-based language...