Author: Benjamin Herrera
Resource ID: CS499G
Resource Type: $\color{a7e8c8}\footnotesize\textsf{GUIDE}$
Hello there!
This guide is a compiled resource of multiple different learning resources, papers, and articles that help you understand the terminology, concepts, and paradigms of Natural Language Processing (NLP). These resources can be quite long, so please take your time. Pace yourself, you don’t want to burn yourself. You will need to learn all of these concepts in order to get a sturdy foundation of the NLP scene.
For this guide, we will start off learning basic machine learning concepts. This includes neural networks, simple gradient descents, concepts of backpropagation, and basic terminology. We will then transition to basic topics of NLP. This includes one-hot encoding, word2vec, and intermediary terms for NLP. Afterwards, we get to the crux of advanced NLP research. This will include transformers, transfer learning, and advanced terminologies.
This guide isn’t meant to be the ultimate course on NLP. Rather, it is a resource to guide you on where you would need to look if you want to see the forefront of the current NLP scene. I highly recommend that after you read a chapter, you should do a little digging of your own and build more conceptualizations yourself. That way, your understanding of the subject you just learned is not bounded by the explanations I give you. Getting different correct perspectives on the same topic will give you the full picture of what you are trying to learn, helping you fully understand the subject at hand.
These concepts can get very confusing, very fast. If you have any questions, please do not hesitate to contact me via Discord at @bherrera or via email at [email protected]. Additionally, if you see an incorrect information or discrepancies, please let me know right away, so I can fix it as soon as possible.
Now let’s get to it!
To keep things organized a clean, each major sub-section is placed under a “toggle” button. You will initially see these toggle buttons with their content hidden. To unhide its content, click on the arrow that is to the left of the green text title. There you will be able to see the full contents behind the subsection. Try it here:
For embedded video content, I suggest that you play the content and click on the YouTube logo so that the video plays on its own tab or press on the “Watch on YouTube” button on the bottom left. Try to open this video here:
https://youtu.be/VZrDxD0Za9I?list=PLu4wnki9NI_8VmJ7Qz_byhKwCquXcy6u9
For embedded paper content, you can just click on the content to open the ArXiV paper. Try to open this paper below:
"I'll Finish It This Week" And Other Lies
For embedded page content, you can just click on the embed box. If the page cannot be opened simply by clicking it, hover over the page and press the “Original” button on the top right (indicated with an arrow pointing to the top right). Try to open this blog below:
https://jalammar.github.io/illustrated-stable-diffusion/
Don’t know absolutely anything about machine learning? Clueless about what a neural network is? Don’t fret! This chapter will give you a quick and simple rundown about the simple stuff regarding machine learning.
How do we represent words into readable information that a neural network model can understand? This chapter explores the two main ways we can solve this problem.
Don’t know anything about linear algebra or matrixes? Don’t worry, this chapter covers some useful topics regarding the subject.
Get it? Matrix, Red Pill, Blue pill? Ok, I’ll stop now
This chapter quickly goes over the basic concepts of an information retrieval system. This should set up some intuition for later chapters as they are analogous to aforementioned concepts.
Goes over the concepts of what an autoencoder is. Ideas from this chapter will transfer over into the next chapter where there are clear parallels between the content in this chapter and the next.
The content from this chapter reviews the current trend in NLP, the transformer. It’s a pretty hefty topic, so strap in with the content you’ve learned in the past 4 chapters, and get ready to dive right into it.
The Bidirectional Encoder Representations for Transformers, a.k.a. BERT, is a transformer encoder language model used for a variety of different NLP tasks. This is the first application of the transformer technique that we’ll look into, and we’ll see how this model was formulated.
The Generative Pre-trained Transformer, a.k.a. GPT, is another application of the transformer technique. Unlike its BERT counterpart that uses encoder transformers, GPT uses decoder transformers. We’ll dive deep into its intricacies and see how BERT and GPT are different from each other.