Learn Generative AI - Part 1

I’ve just started learning about AI in public, as usual, I’ll be sharing the things I learn as I progress through the course materials I’m using. The first thing I wanted to get out of the way is understanding the jargons in this space so I can make sense of the things I read. It is a huge space and there’s a lot to learn so when you’re starting off like me, it can be overwhelming. If you know nothing about the AI engineering landscape and you’re looking for a beginner friendly first step, this post is for you. Let’s dive into it.

The images in this post are from the free Google Generative AI course that I'm taking.

What is AI

Because definitions vary with context and subject, I prefer to draw my own definition of things based on my understanding of the underlying systems that make up said thing. That said, this is how I understand it to be:

AI is a branch of computer science that deals with the theory and development of computer systems that are able to operate autonomously, adapt to new inputs, and mimic human behavioural patterns and thought processes. AI systems are engineered to simulate different aspects of human cognition like learning, problem-solving, reasoning, perception, and language understanding, among others. As a result, AI systems are designed to carry out tasks and make decisions that typically require human-level intelligence. Hence the name, artificial intelligence. So my high-level definition without the jargons is AI is a subset of computer science that focuses on how computer systems can learn and perform tasks like humans.

A fundamental part of this process is that computer systems need to “learn”. Which begs the question, how do computer programs learn? This is where Machine Learning comes in.

---

Machine Learning

Machine Learning is a branch of AI that focuses on the development of algorithms and statistical models, enabling computer systems to perform specific tasks by learning from data, without explicit programming.

That definition is great but not very helpful to a beginner. So to simplify it, Machine Learning is a subset of the AI umbrella that is responsible for how computer systems are ‘trained’. It encompasses different techniques, algorithms, and methods for teaching computers to learn from data to make predictions or decisions. Supervised and Unsupervised learning are two of the primary ways in which this learning process occurs:

Supervised learning

In supervised learning, the machine learning model is trained using a labeled (or structured) dataset. This means that the input data is paired with the corresponding correct output or label. The model learns to make predictions or classifications based on the input data and is guided by the labeled examples provided during training. Common tasks in supervised learning include classification (assigning input data to predefined categories) and regression (predicting numerical values).

Here’s a human-freindly example of supervised learning; Imagine you have a bunch of colorful tennis balls, and you want to sort them into groups.

Supervised Learning is like having a teacher who tells you the name or color of each tennis ball as you sort them. So, if you have a red ball, your teacher says, "This is a red ball," and you put it in the red group. If you have a blue ball, your teacher says, "This is a blue ball," and you put it in the blue group. Your teacher helps you learn how to sort the balls by giving you hints and examples.

Unsupervised Learning

Unsupervised learning, on the other hand, deals with unlabeled data. In this case, the model is not provided with explicit labels or categories. Instead, it aims to discover patterns, structures, or relationships within the data on its own. Common unsupervised learning tasks include clustering (grouping similar data points together) and dimensionality reduction (reducing the number of features while preserving important information).

Consequently, a more human explanation is like sorting the tennis balls without a teacher. You don't know the names or colors of the balls at first. So, you might start by looking at the balls and noticing that some of them look similar. You put the ones that look similar together in a group, but you don't know what to call these groups yet. You're figuring it out on your own, just by looking at the similarities among the balls.

Semi-Supervised Learning

Semi-supervised learning is an approach that falls between supervised and unsupervised learning, utilizing both labeled and unlabeled data to train models and make predictions or classifications.

In semi-supervised learning, the training dataset includes a mix of labeled and unlabeled data. This means that the proportion of the dataset could be 10-20% labeled data and 90-80% unlabeled data. While a small portion of the data has known labels, a significant portion lacks labels. The model leverages the labeled data to learn and make predictions or classifications while also attempting to extract useful information from the unlabeled data to improve its performance. Deep learning models leverage semi-supervised learning along with supervised and unsupervised learning to process complex patterns.

---

Deep Learning

Deep learning is a subset of Machine Learning that focuses on using artificial neural networks to process more complex patterns than regular machine learning models do. It is inspired by the structure and function of the human brain as it employs interconnected nodes (neurons) that are organized into layers, including an input layer, one or more hidden layers, and an output layer.

Deep Learning can often be classified into two Model Types:

• Generative and

• Discriminative

In very simple terms:

Discriminative models are primarily concerned with differentiating between different types of data points. They take input data and assign it to certain categories or labels. They are trained with labeled (structured) data, as such, they do not care about how the data was generated; they simply want to make accurate predictions or classifications based on what they see.

Generative models take a different approach. They are trained with unlabeled data such that they learn the patterns and distributions within data, enabling them to generate new data that is similar to the training data. Instead of just recognizing features (like in discrimintive models), they also understand and replicate the data creation process.

Here’s a more human-centric explanation

Imagine two artists: one who creates paintings (the “generator”) and another who is an expert at spotting forgeries (the “discriminator”). The generator creates a painting and tries to pass it off as a work of a famous artist. The discriminator tries to determine whether the painting is fake. They keep going back and forth, with the generator trying to make better 'forgeries' and the discriminator becoming better at spotting them. Over time, the generator becomes so skilled it can produce works indistinguishable from the original artist's.

Through this competition, the models train each other to improve in creating new data and evaluating it, making it possible for generative AI systems to work at a near perfect capacity.

---

This is our first mention of generative AI, which is the technology we have been drilling towards this whole time. Deep learning is the technology that allows generative AI’s to understand patterns and features in massive amounts of data, which is needed when you want to generate new content. Lets look at it in more detail.

Generative AI

Generative AI is a subset of deep learning systems that can create content or generate new data that is similar to, or indistinguishable from, human-generated content. These systems can produce various types of output, including but not limited to images, music, voice, and text. Being a subset of deep learning, it leverage deep learning capabilities such that it uses artificial neural networks to process labeled and unlabeled data using supervised, unsupervised, and semi-supervised learning methods.

You can tell generative AI apart from a discriminative / predictive AI by their outputs. Gen AI can take unstructured data as input and produce an output in the form of new content or new data while discrimative models can take labeled data as input to produce a classified label.

Here's some output based differences between Gen AI and Discriminative AI. It is not Generative AI when the output is:

• A number

• Class

• Probability

• Discreet

It is Generative AI when the output is:

• Natural language

• Audio

• Video

• Image

There's so much more to tell but I'm going to stop here for now. As I piece it together and connect the dots, I'll share more findings in this series, so we can learn generative AI together. Stay tuned!