Gen-AI, or generative AI, is the new hot topic in the market. From ChatGPT writing your assignments to DALL-E creating art, its complexity is growing as rapidly as its use cases. So let's break down the technology.
We start by understanding AI.
Artificial intelligence can be understood as a discipline, or a field of study, just like physics or sociology. Like other subjects, it has a broad spectrum of topics under its belt. ML, or machine learning, is one of the subfields we'll zoom in on. Although ML is more about statistical math than computer science, both are equally essential for its never-ending use cases.
Think of a machine learning model as a magic box: you put in a carrot (input data), rotate the box once (run the function), and pull out a rabbit (output data). The carrot is like any other carrot in the farm, and the rabbit isn't magical either — it's the structure and making of the magic box that needs a closer inspection.
There are numerous different ML models (or magic boxes), but what's common across all of them is data. An ML model needs data like a car needs fuel — data is paramount. We categorize ML models based on the kind of data and how it's fed into the system: Supervised, Unsupervised, Semi-supervised, and Reinforcement learning. To put it simply:
Supervised learning models use labeled data.
A teacher shows this to a class (assuming the students know how to count):
f(0) = 2
f(3) = 5
f(12) = 14
f(48) = 50
f(30) = 32
f(2) = 4
The students know there's an input inside f() and an output after =, and that for every input, the teacher is adding 2 — so they can perform x + 2 based on what they recognized.
Unsupervised learning models use data that isn't labeled as input or output.
A teacher shows this to her students (again, assuming they know how to count):
1 < 2
3 > 2
9 < 11
5 > 4
1 > 0
Although the students don't understand what the symbols mean, they recognize the pattern, and put > when the left number is greater and < when the right number is greater.
Semi-supervised learning is a mix of both.
Say a student is studying for a test and gets a question like "if x > 9 and x + y = 10, solve for x." They'll use a little bit of supervised learning (addition) and a little bit of unsupervised learning (greater than).
Let's get an overview of deep learning.
Deep learning is a shade of semi-supervised learning. In fact, it's a subset of artificial intelligence inspired by the human brain. If you know a little neurology, you'll know our brains are wired so that several billion neurons, or nodes, create a big mesh network, sending and receiving data to understand and communicate better. (To be fair, it's far more complex than that, but humor me.) Deep learning works similarly, and we can classify these models into the discriminative kind and the generative kind.
Discriminative Deep Learning Models
Discriminative models are classification or prediction models. They're typically supervised, and learn the relationship between the features of data points and their labels. The output of such models is generally a number, a category, a probability, or a class. For instance, a model that predicts whether a picture is of a cat or a dog is a discriminative model.
Generative Deep Learning Models
Generative models generate new content based on previously fed data. They're typically unsupervised, and can generate output like natural language, an image, audio, or video. For example, given enough pictures of cats, a generative model can produce a picture of a cat.
We'll zoom in a little more on generative AI.
Gen-AI is a subset of deep learning. It combines artificial neural networks with semi-structured data, generating new content based on the dataset it's given. Like other machine learning models, generative AI feeds on data — there are two kinds provided to the model:
- Training data — consumed for pre-training a generative transformer model, also called a foundational model, which recognizes and adapts to patterns and learns "what to generate." Think of it like dancing: a dancer watches and learns 10 different styles from thousands of videos, picks up the basic pattern of each dance form, and comes up with their own choreography for new songs. The extensive videos they studied were the training data.
- Input data — primarily given in the form of prompts, or small bits of text that provide context and constraints for the new content to be generated. Going back to the dancing example, a prompt would be someone asking the dancer to perform a classical dance for two minutes.
Just like us humans, machines too misunderstand patterns and jump to wrong conclusions. And just like humans, when a machine answers like it's lost the plot, we say it's having "hallucinations." Hallucinations can occur largely for four reasons:
- The training data was too little.
- The training data was noisy or dirty.
- The prompt data didn't have enough context.
- The prompt data didn't have enough constraints.
Another thing to keep in mind is that the quality of the input, or prompt, highly influences the quality of the output — hence, prompt design is cardinal. Based on the input and expected output, there are multiple types of generative AI models. Let's look at a few, with their examples.
Text to Text
Generation, classification, summarization, translation, and (re)search are all text-to-text tasks, where both input and output are text. ChatGPT and Bard are prime examples of this model type.
Text to Image
These models take text as input and produce an image as output — image generation and image editing are prime use cases today. There are a number of controversies around copyright regulations for these images, which are creating panic in both the art and technology worlds at the moment.
Text to Video / 3D
Video generation and editing has long been a pain point for creators; with text-to-video or text-to-3D gen-AI models, their lives get a little simpler. At the same time, game developers can quickly create game assets and non-playable characters using just text prompts, and even 3D modeling and rendering animations has become easier with gen-AI.
Text to Task
Text-to-task has been around in the industry for a while, in the form of virtual assistants, automation, and software agents. What's new with gen-AI is that tasks no longer need to be custom-made and saved on these assistants — the system should be smart enough to adapt and complete tasks on its own.
Generative AI is the next big thing in the world. It might just burst like the dot-com bubble, or it might revolutionize the world like the Industrial Revolution. While we wait to see which happens, we can learn more about the what, how, and why of what's coming next.