Generative AI is a field of artificial intelligence focused on creating new content that resembles real data. By learning patterns and structures from existing datasets, it can generate fresh outputs such as text, images, audio, video, or even computer code.
Generator: Produces synthetic data from random noise.
Discriminator: Evaluates whether the data is real (from training) or fake (from the generator).
Both are trained in a minimax game until the generator produces realistic data.
A VAE is a generative model that compresses data into a latent space and then reconstructs it back. Unlike standard autoencoders, VAEs generate new samples by sampling from the latent distribution.
Latent space is the compressed representation of data. Each point in this space corresponds to a meaningful variation in the generated output (e.g., changing face shape or smile in image generation).
Mode collapse occurs when a GAN’s generator creates only a narrow set of outputs, repeating similar patterns instead of representing the complete variety present in the training data.
The discriminator distinguishes between real and generated data, providing feedback to the generator to improve output quality.
Overfitting happens when a generative model memorizes training data instead of learning general patterns, producing poor generalization.
Deepfakes are synthetic videos or images created using generative AI, typically replacing one person’s face or voice with another’s.
Data augmentation is the process of expanding a dataset by applying transformations like rotation, flipping, or adding noise. This technique helps prevent overfitting and enhances the model’s ability to generalize to new data.
Self-attention is a mechanism where each element of input (e.g., a word in a sentence) relates to every other element, helping the model understand context and dependencies.
A language model predicts the next word in a sequence based on context. Examples include GPT, BERT, and LLaMA.
Autoregressive models generate outputs step by step, predicting each new element based on previous ones. Example: GPT generates text one word at a time.
Generative Pre-trained Transformer (GPT) is a powerful language model trained on massive text data, enabling it to generate human-like responses, answer queries, and perform tasks that require context understanding and reasoning.
BERT (Bidirectional Encoder Representations from Transformers) is a transformer model trained using masked language modeling, enabling it to understand context in both directions.
The generator produces synthetic data from random noise with the goal of making it realistic enough to trick the discriminator into recognizing it as genuine.
Pixel-wise loss measures the difference between generated and target images at the pixel level (e.g., Mean Squared Error).
GANs generate highly realistic data (images, videos, text) compared to older generative methods.
The discriminator learns by classifying inputs as real or fake, adjusting its weights based on classification errors, and providing gradients to improve the generator.
A proper learning rate ensures stable training:
Too high → instability, poor convergence.
Too low → slow training, mode collapse risk.
Wasserstein loss, used in WGANs, measures how far apart the real and generated data distributions are. Unlike standard GAN loss, it provides smoother gradients, leading to more stable and reliable training.
WGANs address common GAN issues like mode collapse and unstable training by using Wasserstein distance instead of binary cross-entropy, resulting in better convergence and more reliable outputs.
Spectral normalization controls the weights of the discriminator to keep training stable. It prevents the discriminator from overpowering the generator, ensuring balanced and effective learning between both networks.
Attention helps models identify and focus on the most important parts of the input. In image generation, it captures fine details, while in text tasks, it preserves context and improves coherence.
