Definition: A model architecture using transformers and self-attention for text representation, bidirectional context processing, and unsupervised training. BERT: The “Reader” of the AI World GPT vs. BERT Why BERT is still King in Vibe Coding You use GPT to write code. You use BERT to search code. The Retrieval Augmented Generation (RAG) Stack Vibe coding…
Definition: A language model determining token probability based on both preceding and following context text. Uni-directional vs. Bi-directional Coding is Bidirectional Code is highly interdependent. A function at the bottom of the file might call a variable defined at the top. A variable type might be inferred from how it’s used 50 lines later. In-Filling…
Definition: A model inferring predictions based on its own previous predictions, as seen in transformer-based language models. How GPT Actually Writes Code Autoregressive means “predicting the next thing based on the past things.” Why This Matters for Debugging Because the model writes linearly (left to right), it cannot “go back” and fix a mistake it…
Definition: AI systems capable of operating independently to pursue goals, make decisions, and interact with environments with minimal human supervision. The Dream You define the “What” (Build a landing page). The Agent figures out the “How” (Write HTML, write CSS, deploy to Vercel). The Reality in 2025 Autonomous agents (like AutoGPT or Devin) are impressive…
Definition: Automated processes for building ML models, including hyperparameter tuning, feature engineering, and model deployment. AutoML: AI Building AI The Ultimate Vibe If Vibe Coding is “using AI to write code,” AutoML is “using AI to write AI.” Instead of manually picking a model (Random Forest vs. Neural Net) and tuning learning rates, you just…
Definition: A neural network system learning to extract important information through compression, consisting of an encoder and decoder component. What is it? An Auto-Encoder takes a big input (like an image), squashes it into a tiny vector (Latent Space), and then tries to recreate the original image from that vector. It learns to keep only…
Definition: A metric between 0.0 and 1.0 representing binary classification models’ ability to separate positive from negative classes. What is AUC? Area Under the ROC Curve (AUC) is the single best number to tell if your binary classifier (e.g., Spam vs. Ham) is actually working. Vibe Coding Context In Vibe Coding, you often build simple…
Definition: A quality or characteristic describing an observation, such as colour or size, equivalent to feature in machine learning contexts. Attributes in Code vs. AI Prompting for Attributes When asking AI to generate data or mock objects, you must be explicit about the attributes. Feature Engineering with AI If you are doing ML, you can…
Definition: A neural network mechanism indicating the importance of particular words or parts, compressing information needed to predict next tokens in transformers. The Breakthrough Before “Attention,” AI read sentences left-to-right and often forgot the beginning by the time it reached the end. Attention allows the model to look at every word at once and decide…
Definition: Computer systems designed to mimic brain structures, composed of interconnected nodes organized in layers for processing information. How it Works (Simply) An ANN is a giant voting machine. Why Vibe Coders Should Care You don’t need to build ANNs to use them, but understanding their structure explains their failures. Coding with ANNs The Limitation…