Skip to content

What Is A Model

What This Is

Before any optimizer, split, or training loop, there is one question that quietly sits underneath every academy page:

  • what is a "model", actually, and what do I need it to learn

This page treats a model as a function with adjustable knobs. The knobs are called parameters. Training is the process that turns data into knob settings. Inference is the process of using the tuned function to produce an output.

Everything else — architectures, optimizers, loss functions, schedulers — is machinery for doing that one thing well.

This page is for absolute beginners. If you already know what gradient descent is, skip to Loss and Gradients Intuition.

When You Use It

  • you have heard the word "model" for months and want a stable mental picture
  • you can run a scikit-learn fit but cannot explain what .fit actually did
  • you are about to start Phase 4 of the Study Plan and want the deep-learning bridge
  • someone says "the model learned..." and you would like to know what that sentence means

A Model Is A Function

Pick any prediction task you can name:

  • given a photo, is there a cat?
  • given a sentence, is the sentiment positive?
  • given patient vitals, what is the disease risk?

A model is a function that maps the input to the output:

photo → model → "cat" or "no cat"
sentence → model → positive/negative
vitals → model → risk score

The shape of the function — how it combines the input into an answer — is the architecture. The specific numbers inside the function that make it produce this answer for this photo are the parameters or weights.

Training does not change the architecture. Training changes the parameters.

The Simplest Possible Model

A linear model predicts a single number from an input:

y = w · x + b

x is the input (one or several numbers). w is a weight (or a vector of weights). b is a bias. Together, w and b are the parameters.

If you are predicting whether someone will buy a coffee based on the temperature, you could write:

prob = sigmoid(w * temperature + b)

sigmoid squeezes any number into the range (0, 1) so we can read it as a probability. At this point you already have a complete machine-learning model. The rest of the course is about making this same idea scale.

Parameters: The Knobs

In the linear model above, there are exactly 2 parameters: w and b. A neural network is the same idea, scaled up:

  • a tiny multi-layer perceptron (MLP) for the MNIST digits: ~20,000 parameters
  • a ResNet-50 for image classification: ~25 million parameters
  • a modern small LLM: 7 billion parameters
  • a large modern LLM: hundreds of billions

Every number in a trained model's weight file is one knob the training procedure adjusted. This is the thing that is being "learned" in machine learning. Not facts. Not intuitions. Numbers.

What Does It Mean For A Model To "Learn"

A model "learns" when its parameters are adjusted so that its output on training examples moves closer to the desired output.

The recipe has three pieces:

  1. Prediction — run the current parameters on an input, get an output
  2. Loss — compare the output to the truth; the loss is a single number that says "how wrong"
  3. Update — nudge the parameters in the direction that reduces the loss

Repeat a lot. That is training.

The update rule is called gradient descent. The nudge direction is the gradient. The page after this one, Loss and Gradients Intuition, walks through the nudge step.

A Model In 10 Lines

This is a complete machine-learning pipeline. Every piece here has a name that appears on dozens of later pages.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

X, y = load_iris(return_X_y=True)                               # data
X_train, X_valid, y_train, y_valid = train_test_split(X, y, random_state=0)
model = LogisticRegression(max_iter=1000)                       # architecture
model.fit(X_train, y_train)                                     # training
preds = model.predict(X_valid)                                  # inference
accuracy = (preds == y_valid).mean()                            # evaluation

What happened:

  • LogisticRegression(...) created a function with adjustable knobs — the weights and biases
  • model.fit(...) adjusted those knobs so that predictions on X_train moved closer to y_train
  • model.predict(...) ran the tuned function on data it had never seen
  • comparing predictions to y_valid told us how well it generalized

Every model in the academy, no matter how large, follows this exact shape.

Inputs, Outputs, And Labels

Three words you will hear everywhere:

  • features (also: inputs, X) — the numbers you feed into the model
  • label (also: target, y) — the correct answer you want the model to produce
  • prediction — the model's current guess for the answer

Training needs features and labels. Inference only needs features.

Supervised learning is learning from pairs (features, label). Unsupervised learning is finding structure in features without labels. Self-supervised learning makes up labels from the features themselves (predict the next word, predict the missing pixel).

Why Linear Models Are The Right Place To Start

  • they have exactly the pieces every model has (architecture, parameters, loss, fit, predict) and no extra complexity
  • their behavior is fully inspectable — you can literally print the weights
  • they are a strong floor for many problems, which makes the ladder "is the non-linear model actually beating the linear one?" meaningful
  • every deep-learning architecture is just "a stack of linear layers with nonlinearities and connections between them"

If a linear model already works, a deeper one is overkill. This habit is repeated on every later page.

What A Neural Network Adds

A linear model can only draw a single straight boundary between classes. A neural network draws many boundaries and combines them:

input → linear layer → nonlinearity → linear layer → nonlinearity → ... → output

The nonlinearity is what makes the network not collapse into a single linear function. Common choices are ReLU, GELU, and sigmoid. Without it, no matter how many layers you stack, you have still just built one big linear model.

That is the whole secret. A neural network is a sequence of simple functions with nonlinearities that lets it approximate more complicated input-output relationships.

The rest of deep learning is about making that approximation work on realistic data: choosing the architecture (CNN, transformer, RNN), choosing the training recipe (optimizer, schedule, regularization), and choosing the evaluation (splits, metrics, slices).

Evaluation Is Part Of The Model

A model that gets 99% accuracy on data it already saw is useless if it gets 60% on data it has never seen.

Because of this, a "model" in the academy usually means the combination of:

  • the architecture and trained parameters
  • the preprocessing applied to inputs
  • the split and evaluation rule that was used to decide the model is ready

The Honest Splits and Baselines page makes the evaluation part concrete.

What To Inspect After Your First Fit

  • how many parameters the model has
  • the accuracy on training data vs held-out data
  • whether a trivial "predict the most common class" model would do almost as well
  • a few actual predictions — right ones and wrong ones
  • what the model is doing wrong on the wrong ones

If you cannot answer all five, you have not trained a model; you have run some code.

Failure Pattern

The most common beginner failure is treating a model as a black box whose behavior is determined by the library. Every model has knobs, a loss, a training procedure, and an evaluation rule. If you cannot describe those four things for the model you just trained, you are not ready to defend any result it produces.

The second most common failure is thinking bigger means better. A huge model on 50 training examples is almost always worse than a tiny model on the same examples.

Common Mistakes

  • calling every machine-learning system a "model" without distinguishing architecture, weights, and pipeline
  • confusing "learned" with "memorized"
  • treating a good training accuracy as evidence the model works
  • starting with a neural network on a problem a linear model would solve
  • skipping an honest split and letting the model look better than it is

Practice

  1. Train a logistic regression on the iris dataset and print the weights. Explain what each weight means.
  2. Count the parameters in your model. (Hint: sum(p.numel() for p in model.parameters()) for PyTorch, or read the coef_ shape for scikit-learn.)
  3. Train a dummy classifier that always predicts the most common class. Compare its accuracy against your model.
  4. Explain in one sentence what "learning" did to your model's parameters.
  5. Pick one wrong prediction and explain why you think the model got it wrong.

Runnable Example

See examples/first-steps/first_model.py for the simplest possible training and inspection loop.

Longer Connection

Continue with Loss and Gradients Intuition for the update step inside .fit(), Probability and Linear Algebra Refresher for the math you will keep running into, and Honest Splits and Baselines for how to know whether your model has actually learned anything.