Advanced Clustering and Dimensionality Reduction¶

What This Is¶

The basic clustering page is about picking a method for obvious group shapes under a stable view. This page is the next step: the data is high-dimensional, the groups are not compact, the visible picture depends on the projection, and the decision you are asked to defend is which method is producing the story versus which method is being produced by the data.

Two questions drive this page:

is the visible structure a property of the data, or a property of the projection?
is the cluster assignment stable under perturbation, or just lucky on one run?

Every tool here — HDBSCAN, UMAP, kernel PCA, stability scoring — is there to help you answer one of those two questions honestly.

When You Use It¶

high-dimensional data where PCA alone looks ambiguous
non-convex or nested groupings where KMeans collapses shape
embeddings from a pretrained encoder where you need to inspect neighborhood structure
anomaly or outlier work where "not in any cluster" is a valid label
cluster counts are not known, and picking k would be faking a decision
you already have a basic clustering from Clustering and Low-Dimensional Views and need to stress-test it

Tooling¶

HDBSCAN (hdbscan package, or sklearn.cluster.HDBSCAN in recent scikit-learn)
sklearn.cluster.DBSCAN and sklearn.cluster.OPTICS for density comparisons
sklearn.cluster.AgglomerativeClustering with linkage choice for hierarchy
sklearn.decomposition.KernelPCA for non-linear linear-style reduction
sklearn.random_projection.GaussianRandomProjection for cheap high-dimensional reduction
sklearn.manifold.TSNE for local neighborhood inspection
umap.UMAP for faster, more global-respecting manifold learning
sklearn.metrics.silhouette_samples and sklearn.metrics.calinski_harabasz_score for non-visual scoring
sklearn.metrics.pairwise_distances for distance sanity checks

Library Notes¶

HDBSCAN picks a density threshold automatically across the data, so it can find clusters of different densities. It exposes labels_, probabilities_, and outlier_scores_, which matter when you want to defend "this point is not a cluster member" as a real call.
OPTICS is the reachability-plot cousin of DBSCAN — it is slower but gives you a hierarchy of densities to inspect.
AgglomerativeClustering with linkage='ward' assumes spherical variance; linkage='average' handles more irregular shapes; linkage='single' chains along elongated structures. Changing linkage without changing data is often the single most informative ablation.
KernelPCA is a linear reduction in a kernel-induced space. It gives you PCA's stability without PCA's linearity constraint, at the cost of the kernel and gamma choices.
Gaussian random projection is a cheap first pass when the original dimension is too large for PCA to finish. By the Johnson-Lindenstrauss lemma, pairwise distances are approximately preserved for a modest number of projection dimensions.
t-SNE is a local-neighborhood optimizer. Points close in t-SNE were probably close in the original space; points far apart in t-SNE are not necessarily far in the original space. Treat it as inspection, not ground truth.
UMAP preserves more global structure than t-SNE and is much faster on large data. Its n_neighbors knob controls how local the picture is; min_dist controls how tight visible clusters look.
Silhouette samples is a per-point score: the gap between a cluster's cohesion and its separation from the nearest other cluster. Averaging hides bad slices, so inspect the distribution, not only the mean.

Method Choice Heuristics¶

use HDBSCAN when you want clusters of varying density and are willing to accept unlabeled points as a valid outcome
use OPTICS when you want to inspect the reachability plot before committing to one density threshold
use AgglomerativeClustering when you want a dendrogram to cut, or when linkage comparison is the core experiment
use KernelPCA when the structure you expect is non-linear but you still want PCA-style stability
use random projection when the dimension is too large for any exact method and you need a first pass
use t-SNE for inspection only, with several perplexities, and always compare it against one other view
use UMAP when you need a manifold embedding that scales past t-SNE and respects global structure better

Minimal Example¶

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import HDBSCAN
from sklearn.decomposition import KernelPCA
import matplotlib.pyplot as plt

X_scaled = StandardScaler().fit_transform(X)

reduced = KernelPCA(n_components=10, kernel="rbf", gamma=None).fit_transform(X_scaled)

labels = HDBSCAN(
    min_cluster_size=25,
    min_samples=5,
).fit_predict(reduced)

plt.hist(labels, bins=len(set(labels)))

Three inspection moves already appear here: scaling before reduction, reducing before clustering, and looking at the label histogram before trusting the assignment.

The Core Decision Ladder¶

Use the ladder before reaching for an advanced method:

Does the basic ladder already work? KMeans with a reasonable k and a PCA view is the floor. If it already answers the question, advanced methods are overkill.
Does the shape look non-convex or varying-density? If yes, move to HDBSCAN / DBSCAN / OPTICS before trying manifold embeddings.
Is the dimension too large for stable distances? If yes, reduce with PCA, KernelPCA, or random projection first, then cluster. Clustering in 500+ dimensions directly is almost always a mistake.
Do the clusters persist under reseeding, subsampling, and perturbation? If not, the finding is not a cluster — it is a local minimum.
Can the story be defended visually AND numerically? A good clustering survives both a silhouette check and a plot inspection. One without the other is not defensible.

Non-Linear Reduction: KernelPCA, t-SNE, UMAP¶

These three are the working set for non-linear structure.

KernelPCA¶

from sklearn.decomposition import KernelPCA
kpca = KernelPCA(n_components=2, kernel="rbf", gamma=0.05)
X_kpca = kpca.fit_transform(X_scaled)

What to inspect:

how the picture changes as gamma moves from 0.01 to 1.0
whether linear PCA already shows the same structure (if yes, kernel adds nothing)
the eigenvalues_ attribute to see how much signal each component carries

t-SNE¶

from sklearn.manifold import TSNE
X_tsne = TSNE(n_components=2, perplexity=30, random_state=0).fit_transform(X_scaled)

What to inspect:

run at three perplexities: 5, 30, and 100. Stable findings survive all three.
the initialization: PCA-init tends to be steadier than random-init
the "islands" — are they really separate, or is t-SNE exaggerating gaps?

UMAP¶

import umap
reducer = umap.UMAP(n_neighbors=15, min_dist=0.1, n_components=2, random_state=0)
X_umap = reducer.fit_transform(X_scaled)

What to inspect:

n_neighbors small (5) vs large (50): small gives more local, large gives more global
whether the reduction is stable under resampling (rerun with 80% of rows, compare)
whether UMAP and PCA tell the same story at the high level

HDBSCAN In Depth¶

HDBSCAN is the default advanced choice for most real workloads because it:

handles clusters of different density in the same dataset
does not require a cluster count
marks outliers explicitly (label -1)
gives you a probability of membership per point

from sklearn.cluster import HDBSCAN
model = HDBSCAN(
    min_cluster_size=30,
    min_samples=10,
    cluster_selection_epsilon=0.0,
).fit(X)
print(model.labels_)
print(model.probabilities_)

Tuning notes:

min_cluster_size is usually the single most important knob — set it to the smallest group size you actually care about
min_samples controls how conservative the density call is; higher values mark more points as noise
if too many points get label -1, raise cluster_selection_epsilon or lower min_samples

Cluster Stability — Non-Negotiable For Advanced Work¶

A single clustering run is one sample. Stability analysis tells you whether the sample is typical.

from sklearn.utils import resample
from sklearn.metrics import adjusted_rand_score

def stability(cluster_fn, X, n_trials=20, frac=0.8, seed=0):
    rng = np.random.default_rng(seed)
    baseline = cluster_fn(X)
    scores = []
    for _ in range(n_trials):
        idx = rng.choice(len(X), size=int(frac * len(X)), replace=False)
        labels_s = cluster_fn(X[idx])
        scores.append(adjusted_rand_score(baseline[idx], labels_s))
    return scores

The point is not the implementation detail; it is the habit. A clustering that moves every rerun is a finding about the optimizer, not about the data.

Practical stability moves:

rerun with 3 different random seeds and compare label agreement (adjusted Rand index)
subsample 80% of rows 20 times and measure how often pairs stay in the same cluster
add 2% noise and check whether cluster centers move more than one inter-cluster distance

Evaluation Without Labels¶

silhouette_samples — per-point score, inspect the distribution not only the mean
calinski_harabasz_score — ratio of between- to within-cluster variance, rewards compactness
davies_bouldin_score — average similarity between each cluster and its most similar counterpart (lower is better)
stability scores from the resampling recipe above
external validation — if even a handful of points have known labels, compare them to the cluster assignment with adjusted Rand or homogeneity

Never trust one of these alone. A clustering that wins on silhouette but loses on stability is fragile.

Comparing Projections Honestly¶

When two projections disagree, you have one of:

real non-linear structure that linear PCA cannot see
a projection artifact (especially common in t-SNE)
scaling or preprocessing disagreement between the pipelines

Recipe to resolve:

check that PCA, KernelPCA, and UMAP all used the same scaled input
color each projection by the same candidate label set — do the colors move together?
run HDBSCAN in the original space (not the projection) and re-color each projection by those labels
if the projections still disagree under labels from the original space, the disagreement is real — prefer the less-dramatic view as the reporting default

What To Inspect¶

the label histogram — very skewed histograms often mean one blob and many tiny artifacts
the fraction of points marked as noise (label -1) for density-based methods
the silhouette-sample distribution, not only the mean
cluster stability under resampling and reseeding
how much the projection changes when hyperparameters change by one step
whether the cluster story matches any outside knowledge (a time column, a source column, a known tag)

Failure Pattern¶

The single most common failure is trusting a t-SNE picture as proof. A cluster that looks perfectly separated at perplexity=30 can vanish at perplexity=5 and perplexity=100. If the finding does not survive all three, it is not a finding.

The second most common failure is clustering in the original high-dimensional space without thinking about distance concentration. Past ~50 dimensions, Euclidean distances between random points become nearly equal, and any clustering based on them is dominated by noise.

The third: picking k for KMeans by elbow plot and treating the answer as ground truth. The elbow is a heuristic, not evidence.

Common Mistakes¶

fitting dimensionality reduction on the full dataset before splitting (if labels exist downstream)
comparing clustering methods under different preprocessing
using t-SNE for anything except visual inspection
interpreting UMAP distances between clusters as real distances (the method does not guarantee that)
declaring "no clusters" based on one HDBSCAN run with default parameters
choosing between methods by aesthetic preference rather than by stability
forgetting to set random_state and then wondering why the picture changes
comparing silhouette scores across different numbers of clusters without the same preprocessing

Practice¶

Run HDBSCAN and KMeans on the same scaled dataset. Report the silhouette distribution for each and decide which method's story you would defend.
Plot the same data with PCA, t-SNE at three perplexities, and UMAP. Name one finding that survives all five views and one that does not.
Measure clustering stability by resampling 80% of the rows 20 times. Report how often the baseline cluster assignment agrees with the subsample assignment.
Show what happens to pairwise distances as you go from 5 to 50 to 500 dimensions on random Gaussian data. Explain why clustering in 500-D directly is risky.
Fit HDBSCAN with three different min_cluster_size values. Defend one choice using the label histogram and the noise fraction, not just the silhouette.
Construct a dataset where KernelPCA finds structure that linear PCA misses, and explain which kernel / gamma exposed it.

Runnable Example¶

See examples/classical-ml-recipes/advanced_clustering_demo.py for a complete workflow that compares HDBSCAN, KMeans, UMAP, and t-SNE under a single scaled input.

Longer Connection¶

Continue with Dimensionality Reduction for the linear-method foundation this page builds on, Evaluation Metrics Deep Dive for how to reason about the internal scores honestly, and Advanced Unsupervised and Manifold Workflows for the full track where these pieces are assembled into a defended workflow.