Have you ever run a GridSearchCV with 3 parameters and 5 folds, grabbed a coffee, and when you came back… it was still running? Worse: what if it finds the right combination, but you have no idea if another configuration would have been better?

In 2026, this is no longer acceptable. Optuna — the hyperparameter optimization framework that has surpassed 12,400 stars on GitHub and accumulated over 10,400 academic citations (KDD 2019) — has become the de facto standard in the Python community. While GridSearchCV blindly tests combinations, Optuna learns with each trial and converges on the best configurations with up to 90% fewer executions (Source: GitHub optuna/optuna).

"Optuna isn't just faster — it changes how you think about optimization. Instead of 'which combination works?', the question becomes 'what's the best possible configuration given my computational budget?'" — Prefect Labs, Optuna Case Study (2025)

This quick guide will show you, in 10 minutes of code, how to:

• Build modular pipelines with Pipeline + ColumnTransformer from scikit-learn 1.8+
• Integrate everything with Optuna 4.8+ using Bayesian search
• Apply intelligent pruning to abort unpromising trials
• Visualize results and log everything in MLflow

Why GridSearchCV No Longer Scales?

Let's look at the numbers. A GridSearchCV with:

• 4 values for n_estimators (100, 200, 300, 400)
• 3 values for max_depth (5, 10, 15)
• 3 values for min_samples_split (2, 5, 10)
• 5 cross-validation folds

Result: 4 × 3 × 3 × 5 = 180 trainings. If each training takes 30 seconds, that's 90 minutes of waiting.

Now, Optuna with Bayesian search (TPESampler) solves the same problem with 20 to 40 trials — that is, 10 to 20 minutes. And it finds better configurations.

HPO (Hyperparameter Optimization) automation finds configurations that surpass expert intuition by 5% to 15%, according to a 2026 MLOps market analysis (Source: Kindatechnical.com, March/2026). In machine learning, a 5% gain can decide between a production model and a shelved pilot.

Environment Setup

You'll need Python 3.11+ and install four libraries:

pip install optuna==4.8.0 scikit-learn==1.8.0 pandas numpy

If you want the logging bonus, add:

pip install mlflow

Note: scikit-learn 1.8.0 was released in December 2025, and version 1.9.0rc1 has been available since May 2026. Optuna 4.8.0 came out in March 2026 with support for multi-objective GPSampler. (Sources: scikit-learn.org, GitHub optuna/optuna)

Modular Pipeline with scikit-learn

The foundation is a well-built pipeline. You use ColumnTransformer to handle numerical and categorical columns separately, and package everything into a Pipeline:

import pandas as pd
import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score, train_test_split

Titanic dataset — classic, fast, everyone knows it

url = "https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv" df = pd.read_csv(url)

Basic feature engineering

df["Age"] = df["Age"].fillna(df["Age"].median()) df["Embarked"] = df["Embarked"].fillna("S") df["FamilySize"] = df["SibSp"] + df["Parch"] + 1

features = ["Pclass", "Sex", "Age", "Fare", "Embarked", "FamilySize"] target = "Survived"

X = df[features] y = df[target]

Split column types

num_cols = ["Age", "Fare", "FamilySize"] cat_cols = ["Pclass", "Sex", "Embarked"]

Modular preprocessing

preprocessor = ColumnTransformer( transformers=[ ("num", StandardScaler(), num_cols), ("cat", OneHotEncoder(drop="first"), cat_cols), ] )

Complete pipeline

pipeline = Pipeline( steps=[ ("preprocessor", preprocessor), ("classifier", RandomForestClassifier(random_state=42)), ] )

This pipeline is already functional. Run it with pipeline.fit(X, y). But we're just getting started.

Defining the Search Space with Optuna

Here's where the magic happens. Optuna uses suggest_* methods to define the search space. Each parameter becomes a distribution:

import optuna

def objective(trial): # Suggest values for hyperparameters n_estimators = trial.suggest_int("n_estimators", 50, 500, step=50) max_depth = trial.suggest_int("max_depth", 3, 20) min_samples_split = trial.suggest_int("min_samples_split", 2, 20) min_samples_leaf = trial.suggest_int("min_samples_leaf", 1, 10) max_features = trial.suggest_float("max_features", 0.3, 1.0) bootstrap = trial.suggest_categorical("bootstrap", [True, False])

# Update the pipeline with the suggested parameters
pipeline.set_params(
    classifier__n_estimators=n_estimators,
    classifier__max_depth=max_depth,
    classifier__min_samples_split=min_samples_split,
    classifier__min_samples_leaf=min_samples_leaf,
    classifier__max_features=max_features,
    classifier__bootstrap=bootstrap,
)

# Cross-validation with 5 folds
scores = cross_val_score(pipeline, X, y, cv=5, scoring="accuracy")
return scores.mean()

Notice the advantage: suggest_float() can use a logarithmic scale with log=True, ideal for parameters like learning_rate or SVM's C. GridSearch doesn't have this flexibility.

The Execution: Creating the Study

# Create the study with TPE sampler (default, but let's be explicit)
study = optuna.create_study(
    direction="maximize",
    sampler=optuna.samplers.TPESampler(seed=42),
    pruner=optuna.pruners.MedianPruner(),
)

Run the optimization

study.optimize(objective, n_trials=50, timeout=600) # 50 trials or 10 min

print(f"Best trial: {study.best_trial.value:.4f}") print(f"Best parameters: {study.best_trial.params}")

In 50 trials — compared to 180 for GridSearch — Optuna already converges. And with timeout=600, you ensure you won't blow your budget.

Comparison: Which Sampler to Use?

This is the question we hear most often. Here's the definitive comparison:

Rule of thumb: start with TPESampler. Switch to GPSampler if you have few parameters and want faster convergence. Use RandomSampler only as a baseline for comparison.

"By combining TPESampler with MedianPruner, we reduced tuning time by 70% in Databricks projects without losing final model quality." — Databricks Documentation, May/2026

Optuna also offers native integration with MLflow for distributed tuning on Spark clusters (Source: Databricks Documentation, May/2026).

Pruning: The Secret to Saving Hours

Pruning is Optuna's biggest differentiator from traditional methods. The idea is simple: if a trial is performing poorly in the early folds, why continue?

from optuna.pruners import MedianPruner, HyperbandPruner

MedianPruner: aborts trials below the historical median

pruner = MedianPruner(n_startup_trials=5, n_warmup_steps=10)

HyperbandPruner: adaptive resource allocation

pruner = HyperbandPruner(min_resource=1, max_resource=100, reduction_factor=3)

study = optuna.create_study( direction="maximize", sampler=optuna.samplers.TPESampler(seed=42), pruner=pruner, )

Here's how MedianPruner works:

1. In the first 5 trials (n_startup_trials), it lets all run — it needs a baseline
2. From trial 6 onwards, it compares intermediate performance with the historical median
3. If the trial is below the median at the same point, it aborts

In practice, you eliminate 30% to 50% of trials before they consume resources.

Visualization: Understanding What Happened

Optuna has native visualizations that show exactly what happened during optimization:

from optuna.visualization import (
    plot_optimization_history,
    plot_param_importances,
    plot_parallel_coordinate,
    plot_contour,
)

Convergence history

fig1 = plot_optimization_history(study) fig1.show()

Hyperparameter importance

fig2 = plot_param_importances(study) fig2.show()

Parallel coordinates (how parameters relate)

fig3 = plot_parallel_coordinate(study) fig3.show()

plot_param_importances is especially useful: it quickly reveals that, for example, max_depth matters much more than min_samples_leaf in your dataset. Knowing this allows you to refine the search space in subsequent rounds.

Case Study: Titanic with Random Forest

Running the complete code on the Titanic dataset, typical results are:

(Simulated results with Random Forest on Titanic, scikit-learn 1.8, Intel i7-12700 CPU)

A 1.6 percentage point gain might not seem like much. But multiply it by millions of predictions — or consider the cost of a false positive in a fraud detection system.

Bonus: Logging Everything in MLflow

If you use MLflow (and you should), Optuna integrates trivially:

import mlflow

mlflow.set_experiment("titanic-optuna")

with mlflow.start_run(run_name="optuna-tuning"): study = optuna.create_study( direction="maximize", sampler=optuna.samplers.TPESampler(seed=42), pruner=optuna.pruners.MedianPruner(), )

# Optuna callback that logs each trial to MLflow
def mlflow_callback(study, trial):
    mlflow.log_params(trial.params)
    mlflow.log_metric("accuracy", trial.value)

study.optimize(
    objective,
    n_trials=50,
    callbacks=[mlflow_callback],
)

# Log the best model
mlflow.log_params(study.best_trial.params)
mlflow.log_metric("best_accuracy", study.best_trial.value)
mlflow.sklearn.log_model(pipeline, "model")

print("Everything logged in MLflow! Check the dashboard.")

With this, you have complete traceability: every trial, every parameter, every metric. If the model degrades in production, you can go back and discover exactly what changed.

Want to see the live dashboard? The Optuna Dashboard is a web interface that lets you explore studies without writing a single line of code. Run optuna-dashboard sqlite:///optuna.db and you're set.

What's Next: Optuna v5

The Optuna v5 roadmap is already in development. The main expected features include:

• Native multi-objective search with support for trade-offs (e.g., maximize accuracy AND minimize latency)
• Deeper integration with deep learning frameworks (PyTorch Lightning, JAX)
• Transparent parallelization on Kubernetes clusters without additional configuration

With over 16,000 applications using Optuna according to Preferred Networks, the framework is no longer an exotic choice. It's the market standard.

Summary: Your Migration Checklist

If you're still using GridSearchCV in your projects, here's the plan:

• Install optuna>=4.8 and scikit-learn>=1.8
• Build modular pipelines with ColumnTransformer
• Replace GridSearchCV with study.optimize() using TPESampler
• Add MedianPruner to cut bad trials
• Visualize with plot_param_importances and plot_optimization_history
• Log to MLflow for traceability

It will take an hour to migrate. After that, you'll never wait 90 minutes for a grid search again.

The complete code for this tutorial is available on our NeuralPulse GitHub. It runs on any machine with Python 3.11+ and pip install. Give it a try and tell us in the comments what gain you achieved.

Enjoyed the quick guide format? Is there a topic you'd like to see covered here on NeuralPulse? Send your suggestion — we read every one.