lunax.hyper_opt

This module provides hyperparameter optimization functionality using the Optuna framework.

class lunax.hyper_opt.optuna_tuner.OptunaTuner(BaseTuner)

Hyperparameter optimization using Optuna framework.

Parameters:

• param_space (Dict[str, Tuple], optional) – Custom parameter search space definition

• n_trials (int) – Number of optimization trials

• model_class (str) – Model class name to optimize

• metric_name (str, optional) – Metric name for optimization

• timeout (int, optional) – Maximum optimization time in seconds

Methods:

optimize(X_train: pd.DataFrame, y_train: pd.Series, X_val: pd.DataFrame, y_val: pd.Series) → Dict

Perform hyperparameter optimization.

Parameters:

• X_train – Training features

• y_train – Training labels

• X_val – Validation features

• y_val – Validation labels

Returns:

Optimization results dictionary

Return type:

Dict

Supported Models:

• XGBoost:

  • XGBRegressor

  • XGBClassifier

• LightGBM:

  • LGBMRegressor

  • LGBMClassifier

• CatBoost:

  • CatRegressor

  • CatClassifier

Default Parameter Ranges:

XGBoost Models:

• max_depth: [3, 18]

• learning_rate: [0.01, 0.2]

• n_estimators: [50, 1000]

• min_child_weight: [0, 10]

• subsample: [0.6, 1.0]

• colsample_bytree: [0.5, 1.0]

• reg_alpha: [0, 1]

• reg_lambda: [0, 1]

• grow_policy: [‘depthwise’, ‘lossguide’]

LightGBM Models:

• max_depth: [3, 10]

• learning_rate: [0.01, 0.3]

• n_estimators: [50, 1000]

• num_leaves: [31, 127]

• subsample: [0.5, 1.0]

• colsample_bytree: [0.5, 1.0]

• reg_alpha: [0, 1]

• reg_lambda: [0, 1]

CatBoost Models:

• depth: [1, 12]

• learning_rate: [0.01, 0.3]

• iterations: [50, 1000]

• l2_leaf_reg: [1, 10]

• bootstrap_type: [‘Bayesian’, ‘Bernoulli’, ‘MVS’]

• subsample: [0.5, 1.0] (for Bernoulli)

• bagging_temperature: [0, 10] (for Bayesian)

Supported Metrics:

Regression:

• mse (default)

• mae

• rmse

• r2

Classification:

• f1 (default)

• accuracy

• precision

• recall

Example Usage:

from lunax.hyper_opt import OptunaTuner
from lunax.models import xgb_clf

# Basic usage
tuner = OptunaTuner(
    n_trials=50,
    model_class='XGBClassifier',
    metric_name='f1'
)

# Optimize hyperparameters
results = tuner.optimize(X_train, y_train, X_val, y_val)

# Get best parameters and create model
best_params = results['best_params']
model = xgb_clf(best_params)
model.fit(X_train, y_train)

# Custom parameter space
param_space = {
    'max_depth': ('int', 3, 10),
    'learning_rate': ('float', 0.01, 0.1),
    'n_estimators': ('int', 100, 500)
}
tuner = OptunaTuner(param_space=param_space)

# Multiple metrics optimization
tuner = OptunaTuner(
    n_trials=50,
    model_class='LGBMClassifier',
    metric_name=['accuracy', 'f1']  # Will use mean of metrics
)

# With time limit
tuner = OptunaTuner(
    n_trials=50,
    model_class='CatClassifier',
    metric_name='accuracy',
    timeout=3600  # 1 hour limit
)