Optuna, hyper paramerter tuning library, doesn’t allow non primitive (str, int, float, None) type of values in categorical proposal. See Support tuples for Categorical values #4893.

The post there has a workaround, and here I’d like to write down a bit more generic approach. The idea is feeding string values of tuples, i.e., [‘(1, 2, 3)’, ‘(4, 5, 6)’] instead of [(1, 2, 3), (4, 5, 6)], and then turn them back to tuples after optuna suggesting a value.

First, compose the hyper parameter space using tuple.

model = {"a": [(1, 2, 3), (4, 5, 6)]}

When calling optuna, convert them as strings:

def convert_to_str(value: list[Any]) -> list[str]:
  return [str(v) for v in value]

model["a"] = convert_to_str(model["a"])

When needing to suggest, let the optimzier call our own suggest which looks like this:

result = {}
for k, v in model.items():
  v = trial.suggest_categorical(k, v)
  if k == "a":
    assert isinstance(v, str)
    v = eval(v)   # back to original type
  result[k] = v

This can be generalized with some efforts.

1. Identify all keys containing non primitives by inspecting item’s type in the value (which is list[Any]) to turn the value to list[str].
2. Call trial.suggest_catogrical() with the list[str].
3. If a key belongs to previously identified keys with non primitives items, turn the chosen item back to tuple using eval(). This should be safe as long as your hyper parameters are your own safe code.
4. Now you have a dict of params that can be easily passed to your ML model.