Multi Arm Bandit

[garitrik]

I was going through the examples of this package to find out how it can be used to solve multi-arm bandit problems, but I was not able to find any such examples. Is it possible to have an example to solve multiarm bandit problems with this package? That will be really helpful.

[apaleyes]

It would be indeed. We do have BanditParameter thanks to @ekalosak , but no examples or tutorials of using it. Eric, since you were the one to kindly contribute this parameter, maybe you would be able to share some realistic example of solving some problem with it? Not asking for a PR (although that would be lovely!), but a mere post here to help @garitrik . Thanks!

[garitrik]

I was using the bandit parameter in the param space of BayesianOptimisationLoop class. But when I tried running the loop there was an attribute error -' 'BanditParameter' object has no attribute 'dimensionality' ' as the BayesianOptimisationLoop was trying to use the context manager. So, I realized I am not able to use the bandit parameter properly to solve a multi-arm bandit problem.

[apaleyes]

If you could post a code sample, maybe we could look into it. It can be a reasonable bug in Emukit

[garitrik]

from emukit.core.bandit_parameter import BanditParameter
param = BanditParameter('Maxt',sensor_coordinates)
#'sensor_coordinates' is a 2d numpy array containing the coordinates of sensors

X=np.zeros((data_points,2)) #data_points are no of data points number of initial points that are collected jointly before start running the optimization.
Y=np.zeros(data_points)
for i in range (data_points):
X[i]=sensor_coordinates[i]
Y[i]=f(sensor_coordinates[i]) #funtion f returns a float value corresponding to the particular sensor.
Y_init = Y[:, None]

arms_bandit = []
arms_bandit+=tuple(map(tuple, X))
bandit=np.asarray(arms_bandit)

model_gpy = GPRegression(bandit,-Y_init) # Train and wrap the model in Emukit
model_emukit = GPyModelWrapper(model_gpy)

dynamic_lcb = DynamicNegativeLowerConfidenceBound(model = model_emukit,input_space_size=100, delta=0.2)
#For acquisition function I chose the DynamicNegativeLowerConfidenceBound.
#acquisition_optimizer = RandomSearchAcquisitionOptimizer(param, 1)

bayesopt_loop_cust = BayesianOptimizationLoop(
model = model_emukit,
space = param,
acquisition = dynamic_lcb,
batch_size = 1)

bayesopt_loop_cust.run_loop(func.f, 10)
#but when running the loop the following error is coming up.
"
44 all_idxs = list(range(space.dimensionality))
45 self.non_context_idxs = [idx for idx in all_idxs if idx not in self.context_idxs]
46

AttributeError: 'BanditParameter' object has no attribute 'dimensionality'
"
And I don't think I can use banditparameter as an input to the ParameterSpace then maybe a type error will occur because it expects the list as an input. So, I am just stuck in this loop. :( :(

#I have also imported all the required modules and functions to run DynamicNegativeLowerConfidenceBound , to make things look a little clean I didn't include that part.

Will this work?