On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

Julien Aubert; Luc Lehéricy; Patricia Reynaud-Bouret

Proceedings Year : 2023

On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

(1) , (1) , (1)

Julien Aubert

Function : Author

Université Côte d'Azur

Luc Lehéricy

Function : Author

Université Côte d'Azur

Patricia Reynaud-Bouret

Function : Author

Université Côte d'Azur

Abstract

When fitting the learning data of an individual to algorithm-like learning models, the observations are so dependent and non-stationary that one may wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if it is the usual tool applied to experimental cognition. Our objective in this work is to show that the estimation of the learning rate cannot be efficient if the learning rate is constant in the classical Exp3 (Exponential weights for Exploration and Exploitation) algorithm. Secondly, we show that if the learning rate decreases polynomially with the sample size, then the prediction error and in some cases the estimation error of the MLE satisfy bounds in probability that decrease at a polynomial rate.

Keywords

statistical estimation maximum likelihood estimator adversarial multi armed bandits Exp3 cognition Online learning

Domains

General Mathematics [math.GM]

Fichier principal

Exp3___ICML (6).pdf (560.78 Ko)

Boxplot_etaconstant.pdf (7.21 Ko)

Boxplot_etaconstant1000.png (17.56 Ko)

Boxplot_etadec_K2_est.pdf (6.73 Ko)

Boxplot_etadec_K2_est1000.png (15.21 Ko)

Boxplot_etadec_K2_pred.pdf (10.78 Ko)

Boxplot_etadec_K2_pred1000.png (20.49 Ko)

Boxplot_etadec_K4_est.pdf (5.91 Ko)

Boxplot_etadec_K4_est1000.png (13.15 Ko)

Boxplot_etadec_K4_pred.pdf (8.55 Ko)

Boxplot_etadec_K4_pred1000.png (23.27 Ko)

RebuttalEXP3.pdf (68.45 Ko)

Tmax_plot.pdf (8.4 Ko)

Tmax_plotNsim1000.png (97.84 Ko)

Tmax_plotNsimbis.png (21.64 Ko)

differencepdelta2arms.png (3.54 Ko)

erreuretaconstant2bras.png (3.98 Ko)

erreuretaconstant4bras.png (4.57 Ko)

estimationerrorplot1.png (3.06 Ko)

example_paper.pdf (204.17 Ko)

icml_numpapers.pdf (2.76 Ko)

predictionerror.png (4.23 Ko)

proba.pdf (5.26 Ko)

vrais.pdf (6.86 Ko)

Origin	Files produced by the author(s)

Origin	Files produced by the author(s)

Origin	Files produced by the author(s)

Julien AUBERT : Connect in order to contact the contributor

https://hal.univ-cotedazur.fr/hal-04083986

Submitted on : Wednesday, October 11, 2023-1:04:43 PM

Last modification on : Thursday, March 14, 2024-3:16:21 AM

Dates and versions

hal-04083986 , version 1 (10-05-2023)

hal-04083986 , version 2 (11-10-2023)

Licence

Attribution

Identifiers

HAL Id : hal-04083986 , version 2
ARXIV : 2305.06660

Cite

Julien Aubert, Luc Lehéricy, Patricia Reynaud-Bouret. On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm. 2023. ⟨hal-04083986v2⟩

Export

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

UNIV-COTEDAZUR IDEX-UNIV-COTEDAZUR 3IA-COTEDAZUR ANR

146 View

65 Download

On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

Abstract

Keywords

Domains

Dates and versions

Licence

Identifiers

Cite

Export

Collections

Altmetric

Share