1. Online-to-batch Conversions: In this exercise we demonstrate how a successful online learning algorithm can be used to derive a successful PAC learner as well. Consider a PAC learning problem for binary classification parameterized by an instance domain, X, and a hypothesis class, H. Suppose that there exists an online learning algorithm, A, which enjoys a mistake bound MA(H) ∞. Consider running this algorithm on a sequence of T examples which are sampled i.i.d. from a distribution D over the instance space X, and are labeled by some h_ ∈ H. Suppose that for every round t, the prediction of the algorithm is based on a hypothesis ht : X →{0,1}. Show that E[LD(hr )] ≤ MA(H) T , where the expectation is over the random choice of the instances as well as a random choice of r according to the uniform distribution over [T ].
Hint: Use similar arguments to the ones appearing in the proof of Theorem 14.8.
