1. Prove that Memorize is a consistent learner for every class of (binary-valued) functions over any countable domain.

2. Let H be the class of intervals on the line (formally equivalent to axis aligned rectangles in dimension n = 1). Propose an implementation of the ERMH learning rule (in the agnostic case) that given a training set of size m, runs in time O(m2). Hint: Use dynamic programming.

3. Let H1,H2, . . . be a sequence of hypothesis classes for binary classification. Assume that there is a learning algorithm that implements the ERM rule in the realizable case such that the output hypothesis of the algorithm for each class Hn only depends on O(n) examples out of the training set. Furthermore, assume that such a hypothesis can be calculated given these O(n) examples in time O(n), and that the empirical risk of each such hypothesis can be evaluated in time O(mn). For example, if Hn is the class of axis aligned rectangles in Rn, we saw that it is possible to find an ERM hypothesis in the realizable case that is defined by at most 2n examples. Prove that in such cases, it is possible to find an ERMhypothesis forHn in the unrealizable case in time O(mnmO(n)).