Estimation of the error for small-sample optimal binary filter design using prior knowledge

Abstract

Optimal binary filters estimate an unobserved ideal quantity from observed quantities. Optimality is with respect to some error criterion, which is usually mean absolute error MAE (or equivalently mean square error) for the binary values. Both the ideal and observed quantities are random and they are governed by probability laws. The optimal filter is found relative to these laws. Given an observation random vector (in generics, the expression levels of certain genes), we would like to estimate the MAE of the optimal filter for a random variable (the target gene). Because we are dealing with very small samples, we cannot estimate the optimal filter itself. But our genomic application only requires that we estimate its MAE. It will give us an idea on the predictive quality of the input genes with respect to the target gene. The problem is difficult because if one estimates directly from the data without any kind of prior knowledge on the probabilistic distribution of the real-world data, one obtains a poor estimate: the estimation error is big since the number of samples provided is very small. The present work proposes a partial solution to this problem by estimating the estimation error. But this estimation has to be done in a consistent way to avoid being too optimistic about the prediction. This consistency can be achieved by finding a majoring of the bias in the estimation of estimation error. To be able to do that, we have to guess the distribution of the real-world data and inject some prior knowledge about this distribution. This thesis gives an analytic expression of this majoring under some assumptions and shows that, using such a method, we derive a very good estimate of tile optimal error, even for a very limited amount of data.