Evaluating Misclassification Probability Using Empirical Risk

* The work is supported by RFBR, grant 04-01-00858-a

An inequality for uniform deviations of sample averages from their means

We derive a new inequality for uniform deviations of averages from their means. The inequality is a common generalization of previous results of Vapnik and Chervonenkis (1974) and Pollard (1986). Usingthe new inequality we obtain tight bounds for empirical loss minimization learning.

Model selection and error estimation

We study model selection strategies based on penalized empirical loss minimization. We point out a tight relationship between error estimation and data-based complexity penalization: any good error estimate may be converted into a data-based penalty function and the performance of the estimate is governed by the quality of the error estimate. We consider several penalty functions, involving error estimates on independent test data, empirical {\sc vc} dimension, empirical {\sc vc} entropy, andmargin-based quantities. We also consider the maximal difference between the error on the first half of the training data and the second half, and the expected maximal discrepancy, a closely related capacity estimate that can be calculated by Monte Carlo integration. Maximal discrepancy penalty functions are appealing for pattern classification problems, since their computation is equivalent to empirical risk minimization over the training data with some labels flipped.

Adaptive model selection using empirical complexities

Given $n$ independent replicates of a jointly distributed pair $(X,Y)\in {\cal R}^d \times {\cal R}$, we wish to select from a fixed sequence of model classes ${\cal F}_1, {\cal F}_2, \ldots$ a deterministic prediction rule $f: {\cal R}^d \to {\cal R}$ whose risk is small. We investigate the possibility of empirically assessingthe {\em complexity} of each model class, that is, the actual difficulty of the estimation problem within each class. The estimated complexities are in turn used to define an adaptive model selection procedure, which is based on complexity penalized empirical risk.The available data are divided into two parts. The first is used to form an empirical cover of each model class, and the second is used to select a candidate rule from each cover based on empirical risk. The covering radii are determined empirically to optimize a tight upper bound on the estimation error. An estimate is chosen from the list of candidates in order to minimize the sum of class complexity and empirical risk. A distinguishing feature of the approach is that the complexity of each model class is assessed empirically, based on the size of its empirical cover.Finite sample performance bounds are established for the estimates, and these bounds are applied to several non-parametric estimation problems. The estimates are shown to achieve a favorable tradeoff between approximation and estimation error, and to perform as well as if the distribution-dependent complexities of the model classes were known beforehand. In addition, it is shown that the estimate can be consistent,and even possess near optimal rates of convergence, when each model class has an infinite VC or pseudo dimension.For regression estimation with squared loss we modify our estimate to achieve a faster rate of convergence.

Market Index Biases and Minimum Risk Indices

Markets, in the real world, are not efficient zero-sum games where hypotheses of the CAPM are fulfilled. Then, it is easy to conclude the market portfolio is not located on Markowitz"s efficient frontier, and passive investments (and indexing) are not optimal but biased. In this paper, we define and analyze biases suffered by passive investors: the sample, construction, efficiency and active biases and tracking error are presented. We propose Minimum Risk Indices (MRI) as an alternative to deal with to market index biases, and to provide investors with portfolios closer to the efficient frontier, that is, more optimal investment possibilities. MRI (using a Parametric Value-at-Risk Minimization approach) are calculated for three stock markets achieving interesting results. Our indices are less risky and more profitable than current Market Indices in the Argentinean and Spanish markets, facing that way the Efficient Market Hypothesis. Two innovations must be outlined: an error dimension has been included in the backtesting and the Sharpe"s Ratio has been used to select the"best" MRI

Market Index Biases and Minimum Risk Indices

Markets, in the real world, are not efficient zero-sum games where hypotheses of the CAPM are fulfilled. Then, it is easy to conclude the market portfolio is not located on Markowitz"s efficient frontier, and passive investments (and indexing) are not optimal but biased. In this paper, we define and analyze biases suffered by passive investors: the sample, construction, efficiency and active biases and tracking error are presented. We propose Minimum Risk Indices (MRI) as an alternative to deal with to market index biases, and to provide investors with portfolios closer to the efficient frontier, that is, more optimal investment possibilities. MRI (using a Parametric Value-at-Risk Minimization approach) are calculated for three stock markets achieving interesting results. Our indices are less risky and more profitable than current Market Indices in the Argentinean and Spanish markets, facing that way the Efficient Market Hypothesis. Two innovations must be outlined: an error dimension has been included in the backtesting and the Sharpe"s Ratio has been used to select the"best" MRI

Minimization of radiation exposure due to computed tomography in inflammatory bowel disease

Patient awareness and concern regarding the potential health risks from ionizing radiation have peaked recently (Coakley et al., 2011) following widespread press and media coverage of the projected cancer risks from the increasing use of computed tomography (CT) (Berrington et al., 2007). The typical young and educated patient with inflammatory bowel disease (IBD) may in particular be conscious of his/her exposure to ionising radiation as a result of diagnostic imaging. Cumulative effective doses (CEDs) in patients with IBD have been reported as being high and are rising, primarily due to the more widespread and repeated use of CT (Desmond et al., 2008). Radiologists, technologists, and referring physicians have a responsibility to firstly counsel their patients accurately regarding the actual risks of ionizing radiation exposure; secondly to limit the use of those imaging modalities which involve ionising radiation to clinical situations where they are likely to change management; thirdly to ensure that a diagnostic quality imaging examination is acquired with lowest possible radiation exposure. In this paper, we synopsize available evidence related to radiation exposure and risk and we report advances in low-dose CT technology and examine the role for alternative imaging modalities such as ultrasonography or magnetic resonance imaging which avoid radiation exposure.

Avaliação das melhores técnicas disponíveis relativas à refrigeração industrial e às emissões resultantes da armazenagem

Esta dissertação foi realizada em colaboração com o grupo empresarial Monteiro, Ribas e teve como principais objetivos efetuar uma avaliação das melhores técnicas disponíveis relativas à refrigeração industrial e às emissões resultantes da armazenagem. O primeiro objetivo teve como alvo todas as instalações da Monteiro, Ribas enquanto que o segundo objetivo se debruçou sobre Monteiro, Ribas, Embalagens Flexíveis, S.A.. Para cumprir estes objetivos, inicialmente efetuou-se um levantamento das melhores técnicas disponíveis apresentadas nos respetivos documentos de referência. Em seguida selecionaram-se as técnicas que se adequavam às condições e às instalações em estudo e procedeu-se a uma avaliação de forma a verificar o grau de implementação das medidas sugeridas no BREF (Best Available Techniques Reference Document). Relativamente aos sistemas de refrigeração industrial verificou-se que estão implementadas quase todas as medidas referenciadas no respetivo documento de referência. Isto prende-se com o facto dos sistemas de refrigeração existentes no complexo industrial Monteiro, Ribas serem relativamente recentes. Foram implementados no ano de 2012, e são caracterizados por apresentarem uma conceção moderna com elevada eficiência. No que diz respeito à armazenagem de produtos químicos perigosos, a instalação em estudo, apresenta algumas inconformidades, uma vez que a maioria das técnicas mencionadas no BREF não se encontram implementadas, pelo que foi necessário efetuar uma avaliação de riscos ambientais, com recurso à metodologia proposta pela Norma Espanhola UNE 150008:2008 – Análise e Avaliação do Risco Ambiental. Para isso procedeu-se então à formulação de vários cenários de riscos e à quantificação de riscos para à Monteiro, Ribas Embalagens Flexíveis S.A., tendo-se apurado que os riscos estavam avaliados como moderados a altos. Por fim foram sugeridas algumas medidas de prevenção e de minimização do risco que a instalação deve aplicar, como por exemplo, o parque de resíduos perigosos deve ser equipado com kits de contenção de derrames (material absorvente), procedimentos a realizar em caso de emergência, fichas de dados de segurança e o extintor deve ser colocado num local de fácil visualização. No transporte de resíduos perigosos, para o respetivo parque, é aconselhável utilizar bacias de contenção de derrames portáteis e kits de contenção de derrames. Relativamente ao armazém de produtos químicos perigosos é recomendado que se proceda a sua reformulação tendo em conta as MTD apresentadas no subcapítulo 5.2.3 desta dissertação.

A data-dependent skeleton estimate and a scale-sensitive dimension for classification

The classical binary classification problem is investigatedwhen it is known in advance that the posterior probability function(or regression function) belongs to some class of functions. We introduceand analyze a method which effectively exploits this knowledge. The methodis based on minimizing the empirical risk over a carefully selected``skeleton'' of the class of regression functions. The skeleton is acovering of the class based on a data--dependent metric, especiallyfitted for classification. A new scale--sensitive dimension isintroduced which is more useful for the studied classification problemthan other, previously defined, dimension measures. This fact isdemonstrated by performance bounds for the skeleton estimate in termsof the new dimension.

Painesuodattimen automatisoidun kankaanvaihtolaitteen kehittäminen ja tekninen tuotteistaminen

Tässä työssä tarkastellaan CE-merkintään vaadittavia teknisen tuotteistamisen vaiheita käyttäen esimerkkinä painesuodattimen automatisoidun kankaanvaihtolaitteen suunnitteluprosessia. Työssä selvitetään, mitä vaihtoehtoja on painesuodattimen lisälaitteiden luokitteluksi, että ne saadaan tuotteistettua Euroopan talousalueella (ETA). Esimerkkinä käytettävä kankaanvaihtolaite on suunniteltu käyttäen järjestelmällisen koneensuunnittelun menetelmää. CE-merkinnän vaatima riskianalyysi on tehty laitteelle standardin SFS-EN ISO 12100:2010 mukaisesti. Tuloksena saatu laitteen prototyyppi täyttää pääosin laitteelle asetettavat vaatimukset. Kustannusarvio ylittää kuitenkin toivotun omakustannehinnan valoverhojen suhteellisen kalliin hinnan takia. Kustannusarvion mukaan prototyyppi voidaan kuitenkin valmistaa edullisesti, sillä valoverhot eivät ole pakollisia laitteen toiminnallisissa testeissä. Ennen tuotteistamista valoverhojen korvaamisen mahdollisuutta muulla turvatekniikalla on kuitenkin tutkittava. Suunnitteluvaiheen jälkeen laitteen turvallisuuden voidaan todeta olevan vähintään riittävällä tasolla. Riskianalyysi on kuitenkin päivitettävä dokumentti, ja laitteen turvallisuus täytyy varmistaa prototyyppiä testaamalla. Työn perusteella voidaan todeta, että huomioimalla laitteen mahdollisesti aiheuttamat vaaratilanteet jo tuotesuunnittelun alussa, voidaan tuotekehitysprosessia nopeuttaa. Tunnistamalla vaaratilanteet suunnittelun varhaisessa vaiheessa voidaan vähentää riskien määrää, ja siten tarvetta riskien pienentämiselle. Näin vähennetään rakenteen suunnittelun ja riskianalyysin iterointikierrosten määrää, jolloin myös tuotteistamisprosessi nopeutuu.

Les processus additifs markoviens et leurs applications en finance mathématique

Cette thèse porte sur les questions d'évaluation et de couverture des options dans un modèle exponentiel-Lévy avec changements de régime. Un tel modèle est construit sur un processus additif markovien un peu comme le modèle de Black- Scholes est basé sur un mouvement Brownien. Du fait de l'existence de plusieurs sources d'aléa, nous sommes en présence d'un marché incomplet et ce fait rend inopérant les développements théoriques initiés par Black et Scholes et Merton dans le cadre d'un marché complet. Nous montrons dans cette thèse que l'utilisation de certains résultats de la théorie des processus additifs markoviens permet d'apporter des solutions aux problèmes d'évaluation et de couverture des options. Notamment, nous arrivons à caracté- riser la mesure martingale qui minimise l'entropie relative à la mesure de probabilit é historique ; aussi nous dérivons explicitement sous certaines conditions, le portefeuille optimal qui permet à un agent de minimiser localement le risque quadratique associé. Par ailleurs, dans une perspective plus pratique nous caract érisons le prix d'une option Européenne comme l'unique solution de viscosité d'un système d'équations intégro-di érentielles non-linéaires. Il s'agit là d'un premier pas pour la construction des schémas numériques pour approcher ledit prix.

An Equivalence Between Sparse Approximation and Support Vector Machines

In the first part of this paper we show a similarity between the principle of Structural Risk Minimization Principle (SRM) (Vapnik, 1982) and the idea of Sparse Approximation, as defined in (Chen, Donoho and Saunders, 1995) and Olshausen and Field (1996). Then we focus on two specific (approximate) implementations of SRM and Sparse Approximation, which have been used to solve the problem of function approximation. For SRM we consider the Support Vector Machine technique proposed by V. Vapnik and his team at AT&T Bell Labs, and for Sparse Approximation we consider a modification of the Basis Pursuit De-Noising algorithm proposed by Chen, Donoho and Saunders (1995). We show that, under certain conditions, these two techniques are equivalent: they give the same solution and they require the solution of the same quadratic programming problem.

Support Vector Machines: Training and Applications

The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Labs. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and Multi-Layer Perceptron classifiers. An interesting property of this approach is that it is an approximate implementation of the Structural Risk Minimization (SRM) induction principle. The derivation of Support Vector Machines, its relationship with SRM, and its geometrical insight, are discussed in this paper. Training a SVM is equivalent to solve a quadratic programming problem with linear and box constraints in a number of variables equal to the number of data points. When the number of data points exceeds few thousands the problem is very challenging, because the quadratic form is completely dense, so the memory needed to store the problem grows with the square of the number of data points. Therefore, training problems arising in some real applications with large data sets are impossible to load into memory, and cannot be solved using standard non-linear constrained optimization algorithms. We present a decomposition algorithm that can be used to train SVM's over large data sets. The main idea behind the decomposition is the iterative solution of sub-problems and the evaluation of, and also establish the stopping criteria for the algorithm. We present previous approaches, as well as results and important details of our implementation of the algorithm using a second-order variant of the Reduced Gradient Method as the solver of the sub-problems. As an application of SVM's, we present preliminary results we obtained applying SVM to the problem of detecting frontal human faces in real images.

Gerenciamento de risco no licenciamento ambiental de atividades potencialmente poluidoras

A presente monografia tem como objetivo identificar, avaliar e, por fim, sugerir mecanismos de controle dos Riscos inerentes aos processos de Licenciamento Ambiental realizados no âmbito do Instituto de Estadual do Ambiente – INEA