New definitions of margin for multi-class Boosting algorithms

La familia de algoritmos de Boosting son un tipo de técnicas de clasificación y regresión que han demostrado ser muy eficaces en problemas de Visión Computacional. Tal es el caso de los problemas de detección, de seguimiento o bien de reconocimiento de caras, personas, objetos deformables y acciones. El primer y más popular algoritmo de Boosting, AdaBoost, fue concebido para problemas binarios. Desde entonces, muchas han sido las propuestas que han aparecido con objeto de trasladarlo a otros dominios más generales: multiclase, multilabel, con costes, etc. Nuestro interés se centra en extender AdaBoost al terreno de la clasificación multiclase, considerándolo como un primer paso para posteriores ampliaciones. En la presente tesis proponemos dos algoritmos de Boosting para problemas multiclase basados en nuevas derivaciones del concepto margen. El primero de ellos, PIBoost, está concebido para abordar el problema descomponiéndolo en subproblemas binarios. Por un lado, usamos una codificación vectorial para representar etiquetas y, por otro, utilizamos la función de pérdida exponencial multiclase para evaluar las respuestas. Esta codificación produce un conjunto de valores margen que conllevan un rango de penalizaciones en caso de fallo y recompensas en caso de acierto. La optimización iterativa del modelo genera un proceso de Boosting asimétrico cuyos costes dependen del número de etiquetas separadas por cada clasificador débil. De este modo nuestro algoritmo de Boosting tiene en cuenta el desbalanceo debido a las clases a la hora de construir el clasificador. El resultado es un método bien fundamentado que extiende de manera canónica al AdaBoost original. El segundo algoritmo propuesto, BAdaCost, está concebido para problemas multiclase dotados de una matriz de costes. Motivados por los escasos trabajos dedicados a generalizar AdaBoost al terreno multiclase con costes, hemos propuesto un nuevo concepto de margen que, a su vez, permite derivar una función de pérdida adecuada para evaluar costes. Consideramos nuestro algoritmo como la extensión más canónica de AdaBoost para este tipo de problemas, ya que generaliza a los algoritmos SAMME, Cost-Sensitive AdaBoost y PIBoost. Por otro lado, sugerimos un simple procedimiento para calcular matrices de coste adecuadas para mejorar el rendimiento de Boosting a la hora de abordar problemas estándar y problemas con datos desbalanceados. Una serie de experimentos nos sirven para demostrar la efectividad de ambos métodos frente a otros conocidos algoritmos de Boosting multiclase en sus respectivas áreas. En dichos experimentos se usan bases de datos de referencia en el área de Machine Learning, en primer lugar para minimizar errores y en segundo lugar para minimizar costes. Además, hemos podido aplicar BAdaCost con éxito a un proceso de segmentación, un caso particular de problema con datos desbalanceados. Concluimos justificando el horizonte de futuro que encierra el marco de trabajo que presentamos, tanto por su aplicabilidad como por su flexibilidad teórica. Abstract The family of Boosting algorithms represents a type of classification and regression approach that has shown to be very effective in Computer Vision problems. Such is the case of detection, tracking and recognition of faces, people, deformable objects and actions. The first and most popular algorithm, AdaBoost, was introduced in the context of binary classification. Since then, many works have been proposed to extend it to the more general multi-class, multi-label, costsensitive, etc... domains. Our interest is centered in extending AdaBoost to two problems in the multi-class field, considering it a first step for upcoming generalizations. In this dissertation we propose two Boosting algorithms for multi-class classification based on new generalizations of the concept of margin. The first of them, PIBoost, is conceived to tackle the multi-class problem by solving many binary sub-problems. We use a vectorial codification to represent class labels and a multi-class exponential loss function to evaluate classifier responses. This representation produces a set of margin values that provide a range of penalties for failures and rewards for successes. The stagewise optimization of this model introduces an asymmetric Boosting procedure whose costs depend on the number of classes separated by each weak-learner. In this way the Boosting procedure takes into account class imbalances when building the ensemble. The resulting algorithm is a well grounded method that canonically extends the original AdaBoost. The second algorithm proposed, BAdaCost, is conceived for multi-class problems endowed with a cost matrix. Motivated by the few cost-sensitive extensions of AdaBoost to the multi-class field, we propose a new margin that, in turn, yields a new loss function appropriate for evaluating costs. Since BAdaCost generalizes SAMME, Cost-Sensitive AdaBoost and PIBoost algorithms, we consider our algorithm as a canonical extension of AdaBoost to this kind of problems. We additionally suggest a simple procedure to compute cost matrices that improve the performance of Boosting in standard and unbalanced problems. A set of experiments is carried out to demonstrate the effectiveness of both methods against other relevant Boosting algorithms in their respective areas. In the experiments we resort to benchmark data sets used in the Machine Learning community, firstly for minimizing classification errors and secondly for minimizing costs. In addition, we successfully applied BAdaCost to a segmentation task, a particular problem in presence of imbalanced data. We conclude the thesis justifying the horizon of future improvements encompassed in our framework, due to its applicability and theoretical flexibility.

Globally optimal on-line learning rules for multi-layer neural networks

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Direct current ripple compensation for multi-phase fault-tolerant machines

A second-harmonic direct current (DC) ripple compensation technique is presented for a multi-phase, fault-tolerant, permanent magnet machine. The analysis has been undertaken in a general manner for any pair of phases in operation with the remaining phases inactive. The compensation technique determines the required alternating currents in the machine to eliminate the second-harmonic DC-link current, while at the same time minimising the total rms current in the windings. An additional benefit of the compensation technique is a reduction in the magnitude of the electromagnetic torque ripple. Practical results are included from a 70 kW, five-phase generator system to validate the analysis and illustrate the performance of the compensation technique.

Adaptive Control and Multi-agent Interface for Infotelecommunication Systems of New Generation

Problems for intellectualisation for man-machine interface and methods of self-organization for network control in multi-agent infotelecommunication systems have been discussed. Architecture and principles for construction of network and neural agents for telecommunication systems of new generation have been suggested. Methods for adaptive and multi-agent routing for information flows by requests of external agents- users of global telecommunication systems and computer networks have been described.

Multi-model fitting based on minimum spanning tree

This paper presents a novel approach to the computation of primitive geometrical structures, where no prior knowledge about the visual scene is available and a high level of noise is expected. We based our work on the grouping principles of proximity and similarity, of points and preliminary models. The former was realized using Minimum Spanning Trees (MST), on which we apply a stable alignment and goodness of fit criteria. As for the latter, we used spectral clustering of preliminary models. The algorithm can be generalized to various model fitting settings, without tuning of run parameters. Experiments demonstrate the significant improvement in the localization accuracy of models in plane, homography and motion segmentation examples. The efficiency of the algorithm is not dependent on fine tuning of run parameters like most others in the field.

A nested Petri net framework for modeling and analyzing multi-agent systems

In the past two decades, multi-agent systems (MAS) have emerged as a new paradigm for conceptualizing large and complex distributed software systems. A multi-agent system view provides a natural abstraction for both the structure and the behavior of modern-day software systems. Although there were many conceptual frameworks for using multi-agent systems, there was no well established and widely accepted method for modeling multi-agent systems. This dissertation research addressed the representation and analysis of multi-agent systems based on model-oriented formal methods. The objective was to provide a systematic approach for studying MAS at an early stage of system development to ensure the quality of design. ^ Given that there was no well-defined formal model directly supporting agent-oriented modeling, this study was centered on three main topics: (1) adapting a well-known formal model, predicate transition nets (PrT nets), to support MAS modeling; (2) formulating a modeling methodology to ease the construction of formal MAS models; and (3) developing a technique to support machine analysis of formal MAS models using model checking technology. PrT nets were extended to include the notions of dynamic structure, agent communication and coordination to support agent-oriented modeling. An aspect-oriented technique was developed to address the modularity of agent models and compositionality of incremental analysis. A set of translation rules were defined to systematically translate formal MAS models to concrete models that can be verified through the model checker SPIN (Simple Promela Interpreter). ^ This dissertation presents the framework developed for modeling and analyzing MAS, including a well-defined process model based on nested PrT nets, and a comprehensive methodology to guide the construction and analysis of formal MAS models.^

Multi-scale colorectal tumour segmentation using a novel coarse to fine strategy

This paper addresses the problem of colorectal tumour segmentation in complex real world imagery. For efficient segmentation, a multi-scale strategy is developed for extracting the potentially cancerous region of interest (ROI) based on colour histograms while searching for the best texture resolution. To achieve better segmentation accuracy, we apply a novel bag-of-visual-words method based on rotation invariant raw statistical features and random projection based l2-norm sparse representation to classify tumour areas in histopathology images. Experimental results on 20 real world digital slides demonstrate that the proposed algorithm results in better recognition accuracy than several state of the art segmentation techniques.

Evaluating a School Community Linked Physical Activity Intervention Programme: a Multi-level Analysis.

Previous research has highlighted the importance of positive physical activity (PA) behaviors during childhood to promote sustained active lifestyles throughout the lifespan (Telama et al. 2005; 2014). It is in this context that the role of schools and teachers in facilitating PA education is promoted. Research suggests that teachers play an important role in the attitudes of children towards PA (Figley 1985) and schools may be an efficient vehicle for PA provision and promotion (McGinnis, Kanner and DeGraw, 1991; Wechsler, Deveraux, Davis and Collins, 2000). Yet despite consensus that schools represent an ideal setting from which to ‘reach’ young people (Department of Health and Human Services, UK, 2012) there remains conceptual (e.g. multi-component intervention) and methodological (e.g. duration, intensity, family involvement) ambiguity regarding the mechanisms of change claimed by PA intervention programmes. This may, in part, contribute to research findings that suggest that PA interventions have had limited impact on children’s overall activity levels and thereby limited impact in reducing children’s metabolic health (Metcalf, Henley & Wilkin, 2012). A marked criticism of the health promotion field has been the focus on behavioural change while failing to acknowledge the impact of context in influencing health outcomes (Golden & Earp, 2011). For years, the trans-theoretical model of behaviour change has been ‘the dominant model for health behaviour change’ (Armitage, 2009); this model focusses primarily on the individual and the psychology of the change process. Arguably, this model is limited by the individual’s decision-making ability and degree of self-efficacy in order to achieve sustained behavioural change and does not take account of external factors that may hinder their ability to realise change. Similar to the trans-theoretical model, socio-ecological models identify the individual at the focal point of change but also emphasises the importance of connecting multiple impacting variables, in particular, the connections between the social environment, the physical environment and public policy in facilitating behavioural change (REF). In this research, a social-ecological framework was used to connect the ways a PA intervention programme had an impact (or not) on participants, and to make explicit the foundational features of the programme that facilitated positive change. In this study, we examined the evaluation of a multi-agency approach to a PA intervention programme which aimed to increase physical activity, and awareness of the importance of physical activity to key stage 2 (age 7-12) pupils in three UK primary schools. The agencies involved were the local health authority, a community based charitable organisation, a local health administrative agency, and the city school district. In examining the impact of the intervention, we adopted a process evaluation model in order to better understand the mechanisms and context that facilitated change. Therefore, the aim of this evaluation was to describe the provision, process and impact of the intervention by 1) assessing changes in physical activity levels 2) assessing changes in the student’s attitudes towards physical activity, 3) examining student’s perceptions of the child size fitness equipment in school and their likelihood of using the equipment outside of school and 4) exploring staff perceptions, specifically the challenges and benefits, of facilitating equipment based exercise sessions in the school environment. Methodology, Methods, Research Instruments or Sources Used Evaluation of the intervention was designed as a matched-control study and was undertaken over a seven-month period. The school-based intervention involved 3 intervention schools (n =436; 224 boys) and one control school (n=123; 70 boys) in a low socioeconomic and multicultural urban setting. The PA intervention was separated into two phases: a motivation DVD and 10 days of circuit based exercise sessions (Phase 1) followed by a maintenance phase (Phase 2) that incorporated a PA reward program and the use of specialist kid’s gym equipment located at each school for a period of 4 wk. Outcome measures were measured at baseline (January) and endpoint (July; end of academic school year) using reliable and valid self-report measures. The children’s attitudes towards PA were assessed using the Children’s Attitudes towards Physical Activity (CATPA) questionnaire. The Physical Activity Questionnaire for Children (PAQ-C), a 7-day recall questionnaire, was used to assess PA levels over a school week. A standardised test battery (Fitnessgram®) was used to assess cardiovascular fitness, body composition, muscular strength and endurance, and flexibility. After the 4 wk period, similar kid’s equipment was available for general access at local community facilities. The control school did not receive any of the interventions. All physical fitness tests and PA questionnaires were administered and collected prior to the start of the intervention (January) and following the intervention period (July) by an independent evaluation team. Evaluation testing took place at the individual schools over 2-3 consecutive days (depending on the number of children to be tested at the school). Staff (n=19) and student perceptions (n = 436) of the child sized fitness equipment were assessed via questionnaires post-intervention. Students completed a questionnaire to assess enjoyment, usage, ease of use and equipment assess and usage in the community. A questionnaire assessed staff perceptions on the delivery of the exercise sessions, classroom engagement and student perceptions. Conclusions, Expected Outcomes or Findings Findings showed that both the intervention (16.4%) and control groups increased their PAQ-C score by post-intervention (p < 0.05); with the intervention (17.8%) and control (21.3%) boys showing the greatest increase in physical activity levels. At post-intervention, there was a 5.5% decline in the intervention girls’ attitudes toward PA in the aesthetic subdomains (p = 0.009); whereas the control boys had an increase in positive attitudes in the health domain (p = 0.003). No significant differences in attitudes towards physical activity were observed in any other domain for either group at post-intervention (p > 0.05). The results of the equipment questionnaire, 96% of the children stated they enjoyed using the equipment and would like to use the equipment again in the future; however at post-intervention only 27% reported using the equipment outside of school in the last 7 days. Students identified the ski walker (34%) and cycle (32%) as their favorite pieces of equipment; with the single joint exercises such as leg extension and bicep/tricep machine (<3%) as their least favorite. Key themes from staff were that the equipment sessions were enjoyable, a novel activity, children felt very grown-up, and the activity was linked to a real fitness experience. They also expressed the need for more support to deliver the sessions and more time required for each session. Findings from this study suggest that a more integrated approach within the various agencies is required, particularly more support to increase teachers pedagogical content knowledge in physical activity instruction which is age appropriate. Future recommendations for successful implementation include sufficient time period for all students to access and engage with the equipment; increased access and marketing of facilities to parents within the local community, and professional teacher support strategies to facilitate the exercise sessions.

Dynamic load balancing of distributed memory parallel computational mechanics using unstructured meshes for multi-physical modelling

As the complexity of parallel applications increase, the performance limitations resulting from computational load imbalance become dominant. Mapping the problem space to the processors in a parallel machine in a manner that balances the workload of each processors will typically reduce the run-time. In many cases the computation time required for a given calculation cannot be predetermined even at run-time and so static partition of the problem returns poor performance. For problems in which the computational load across the discretisation is dynamic and inhomogeneous, for example multi-physics problems involving fluid and solid mechanics with phase changes, the workload for a static subdomain will change over the course of a computation and cannot be estimated beforehand. For such applications the mapping of loads to process is required to change dynamically, at run-time in order to maintain reasonable efficiency. The issue of dynamic load balancing are examined in the context of PHYSICA, a three dimensional unstructured mesh multi-physics continuum mechanics computational modelling code.

Design and Pilot Study for an Efficient High-Throughput Automated Computer-Vision Guided Intelligent De-Calyxing Machine for Post-Harvest Strawberry Processing

Strawberries harvested for processing as frozen fruits are currently de-calyxed manually in the field. This process requires the removal of the stem cap with green leaves (i.e. the calyx) and incurs many disadvantages when performed by hand. Not only does it necessitate the need to maintain cutting tool sanitation, but it also increases labor time and exposure of the de-capped strawberries before in-plant processing. This leads to labor inefficiency and decreased harvest yield. By moving the calyx removal process from the fields to the processing plants, this new practice would reduce field labor and improve management and logistics, while increasing annual yield. As labor prices continue to increase, the strawberry industry has shown great interest in the development and implementation of an automated calyx removal system. In response, this dissertation describes the design, operation, and performance of a full-scale automatic vision-guided intelligent de-calyxing (AVID) prototype machine. The AVID machine utilizes commercially available equipment to produce a relatively low cost automated de-calyxing system that can be retrofitted into existing food processing facilities. This dissertation is broken up into five sections. The first two sections include a machine overview and a 12-week processing plant pilot study. Results of the pilot study indicate the AVID machine is able to de-calyx grade-1-with-cap conical strawberries at roughly 66 percent output weight yield at a throughput of 10,000 pounds per hour. The remaining three sections describe in detail the three main components of the machine: a strawberry loading and orientation conveyor, a machine vision system for calyx identification, and a synchronized multi-waterjet knife calyx removal system. In short, the loading system utilizes rotational energy to orient conical strawberries. The machine vision system determines cut locations through RGB real-time feature extraction. The high-speed multi-waterjet knife system uses direct drive actuation to locate 30,000 psi cutting streams to precise coordinates for calyx removal. Based on the observations and studies performed within this dissertation, the AVID machine is seen to be a viable option for automated high-throughput strawberry calyx removal. A summary of future tasks and further improvements is discussed at the end.

Dynamic Multi-Objective Optimization With jMetal and Spark: a Case Study

Technologies for Big Data and Data Science are receiving increasing research interest nowadays. This paper introduces the prototyping architecture of a tool aimed to solve Big Data Optimization problems. Our tool combines the jMetal framework for multi-objective optimization with Apache Spark, a technology that is gaining momentum. In particular, we make use of the streaming facilities of Spark to feed an optimization problem with data from different sources. We demonstrate the use of our tool by solving a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on near real-time traffic data from New York City, which is updated several times per minute. Our experiment shows that both jMetal and Spark can be integrated providing a software platform to deal with dynamic multi-optimization problems.

Multi-phase electric drives for mobility

Multi-phase electrical drives are potential candidates for the employment in innovative electric vehicle powertrains, in response to the request for high efficiency and reliability of this type of application. In addition to the multi-phase technology, in the last decades also, multilevel technology has been developed. These two technologies are somewhat complementary since both allow increasing the power rating of the system without increasing the current and voltage ratings of the single power switches of the inverter. In this thesis, some different topics concerning the inverter, the motor and the fault diagnosis of an electric vehicle powertrain are addressed. In particular, the attention is focused on multi-phase and multilevel technologies and their potential advantages with respect to traditional technologies. First of all, the mathematical models of two multi-phase machines, a five-phase induction machine and an asymmetrical six-phase permanent magnet synchronous machines are developed using the Vector Space Decomposition approach. Then, a new modulation technique for multi-phase multilevel T-type inverters, which solves the voltage balancing problem of the DC-link capacitors, ensuring flexible management of the capacitor voltages, is developed. The technique is based on the proper selection of the zero-sequence component of the modulating signals. Subsequently, a diagnostic technique for detecting the state of health of the rotor magnets in a six-phase permanent magnet synchronous machine is established. The technique is based on analysing the electromotive force induced in the stator windings by the rotor magnets. Furthermore, an innovative algorithm able to extend the linear modulation region for five-phase inverters, taking advantage of the multiple degrees of freedom available in multi-phase systems is presented. Finally, the mathematical model of an eighteen-phase squirrel cage induction motor is defined. This activity aims to develop a motor drive able to change the number of poles of the machine during the machine operation.

Hierarchical Multi-Label Text Classification in a Low-Resource Setting

In this thesis we address a multi-label hierarchical text classification problem in a low-resource setting and explore different approaches to identify the best one for our case. The goal is to train a model that classifies English school exercises according to a hierarchical taxonomy with few labeled data. The experiments made in this work employ different machine learning models and text representation techniques: CatBoost with tf-idf features, classifiers based on pre-trained models (mBERT, LASER), and SetFit, a framework for few-shot text classification. SetFit proved to be the most promising approach, achieving better performance when during training only a few labeled examples per class are available. However, this thesis does not consider all the hierarchical taxonomy, but only the first two levels: to address classification with the classes at the third level further experiments should be carried out, exploring methods for zero-shot text classification, data augmentation, and strategies to exploit the hierarchical structure of the taxonomy during training.

Doehlert Design-desirability Function Multi-criteria Optimal Separation Of 17 Phenolic Compounds From Extra-virgin Olive Oil By Capillary Zone Electrophoresis.

In Brazil, the consumption of extra-virgin olive oil (EVOO) is increasing annually, but there are no experimental studies concerning the phenolic compound contents of commercial EVOO. The aim of this work was to optimise the separation of 17 phenolic compounds already detected in EVOO. A Doehlert matrix experimental design was used, evaluating the effects of pH and electrolyte concentration. Resolution, runtime and migration time relative standard deviation values were evaluated. Derringer's desirability function was used to simultaneously optimise all 37 responses. The 17 peaks were separated in 19min using a fused-silica capillary (50μm internal diameter, 72cm of effective length) with an extended light path and 101.3mmolL(-1) of boric acid electrolyte (pH 9.15, 30kV). The method was validated and applied to 15 EVOO samples found in Brazilian supermarkets.

Behavior Of Listeria Monocytogenes In A Multi-species Biofilm With Enterococcus Faecalis And Enterococcus Faecium And Control Through Sanitation Procedures.

The formation of mono-species biofilm (Listeria monocytogenes) and multi-species biofilms (Enterococcus faecium, Enterococcus faecalis, and L. monocytogenes) was evaluated. In addition, the effectiveness of sanitation procedures for the control of the multi-species biofilm also was evaluated. The biofilms were grown on stainless steel coupons at various incubation temperatures (7, 25 and 39°C) and contact times (0, 1, 2, 4, 6 and 8days). In all tests, at 7°C, the microbial counts were below 0.4 log CFU/cm(2) and not characteristic of biofilms. In mono-species biofilm, the counts of L. monocytogenes after 8days of contact were 4.1 and 2.8 log CFU/cm(2) at 25 and 39°C, respectively. In the multi-species biofilms, Enterococcus spp. were present at counts of 8 log CFU/cm(2) at 25 and 39°C after 8days of contact. However, the L. monocytogenes in multi-species biofilms was significantly affected by the presence of Enterococcus spp. and by temperature. At 25°C, the growth of L. monocytogenes biofilms was favored in multi-species cultures, with counts above 6 log CFU/cm(2) after 8days of contact. In contrast, at 39°C, a negative effect was observed for L. monocytogenes biofilm growth in mixed cultures, with a significant reduction in counts over time and values below 0.4 log CFU/cm(2) starting at day 4. Anionic tensioactive cleaning complemented with another procedure (acid cleaning, disinfection or acid cleaning+disinfection) eliminated the multi-species biofilms under all conditions tested (counts of all micro-organisms<0.4 log CFU/cm(2)). Peracetic acid was the most effective disinfectant, eliminating the multi-species biofilms under all tested conditions (counts of the all microorganisms <0.4 log CFU/cm(2)). In contrast, biguanide was the least effective disinfectant, failing to eliminate biofilms under all the test conditions.