Sample Size vs. Bias in Defect Prediction

Most empirical disciplines promote the reuse and sharing of datasets, as it leads to greater possibility of replication. While this is increasingly the case in Empirical Software Engineering, some of the most popular bug-fix datasets are now known to be biased. This raises two significants concerns: first, that sample bias may lead to underperforming prediction models, and second, that the external validity of the studies based on biased datasets may be suspect. This issue has raised considerable consternation in the ESE literature in recent years. However, there is a confounding factor of these datasets that has not been examined carefully: size. Biased datasets are sampling only some of the data that could be sampled, and doing so in a biased fashion; but biased samples could be smaller, or larger. Smaller data sets in general provide less reliable bases for estimating models, and thus could lead to inferior model performance. In this setting, we ask the question, what affects performance more? bias, or size? We conduct a detailed, large-scale meta-analysis, using simulated datasets sampled with bias from a high-quality dataset which is relatively free of bias. Our results suggest that size always matters just as much bias direction, and in fact much more than bias direction when considering information-retrieval measures such as AUC and F-score. This indicates that at least for prediction models, even when dealing with sampling bias, simply finding larger samples can sometimes be sufficient. Our analysis also exposes the complexity of the bias issue, and raises further issues to be explored in the future.

Estudio acústico del auditorio Rafael Frühbeck, Burgos

El presente proyecto desarrolla un estudio acústico del recinto: Auditorio Rafael Frühbeck de Burgos, cumpliendo con las indicaciones exigidas por la norma UNE-EN ISO 3382-1: 2010, “Medición de parámetros acústicos en recintos, Parte 1: Salas de Espectáculos”. Se desarrollan dos estudios acústicos sobre el mismo recinto. En el primero de ellos, el recinto está configurado para la realización de eventos tales como conferencias o congresos, donde la inteligibilidad de la palabra es un factor determinante. En el segundo estudio, el recinto se configura para espectáculos musicales como conciertos de orquesta sinfónica o música de cámara. En esta configuración, la palabra ya no es tan determinante como la correcta interpretación y disfrute de la música por parte de la audiencia. Para ambas configuraciones del recinto se ha realizado un procesado estadístico de los datos con el fin de obtener un valor único de cada parámetro acústico estudiado. De esta forma, se comparan los resultados para ambas configuraciones, y se evalúan los valores obtenidos de cada uno de los parámetros acústicos con el fin de conocer si se adecuan a las necesidades acústicas exigidas por el tipo de evento desarrollado. Además, se ha construido un modelo geométrico del recinto por ordenador, para ambas configuraciones acústicas, haciendo uso del software profesional de predicción y simulación acústica EASE. Se realiza un estudio acústico sobre el modelo geométrico mediante simulación, siguiendo las pautas llevadas a cabo durante la medición “in situ”. Los resultados obtenidos por simulación se comparan con los obtenidos de las mediciones “in situ”, para estudiar la validación del modelo geométrico. El parámetro acústico elegido para validar el modelo, en un primer momento, será el tiempo de reverberación. Si se consigue una buena validación del modelo geométrico, este puede ser utilizado para realizar predicciones acústicas mediante simulación, cuando un sistema de refuerzo sonoro sea utilizado dentro del recinto. El sistema de refuerzo sonoro ubicado en el recinto sometido a estudio, no ha sido utilizado en el presente proyecto. ABSTRACT. The present projects carry out an acoustic study of enclosure: Rafael Frühbeck Concert Hall, in Burgos, fulfilling the indications demanded by the standard UNE-EN ISO 3382-1:2010 “Measurement of room acoustic parameters – Part 1: Performance spaces. Two acoustics studies are developed on the same enclosure. In first of them, the enclosure is formed for the accomplishment of events such as conferences or congresses, where speech intelligibility is a determining factor. In the second study, the enclosure forms for musical performances like concerts of symphony orchestra or chamber music. In this acoustic configuration, speech intelligibility is not as determining as the correct interpretation and enjoyment of music in audience areas. For both configurations of the enclosure, a statistical processing of the data has been realised with the purpose of obtaining a unique value of each studied acoustic parameter. In this way, the results for both configurations are compared, and the obtained values of each one of the acoustic parameters are evaluated with the purpose of knowing if they are adapted to the acoustic needs demanded by the type of developed event. In addition, a geometric model of the enclosure has been constructed by computer, for both acoustic configurations; making use of the professional software of prediction and acoustical simulation, EASE. An acoustic study is developed on the geometric model by means of simulation, following the rules carried out during the measurement “in situ”. The results obtained by simulation are compared with the obtained ones from the measurement “in situ”, to study the validation of the geometric model. Initially the acoustic parameter chosen to validate the model is Reverberation Time. If a good validation of the geometric model is reached, it can be used to realize acoustic predictions by simulation, when a sound reinforcement system is used within the enclosure. The sound reinforcement system located in the enclosure under study has not been used in the present project.

Tractor PT: A traction prediction software for agricultural tractor

Traction prediction modelling, a key factor in farm tractor design, has been driven by the need to find the answer to this question without having to build physical prototypes. A wide range of theories and their respective algorithms can be used in such predictions. The “Tractors and Tillage” research team at the Polytechnic University of Madrid, which engages, among others, in traction prediction for farm tractors, has developed a series of programs based on the cone index as the parameter representative of the terrain. With the software introduced in the present paper, written in Visual Basic, slip can be predicted in two- and four-wheel drive tractors using any one of four models. It includes databases for tractors, front tyres, rear tyres and working conditions (soil cone index and drawbar pull exerted). The results can be exported in spreadsheet format.

Advanced predictive quality control strategy involving different facilities

There are many industries that use highly technological solutions to improve quality in all of their products. The steel industry is one example. Several automatic surface-inspection systems are used in the steel industry to identify various types of defects and to help operators decide whether to accept, reroute, or downgrade the material, subject to the assessment process. This paper focuses on promoting a strategy that considers all defects in an integrated fashion. It does this by managing the uncertainty about the exact position of a defect due to different process conditions by means of Gaussian additive influence functions. The relevance of the approach is in making possible consistency and reliability between surface inspection systems. The results obtained are an increase in confidence in the automatic inspection system and an ability to introduce improved prediction and advanced routing models. The prediction is provided to technical operators to help them in their decision-making process. It shows the increase in improvement gained by reducing the 40 % of coils that are downgraded at the hot strip mill because of specific defects. In addition, this technology facilitates an increase of 50 % in the accuracy of the estimate of defect survival after the cleaning facility in comparison to the former approach. The proposed technology is implemented by means of software-based, multi-agent solutions. It makes possible the independent treatment of information, presentation, quality analysis, and other relevant functions.

Estudio de la fiabilidad software, según defectos, en controladores de velocidad para robots autónomos móviles

Tanto los robots autónomos móviles como los robots móviles remotamente operados se utilizan con éxito actualmente en un gran número de ámbitos, algunos de los cuales son tan dispares como la limpieza en el hogar, movimiento de productos en almacenes o la exploración espacial. Sin embargo, es difícil garantizar la ausencia de defectos en los programas que controlan dichos dispositivos, al igual que ocurre en otros sectores informáticos. Existen diferentes alternativas para medir la calidad de un sistema en el desempeño de las funciones para las que fue diseñado, siendo una de ellas la fiabilidad. En el caso de la mayoría de los sistemas físicos se detecta una degradación en la fiabilidad a medida que el sistema envejece. Esto es debido generalmente a efectos de desgaste. En el caso de los sistemas software esto no suele ocurrir, ya que los defectos que existen en ellos generalmente no han sido adquiridos con el paso del tiempo, sino que han sido insertados en el proceso de desarrollo de los mismos. Si dentro del proceso de generación de un sistema software se focaliza la atención en la etapa de codificación, podría plantearse un estudio que tratara de determinar la fiabilidad de distintos algoritmos, válidos para desempeñar el mismo cometido, según los posibles defectos que pudieran introducir los programadores. Este estudio básico podría tener diferentes aplicaciones, como por ejemplo elegir el algoritmo menos sensible a los defectos, para el desarrollo de un sistema crítico o establecer procedimientos de verificación y validación, más exigentes, si existe la necesidad de utilizar un algoritmo que tenga una alta sensibilidad a los defectos. En el presente trabajo de investigación se ha estudiado la influencia que tienen determinados tipos de defectos software en la fiabilidad de tres controladores de velocidad multivariable (PID, Fuzzy y LQR) al actuar en un robot móvil específico. La hipótesis planteada es que los controladores estudiados ofrecen distinta fiabilidad al verse afectados por similares patrones de defectos, lo cual ha sido confirmado por los resultados obtenidos. Desde el punto de vista de la planificación experimental, en primer lugar se realizaron los ensayos necesarios para determinar si los controladores de una misma familia (PID, Fuzzy o LQR) ofrecían una fiabilidad similar, bajo las mismas condiciones experimentales. Una vez confirmado este extremo, se eligió de forma aleatoria un representante de clase de cada familia de controladores, para efectuar una batería de pruebas más exhaustiva, con el objeto de obtener datos que permitieran comparar de una forma más completa la fiabilidad de los controladores bajo estudio. Ante la imposibilidad de realizar un elevado número de pruebas con un robot real, así como para evitar daños en un dispositivo que generalmente tiene un coste significativo, ha sido necesario construir un simulador multicomputador del robot. Dicho simulador ha sido utilizado tanto en las actividades de obtención de controladores bien ajustados, como en la realización de los diferentes ensayos necesarios para el experimento de fiabilidad. ABSTRACT Autonomous mobile robots and remotely operated robots are used successfully in very diverse scenarios, such as home cleaning, movement of goods in warehouses or space exploration. However, it is difficult to ensure the absence of defects in programs controlling these devices, as it happens in most computer sectors. There exist different quality measures of a system when performing the functions for which it was designed, among them, reliability. For most physical systems, a degradation occurs as the system ages. This is generally due to the wear effect. In software systems, this does not usually happen, and defects often come from system development and not from use. Let us assume that we focus on the coding stage in the software development pro¬cess. We could consider a study to find out the reliability of different and equally valid algorithms, taking into account any flaws that programmers may introduce. This basic study may have several applications, such as choosing the algorithm less sensitive to pro¬gramming defects for the development of a critical system. We could also establish more demanding procedures for verification and validation if we need an algorithm with high sensitivity to programming defects. In this thesis, we studied the influence of certain types of software defects in the reliability of three multivariable speed controllers (PID, Fuzzy and LQR) designed to work in a specific mobile robot. The hypothesis is that similar defect patterns affect differently the reliability of controllers, and it has been confirmed by the results. From the viewpoint of experimental planning, we followed these steps. First, we conducted the necessary test to determine if controllers of the same family (PID, Fuzzy or LQR) offered a similar reliability under the same experimental conditions. Then, a class representative was chosen at ramdom within each controller family to perform a more comprehensive test set, with the purpose of getting data to compare more extensively the reliability of the controllers under study. The impossibility of performing a large number of tests with a real robot and the need to prevent the damage of a device with a significant cost, lead us to construct a multicomputer robot simulator. This simulator has been used to obtain well adjusted controllers and to carry out the required reliability experiments.

Software tool for simulation of aircraft trajectories

New concepts in air navigation have been introduced recently. Among others, are the concepts of trajectory optimization, 4D trajectories, RBT (Reference Business Trajectory), TBO (trajectory based operations), CDA (Continuous Descent Approach) and ACDA (Advanced CDA), conflict resolution, arrival time (AMAN), introduction of new aircraft (UAVs, UASs) in air space, etc. Although some of these concepts are new, the future Air Traffic Management will maintain the four ATM key performance areas such as Safety, Capacity, Efficiency, and Environmental impact. So much, the performance of the ATM system is directly related to the accuracy with which the future evolution of the traffic can be predicted. In this sense, future air traffic management will require a variety of support tools to provide suitable help to users and engineers involved in the air space management. Most of these tools are based on an appropriate trajectory prediction module as main component. Therefore, the purposes of these tools are related with testing and evaluation of any air navigation concept before they become fully operative. The aim of this paper is to provide an overview to the design of a software tool useful to estimate aircraft trajectories adapted to air navigation concepts. Other usage of the tool, like controller design, vertical navigation assessment, procedures validation and hardware and software in the loop are available in the software tool. The paper will show the process followed to design the tool, the software modules needed to perform accurately and the process followed to validate the output data.

Estudio de una metodología de software para la predicción y análisis del balance paramétrico de buques para su aplicación en los criterios de estabilidad de segunda generación

El objetivo de ésta tesis es estudiar cómo desarrollar una aplicación informática que implemente algoritmos numéricos de evaluación de características hidrodinámicas de modelos geométricos representativos de carenas de buques. Se trata de especificar los requisitos necesarios que debe cumplir un programa para informático orientado a dar solución a un determinado problema hidródinámico, como es simular el comportamiento en balance de un buque sometido a oleaje, de popa o proa. una vez especificada la aplicación se realizará un diseño del programa; se estudiarán alternativas para implementar la aplicación; se explicará el proceso que ha de seguirse para obtener la aplicación en funcionamiento y se contrastarán los resultados obtenidos en la medida que sea posible. Se pretende sistematizar y sintetizar todo el proceso de desarrollo de software, orientado a la simulación del comportamiento hidrodinámico de un buque, en una metodología que se pondrá a disposición de la comunidad académica y científica en la forma que se considere más adecuada. Se trata, por tanto, de proponer una metodología de desarrollo de software para obetener una aplicación que facilite la evaluación de diferentes alternativas de estudio variando parámetros relativos al problema en estudio y que sea capaz de proporcionar resultados para su análisis. Así mismo se incide en cómo ha de conducirse en el proceso para que dicha aplicación pueda crecer, incorporando soluciones existentes no implementadas o nuevas soluciones que aparezcan en este ámbito de conocimiento. Como aplicación concreta de la aplicación se ha elegido implementar los algoritmos necesarios para evaluar la aparición del balance paramétrico en un buque. En el análisis de éste problema se considera de interés la representación geométrica que se hace de la carena del buque. Además de la carena aparecen otros elementos que tienen influencia determinante en éste estudio, como son las situación de mar y las situaciones de carga. Idealmente, el problema sería resuelto si se consiguiera determinar el ángulo de balance que se produce al enfrentar un buque a las diferentes condiciones de mar. Se pretende preparar un programa utilizando el paradigma de la orientación a objetos. Considero que es la más adecuada forma de modularizar el programa para poder utilizar diferentes modelos de una misma carena y así comparar los resultados de la evaluación del balance paramétrico entre sí. En una etapa posterior se podrían comparar los resultados con otros obtenidos empíricamente. Hablo de una nueva metodología porque pretendo indicar cómo se ha de construir una aplicación de software que sea usable y sobre la que se pueda seguir desarrollando. Esto justifica la selección del lenguaje de programación C++. Se seleccionará un núcleo geométrico de software que permita acoplar de forma versátil los distintos componentes de software que van a construir el programa. Este trabajo pretende aplicar el desarrollo de software a un aspecto concreto del área de conocimiento de la hidrodinámica. No se pretende aportar nuevos algoritmos para resolver problemas de hidrodinámica, sino diseñar un conjunto de objetos de software que implementen soluciones existentes a conocidas soluciones numéricas a dichos problemas. Se trata fundamentalmente de un trabajo de software, más que de hidrodinámica. Lo que aporta de novedad es una nueva forma de realizar un programa aplicado a los cálculos hidrodinámicos relativos a la determinación del balance paramétrico, que pueda crecer e incorporar cualquier novedad que pueda surgir más adelante. Esto será posible por la programación modular utilizada y los objetos que representan cada uno de los elementos que intervienen en la determinación del balance paramétrico. La elección de aplicar la metodología a la predicción del balance paramétrico se debe a que este concepto es uno de los elementos que intervienen en la evaluación de criterios de estabilidad de segunda generación que estan en estudio para su futura aplicación en el ámbito de la construcción naval. Es por tanto un estudio que despierta interés por su próxima utilidad. ABSTRACT The aim of this thesis is to study how to develop a computer application implementing numerical algorithms to assess hydrodynamic features of geometrical models of vessels. It is therefore to propose a methodology for software development applied to an hydrodynamic problem, in order to evaluate different study alternatives by varying different parameters related to the problem and to be capable of providing results for analysis. As a concrete application of the program it has been chosen to implement the algorithms necessary for evaluating the appearance of parametric rolling in a vessel. In the analysis of this problem it is considered of interest the geometrical representation of the hull of the ship and other elements which have decisive influence in this phenomena, such as the sea situation and the loading condition. Ideally, the application would determine the roll angle that occurs when a ship is on waves of different characteristics. It aims to prepare a program by using the paradigm of object oriented programming. I think it is the best methodology to modularize the program. My intention is to show how face the global process of developing an application from the initial specification until the final release of the program. The process will keep in mind the spefici objetives of usability and the possibility of growing in the scope of the software. This work intends to apply software development to a particular aspect the area of knowledge of hydrodynamics. It is not intended to provide new algorithms for solving problems of hydrodynamics, but designing a set of software objects that implement existing solutions to these problems. This is essentially a job software rather than hydrodynamic. The novelty of this thesis stands in this work focuses in describing how to apply the whole proccess of software engineering to hydrodinamics problems. The choice of the prediction of parametric balance as the main objetive to be applied to is because this concept is one of the elements involved in the evaluation of the intact stability criteria of second generation. Therefore, I consider this study as relevant usefull for the future application in the field of shipbuilding.

Building intuition of iron evolution during solar cell processing through analysis of different process models

An important aspect of Process Simulators for photovoltaics is prediction of defect evolution during device fabrication. Over the last twenty years, these tools have accelerated process optimization, and several Process Simulators for iron, a ubiquitous and deleterious impurity in silicon, have been developed. The diversity of these tools can make it difficult to build intuition about the physics governing iron behavior during processing. Thus, in one unified software environment and using self-consistent terminology, we combine and describe three of these Simulators. We vary structural defect distribution and iron precipitation equations to create eight distinct Models, which we then use to simulate different stages of processing. We find that the structural defect distribution influences the final interstitial iron concentration ([Fe-i]) more strongly than the iron precipitation equations. We identify two regimes of iron behavior: (1) diffusivity-limited, in which iron evolution is kinetically limited and bulk [Fe-i] predictions can vary by an order of magnitude or more, and (2) solubility-limited, in which iron evolution is near thermodynamic equilibrium and the Models yield similar results. This rigorous analysis provides new intuition that can inform Process Simulation, material, and process development, and it enables scientists and engineers to choose an appropriate level of Model complexity based on wafer type and quality, processing conditions, and available computation time.