Optimal Data Distributions in Machine Learning

In the first part of the thesis we explore three fundamental questions that arise naturally when we conceive a machine learning scenario where the training and test distributions can differ. Contrary to conventional wisdom, we show that in fact mismatched training and test distribution can yield better out-of-sample performance. This optimal performance can be obtained by training with the dual distribution. This optimal training distribution depends on the test distribution set by the problem, but not on the target function that we want to learn. We show how to obtain this distribution in both discrete and continuous input spaces, as well as how to approximate it in a practical scenario. Benefits of using this distribution are exemplified in both synthetic and real data sets.

In order to apply the dual distribution in the supervised learning scenario where the training data set is fixed, it is necessary to use weights to make the sample appear as if it came from the dual distribution. We explore the negative effect that weighting a sample can have. The theoretical decomposition of the use of weights regarding its effect on the out-of-sample error is easy to understand but not actionable in practice, as the quantities involved cannot be computed. Hence, we propose the Targeted Weighting algorithm that determines if, for a given set of weights, the out-of-sample performance will improve or not in a practical setting. This is necessary as the setting assumes there are no labeled points distributed according to the test distribution, only unlabeled samples.

Finally, we propose a new class of matching algorithms that can be used to match the training set to a desired distribution, such as the dual distribution (or the test distribution). These algorithms can be applied to very large datasets, and we show how they lead to improved performance in a large real dataset such as the Netflix dataset. Their computational complexity is the main reason for their advantage over previous algorithms proposed in the covariate shift literature.

In the second part of the thesis we apply Machine Learning to the problem of behavior recognition. We develop a specific behavior classifier to study fly aggression, and we develop a system that allows analyzing behavior in videos of animals, with minimal supervision. The system, which we call CUBA (Caltech Unsupervised Behavior Analysis), allows detecting movemes, actions, and stories from time series describing the position of animals in videos. The method summarizes the data, as well as it provides biologists with a mathematical tool to test new hypotheses. Other benefits of CUBA include finding classifiers for specific behaviors without the need for annotation, as well as providing means to discriminate groups of animals, for example, according to their genetic line.

Even aberration measurement of lithographic projection system based on optimized phase-shifting marks

In the present paper, we propose a novel method for measuring the even aberrations of lithographic projection optics by use of optimized phase-shifting marks on the test mask. The line/space ratio of the phase-shifting marks is optimized to obtain the maximum sensitivities of Zernike coefficients corresponding to even aberrations. Spherical aberration and astigmatism can be calculated from the focus shifts of phase-shifting gratings oriented at 0 degrees, 45 degrees, 90 degrees and 135 degrees at multiple illumination settings. The PROLITH simulation results show that, the measurement accuracy of spherical aberration and astigmatism obviously increase, after the optimization of the measurement mark. (C) 2008 Elsevier B.V. All rights reserved.

System error removal and structure optimization based on Fizeau interferometer

In the process of interferometric testing, the measurement result is influenced by the system structure, which reduces the measurement accuracy. To obtain an accurate test result, it is necessary to analyze the test system, and build the relationship between the measurement error and the system parameters. In this paper, the influences of the system elements which include the collimated lens and the standard surface on the interferometric testing are analyzed, the expressions of phase distribution and wavefront error on the detector are obtained, the method to remove some element errors is introduced, and the optimization structure relationships are given. (C) 2006 Elsevier GmbH. All rights reserved.

Length-weight relationship of marine fish species off Sao Sebastiao system, Sao Paulo, Southeastern Brazil

The parameters a and b of the length-weight relationship of the form W = a . L super(b) were estimated for 57 fish species sampled in Sao Sebastiao Channel and shelf system in 1997, Sao Paulo, Brazil. The b values ranged from 2.746 to 3.617. The Student's t-test revealed that mot (44) species had b values significantly different from 3. A normal distribution of the calculated LWR exponents (b) was obtained.

A High-Speed Fish Evisceration System (FES) for Bycatch and Underutilized Fish Stocks

Development of a high-speed and high-yield water-powered fish evisceration system (FES) to efficiently preprocess small fish and bycatch for producing minced fish meat is described. The concept of the system is propelling fish in a stream of water through an arrangement of cutting blades and brushes. Eviscerated fish are separated from the viscera and water stream in a dual screen rotary sieve. The FES processed head off fish, weighing 170–500 g, at the rate of 300 fish/min when used with an automatic heading machine. Yields of mince produced from walleye pollock, Theragra chalcogramma; and Pacific whiting, Merluccius productus; processed by the FES ranged between 43% and 58%. The maximum yield of minced muscle from fish weighing over 250 g was 52%, and the yield of 250 g was 58%. Test results indicated that surimi made from minced meat recovered from fish processed with the FES was comparable in quality to commercial grade surimi from conventional systems. Redesigned for commercial operation in the Faeroe Islands (Denmark), the system effectively processed North Atlantic blue whiting, Micromesistius poutassou, with an average weight of 110 g at a constant rate of 500–600 fish/min, producing deboned mince feeding a surimi processing line at a rate of 2.0 t/h. Yields of mince ranged from 55% to 63% from round fish. Surimi made from the blue whiting mince meat produced by the FES was comparable to surimi commercially produced from blue whiting by Norway and France and sold into European markets.

A Test of Computer-assisted Matching Using the North Pacific Humpback Whale, Megaptera novaeangliae, Tail Flukes Photograph Collection

Testing was conducted of a computer-assisted system for matching humpback whale tail flukes photographs. Trials with a 12,000-photographs database found no differences in match success between matching by computer and matching by comparing smaller catalogs ranging in size from 200 to 400 photographs. Tests with a 24,000-photographs database showed that, on average, the first match was found after examining about 130 photographs whether the photograph quality was excellent, good, or poor. Match success did not appear to be strongly related to whether the tail flukes had especially distinctive markings or pigment patterns (recognition quality). An advantage of computer-assisted matching is the ability to compare new photographs to the entire North Pacific collection, where no bias is introduced based on expectation of resightings within or between specific areas, or based on expectation of behavioral role (e.g. matching “known” females to “known” females).

Monitoring Hawaii’s marine protected areas: examining spatial and temporal trends using a seascape approach

Hawaii’s coastal marine resources have declined dramatically over the past 100 years due to multiple anthropogenic stressors including overfishing, coastal development, pollution, overuse, invasive species and climate change. It is now becoming evident that ecosystem-based management, in the form of marine protected areas (MPAs), is necessary to conserve biodiversity, maintain viable fisheries, and deliver a broad suite of ecosystem services. Over the past four decades, Hawaii has developed a system of MPAs to conserve and replenish marine resources around the state. These Marine Life Conservation Districts (MLCDs) vary in size, habitat quality, and management regimes, providing an excellent opportunity to test hypotheses concerning MPA design and function using multiple discreet sampling units. NOAA/NOS/NCCOS/Center for Coastal Monitoring and Assessment’s Biogeography Branch used digital benthic habitat maps coupled with comprehensive ecological studies between 2002 and 2004 to evaluate the efficacy of all existing MLCDs using a spatially-explicit stratified random sampling design. The results from this work have shown that areas fully protected from fishing had higher fish biomass, larger overall fish size, and higher biodiversity than adjacent areas of similar habitat quality. Other key findings demonstrated that top predators and other important fisheries species were more abundant and larger in the MPAs, illustrating the effectiveness of these closures in conserving these populations. Habitat complexity, protected area size and habitat diversity were the major factors in determining effectiveness among MPAs.

Semi-intensive shrimp farming system of Kerala, India: a production function analysis

An analysis of the factor-product relationship in the semi-intensive shrimp farming system of Kerala, farm basis and hectare basis, we are attempted and the results reported in this paper. The Cobb-Douglas model, in which the physical relationship between input and output is estimated, and the marginal analysis then employed to evaluate the producer behaviour, was used for the analysis. The study was based on empirical data collected during November 1994 to May 1996, covering three seasons, from 21 farms spread over Alappuzha, Ernakulam and Kasaragod districts of the state. The sample covered a total area of 61.06 ha. Of the 11 explanatory variables considered in the model, the size of the farm, casual labour and chemical fertilizers were found statistically significant. It was also observed that the factors such as age of pond, experience of the farmer, feed, miscellaneous costs, number of seed stocked and skilled labour contributed positively to the output. The estimated industry production function exhibited unitary economies of scale. The estimated mean output was 3937 kg/ha. The test of multi-co-linearity showed that there is no problem of dominant variable. On the basis of the marginal product and the given input-output prices, the optimum amounts of seed, feed and casual labour were estimated. They were about 97139 seed, 959 kg of feed and 585 man-days of casual labour per farm. This indicated the need for reducing the stocking density and amount of feed from the present levels, in order to maximise profit. Based on the finding of the study, suggestions for improving the industry production function are proposed.

Using Level Sets as the basis for a scalable, parallel geometry engine and mesh generation system

The background to this review paper is research we have performed over recent years aimed at developing a simulation system capable of handling large scale, real world applications implemented in an end-to-end parallel, scalable manner. The particular focus of this paper is the use of a Level Set solid modeling geometry kernel within this parallel framework to enable automated design optimization without topological restrictions and on geometries of arbitrary complexity. Also described is another interesting application of Level Sets: their use in guiding the export of a body-conformal mesh from our basic cut-Cartesian background octree - mesh - this permits third party flow solvers to be deployed. As a practical demonstrations meshes of guaranteed quality are generated and flow-solved for a B747 in full landing configuration and an automated optimization is performed on a cooled turbine tip geometry. Copyright © 2009 by W.N.Dawes.

An advanced multi-configuration stator well cooling test facility

Optimisation of cooling systems within gas turbine engines is of great interest to engine manufacturers seeking gains in performance, efficiency and component life. The effectiveness of coolant delivery is governed by complex flows within the stator wells and the interaction of main annulus and cooling air in the vicinity of the rim seals. This paper reports the development of a test facility which allows the interaction of cooling air and main gas paths to be measured at conditions representative of those found in modern gas turbine engines. The test facility features a two stage turbine with an overall pressure ratio of approximately 2.6:1. Hot air is supplied to the main annulus using a Rolls-Royce Dart compressor driven by an aero-derivative engine plant. Cooling air can be delivered to the stator wells at multiple locations and at a range of flow rates which cover bulk ingestion through to bulk egress. The facility has been designed with adaptable geometry to enable rapid changes of cooling air path configuration. The coolant delivery system allows swift and accurate changes to the flow settings such that thermal transients may be performed. Particular attention has been focused on obtaining high accuracy data, using a radio telemetry system, as well as thorough through-calibration practices. Temperature measurements can now be made on both rotating and stationary discs with a long term uncertainty in the region of 0.3 K. A gas concentration measurement system has also been developed to obtain direct measurement of re-ingestion and rim seal exchange flows. High resolution displacement sensors have been installed in order to measure hot running geometry. This paper documents the commissioning of a test facility which is unique in terms of rapid configuration changes, non-dimensional engine matching and the instrumentation density and resolution. Example data for each of the measurement systems is presented. This includes the effect of coolant flow rate on the metal temperatures within the upstream cavity of the turbine stator well, the axial displacement of the rotor assembly during a commissioning test, and the effect of coolant flow rate on mixing in the downstream cavity of the stator well. Copyright © 2010 by ASME.

Impacts of concurrency, iteration, design review, and problem complexity on design project lead time and error generation

A case study of an aircraft engine manufacturer is used to analyze the effects of management levers on the lead time and design errors generated in an iteration-intensive concurrent engineering process. The levers considered are amount of design-space exploration iteration, degree of process concurrency, and timing of design reviews. Simulation is used to show how the ideal combination of these levers can vary with changes in design problem complexity, which can increase, for instance, when novel technology is incorporated in a design. Results confirm that it is important to consider multiple iteration-influencing factors and their interdependencies to understand concurrent processes, because the factors can interact with confounding effects. The article also demonstrates a new approach to derive a system dynamics model from a process task network. The new approach could be applied to analyze other concurrent engineering scenarios. © The Author(s) 2012.

Spoken Dialog Challenge 2010: Comparison of live and control test results

The Spoken Dialog Challenge 2010 was an exercise to investigate how different spoken dialog systems perform on the same task. The existing Let's Go Pittsburgh Bus Information System was used as a task and four teams provided systems that were first tested in controlled conditions with speech researchers as users. The three most stable systems were then deployed to real callers. This paper presents the results of the live tests, and compares them with the control test results. Results show considerable variation both between systems and between the control and live tests. Interestingly, relatively high task completion for controlled tests did not always predict relatively high task completion for live tests. Moreover, even though the systems were quite different in their designs, we saw very similar correlations between word error rate and task completion for all the systems. The dialog data collected is available to the research community. © 2011 Association for Computational Linguistics.

Auto-calibration of a camera system using Image Alignment

Calibration of a camera system is a necessary step in any stereo metric process. It correlates all cameras to a common coordinate system by measuring the intrinsic and extrinsic parameters of each camera. Currently, manual calibration of a camera system is the only way to achieve calibration in civil engineering operations that require stereo metric processes (photogrammetry, videogrammetry, vision based asset tracking, etc). This type of calibration however is time-consuming and labor-intensive. Furthermore, in civil engineering operations, camera systems are exposed to open, busy sites. In these conditions, the position of presumably stationary cameras can easily be changed due to external factors such as wind, vibrations or due to an unintentional push/touch from personnel on site. In such cases manual calibration must be repeated. In order to address this issue, several self-calibration algorithms have been proposed. These algorithms use Projective Geometry, Absolute Conic and Kruppa Equations and variations of these to produce processes that achieve calibration. However, most of these methods do not consider all constraints of a camera system such as camera intrinsic constraints, scene constraints, camera motion or varying camera intrinsic properties. This paper presents a novel method that takes all constraints into consideration to auto-calibrate cameras using an image alignment algorithm originally meant for vision based tracking. In this method, image frames are taken from cameras. These frames are used to calculate the fundamental matrix that gives epipolar constraints. Intrinsic and extrinsic properties of cameras are acquired from this calculation. Test results are presented in this paper with recommendations for further improvement.

A holographic projection system with an electrically tuning and continuously adjustable optical zoom

A holographic projection system with optical zoom is demonstrated. By using a combination of a LC lens and an encoded Fresnel lens on the LCoS panel, we can control zoom in a holographic projector. The magnification can be electrically adjusted by tuning the focal length of the combination of the two lenses. The zoom ratio of the holographic projection system can reach 3.7:1 with continuous zoom function. The optical zoom function can decrease the complexity of the holographic projection system.

Demonstration of Improved Passive UHF RFID Coverage using Optically-Fed Distributed Multi-Antenna System

Optically-fed distributed antenna system (DAS) technology is combined with passive ultra high frequency (UHF) radio frequency identification (RFID). It is shown that RFID signals can be carried on directly modulated radio over fiber links without impacting their performance. It is also shown that a multi-antenna DAS can greatly reduce the number of nulls experienced by RFID in a complex radio environment, increasing the likelihood of successful tag detection. Consequently, optimization of the DAS reduces nulls further. We demonstrate RFID tag reading using a three antenna DAS system over a 20mx6m area, limited by building constraints, where 100% of the test points can be successfully read. The detected signal strength from the tag is also observed to increase by an average of approximately 10dB compared with a conventional switched multi-antenna RFID system. This improvement is achieved at +31dBm equivalent isotropically radiated power (EIRP) from all three antenna units (AUs).