Automatic Food Intake Assessment Using Camera Phones

Obesity is becoming an epidemic phenomenon in most developed countries. The fundamental cause of obesity and overweight is an energy imbalance between calories consumed and calories expended. It is essential to monitor everyday food intake for obesity prevention and management. Existing dietary assessment methods usually require manually recording and recall of food types and portions. Accuracy of the results largely relies on many uncertain factors such as user's memory, food knowledge, and portion estimations. As a result, the accuracy is often compromised. Accurate and convenient dietary assessment methods are still blank and needed in both population and research societies. In this thesis, an automatic food intake assessment method using cameras, inertial measurement units (IMUs) on smart phones was developed to help people foster a healthy life style. With this method, users use their smart phones before and after a meal to capture images or videos around the meal. The smart phone will recognize food items and calculate the volume of the food consumed and provide the results to users. The technical objective is to explore the feasibility of image based food recognition and image based volume estimation. This thesis comprises five publications that address four specific goals of this work: (1) to develop a prototype system with existing methods to review the literature methods, find their drawbacks and explore the feasibility to develop novel methods; (2) based on the prototype system, to investigate new food classification methods to improve the recognition accuracy to a field application level; (3) to design indexing methods for large-scale image database to facilitate the development of new food image recognition and retrieval algorithms; (4) to develop novel convenient and accurate food volume estimation methods using only smart phones with cameras and IMUs. A prototype system was implemented to review existing methods. Image feature detector and descriptor were developed and a nearest neighbor classifier were implemented to classify food items. A reedit card marker method was introduced for metric scale 3D reconstruction and volume calculation. To increase recognition accuracy, novel multi-view food recognition algorithms were developed to recognize regular shape food items. To further increase the accuracy and make the algorithm applicable to arbitrary food items, new food features, new classifiers were designed. The efficiency of the algorithm was increased by means of developing novel image indexing method in large-scale image database. Finally, the volume calculation was enhanced through reducing the marker and introducing IMUs. Sensor fusion technique to combine measurements from cameras and IMUs were explored to infer the metric scale of the 3D model as well as reduce noises from these sensors.

Factors to consider in Micronesian food-based interventions: a case study of preventing Vitamin A deficiency

Background: Many factors need to be considered in a food-based intervention. Vitamin A deficiency and chronic diseases, such as diabetes, heart disease and cancer, have become serious problems in the Federated States of Micronesia (FSM) following the decreased production and consumption of locally grown foods. However, agricultural and social conditions are still favourable for local food production. Aim: To identify key factors to consider in a Micronesian food-based intervention focusing on increased production and consumption of four major Micronesian staple foods: banana, breadfruit, giant swamp taro and pandanus. Methods: Ethnographic methods including key informant interviews and a literature review. Results: Pacific and Micronesian values, concepts of food and disease, and food classifications differ sharply from Western concepts. There are few FSM professionals with nutrition expertise. Traditional foods and food cultivars vary in nutrient content, consumption level, cost, availability, status, convenience in growing, storing and cooking, and organoleptic factors. Conclusions: A systematic consideration of the factors that relate to a food-based intervention is critical to its success. The evaluation of which food and cultivar of that food that might be most effectively promoted is also critical. Regional differences, for example FSM inter-island differences between the staple foods and cultivars, must be considered carefully. The evaluation framework presented here may be relevant to Pacific island and other countries with similar foods where food-based interventions are being planned. An ethnographic approach was found to be essential in understanding the cultural context and in data collection and analysis.

Increasing consumption of ultra-processed foods and likely impact on human health: evidence from Brazil

Objective: To assess time trends in the contribution of processed foods to food purchases made by Brazilian households and to explore the potential impact on the overall quality of the diet. Design: Application of a new classification of foodstuffs based on extent and purpose of food processing to data collected by comparable probabilistic household budget surveys. The classification assigns foodstuffs to the following groups: unprocessed/minimally processed foods (Group 1); processed culinary ingredients (Group 2); or ultra-processed ready-to-eat or ready-to-heat food products (Group 3). Setting: Eleven metropolitan areas of Brazil. Subjects: Households; n 13 611 in 1987-8, n 16 014 in 1995-5 and n 13 848 in 2002-3. Results: Over the last three decades, the household consumption of Group 1 and Group 2 foods has been steadily replaced by consumption of Group 3 ultra-processed food products, both overall and in lower- and upper-income groups. In the 2002-3 survey, Group 3 items represented more than one-quarter of total energy (more than one-third for higher-income households). The overall nutrient profile of Group 3 items, compared with that of Group 1 and Group 2 items, revealed more added sugar, more saturated fat, more sodium, less fibre and much higher energy density. Conclusions: The high energy density and the unfavourable nutrition profiling of Group 3 food products, and also their potential harmful effects on eating and drinking behaviours, indicate that governments and health authorities should use all possible methods, including legislation and statutory regulation, to halt and reverse the replacement of minimally processed foods and processed culinary ingredients by ultra-processed food products.

Digital signal processing and artificial intelligence for the automated classification of food allergy

As a by-product of the ‘information revolution’ which is currently unfolding, lifetimes of man (and indeed computer) hours are being allocated for the automated and intelligent interpretation of data. This is particularly true in medical and clinical settings, where research into machine-assisted diagnosis of physiological conditions gains momentum daily. Of the conditions which have been addressed, however, automated classification of allergy has not been investigated, even though the numbers of allergic persons are rising, and undiagnosed allergies are most likely to elicit fatal consequences. On the basis of the observations of allergists who conduct oral food challenges (OFCs), activity-based analyses of allergy tests were performed. Algorithms were investigated and validated by a pilot study which verified that accelerometer-based inquiry of human movements is particularly well-suited for objective appraisal of activity. However, when these analyses were applied to OFCs, accelerometer-based investigations were found to provide very poor separation between allergic and non-allergic persons, and it was concluded that the avenues explored in this thesis are inadequate for the classification of allergy. Heart rate variability (HRV) analysis is known to provide very significant diagnostic information for many conditions. Owing to this, electrocardiograms (ECGs) were recorded during OFCs for the purpose of assessing the effect that allergy induces on HRV features. It was found that with appropriate analysis, excellent separation between allergic and nonallergic subjects can be obtained. These results were, however, obtained with manual QRS annotations, and these are not a viable methodology for real-time diagnostic applications. Even so, this was the first work which has categorically correlated changes in HRV features to the onset of allergic events, and manual annotations yield undeniable affirmation of this. Fostered by the successful results which were obtained with manual classifications, automatic QRS detection algorithms were investigated to facilitate the fully automated classification of allergy. The results which were obtained by this process are very promising. Most importantly, the work that is presented in this thesis did not obtain any false positive classifications. This is a most desirable result for OFC classification, as it allows complete confidence to be attributed to classifications of allergy. Furthermore, these results could be particularly advantageous in clinical settings, as machine-based classification can detect the onset of allergy which can allow for early termination of OFCs. Consequently, machine-based monitoring of OFCs has in this work been shown to possess the capacity to significantly and safely advance the current state of clinical art of allergy diagnosis

Development of thermo-analytical prediction and classification models for food in thermal devices using a multi sensor system and artificial neural networks

Die thermische Verarbeitung von Lebensmitteln beeinflusst deren Qualität und ernährungsphysiologischen Eigenschaften. Im Haushalt ist die Überwachung der Temperatur innerhalb des Lebensmittels sehr schwierig. Zudem ist das Wissen über optimale Temperatur- und Zeitparameter für die verschiedenen Speisen oft unzureichend. Die optimale Steuerung der thermischen Zubereitung ist maßgeblich abhängig von der Art des Lebensmittels und der äußeren und inneren Temperatureinwirkung während des Garvorgangs. Das Ziel der Arbeiten war die Entwicklung eines automatischen Backofens, der in der Lage ist, die Art des Lebensmittels zu erkennen und die Temperatur im Inneren des Lebensmittels während des Backens zu errechnen. Die für die Temperaturberechnung benötigten Daten wurden mit mehreren Sensoren erfasst. Hierzu kam ein Infrarotthermometer, ein Infrarotabstandssensor, eine Kamera, ein Temperatursensor und ein Lambdasonde innerhalb des Ofens zum Einsatz. Ferner wurden eine Wägezelle, ein Strom- sowie Spannungs-Sensor und ein Temperatursensor außerhalb des Ofens genutzt. Die während der Aufheizphase aufgenommen Datensätze ermöglichten das Training mehrerer künstlicher neuronaler Netze, die die verschiedenen Lebensmittel in die entsprechenden Kategorien einordnen konnten, um so das optimale Backprogram auszuwählen. Zur Abschätzung der thermische Diffusivität der Nahrung, die von der Zusammensetzung (Kohlenhydrate, Fett, Protein, Wasser) abhängt, wurden mehrere künstliche neuronale Netze trainiert. Mit Ausnahme des Fettanteils der Lebensmittel konnten alle Komponenten durch verschiedene KNNs mit einem Maximum von 8 versteckten Neuronen ausreichend genau abgeschätzt werden um auf deren Grundlage die Temperatur im inneren des Lebensmittels zu berechnen. Die durchgeführte Arbeit zeigt, dass mit Hilfe verschiedenster Sensoren zur direkten beziehungsweise indirekten Messung der äußeren Eigenschaften der Lebensmittel sowie KNNs für die Kategorisierung und Abschätzung der Lebensmittelzusammensetzung die automatische Erkennung und Berechnung der inneren Temperatur von verschiedensten Lebensmitteln möglich ist.

Drying condition effects on fluidization quality and fluidization velocity at initial and final stages of cubical particulates for pet food

In this research fluidization behavior of cubical Bovine intestine samples was studied. Bovine intestine samples were heat pump dried at atmospheric pressure and at emperatures below and above the material freezing points. Experiments were conducted to study fluidization characteristics and drying kinetics at different drying conditions. Bovine particles were characterized according to Geldart classification and minimum fluidization velocity was calculated using Ergun Equation and generalized equation for all drying conditions at the beginning of the trials and end of the trials. Walli’s model was used to categorize stability of the fluidization at the beginning and end of the drying for each trial. Walli’s values determined were positive at the beginning and end of all trials indicating stable fluidisation at the beginning and end for each drying condition.

Influence of drying conditions on the moisture diffusion and fluidization quality during multi-stage fluidized bed drying of bovine intestine for pet food

In order to establish the influence of the drying air characteristics on the drying performance and fluidization quality of bovine intestine for pet food, several drying tests have been carried out in a laboratory scale heat pump assisted fluid bed dryer. Bovine intestine samples were heat pump fluidized bed dried at atmospheric pressure and at temperatures below and above the materials freezing points, equipped with a continuous monitoring system. The investigation of the drying characteristics have been conducted in the temperature range −10 to 25 ◦C and the airflow in the range 1.5–2.5 m/s. Some experiments were conducted as single temperature drying experiments and others as two stage drying experiments employing two temperatures. An Arrhenius-type equation was used to interpret the influence of the drying air temperature on the effective diffusivity, calculated with the method of slopes in terms of energy activation, and this was found to be sensitive to the temperature. The effective diffusion coefficient of moisture transfer was determined by the Fickian method using uni-dimensional moisture movement in both moisture, removal by evaporation and combined sublimation and evaporation. Correlations expressing the effective moisture diffusivity and drying temperature are reported. Bovine particles were characterized according to the Geldart classification and the minimum fluidization velocity was calculated using the Ergun Equation and generalized equation for all drying conditions at the beginning and end of the trials. Walli’s model was used to categorize stability of the fluidization at the beginning and end of the dryingv for each trial. The determined Walli’s values were positive at the beginning and end of all trials indicating stable fluidization at the beginning and end for each drying condition.

Classification of jinjiang oysters Crassostrea rivularis (Gould, 1861) from China, based on morphology and phylogenetic analysis

The jinjiang oyster Crassostrea rivularis [Gould, 1861. Descriptions of Shells collected in the North Pacific Exploring Expedition under Captains Ringgold and Rodgers. Proc. Boston Soc. Nat. Hist. 8 (April) 33-40] is one of the most important and best-known oysters in China. Based on the color of its flesh, two forms of C rivularis are recognized and referred to as the "white meat" and 11 red meat" oysters. The classification of white and red forms of this species has been a subject of confusion and debate in China. To clarify the taxonomic status of the two forms of C. rivularis, we collected and analyzed oysters from five locations along China's coast using both morphological characters and DNA sequences from mitochondrial 16S rRNA and cytochrome oxidase 1, and the nuclear 28S rRNA genes. Oysters were classified as white or red forms according to their morphological characteristics and then subjected to DNA sequencing. Both morphological and DNA sequence data suggest that the red and white oysters are two separate species. Phylogenetic analysis of DNA sequences obtained in this study and existing sequences of reference species show that the red oyster is the same species as C. ariakensis Wakiya [1929. Japanese food oysters. Jpn. J. Zool. 2, 359-367.], albeit the red oysters from north and south China are genetically distinctive. The white oyster is the same species as a newly described species from Hong Kong, C. hongkongensis Lam and Morton [2003. Mitochondrial DNA and identification of a new species of Crassostrea (Bivalvia: Ostreidae) cultured for centuries in the Pearl River Delta, Hong Kong, China. Aqua. 228, 1-13]. Although the name C. rivularis has seniority over C. ariakensis and C. hongkongensis, the original description of Ostrea rivularis by Gould [1861] does not fit shell characteristics of either the red or the white oysters. We propose that the name of C. rivularis Gould [1861] should be suspended, the red oyster should take the name C. ariakensis, and the white oyster should take the name C. hongkongensis. (C) 2004 Elsevier B.V. All rights reserved.

Classification the geographical origin of corn distillers dried grains with solubles by near infrared reflectance spectroscopy combined with chemometrics: A feasibility study

In this study, 137 corn distillers dried grains with solubles (DDGS) samples from a range of different geographical origins (Jilin Province of China, Heilongjiang Province of China, USA and Europe) were collected and analysed. Different near infrared spectrometers combined with different chemometric packages were used in two independent laboratories to investigate the feasibility of classifying geographical origin of DDGS. Base on the same dataset, one laboratory developed a partial least square discriminant analysis model and another laboratory developed an orthogonal partial least square discriminant analysis model. Results showed that both models could perfectly classify DDGS samples from different geographical origins. These promising results encourage the development of larger scale efforts to produce datasets which can be used to differentiate the geographical origin of DDGS and such efforts are required to provide higher level food security measures on a global scale.

Microbiological quality of bivalves: relevance to food safety

Os moluscos bivalves constituem um recurso haliêutico de elevada importância na economia (inter)nacional pelas suas características organolépticas, valor nutritivo e relevância na gastronomia tradicional. Não obstante, representam um produto alimentar de elevado risco para a saúde pública. A contaminação microbiológica (autóctone e antropogénica), sendo crónica nos bancos de bivalves das zonas estuarino-lagunares, constitui uma das principais preocupações associadas à segurança alimentar. Aquando da filtração inerente aos processos de respiração e alimentação, os bivalves bioacumulam passivamente microrganismos incluindo os patogénicos. A sua colocação no mercado impõe pois, prévia salubrização para níveis microbiológicos compatíveis com a legislação em vigor, salvaguardando a saúde pública. Apesar da monitorização das áreas de apanha e produção, das medidas de prevenção e da depuração, a ocorrência de surtos associados ao consumo de bivalves tem aumentado. Tal deve-se à insuficiente monitorização da contaminação microbiológica dos bivalves, contribuindo para uma gestão ineficaz do produto e consequente sub-valorização. O presente trabalho pretendeu caracterizar o estado de desenvolvimento do sector de exploração de bivalves em Portugal do ponto de vista da segurança alimentar, e analisar os aspectos cruciais da monitorização e da depuração do produto apresentando alternativas abrangentes e aplicáveis ao sector. Assim, desenvolveu-se uma metodologia de base molecular passível de adaptação à monitorização dos bivalves das zonas conquícolas, como alternativa ao método de referência vigente do Número Mais Provável que é baseado apenas na quantificação de Escherichia coli. O mexilhão (Mytilus edulis) da Ria de Aveiro, bivalve de interesse comercial a nível (inter)nacional serviu de modelo para a comparação de protocolos de extração de DNA. Esta metodologia foi desenvolvida de modo a que os métodos de extração de DNA sejam passíveis de aplicação a outras matrizes biológicas ou ambientais. Para além da detecção e quantificação directa de bactérias patogénicas, esta metodologia poderá ser aplicada à monitorização da transferência vertical microbiana nos bancos de bivalves bem como à caracterização da dinâmica espacio-temporal das populações microbianas no ambiente e à monitorização dos processos de depuração. Foi ainda abordado o potencial da aplicação de bacteriófagos ou de enzimas líticas para a optimização dos processos de purificação. O trabalho realizado e as perspectivas futuras propostas pretendem contribuir para a dinamização e requalificação do sector de exploração de bivalves através da melhoria do nível de segurança alimentar dos moluscos bivalves comercializados para alimentação humana, valorizando este recurso.

Feature Engineering in Fine-Grained Image Classification

Thesis (Ph.D.)--University of Washington, 2013

Classification et relations entre les traits fonctionnels des crustacés zooplanctoniques : de l’organisme à l’écosystème

Les écologistes reconnaissent depuis longtemps que les organismes sont soutenus par le flux, l’emmagasinage et le renouvellement d’énergie et de matériel de l’écosystème, puisqu’ils sont nécessaires au métabolisme biologique et à la construction de biomasse. L’importance des organismes dans la régularisation des processus écosystémiques est maintenant de plus en plus considérée. Situé au centre des chaînes trophiques aquatiques, le zooplancton influence les flux d’énergie et de matériel dans les écosystèmes. Plusieurs de leurs caractéristiques sont connues comme étant de bons indicateurs de leur effet sur l’environnement, notamment leur taille, contenu corporel et taux métabolique. La plupart de ces caractéristiques peuvent être appelées « traits fonctionnels ». Alors que l’emploi des traits devient de plus en plus populaire en écologie des communautés aquatiques, peu ont su utiliser cette approche afin de concrètement lier la structure des communautés zooplanctoniques aux processus écosystémiques. Dans cette étude, nous avons colligé les données provenant d’une grande variété de littérature afin de construire une base de données sur les traits du zooplancton crustacé contribuant directement ou indirectement aux flux de C, N et P dans les écosystèmes. Notre méta-analyse a permis d’assembler plus de 9000 observations sur 287 espèces et d’identifier par le fait même ce qu’il manque à nos connaissances. Nous avons examiné une série de corrélations croisées entre 16 traits, dont 35 étaient significatives, et avons exploré les relations entre les unités taxonomiques de même qu’entre les espèces marines et d’eaux douces. Notre synthèse a entre autres révélé des patrons significativement différents entre le zooplancton marin et dulcicole quant à leur taux de respiration et leur allométrie (masse vs. longueur corporelle). Nous proposons de plus une nouvelle classification de traits liant les fonctions des organismes à celles de l’écosystème. Notre but est d’offrir une base de données sur les traits du zooplancton, des outils afin de mieux lier les organismes aux processus écosystémiques et de stimuler la recherche de patrons généraux et de compromis entre les traits.

Online dietary intake estimation: The Food4Me food frequency questionnaire

Background: Dietary assessment methods are important tools for nutrition research. Online dietary assessment tools have the potential to become invaluable methods of assessing dietary intake because, compared with traditional methods, they have many advantages including the automatic storage of input data and the immediate generation of nutritional outputs. Objective: The aim of this study was to develop an online food frequency questionnaire (FFQ) for dietary data collection in the “Food4Me” study and to compare this with the validated European Prospective Investigation of Cancer (EPIC) Norfolk printed FFQ. Methods: The Food4Me FFQ used in this analysis was developed to consist of 157 food items. Standardized color photographs were incorporated in the development of the Food4Me FFQ to facilitate accurate quantification of the portion size of each food item. Participants were recruited in two centers (Dublin, Ireland and Reading, United Kingdom) and each received the online Food4Me FFQ and the printed EPIC-Norfolk FFQ in random order. Participants completed the Food4Me FFQ online and, for most food items, participants were requested to choose their usual serving size among seven possibilities from a range of portion size pictures. The level of agreement between the two methods was evaluated for both nutrient and food group intakes using the Bland and Altman method and classification into quartiles of daily intake. Correlations were calculated for nutrient and food group intakes. Results: A total of 113 participants were recruited with a mean age of 30 (SD 10) years (40.7% male, 46/113; 59.3%, 67/113 female). Cross-classification into exact plus adjacent quartiles ranged from 77% to 97% at the nutrient level and 77% to 99% at the food group level. Agreement at the nutrient level was highest for alcohol (97%) and lowest for percent energy from polyunsaturated fatty acids (77%). Crude unadjusted correlations for nutrients ranged between .43 and .86. Agreement at the food group level was highest for “other fruits” (eg, apples, pears, oranges) and lowest for “cakes, pastries, and buns”. For food groups, correlations ranged between .41 and .90. Conclusions: The results demonstrate that the online Food4Me FFQ has good agreement with the validated printed EPIC-Norfolk FFQ for assessing both nutrient and food group intakes, rendering it a useful tool for ranking individuals based on nutrient and food group intakes.

Online dietary intake estimation : reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record

Background: Advances in nutritional assessment are continuing to embrace developments in computer technology. The online Food4Me food frequency questionnaire (FFQ) was created as an electronic system for the collection of nutrient intake data. To ensure its accuracy in assessing both nutrient and food group intake, further validation against data obtained using a reliable, but independent, instrument and assessment of its reproducibility are required. Objective: The aim was to assess the reproducibility and validity of the Food4Me FFQ against a 4-day weighed food record (WFR). Methods: Reproducibility of the Food4Me FFQ was assessed using test-retest methodology by asking participants to complete the FFQ on 2 occasions 4 weeks apart. To assess the validity of the Food4Me FFQ against the 4-day WFR, half the participants were also asked to complete a 4-day WFR 1 week after the first administration of the Food4Me FFQ. Level of agreement between nutrient and food group intakes estimated by the repeated Food4Me FFQ and the Food4Me FFQ and 4-day WFR were evaluated using Bland-Altman methodology and classification into quartiles of daily intake. Crude unadjusted correlation coefficients were also calculated for nutrient and food group intakes. Results: In total, 100 people participated in the assessment of reproducibility (mean age 32, SD 12 years), and 49 of these (mean age 27, SD 8 years) also took part in the assessment of validity. Crude unadjusted correlations for repeated Food4Me FFQ ranged from .65 (vitamin D) to .90 (alcohol). The mean cross-classification into “exact agreement plus adjacent” was 92% for both nutrient and food group intakes, and Bland-Altman plots showed good agreement for energy-adjusted macronutrient intakes. Agreement between the Food4Me FFQ and 4-day WFR varied, with crude unadjusted correlations ranging from .23 (vitamin D) to .65 (protein, % total energy) for nutrient intakes and .11 (soups, sauces and miscellaneous foods) to .73 (yogurts) for food group intake. The mean cross-classification into “exact agreement plus adjacent” was 80% and 78% for nutrient and food group intake, respectively. There were no significant differences between energy intakes estimated using the Food4Me FFQ and 4-day WFR, and Bland-Altman plots showed good agreement for both energy and energy-controlled nutrient intakes. Conclusions: The results demonstrate that the online Food4Me FFQ is reproducible for assessing nutrient and food group intake and has moderate agreement with the 4-day WFR for assessing energy and energy-adjusted nutrient intakes. The Food4Me FFQ is a suitable online tool for assessing dietary intake in healthy adults.