The role of context in image annotation and recommendation

With the rise of smart phones, lifelogging devices (e.g. Google Glass) and popularity of image sharing websites (e.g. Flickr), users are capturing and sharing every aspect of their life online producing a wealth of visual content. Of these uploaded images, the majority are poorly annotated or exist in complete semantic isolation making the process of building retrieval systems difficult as one must firstly understand the meaning of an image in order to retrieve it. To alleviate this problem, many image sharing websites offer manual annotation tools which allow the user to “tag” their photos, however, these techniques are laborious and as a result have been poorly adopted; Sigurbjörnsson and van Zwol (2008) showed that 64% of images uploaded to Flickr are annotated with < 4 tags. Due to this, an entire body of research has focused on the automatic annotation of images (Hanbury, 2008; Smeulders et al., 2000; Zhang et al., 2012a) where one attempts to bridge the semantic gap between an image’s appearance and meaning e.g. the objects present. Despite two decades of research the semantic gap still largely exists and as a result automatic annotation models often offer unsatisfactory performance for industrial implementation. Further, these techniques can only annotate what they see, thus ignoring the “bigger picture” surrounding an image (e.g. its location, the event, the people present etc). Much work has therefore focused on building photo tag recommendation (PTR) methods which aid the user in the annotation process by suggesting tags related to those already present. These works have mainly focused on computing relationships between tags based on historical images e.g. that NY and timessquare co-exist in many images and are therefore highly correlated. However, tags are inherently noisy, sparse and ill-defined often resulting in poor PTR accuracy e.g. does NY refer to New York or New Year? This thesis proposes the exploitation of an image’s context which, unlike textual evidences, is always present, in order to alleviate this ambiguity in the tag recommendation process. Specifically we exploit the “what, who, where, when and how” of the image capture process in order to complement textual evidences in various photo tag recommendation and retrieval scenarios. In part II, we combine text, content-based (e.g. # of faces present) and contextual (e.g. day-of-the-week taken) signals for tag recommendation purposes, achieving up to a 75% improvement to precision@5 in comparison to a text-only TF-IDF baseline. We then consider external knowledge sources (i.e. Wikipedia & Twitter) as an alternative to (slower moving) Flickr in order to build recommendation models on, showing that similar accuracy could be achieved on these faster moving, yet entirely textual, datasets. In part II, we also highlight the merits of diversifying tag recommendation lists before discussing at length various problems with existing automatic image annotation and photo tag recommendation evaluation collections. In part III, we propose three new image retrieval scenarios, namely “visual event summarisation”, “image popularity prediction” and “lifelog summarisation”. In the first scenario, we attempt to produce a rank of relevant and diverse images for various news events by (i) removing irrelevant images such memes and visual duplicates (ii) before semantically clustering images based on the tweets in which they were originally posted. Using this approach, we were able to achieve over 50% precision for images in the top 5 ranks. In the second retrieval scenario, we show that by combining contextual and content-based features from images, we are able to predict if it will become “popular” (or not) with 74% accuracy, using an SVM classifier. Finally, in chapter 9 we employ blur detection and perceptual-hash clustering in order to remove noisy images from lifelogs, before combining visual and geo-temporal signals in order to capture a user’s “key moments” within their day. We believe that the results of this thesis show an important step towards building effective image retrieval models when there lacks sufficient textual content (i.e. a cold start).

Analysis of the folksonomy of freesound

User generated content shared in online communities is often described using collaborative tagging systems where users assign labels to content resources. As a result, a folksonomy emerges that relates a number of tags with the resources they label and the users that have used them. In this paper we analyze the folksonomy of Freesound, an online audio clip sharing site which contains more than two million users and 150,000 user-contributed sound samplescovering a wide variety of sounds. By following methodologies taken from similar studies, we compute some metrics that characterize the folksonomy both at the globallevel and at the tag level. In this manner, we are able to betterunderstand the behavior of the folksonomy as a whole, and also obtain some indicators that can be used as metadata for describing tags themselves. We expect that such a methodology for characterizing folksonomies can be useful to support processes such as tag recommendation or automatic annotation of online resources.

Brokerage Platform for Media Content Recommendation

Near real time media content personalisation is nowadays a major challenge involving media content sources, distributors and viewers. This paper describes an approach to seamless recommendation, negotiation and transaction of personalised media content. It adopts an integrated view of the problem by proposing, on the business-to-business (B2B) side, a brokerage platform to negotiate the media items on behalf of the media content distributors and sources, providing viewers, on the business-to-consumer (B2C) side, with a personalised electronic programme guide (EPG) containing the set of recommended items after negotiation. In this setup, when a viewer connects, the distributor looks up and invites sources to negotiate the contents of the viewer personal EPG. The proposed multi-agent brokerage platform is structured in four layers, modelling the registration, service agreement, partner lookup, invitation as well as item recommendation, negotiation and transaction stages of the B2B processes. The recommendation service is a rule-based switch hybrid filter, including six collaborative and two content-based filters. The rule-based system selects, at runtime, the filter(s) to apply as well as the final set of recommendations to present. The filter selection is based on the data available, ranging from the history of items watched to the ratings and/or tags assigned to the items by the viewer. Additionally, this module implements (i) a novel item stereotype to represent newly arrived items, (ii) a standard user stereotype for new users, (iii) a novel passive user tag cloud stereotype for socially passive users, and (iv) a new content-based filter named the collinearity and proximity similarity (CPS). At the end of the paper, we present off-line results and a case study describing how the recommendation service works. The proposed system provides, to our knowledge, an excellent holistic solution to the problem of recommending multimedia contents.

A Multi–Agent Brokerage Platform for Media Content Recommendation

Near real time media content personalisation is nowadays a major challenge involving media content sources, distributors and viewers. This paper describes an approach to seamless recommendation, negotiation and transaction of personalised media content. It adopts an integrated view of the problem by proposing, on the business-to-business (B2B) side, a brokerage platform to negotiate the media items on behalf of the media content distributors and sources, providing viewers, on the business-to-consumer (B2C) side, with a personalised electronic programme guide (EPG) containing the set of recommended items after negotiation. In this setup, when a viewer connects, the distributor looks up and invites sources to negotiate the contents of the viewer personal EPG. The proposed multi-agent brokerage platform is structured in four layers, modelling the registration, service agreement, partner lookup, invitation as well as item recommendation, negotiation and transaction stages of the B2B processes. The recommendation service is a rule-based switch hybrid filter, including six collaborative and two content-based filters. The rule-based system selects, at runtime, the filter(s) to apply as well as the final set of recommendations to present. The filter selection is based on the data available, ranging from the history of items watched to the ratings and/or tags assigned to the items by the viewer. Additionally, this module implements (i) a novel item stereotype to represent newly arrived items, (ii) a standard user stereotype for new users, (iii) a novel passive user tag cloud stereotype for socially passive users, and (iv) a new content-based filter named the collinearity and proximity similarity (CPS). At the end of the paper, we present off-line results and a case study describing how the recommendation service works. The proposed system provides, to our knowledge, an excellent holistic solution to the problem of recommending multimedia contents.

Instantaneous characterization of vegetable oils via TAG and FFA profiles by easy ambient sonic-spray ionization mass spectrometry

A fast and reliable method is presented for the analysis of vegetable oils. Easy ambient sonic-spray ionization mass spectrometry (EASI-MS) is shown to efficiently desorb and ionize the main oil constituents from an inert surface under ambient conditions and to provide comprehensive triacylglyceride (TAG) and free fatty acid (FFA) profiles detected mainly as either [ TAG + Na](+) or [FFA - H](-) ions. EASI(+/-)-MS analysis is simple, easily implemented, requires just a tiny droplet of the oil and is performed without any pre-separation or chemical manipulation. It also causes no fragmentation of TAG ions hence diacylglyceride (DAG) and monoacylglyceride (MAG) profiles and contents can also be measured. The EASI(+/-)-MS profiles of TAG and FFA permit authentication and quality control and can be used, for instance, to access levels of adulteration, acidity, oxidation or hydrolysis of vegetable oils in general.

Evaluation of GFP Tag as a Screening Reporter in Directed Evolution of a Hyperthermophilic beta-Glucosidase

By applying a directed evolution methodology specific enzymatic characteristics can be enhanced, but to select mutants of interest from a large mutant bank, this approach requires high throughput screening and facile selection. To facilitate such primary screening of enhanced clones, an expression system was tested that uses a green fluorescent protein (GFP) tag from Aequorea victoria linked to the enzyme of interest. As GFP`s fluorescence is readily measured, and as there is a 1:1 molar correlation between the target protein and GFP, the concept proposed was to determine whether GFP could facilitate primary screening of error-prone PCR (EPP) clones. For this purpose a thermostable beta-glucosidase (BglA) from Fervidobacterium sp. was used as a model enzyme. A vector expressing the chimeric protein BglA-GFP-6XHis was constructed and the fusion protein purified and characterized. When compared to the native proteins, the components of the fusion displayed modified characteristics, such as enhanced GFP thermostability and a higher BglA optimum temperature. Clones carrying mutant BglA proteins obtained by EPP, were screened based on the BglA/GFP activity ratio. Purified tagged enzymes from selected clones resulted in modified substrate specificity.

Histology of dart tag insertion sites in the epaulette shark

Samples of dermal and epidermal tissues of epaulette sharks Hemiscyllium ocellatum were examined histologically to assess damage caused by tagging. Tissues from around tag sites were removed at time intervals ranging from 100 min to 284 days post-tagging. These samples showed acute and chronic responses to tagging. Acute responses consisted of localized tissue breakdown and haemorrhaging, and occurred within the first few hours after tag insertion. At 10 h post-tagging, an intermediate response was apparent. This phase was characterized by further haemorrhaging and red and white blood cell movement into the wound area. The chronic response observed in the 10-284-day post-tagging samples was characterized by fibrous tissue formation to sequester the tag. This tissue presumably protects the adjacent musculature from further trauma produced by movement of the tag and provides a continuous barrier between the internal and external milieu. Tissue repair appeared to progress consistently in all specimens and no secondary infections at the tag site were seen. Tagging produced only localized tissue disruption and did not appear to be detrimental to the long term health of individual sharks. Our findings show that spaghetti style dart tagging is an acceptable method for marking individuals (40-75+ cm total length) of this species. (C) 1997 The Fisheries Society of the British Isles.

Nutritional recommendation alone does not change the obesity profile of health professionals

Objective: To evaluate the frequency of overweight and obesity in health professionals, before and after a single specialized dietary recommendation. Methods: Anthropometric measures of 579 workers of a general hospital in the city of Sao Paulo, Brazil were taken. The weight (f), height (h) and waist circumference (wc) were interpreted according to the WHO and NCEP ATP III guidelines. Nutrition specialist provided dietary and behavioral recommendations. The entire sample underwent a new evaluation one year later. Results: At the first evaluation, 79 employees presente WC >= 102 cm (male) or WC >= 88 cm (female). The association between WC >= 102 cm (men) or WC >= 88 cm (women) and BMI >= 30 kg/m(2) was found in 12.8 % (69 subjects). The BMI distribution per age group indicated that the increase in overweight and obesity was directly proportional to the age increase. Physical activities were not practiced by 75% of the subjects studied. A year later, the evaluation indicated lack of statistical differences regarding the BMI and waist circumference of the sample and only 2.8% started to practice a physical activity. Conclusion: Dietary recommendation alone failed to promote changes in the eating habits of health professionals who work at a general hospital or to encourage them to practice exercise.

Lime and fertilizer recommendation system for coconut trees

Fertilizer recommendation to most agricultural crops is based on response curves. Such curves are constructed from field experimental data, obtained for a particular condition and may not be reliable to be applied to other regions. The aim of this study was to develop a Lime and Fertilizer Recommendation System for Coconut Crop based on the nutritional balance. The System considers the expected productivity and plant nutrient use efficiency to estimate nutrient demand, and effective rooting layer, soil nutrient availability, as well as any other nutrient input to estimate the nutrient supply. Comparing the nutrient demand with the nutrient supply the System defines the nutrient balance. If the balance for a given nutrient is negative, lime and, or, fertilization is recommended. On the other hand, if the balance is positive, no lime or fertilizer is needed. For coconut trees, the fertilization regime is divided in three stages: fertilization at the planting spot, band fertilization and fertilization at the production phase. The data set for the development of the System for coconut trees was obtained from the literature. The recommendations generated by the System were compared to those derived from recommendation tables used for coconut crop in Brazil. The main differences between the two procedures were for the P rate applied in the planting hole, which was higher in the proposed System because the tables do not pay heed to the pit volume, whereas the N and K rates were lower. The crop demand for K is very high, and the rates recommended by the System are superior to the table recommendations for the formation and initial production stage. The fertilizer recommendations by the System are higher for the phase of coconut tree growth as compared to the production phase, because greater amount of biomass is produced in the first phase.

Tourism mobile and recommendation systems - a state of the art

Recommendation systems have been growing in number for the last fifteen years. To evolve and adapt to the demands of the actual society, many paradigms emerged giving birth to even more paradigms and hybrid approaches. Mobile devices have also been under an incredible growth rate in every business area, and there are already lots of mobile based systems to assist tourists. This explosive growth gave birth to different mobile applications, each having their own advantages and disadvantages. Since recommendation and mobile systems might as well be integrated, this work intends to present the current state of the art in tourism mobile and recommendation systems, as well as to state their advantages and disadvantages.

A hybrid recommendation approach for a tourism system

Many current e-commerce systems provide personalization when their content is shown to users. In this sense, recommender systems make personalized suggestions and provide information of items available in the system. Nowadays, there is a vast amount of methods, including data mining techniques that can be employed for personalization in recommender systems. However, these methods are still quite vulnerable to some limitations and shortcomings related to recommender environment. In order to deal with some of them, in this work we implement a recommendation methodology in a recommender system for tourism, where classification based on association is applied. Classification based on association methods, also named associative classification methods, consist of an alternative data mining technique, which combines concepts from classification and association in order to allow association rules to be employed in a prediction context. The proposed methodology was evaluated in some case studies, where we could verify that it is able to shorten limitations presented in recommender systems and to enhance recommendation quality.

Learning objects recommendation in an adaptive educational hypermedia system

The aim of this paper is presenting the modules of the Adaptive Educational Hypermedia System PCMAT, responsible for the recommendation of learning objects. PCMAT is an online collaborative learning platform with a constructivist approach, which assesses the user’s knowledge and presents contents and activities adapted to the characteristics and learning style of students of mathematics in basic schools. The recommendation module and search and retrieval module choose the most adequate learning object, based on the user's characteristics and performance, and in this way contribute to the system’s adaptability.

Recommendation & mobile systems - a state of the art for tourism

Recommendation systems have been growing in number over the last fifteen years. To evolve and adapt to the demands of the actual society, many paradigms emerged giving birth to even more paradigms and hybrid approaches. These approaches contain strengths and weaknesses that need to be evaluated according to the knowledge area in which the system is going to be implemented. Mobile devices have also been under an incredible growth rate in every business area, and there are already lots of mobile based systems to assist tourists. This explosive growth gave birth to different mobile applications, each having their own advantages and disadvantages. Since recommendation and mobile systems might as well be integrated, this work intends to present the current state of the art in tourism mobile and recommendation systems, as well as to state their advantages and disadvantages.

Shopping center tracking and recommendation systems

Shopping centers present a rich and heterogeneous environment, where IT systems can be implemented in order to support the needs of its actors. However, due to the environment complexity, several feasibility issues emerge when designing both the logical and physical architecture of such systems. Additionally, the system must be able to cope with the individual needs of each actor, and provide services that are easily adopted by them, taking into account several sociological and economical aspects. In this sense, we present an overview of current support systems for shopping center environments. From this overview, a high-level model of the domain (involving actors and services) is described along with challenges and possible features in the context of current Semantic Web, mobile device and sensor technologies.

Recommendation and planning through mobile devices in tourism context

In this paper we present a mobile recommendation and planning system, named PSiS Mobile. It is designed to provide effective support during a tourist visit through context-aware information and recommendations about points of interest, exploiting tourist preferences and context. Designing a tool like this brings several challenges that must be addressed. We discuss how these challenges have been overcame, present the overall system architecture, since this mobile application extends the PSiS project website, and the mobile application architecture.