Local and Global Feature Extraction for Invariant Object Recognition

Perceiving the world visually is a basic act for humans, but for computers it is still an unsolved problem. The variability present innatural environments is an obstacle for effective computer vision. The goal of invariant object recognition is to recognise objects in a digital image despite variations in, for example, pose, lighting or occlusion. In this study, invariant object recognition is considered from the viewpoint of feature extraction. Thedifferences between local and global features are studied with emphasis on Hough transform and Gabor filtering based feature extraction. The methods are examined with respect to four capabilities: generality, invariance, stability, and efficiency. Invariant features are presented using both Hough transform and Gabor filtering. A modified Hough transform technique is also presented where the distortion tolerance is increased by incorporating local information. In addition, methods for decreasing the computational costs of the Hough transform employing parallel processing and local information are introduced.

Comparison of Local Feature Detectors and Descriptors for Visual Object Categorization

Local features are used in many computer vision tasks including visual object categorization, content-based image retrieval and object recognition to mention a few. Local features are points, blobs or regions in images that are extracted using a local feature detector. To make use of extracted local features the localized interest points are described using a local feature descriptor. A descriptor histogram vector is a compact representation of an image and can be used for searching and matching images in databases. In this thesis the performance of local feature detectors and descriptors is evaluated for object class detection task. Features are extracted from image samples belonging to several object classes. Matching features are then searched using random image pairs of a same class. The goal of this thesis is to find out what are the best detector and descriptor methods for such task in terms of detector repeatability and descriptor matching rate.

Hardware Accelarated Visual Tracking Algorithms. A Systematic Literature Review

Many industrial applications need object recognition and tracking capabilities. The algorithms developed for those purposes are computationally expensive. Yet ,real time performance, high accuracy and small power consumption are essential measures of the system. When all these requirements are combined, hardware acceleration of these algorithms becomes a feasible solution. The purpose of this study is to analyze the current state of these hardware acceleration solutions, which algorithms have been implemented in hardware and what modifications have been done in order to adapt these algorithms to hardware.

Reconstruction of 3D Planar Objects with Partial Occlusion

Kolmiulotteisten kappaleiden rekonstruktio on yksi konenäön haastavimmista ongelmista, koska kappaleiden kolmiulotteisia etäisyyksiä ei voida selvittää yhdestä kaksiulotteisesta kuvasta. Ongelma voidaan ratkaista stereonäön avulla, jossa näkymän kolmiulotteinen rakenne päätellään usean kuvan perusteella. Tämä lähestymistapa mahdollistaa kuitenkin vain rekonstruktion niille kappaleiden osille, jotka näkyvät vähintään kahdessa kuvassa. Piilossa olevien osien rekonstruktio ei ole mahdollista pelkästään stereonäön avulla. Tässä työssä on kehitetty uusi menetelmä osittain piilossa olevien kolmiulotteisten tasomaisten kappaleiden rekonstruktioon. Menetelmän avulla voidaan selvittää hyvällä tarkkuudella tasomaisista pinnoista koostuvan kappaleen muoto ja paikka käyttäen kahta kuvaa kappaleesta. Menetelmä perustuu epipolaarigeometriaan, jonka avulla selvitetään molemmissa kuvissa näkyvät kappaleiden osat. Osittain piilossa olevien piirteiden rekonstruointi suoritetaan käyttämäen stereonäköä sekä tietoa kappaleen rakenteesta. Esitettyä ratkaisua voitaisiin käyttää esimerkiksi kolmiulotteisten kappaleiden visualisointiin, robotin navigointiin tai esineentunnistukseen.

Evaluation of Bubble Formation and Break Up in Suppression Pools by Using Pattern Recognition Methods

During a possible loss of coolant accident in BWRs, a large amount of steam will be released from the reactor pressure vessel to the suppression pool. Steam will be condensed into the suppression pool causing dynamic and structural loads to the pool. The formation and break up of bubbles can be measured by visual observation using a suitable pattern recognition algorithm. The aim of this study was to improve the preliminary pattern recognition algorithm, developed by Vesa Tanskanen in his doctoral dissertation, by using MATLAB. Video material from the PPOOLEX test facility, recorded during thermal stratification and mixing experiments, was used as a reference in the development of the algorithm. The developed algorithm consists of two parts: the pattern recognition of the bubbles and the analysis of recognized bubble images. The bubble recognition works well, but some errors will appear due to the complex structure of the pool. The results of the image analysis were reasonable. The volume and the surface area of the bubbles were not evaluated. Chugging frequencies calculated by using FFT fitted well into the results of oscillation frequencies measured in the experiments. The pattern recognition algorithm works in the conditions it is designed for. If the measurement configuration will be changed, some modifications have to be done. Numerous improvements are proposed for the future 3D equipment.

Local Features in Image and Video Processing - Object Class Matching and Video Shot Detection

The usage of digital content, such as video clips and images, has increased dramatically during the last decade. Local image features have been applied increasingly in various image and video retrieval applications. This thesis evaluates local features and applies them to image and video processing tasks. The results of the study show that 1) the performance of different local feature detector and descriptor methods vary significantly in object class matching, 2) local features can be applied in image alignment with superior results against the state-of-the-art, 3) the local feature based shot boundary detection method produces promising results, and 4) the local feature based hierarchical video summarization method shows promising new new research direction. In conclusion, this thesis presents the local features as a powerful tool in many applications and the imminent future work should concentrate on improving the quality of the local features.

Semantic and fuzzy modelling for human behaviour recognition in smart spaces : a case study on ambient assisted living.

Human activity recognition in everyday environments is a critical, but challenging task in Ambient Intelligence applications to achieve proper Ambient Assisted Living, and key challenges still remain to be dealt with to realize robust methods. One of the major limitations of the Ambient Intelligence systems today is the lack of semantic models of those activities on the environment, so that the system can recognize the speci c activity being performed by the user(s) and act accordingly. In this context, this thesis addresses the general problem of knowledge representation in Smart Spaces. The main objective is to develop knowledge-based models, equipped with semantics to learn, infer and monitor human behaviours in Smart Spaces. Moreover, it is easy to recognize that some aspects of this problem have a high degree of uncertainty, and therefore, the developed models must be equipped with mechanisms to manage this type of information. A fuzzy ontology and a semantic hybrid system are presented to allow modelling and recognition of a set of complex real-life scenarios where vagueness and uncertainty are inherent to the human nature of the users that perform it. The handling of uncertain, incomplete and vague data (i.e., missing sensor readings and activity execution variations, since human behaviour is non-deterministic) is approached for the rst time through a fuzzy ontology validated on real-time settings within a hybrid data-driven and knowledgebased architecture. The semantics of activities, sub-activities and real-time object interaction are taken into consideration. The proposed framework consists of two main modules: the low-level sub-activity recognizer and the high-level activity recognizer. The rst module detects sub-activities (i.e., actions or basic activities) that take input data directly from a depth sensor (Kinect). The main contribution of this thesis tackles the second component of the hybrid system, which lays on top of the previous one, in a superior level of abstraction, and acquires the input data from the rst module's output, and executes ontological inference to provide users, activities and their in uence in the environment, with semantics. This component is thus knowledge-based, and a fuzzy ontology was designed to model the high-level activities. Since activity recognition requires context-awareness and the ability to discriminate among activities in di erent environments, the semantic framework allows for modelling common-sense knowledge in the form of a rule-based system that supports expressions close to natural language in the form of fuzzy linguistic labels. The framework advantages have been evaluated with a challenging and new public dataset, CAD-120, achieving an accuracy of 90.1% and 91.1% respectively for low and high-level activities. This entails an improvement over both, entirely data-driven approaches, and merely ontology-based approaches. As an added value, for the system to be su ciently simple and exible to be managed by non-expert users, and thus, facilitate the transfer of research to industry, a development framework composed by a programming toolbox, a hybrid crisp and fuzzy architecture, and graphical models to represent and con gure human behaviour in Smart Spaces, were developed in order to provide the framework with more usability in the nal application. As a result, human behaviour recognition can help assisting people with special needs such as in healthcare, independent elderly living, in remote rehabilitation monitoring, industrial process guideline control, and many other cases. This thesis shows use cases in these areas.

Automatic recognition of fish from video sequences

The problem of automatic recognition of the fish from the video sequences is discussed in this Master’s Thesis. This is a very urgent issue for many organizations engaged in fish farming in Finland and Russia because the process of automation control and counting of individual species is turning point in the industry. The difficulties and the specific features of the problem have been identified in order to find a solution and propose some recommendations for the components of the automated fish recognition system. Methods such as background subtraction, Kalman filtering and Viola-Jones method were implemented during this work for detection, tracking and estimation of fish parameters. Both the results of the experiments and the choice of the appropriate methods strongly depend on the quality and the type of a video which is used as an input data. Practical experiments have demonstrated that not all methods can produce good results for real data, whereas on synthetic data they operate satisfactorily.

Deep convolutional neural networks for semantic video object segmentation

In this thesis, we propose to infer pixel-level labelling in video by utilising only object category information, exploiting the intrinsic structure of video data. Our motivation is the observation that image-level labels are much more easily to be acquired than pixel-level labels, and it is natural to find a link between the image level recognition and pixel level classification in video data, which would transfer learned recognition models from one domain to the other one. To this end, this thesis proposes two domain adaptation approaches to adapt the deep convolutional neural network (CNN) image recognition model trained from labelled image data to the target domain exploiting both semantic evidence learned from CNN, and the intrinsic structures of unlabelled video data. Our proposed approaches explicitly model and compensate for the domain adaptation from the source domain to the target domain which in turn underpins a robust semantic object segmentation method for natural videos. We demonstrate the superior performance of our methods by presenting extensive evaluations on challenging datasets comparing with the state-of-the-art methods.

3D-partikkelit aloittelijoille : tarkastelussa 3D-Kalevala-projekti

Opinnäytteeni teososana on 3D-Kalevala projektinimellä tunnetun animaation partikkeliefektit. Tarkastelen tarkemmin elokuvan Lumi-, Kylä-, Paja- ja Luola-kohtauksia. 3D-Kalevala on tietokoneella tehty animaatio, joka kertoo Suomen kansalliseepoksen Kalevalan päähenkilöstä Väinämöisestä. Elokuvassa vanha Väinämöinen muistelee nuoruutensa tapahtumia. 3D-Kalevala-projekti käynnistettiin vuonna 2003, mutta sen alkuperäiset tekijät eivät saaneet sitä valmiiksi, ja projektin teko keskeytettiin vuonna 2005. Vuoden 2006 keväällä projektiin perustettiin uusi kahden verkkoviestinnän opiskelijan projektiryhmä, jonka tehtävänä oli saada elokuva valmiiksi vuoden 2007 kevääseen mennessä. Kun aloitimme projektin tekemisen, olin kolmiulotteisessa mallinnuksessa aloittelija. Tästä johtuen opinnäytteen kirjallinen osa on opas aloittelijoille partikkelien maailmaan. Selvitän raportissani miten elokuvan partikkeliefektit rakennettiin, mitä niiden tekeminen minulta vaati ja miten ne loppujen lopuksi mielestäni onnistuivat. Elokuvan efektit tehtiin 3D Studio Max-ohjelman versiolla 6.0, ja tämän takia kirjoitankin efektien rakentamisesta kyseisen ohjelman keinoin. Projektin suuruuden vuoksi molemmat tekijät pääsivät tekemään monenlaisia töitä, mutta päävastuualueet olivat selvät. Minun osani oli tehdä elokuvaan efektejä. Partikkeliefektit ovat proseduraalisia efektejä, joiden avulla on mahdollista tehdä aidon näköisiä luonnonilmiöitä, kuten tulta, savua, kipinöitä ja veden roiskeita. Koska partikkeliefektit mallintavat reaalimaailman ilmiöitä, on tekijän hyvä olla kiinnostunut selvittämään ilmiöiden käyttäytymistä luonnossa. Raportoin myös projektin aikana huomaamistani hyvistä tavoista opiskella itsenäisesti partikkelien rakentamiseen käytettyjä tekniikoita. On hyvä lukea 3D Studio Maxin tasokasta käyttöohjesovellusta, tutustua Internetissä löytyviin 3D-aiheisiin foorumeihin, käydä aiheesta tutoriaaleja läpi sekä tutustua ohjelman ominaisuuksiin kokeilemalla ja tutkimalla itsenäisesti. Elokuvan efektit onnistuivat mielestäni kiitettävästi ottaen huomioon lähtötasoni. Löysin tapoja kehittää itseäni ja helppoja keinoja toteuttaa realistista jälkeä efektien rakentamisessa. Toivon, että raportistani olisi jollekin 3D-partikkeliefekteistä kiinnostuneelle hyötyä.

Keskeytyneen uusmediatuotannon jatkaminen : kohteena 3D-Kalevala

Opinnäytetyöni aiheena on keskeytyneen uusmediatuotannon jatkaminen. Monimuototyön työosana toteutettiin 7-minuuttinen 3D animaatio, joka kertoo Suomen kansalliseepoksen Kalevalan taruhahmosta Väinämöisestä, muistelemassa menneitä. Projekti käynnistettiin alun perin vuonna 2003, mutta resurssien vähetessä se keskeytyi vuoden 2005 loppupuolella. Keväällä 2006 projekti käynnistettiin uuden projektiryhmän voimin, jossa olin itse mukana vastaten muun muassa tuotannonohjauksesta ja hahmoanimoinneista. Uusi projektiryhmä oli henkilöstöresursseiltaan pieni, joten vastuualueet olivat monipuolisia. Keskeytyneen projektin jatkamisen ja haltuunoton haasteellisuus sai minut kiinnostumaan tutkia aihetta tarkemmin. Tuotannonohjaajana vastasin hyvin pitkälle tuotannon uudesta käynnistämisestä ja projektin saattamisesta vihdoin loppuun. Keskeytyneen projektin haltuunotto oli tilanteena kaikille uusi, mikä heijastui vaikeuksina uudelleen käynnistettyyn tuotantoon. Raportin tarkoituksena ei ole olla projektinhallinnallinen käsikirja, sillä käsittelen vain tämän projektin jatkolle oleellisina pidettyjä asioita. Projekti toivottavasti kuitenkin antaa kuvan huolellisen projektinhallinnan ja onnistuneen tuotannonohjauksen tärkeydestä. Jokainen keskeytynyt projekti ei ole aina välttämättä elvytettävissä - ainakaan alkuperäisessä muodossaan. Projektin jatkamista tulisi katsoa aina tapauskohtaisesti. Keskeytymiseen on useimmiten syynsä, joten ongelmien selvittäminen ja niihin puuttuminen on tärkeää ennen jatkopäätöksen tekemistä. Myös projektityöskentelytavat kehittyvät ja pohdin työssäni uusien projektinhallintatapojen, kuten wikien käyttöä projektinhallinnan työkaluna ja projektiyhteisön välistä viestintää edistävänä työkaluna.