Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

Governing algorithms in the big data era for balancing new digital rights - designing GDPR compliant and trustworthy XAI systems

This thesis investigates the legal, ethical, technical, and psychological issues of general data processing and artificial intelligence practices and the explainability of AI systems. It consists of two main parts. In the initial section, we provide a comprehensive overview of the big data processing ecosystem and the main challenges we face today. We then evaluate the GDPR’s data privacy framework in the European Union. The Trustworthy AI Framework proposed by the EU’s High-Level Expert Group on AI (AI HLEG) is examined in detail. The ethical principles for the foundation and realization of Trustworthy AI are analyzed along with the assessment list prepared by the AI HLEG. Then, we list the main big data challenges the European researchers and institutions identified and provide a literature review on the technical and organizational measures to address these challenges. A quantitative analysis is conducted on the identified big data challenges and the measures to address them, which leads to practical recommendations for better data processing and AI practices in the EU. In the subsequent part, we concentrate on the explainability of AI systems. We clarify the terminology and list the goals aimed at the explainability of AI systems. We identify the reasons for the explainability-accuracy trade-off and how we can address it. We conduct a comparative cognitive analysis between human reasoning and machine-generated explanations with the aim of understanding how explainable AI can contribute to human reasoning. We then focus on the technical and legal responses to remedy the explainability problem. In this part, GDPR’s right to explanation framework and safeguards are analyzed in-depth with their contribution to the realization of Trustworthy AI. Then, we analyze the explanation techniques applicable at different stages of machine learning and propose several recommendations in chronological order to develop GDPR-compliant and Trustworthy XAI systems.

Sistema de visão para percepção e navegação de um UAV Quadrotor

Mestrado em Engenharia Electrotécnica e de Computadores

Coordenação de veículo de inspecção subaquática com veículo de superfície autónomo

O LSA/ISEP(Laboratório de sistemas Autónomos do Instituto Superior de Engenharia do Porto) tem vindo nos últimos anos a desenvolver sistemas robóticos inovadores para operação em ambiente marinho sendo o veículo de superfície autónomo ROAZ II um exemplo de renome internacional. Neste contexto, e tendo em vista a satisfação dos requisitos parciais conducentes à obtenção do grau de Mestre em Eng. Electrotécnica e de Computadores - Ramo de Sistemas Autónomos do ISEP, o presente trabalho visou a integração de um robô submarino operado remotamente (ROV) com o robô de superfície ROAZ II. Esta solução inovadora de operação coordenada e integrada de um ASV/ROV permite dotar o ASV de mobilidade e visão subaquática. Após a caracterização e análise de requisitos de diversos cenários operacionais foi apresentada uma arquitectura de controlo coordenado dos dois veículos baseada em manobras de controlo descritas por autómatos híbridos. Os dois veículos foram modelados e as manobras coordenadas projectadas foram validadas com um simulador em ambiente Matlab/Simulink. Foi desenvolvido um sistema de localização relativa do ROV através da fusão sensorial de um sistema INS com um sistema acústico USBL utilizando um filtro EKF. O veículo ROV (VideoRay) do LSA foi instrumentado com os sensores necessários e efectuada a integração de hardware e software com o ASV ROAZ II permitindo a operação remota. Foi realizada uma missão demonstrativa de inspecção de pilares subaquáticos em cenário real com a operação conjunta dos dois robôs.

Use of negative information in positioning and tracking algorithms

To avoid additional hardware deployment, indoor localization systems have to be designed in such a way that they rely on existing infrastructure only. Besides the processing of measurements between nodes, localization procedure can include the information of all available environment information. In order to enhance the performance of Wi-Fi based localization systems, the innovative solution presented in this paper considers also the negative information. An indoor tracking method inspired by Kalman filtering is also proposed.

Solução de navegação inercial para o sistema plasys

Neste trabalho faz-se uma pesquisa e análise dos conceitos associados à navegação inercial para estimar a distância percorrida por uma pessoa. Foi desenvolvida uma plataforma de hardware para implementar os algoritmos de navegação inercial e estudar a marcha humana. Os testes efetuados permitiram adaptar os algoritmos de navegação inercial para humanos e testar várias técnicas para reduzir o erro na estimativa da distância percorrida. O sistema desenvolvido é um sistema modular que permite estudar o efeito da inserção de novos sensores. Desta forma foram adaptados os algoritmos de navegação para permitir a utilização da informação dos sensores de força colocados na planta do pé do utilizador. A partir desta arquitetura foram efetuadas duas abordagens para o cálculo da distância percorrida por uma pessoa. A primeira abordagem estima a distância percorrida considerando o número de passos. A segunda abordagem faz uma estimação da distância percorrida com base nos algoritmos de navegação inercial. Foram realizados um conjunto de testes para comparar os erros na estimativa da distância percorrida pelas abordagens efetuadas. A primeira abordagem obteve um erro médio de 4,103% em várias cadências de passo. Este erro foi obtido após sintonia para o utilizador em questão. A segunda abordagem obteve um erro de 9,423%. De forma a reduzir o erro recorreu-se ao filtro de Kalman o que levou a uma redução do erro para 9,192%. Por fim, recorreu-se aos sensores de força que permitiram uma redução para 8,172%. A segunda abordagem apesar de ter um erro maior não depende do utilizador pois não necessita de sintonia dos parâmetros para estimar a distância para cada pessoa. Os testes efetuados permitiram, através dos sensores de força, testar a importância da força sentida pela planta do pé para aferir a fase do ciclo de marcha. Esta capacidade permite reduzir os erros na estimativa da distância percorrida e obter uma maior robustez neste tipo de sistemas.

Standard commercial education robotic platform: intermediate solution to teaching robotics

Teaching robotics to students at the beginning of their studies has become a huge challenge. Simulation environments can be an effective solution to that challenge where students can interact with simulated robots and have the first contact with robotic constraints. From our previous experience with simulation environments it was possible to observe that students with lower background knowledge in robotics where able to deal with a limited number of constraints, implement a simulated robotic platform and study several sensors. The question is: after this first phase what should be the best approach? Should the student start developing their own hardware? Hardware development is a very important part of an engineer's education but it can also be a difficult phase that could lead to discouragement and loss of motivation in some students. Considering the previous constraints and first year engineering students’ high abandonment rate it is important to develop teaching strategies to deal with this problem in a feasible way. The solution that we propose is the integration of a low-cost standard robotic platform WowWee Rovio as an intermediate solution between the simulation phase and the stage where the students can develop their own robots. This approach will allow the students to keep working in robotic areas such as: cooperative behaviour, perception, navigation and data fusion. The propose approach proved to be a motivation step not only for the students but also for the teachers. Students and teachers were able to reach an agreement between the level of demand imposed by the teachers and satisfaction/motivation of the students.

New integrative computational approaches unveil the Saccharomyces cerevisiae pheno-metabolomic fermentative profile and allow strain selection for winemaking

During must fermentation by Saccharomyces cerevisiae strains thousands of volatile aroma compounds are formed. The objective of the present work was to adapt computational approaches to analyze pheno-metabolomic diversity of a S. cerevisiae strain collection with different origins. Phenotypic and genetic characterization together with individual must fermentations were performed, and metabolites relevant to aromatic profiles were determined. Experimental results were projected onto a common coordinates system, revealing 17 statistical-relevant multi-dimensional modules, combining sets of most-correlated features of noteworthy biological importance. The present method allowed, as a breakthrough, to combine genetic, phenotypic and metabolomic data, which has not been possible so far due to difficulties in comparing different types of data. Therefore, the proposed computational approach revealed as successful to shed light into the holistic characterization of S. cerevisiae pheno-metabolome in must fermentative conditions. This will allow the identification of combined relevant features with application in selection of good winemaking strains.

Heel and toe clearance estimation for gait analysis using wireless inertial sensors.

Tripping is considered a major cause of fall in older people. Therefore, foot clearance (i.e., height of the foot above ground during swing phase) could be a key factor to better understand the complex relationship between gait and falls. This paper presents a new method to estimate clearance using a foot-worn and wireless inertial sensor system. The method relies on the computation of foot orientation and trajectory from sensors signal data fusion, combined with the temporal detection of toe-off and heel-strike events. Based on a kinematic model that automatically estimates sensor position relative to the foot, heel and toe trajectories are estimated. 2-D and 3-D models are presented with different solving approaches, and validated against an optical motion capture system on 12 healthy adults performing short walking trials at self-selected, slow, and fast speed. Parameters corresponding to local minimum and maximum of heel and toe clearance were extracted and showed accuracy ± precision of 4.1 ± 2.3 cm for maximal heel clearance and 1.3 ± 0.9 cm for minimal toe clearance compared to the reference. The system is lightweight, wireless, easy to wear and to use, and provide a new and useful tool for routine clinical assessment of gait outside a dedicated laboratory.

Recent advances in remote sensing image processing

Remote sensing image processing is nowadays a mature research area. The techniques developed in the field allow many real-life applications with great societal value. For instance, urban monitoring, fire detection or flood prediction can have a great impact on economical and environmental issues. To attain such objectives, the remote sensing community has turned into a multidisciplinary field of science that embraces physics, signal theory, computer science, electronics, and communications. From a machine learning and signal/image processing point of view, all the applications are tackled under specific formalisms, such as classification and clustering, regression and function approximation, image coding, restoration and enhancement, source unmixing, data fusion or feature selection and extraction. This paper serves as a survey of methods and applications, and reviews the last methodological advances in remote sensing image processing.

A global view of cross-border migration

This paper evaluates the global welfare impact of observed levels of migration using a quantitativemulti-sector model of the world economy calibrated to aggregate and firm-level data.Our framework features cross-country labor productivity differences, international trade, remittances,and a heterogeneous workforce. We compare welfare under the observed levels ofmigration to a no-migration counterfactual. In the long run, natives in countries that receiveda lot of migration -such as Canada or Australia- are better o due to greater product varietyavailable in consumption and as intermediate inputs. In the short run the impact of migrationon average welfare in these countries is close to zero, while the skilled and unskilled nativestend to experience welfare changes of opposite signs. The remaining natives in countries withlarge emigration flows -such as Jamaica or El Salvador- are also better off due to migration,but for a different reason: remittances. The welfare impact of observed levels of migration issubstantial, at about 5 to 10% for the main receiving countries and about 10% in countries withlarge incoming remittances. Our results are robust to accounting for imperfect transferabilityof skills, selection into migration, and imperfect substitution between natives and immigrants.

Land use/cover classification in the Brazilian Amazon using satellite images

Land use/cover classification is one of the most important applications in remote sensing. However, mapping accurate land use/cover spatial distribution is a challenge, particularly in moist tropical regions, due to the complex biophysical environment and limitations of remote sensing data per se. This paper reviews experiments related to land use/cover classification in the Brazilian Amazon for a decade. Through comprehensive analysis of the classification results, it is concluded that spatial information inherent in remote sensing data plays an essential role in improving land use/cover classification. Incorporation of suitable textural images into multispectral bands and use of segmentation‑based method are valuable ways to improve land use/cover classification, especially for high spatial resolution images. Data fusion of multi‑resolution images within optical sensor data is vital for visual interpretation, but may not improve classification performance. In contrast, integration of optical and radar data did improve classification performance when the proper data fusion method was used. Among the classification algorithms available, the maximum likelihood classifier is still an important method for providing reasonably good accuracy, but nonparametric algorithms, such as classification tree analysis, have the potential to provide better results. However, they often require more time to achieve parametric optimization. Proper use of hierarchical‑based methods is fundamental for developing accurate land use/cover classification, mainly from historical remotely sensed data.

Robust Detection of Incumbents in Cognitive Radio Networks Using Groups

Cognitive radio is a wireless technology aimed at improvingthe efficiency use of the radio-electric spectrum, thus facilitating a reductionin the load on the free frequency bands. Cognitive radio networkscan scan the spectrum and adapt their parameters to operate in the unoccupiedbands. To avoid interfering with licensed users operating on a givenchannel, the networks need to be highly sensitive, which is achieved byusing cooperative sensing methods. Current cooperative sensing methodsare not robust enough against occasional or continuous attacks. This articleoutlines a Group Fusion method that takes into account the behavior ofusers over the short and long term. On fusing the data, the method is basedon giving more weight to user groups that are more unanimous in their decisions.Simulations have been performed in a dynamic environment withinterferences. Results prove that when attackers are present (both reiterativeor sporadic), the proposed Group Fusion method has superior sensingcapability than other methods.

On-Line Analytical Processing of Inventory Turnover

Operatiivisen tiedon tuottaminen loppukäyttäjille analyyttistä tarkastelua silmällä pitäen aiheuttaa ongelmia useille yrityksille. Diplomityö pyrkii ratkaisemaan ko. ongelman Teleste Oyj:ssä. Työ on jaettu kolmeen pääkappaleeseen. Kappale 2 selkiyttää On-Line Analytical Processing (OLAP)- käsitteen. Kappale 3 esittelee muutamia OLAP-tuotteiden valmistajia ja heidän arkkitehtuurejaan sekä tyypillisten sovellusalueiden lisäksi huomioon otettavia asioita OLAP käyttöönoton yhteydessä. Kappale 4, tuo esille varsinaisen ratkaisun. Teknisellä arkkitehtuurilla on merkittävä asema ratkaisun rakenteen kannalta. Tässä on sovellettu Microsoft:n tietovarasto kehysrakennetta. Kappaleen 4 edetessä, tapahtumakäsittelytieto muutetaan informaatioksi ja edelleen loppukäyttäjien tiedoksi. Loppukäyttäjät varustetaan tehokkaalla ja tosiaikaisella analysointityökalulla moniulotteisessa ympäristössä. Vaikka kiertonopeus otetaan työssä sovellusesimerkiksi, työ ei pyri löytämään optimaalista tasoa Telesten varastoille. Siitä huolimatta eräitä parannusehdotuksia mainitaan.

Review of Robust Cooperative Spectrum Sensing Techniques for Cognitive Radio Networks

Cognitive radio networks sense spectrum occupancy and manage themselvesto operate in unused bands without disturbing licensed users. The detection capability of aradio system can be enhanced if the sensing process is performed jointly by a group of nodesso that the effects of wireless fading and shadowing can be minimized. However, taking acollaborative approach poses new security threats to the system as nodes can report falsesensing data to reach a wrong decision. This paper makes a review of secure cooperativespectrum sensing in cognitive radio networks. The main objective of these protocols is toprovide an accurate resolution about the availability of some spectrum channels, ensuring thecontribution from incapable users as well as malicious ones is discarded. Issues, advantagesand disadvantages of such protocols are investigated and summarized.