Automated Discovery of Social Networks in Online Learning Communities

As a way to gain greater insights into the operation of online communities, this dissertation applies automated text mining techniques to text-based communication to identify, describe and evaluate underlying social networks among online community members. The main thrust of the study is to automate the discovery of social ties that form between community members, using only the digital footprints left behind in their online forum postings. Currently, one of the most common but time consuming methods for discovering social ties between people is to ask questions about their perceived social ties. However, such a survey is difficult to collect due to the high investment in time associated with data collection and the sensitive nature of the types of questions that may be asked. To overcome these limitations, the dissertation presents a new, content-based method for automated discovery of social networks from threaded discussions, referred to as ‘name network’. As a case study, the proposed automated method is evaluated in the context of online learning communities. The results suggest that the proposed ‘name network’ method for collecting social network data is a viable alternative to costly and time-consuming collection of users’ data using surveys. The study also demonstrates how social networks produced by the ‘name network’ method can be used to study online classes and to look for evidence of collaborative learning in online learning communities. For example, educators can use name networks as a real time diagnostic tool to identify students who might need additional help or students who may provide such help to others. Future research will evaluate the usefulness of the ‘name network’ method in other types of online communities.

Commentary on: Are we overpathologizing everyday life? A tenable blueprint for behavioral addiction research: On functional and compulsive aspects of reinforcement pathologies

Background: This paper is a commentary to a debate article entitled: "Are we overpathologizing everyday life? A tenable blueprint for behavioral addiction research", by Billieux et al. (2015). Methods and aim: This brief response focused on the necessity to better characterize psychological and related neurocognitive determinants of persistent deleterious actions associated or not with substance utilization. Results: A majority of addicted people could be driven by psychological functional reasons to keep using drugs, gambling or buying despite the growing number of related negative consequences. In addition, a non-negligible proportion of them would need assistance to restore profound disturbances in basic learning processes involved in compulsive actions. Conclusions: The distinction between psychological functionality and compulsive aspects of addictive behaviors should represent a big step towards more efficient treatments.

Studio e Sperimentazione di Algoritmi di Reinforcement Learning Applicati a Video Game

Nella prima parte del mio lavoro viene presentato uno studio di una prima soluzione "from scratch" sviluppata da Andrew Karpathy. Seguono due miei miglioramenti: il primo modificando direttamente il codice della precedente soluzione e introducendo, come obbiettivo aggiuntivo per la rete nelle prime fasi di gioco, l'intercettazione della pallina da parte della racchetta, migliorando l'addestramento iniziale; il secondo é una mia personale implementazione utilizzando algoritmi più complessi, che sono allo stato dell'arte su giochi dell'Atari, e che portano un addestramento molto più veloce della rete.

Reinforcement learning over encrypted data

Reinforcement learning is a particular paradigm of machine learning that, recently, has proved times and times again to be a very effective and powerful approach. On the other hand, cryptography usually takes the opposite direction. While machine learning aims at analyzing data, cryptography aims at maintaining its privacy by hiding such data. However, the two techniques can be jointly used to create privacy preserving models, able to make inferences on the data without leaking sensitive information. Despite the numerous amount of studies performed on machine learning and cryptography, reinforcement learning in particular has never been applied to such cases before. Being able to successfully make use of reinforcement learning in an encrypted scenario would allow us to create an agent that efficiently controls a system without providing it with full knowledge of the environment it is operating in, leading the way to many possible use cases. Therefore, we have decided to apply the reinforcement learning paradigm to encrypted data. In this project we have applied one of the most well-known reinforcement learning algorithms, called Deep Q-Learning, to simple simulated environments and studied how the encryption affects the training performance of the agent, in order to see if it is still able to learn how to behave even when the input data is no longer readable by humans. The results of this work highlight that the agent is still able to learn with no issues whatsoever in small state spaces with non-secure encryptions, like AES in ECB mode. For fixed environments, it is also able to reach a suboptimal solution even in the presence of secure modes, like AES in CBC mode, showing a significant improvement with respect to a random agent; however, its ability to generalize in stochastic environments or big state spaces suffers greatly.

Unsupervised reinforcement learning via state entropy maximization

Reinforcement Learning (RL) provides a powerful framework to address sequential decision-making problems in which the transition dynamics is unknown or too complex to be represented. The RL approach is based on speculating what is the best decision to make given sample estimates obtained from previous interactions, a recipe that led to several breakthroughs in various domains, ranging from game playing to robotics. Despite their success, current RL methods hardly generalize from one task to another, and achieving the kind of generalization obtained through unsupervised pre-training in non-sequential problems seems unthinkable. Unsupervised RL has recently emerged as a way to improve generalization of RL methods. Just as its non-sequential counterpart, the unsupervised RL framework comprises two phases: An unsupervised pre-training phase, in which the agent interacts with the environment without external feedback, and a supervised fine-tuning phase, in which the agent aims to efficiently solve a task in the same environment by exploiting the knowledge acquired during pre-training. In this thesis, we study unsupervised RL via state entropy maximization, in which the agent makes use of the unsupervised interactions to pre-train a policy that maximizes the entropy of its induced state distribution. First, we provide a theoretical characterization of the learning problem by considering a convex RL formulation that subsumes state entropy maximization. Our analysis shows that maximizing the state entropy in finite trials is inherently harder than RL. Then, we study the state entropy maximization problem from an optimization perspective. Especially, we show that the primal formulation of the corresponding optimization problem can be (approximately) addressed through tractable linear programs. Finally, we provide the first practical methodologies for state entropy maximization in complex domains, both when the pre-training takes place in a single environment as well as multiple environments.

Concepts and methods for efficient decentralized learning in federated settings

Deep Neural Networks (DNNs) have revolutionized a wide range of applications beyond traditional machine learning and artificial intelligence fields, e.g., computer vision, healthcare, natural language processing and others. At the same time, edge devices have become central in our society, generating an unprecedented amount of data which could be used to train data-hungry models such as DNNs. However, the potentially sensitive or confidential nature of gathered data poses privacy concerns when storing and processing them in centralized locations. To this purpose, decentralized learning decouples model training from the need of directly accessing raw data, by alternating on-device training and periodic communications. The ability of distilling knowledge from decentralized data, however, comes at the cost of facing more challenging learning settings, such as coping with heterogeneous hardware and network connectivity, statistical diversity of data, and ensuring verifiable privacy guarantees. This Thesis proposes an extensive overview of decentralized learning literature, including a novel taxonomy and a detailed description of the most relevant system-level contributions in the related literature for privacy, communication efficiency, data and system heterogeneity, and poisoning defense. Next, this Thesis presents the design of an original solution to tackle communication efficiency and system heterogeneity, and empirically evaluates it on federated settings. For communication efficiency, an original method, specifically designed for Convolutional Neural Networks, is also described and evaluated against the state-of-the-art. Furthermore, this Thesis provides an in-depth review of recently proposed methods to tackle the performance degradation introduced by data heterogeneity, followed by empirical evaluations on challenging data distributions, highlighting strengths and possible weaknesses of the considered solutions. Finally, this Thesis presents a novel perspective on the usage of Knowledge Distillation as a mean for optimizing decentralized learning systems in settings characterized by data heterogeneity or system heterogeneity. Our vision on relevant future research directions close the manuscript.

Autonomous Pricing using Policy-Gradient Reinforcement Learning

Nella letteratura economica e di teoria dei giochi vi è un dibattito aperto sulla possibilità di emergenza di comportamenti anticompetitivi da parte di algoritmi di determinazione automatica dei prezzi di mercato. L'obiettivo di questa tesi è sviluppare un modello di reinforcement learning di tipo actor-critic con entropy regularization per impostare i prezzi in un gioco dinamico di competizione oligopolistica con prezzi continui. Il modello che propongo esibisce in modo coerente comportamenti cooperativi supportati da meccanismi di punizione che scoraggiano la deviazione dall'equilibrio raggiunto a convergenza. Il comportamento di questo modello durante l'apprendimento e a convergenza avvenuta aiuta inoltre a interpretare le azioni compiute da Q-learning tabellare e altri algoritmi di prezzo in condizioni simili. I risultati sono robusti alla variazione del numero di agenti in competizione e al tipo di deviazione dall'equilibrio ottenuto a convergenza, punendo anche deviazioni a prezzi più alti.

Influence of different catilever extensions and glass or polyamaramid reinforcement fibers on fracture strength of implant-supported temporary

In long-term oral rehabilitation treatments, resistance of provisional crowns is a very important factor, especially in cases of an extensive edentulous distal space. The aim of this laboratorial study was to evaluate an acrylic resin cantilever-type prosthesis regarding the flexural strength of its in-balance portion as a function of its extension variation and reinforcement by two types of fibers (glass and polyaramid), considering that literature is not conclusive on this subject. Each specimen was composed by 3 total crowns at its mesial portion, each one attached to an implant component (abutment), while the distal portion (cantilever) had two crowns. Each specimen was constructed by injecting acrylic resin into a two-part silicone matrix placed on a metallic base. In each specimen, the crowns were fabricated with either acrylic resin (control group) or acrylic resin reinforced by glass (Fibrante, Angelus) or polyaramide (Kevlar 49, Du Pont) fibers. Compression load was applied on the cantilever, in a point located 7, 14 or 21 mm from the distal surface of the nearest crown with abutment, to simulate different extensions. The specimen was fixed on the metallic base and the force was applied until fracture in a universal test machine. Each one of the 9 sub-groups was composed by 10 specimens. Flexural strength means (in kgf) for the distances of 7, 14 and 21 mm were, respectively, 28.07, 8.27 and 6.39 for control group, 31.89, 9.18 and 5.16 for Kevlar 49 and 30.90, 9.31 and 6.86 for Fibrante. Data analysis ANOVA showed statistically significant difference (p<0.05) only regarding cantilever extension. Tukey's test detected significantly higher flexural strength for the 7 mm-distance, followed by 14 and 21 mm. Fracture was complete only on specimens of non-reinforced groups.

Power quality analysis applying a hybrid methodology with wavelet transforms and neural networks

A hybrid system to automatically detect, locate and classify disturbances affecting power quality in an electrical power system is presented in this paper. The disturbances characterized are events from an actual power distribution system simulated by the ATP (Alternative Transients Program) software. The hybrid approach introduced consists of two stages. In the first stage, the wavelet transform (WT) is used to detect disturbances in the system and to locate the time of their occurrence. When such an event is flagged, the second stage is triggered and various artificial neural networks (ANNs) are applied to classify the data measured during the disturbance(s). A computational logic using WTs and ANNs together with a graphical user interface (GU) between the algorithm and its end user is then implemented. The results obtained so far are promising and suggest that this approach could lead to a useful application in an actual distribution system. (C) 2009 Elsevier Ltd. All rights reserved.

Implementation of a Control Loop Experiment in a Network-Based Control System With LonWorks Technology and IP Networks

Considering the increasing popularity of network-based control systems and the huge adoption of IP networks (such as the Internet), this paper studies the influence of network quality of service (QoS) parameters over quality of control parameters. An example of a control loop is implemented using two LonWorks networks (CEA-709.1) interconnected by an emulated IP network, in which important QoS parameters such as delay and delay jitter can be completely controlled. Mathematical definitions are provided according to the literature, and the results of the network-based control loop experiment are presented and discussed.

Reachability of the synchronous state in a mutually connected PLL network

Second-order phase locked loops (PLLs) are devices that are able to provide synchronization between the nodes in a network even under severe quality restrictions in the signal propagation. Consequently, they are widely used in telecommunication and control. Conventional master-slave (M-S) clock-distribution systems are being, replaced by mutually connected (MC) ones due to their good potential to be used in new types of application such as wireless sensor networks, distributed computation and communication systems. Here, by using an analytical reasoning, a nonlinear algebraic system of equations is proposed to establish the existence conditions for the synchronous state in an MC PLL network. Numerical experiments confirm the analytical results and provide ideas about how the network parameters affect the reachability of the synchronous state. The phase-difference oscillation amplitudes are related to the node parameters helping to design PLL neural networks. Furthermore, estimation of the acquisition time depending on the node parameters allows the performance evaluation of time distribution systems and neural networks based on phase-locked techniques. (c) 2008 Elsevier GmbH. All rights reserved.

Mapping the Contemporary Studies on Alliances and Strategic Networks

The competition among the companies depends on the velocity and efficience they can create and commercialize knowledge in a timely and cost-efficient manner. In this context, collaboration emerges as a reaction to the environmental changes. Although strategic alliances and networks have been exploited in the strategic literature for decades, the complexity and continuous usage of these cooperation structures, in a world of growing competition, justify the continuous interest in both themes. This article presents a scanning of the contemporary academic production in strategic alliances and networks, covering the period from January 1997 to august 2007, based on the top five journals accordingly to the journal of Citation Report 2006 in the business and management categories simultaneously. The results point to a retraction in publications about strategic alliances and a significant growth in the area of strategic. networks. The joint view of strategic alliances and networks, cited by some authors a the evolutionary path of study, still did not appear salient. The most cited topics found in the alliance literature are the governance structure, cooperation, knowledge transfer, culture, control, trust, alliance formation,,previous experience, resources, competition and partner selection. The theme network focuses mainly on structure, knowledge transfer and social network, while the joint vision is highly concentrated in: the subjects of alliance formation and the governance choice.

Flow and axial dispersion simulation for traveling axisymmetric Taylor vortices

Numerical experiments using a finite difference method were carried out to determine the motion of axisymmetric Taylor vortices for narrow-gap Taylor vortex flow. When a pressure gradient is imposed on the flow the vortices are observed to move with an axial speed of 1.16 +/- 0.005 times the mean axial flow velocity. The method of Brenner was used to calculate the long-time axial spread of material in the flow. For flows where there is no pressure gradient, the axial dispersion scales with the square root of the molecular diffusion, in agreement with the results of Rosen-bluth et al. for high Peclet number dispersion in spatially periodic flows with a roll structure. When a pressure gradient is imposed the dispersion increases by an amount approximately equal to 6.5 x 10(-4) (W) over bar(2)d(2)/D-m, where (W) over bar is the average axial velocity in the annulus, analogous to Taylor dispersion for laminar flow in an empty tube.

Why fear snakes and spiders? Evidence for a learning-based account of fear-relevant stimulus-processing

Fear-relevant stimuli, such as snakes, spiders and heights, preferentially capture attention as compared to nonfear-relevant stimuli. This is said to reflect an encapsulated mechanism whereby attention is captured by the simple perceptual features of stimuli that have evolutionary significance. Research, using pictures of snakes and spiders, has found some support for this account; however, participants may have had prior fear of snakes and spiders that influenced results. The current research compared responses of snake and spider experts who had little fear of snakes and spiders, and control participants across a series of affective priming and visual search tasks. Experts discriminated between dangerous and nondangerous snakes and spiders, and expert responses to pictures of nondangerous snakes and spiders differed from those of control participants. The current results dispute that stimulus fear relevance is based purely on perceptual features, and provides support for the role of learning and experience.