Studying Recommender Systems to Enhance Distributed Computing Schedulers

Distributed Computing frameworks belong to a class of programming models that allow developers to

launch workloads on large clusters of machines. Due to the dramatic increase in the volume of

data gathered by ubiquitous computing devices, data analytic workloads have become a common

case among distributed computing applications, making Data Science an entire field of

Computer Science. We argue that Data Scientist's concern lays in three main components: a dataset,

a sequence of operations they wish to apply on this dataset, and some constraint they may have

related to their work (performances, QoS, budget, etc). However, it is actually extremely

difficult, without domain expertise, to perform data science. One need to select the right amount

and type of resources, pick up a framework, and configure it. Also, users are often running their

application in shared environments, ruled by schedulers expecting them to specify precisely their resource

needs. Inherent to the distributed and concurrent nature of the cited frameworks, monitoring and

profiling are hard, high dimensional problems that block users from making the right

configuration choices and determining the right amount of resources they need. Paradoxically, the

system is gathering a large amount of monitoring data at runtime, which remains unused.

In the ideal abstraction we envision for data scientists, the system is adaptive, able to exploit

monitoring data to learn about workloads, and process user requests into a tailored execution

context. In this work, we study different techniques that have been used to make steps toward

such system awareness, and explore a new way to do so by implementing machine learning

techniques to recommend a specific subset of system configurations for Apache Spark applications.

Furthermore, we present an in depth study of Apache Spark executors configuration, which highlight

the complexity in choosing the best one for a given workload.

Development of a New Client-Server Architecture for Context Aware Mobile Computing

This dissertation studies the context-aware application with its proposed algorithms at client side. The required context-aware infrastructure is discussed in depth to illustrate that such an infrastructure collects the mobile user’s context information, registers service providers, derives mobile user’s current context, distributes user context among context-aware applications, and provides tailored services. The approach proposed tries to strike a balance between the context server and mobile devices. The context acquisition is centralized at the server to ensure the usability of context information among mobile devices, while context reasoning remains at the application level. Hence, a centralized context acquisition and distributed context reasoning are viewed as a better solution overall. The context-aware search application is designed and implemented at the server side. A new algorithm is proposed to take into consideration the user context profiles. By promoting feedback on the dynamics of the system, any prior user selection is now saved for further analysis such that it may contribute to help the results of a subsequent search. On the basis of these developments at the server side, various solutions are consequently provided at the client side. A proxy software-based component is set up for the purpose of data collection. This research endorses the belief that the proxy at the client side should contain the context reasoning component. Implementation of such a component provides credence to this belief in that the context applications are able to derive the user context profiles. Furthermore, a context cache scheme is implemented to manage the cache on the client device in order to minimize processing requirements and other resources (bandwidth, CPU cycle, power). Java and MySQL platforms are used to implement the proposed architecture and to test scenarios derived from user’s daily activities. To meet the practical demands required of a testing environment without the impositions of a heavy cost for establishing such a comprehensive infrastructure, a software simulation using a free Yahoo search API is provided as a means to evaluate the effectiveness of the design approach in a most realistic way. The integration of Yahoo search engine into the context-aware architecture design proves how context aware application can meet user demands for tailored services and products in and around the user’s environment. The test results show that the overall design is highly effective,providing new features and enriching the mobile user’s experience through a broad scope of potential applications.

The Impact of Geographic Mobility on Canadian Military families and their Children’s Access Special Education Services

To achieve academic success, children with learning-related disabilities often receive special education supports at school. Currently, Canada does not have a federal department or integrated national system of education. Instead, each province and territory has a separate department or ministry that is responsible for the organization and delivery of education, including special education, at the elementary level. At the macro (national) level, inclusive education is the policy across Canada. However, each province and territory has its own legislation, definitions, and policies mandating special education services. These variations result in little consistency at the micro (individual school) level. Differences between eligibility requirements, supports offered, and delivery methods may present challenges for highly mobile families who must navigate new special education systems on behalf of their children with medical or learning challenges. One of the defining features of the Canadian military lifestyle is geographic mobility. As a result, many families are tasked with navigating new school systems for their children, a task that may be more difficult when children require special education services. The purpose of this study is to explore the impact of geographic mobility on Canadian military families and their children’s access to special education services. The secondary objective was to gain insight into supports that helped facilitate access to services, as well as supports that participants believe would have helped facilitate access. A qualitative approach, interpretive phenomenological analysis (IPA), was employed due to of its focus on individuals’ experiences and their understandings of a particular phenomenon. IPA allowed participants to reflect on the significance of their experiences, while the researcher engaged with these reflections to make sense of the meanings associated with their experiences. Nine semi-structured interviews were conducted with civilian caregivers who have a child with special education needs. An interview guide and probes were used to elicit rich, detailed, first-person accounts of their experiences navigating new special education systems. The main themes that emerged from the participants’ combined experiences addressed the emotional components of experiencing a transition, factors that may facilitate access to special education services, and career implications associated with accessing and maintaining special education services. Findings from the study illustrate that Canadian families experience many, and often times severe, barriers to accessing special education services after a posting. Furthermore, the impacts reported throughout the study echo the existing American literature on geographic mobility and access to special education services. Building on the literature, this study also highlights the need for further research exploring factors that create unique barriers to access in a Canadian context, resulting from the current special education climate, military policies, and military family support services.

Ecuadorian migration and family strategies of social mobility: dialectic relation between emotional and financial capital

This text deals with transnational strategies of social mobility in Ecuadorian migrant households in Spain. We apply the capital accumulation model (Moser, 2009) for this purpose. The main target of this article is, beyond thinking in terms of capital stock and accumulation, the analysis in depth of the dynamics of the different types of capital, that is to say, how they interact with each other in the framework of the social mobility strategies of the migrants and their families. We are bringing into light the way some households adopt investing decisions in capitals that don't translate into any addition or earnings in all cases, on the contrary, concentrating all their efforts on the accumulation of a certain asset they may, in some cases, lead to a loss of another. We will concentrate our analysis primarily on the dynamics between the physical and financial capital and the social and emotional capital, showing the tensions produced between these two types of assets. At the same time, we will highlight how migrants negotiate their family strategies of social mobility in the transnational area. Our study is based in empirical material obtained from qualitative fieldwork (in-depth interviews) with families of migrants in the urban district of Turubamba Bajo -(south of Quito) and in Madrid. A series of households were selected where interviews were carried out in the country of origin as well as in the context of immigration, with different family members, analysing the transnational social and economic strategies of families of migrant members. Family members of migrants established in Spain were interviewed in Quito, as well as key informants in the district (school teachers, nursery members of the staff, etc.). The research was framed within the projects "Impact of migration on the development: gender and transnationalism", Ministry of Science and Innovation (SEJ2007/63179) (Laura Oso, dir. 2007-2010),"Gender, transnationalism and intergenerational strategies of social mobility", Ministry of Economy and Competitiveness (FEM2011/26210) (Laura Oso, dir. 201-1-2015) and “Gender, Crossed Mobilities and Transnational Dynamics”, Ministry of Economy and Competitiveness (FEM2015-67164).

Las estatuas-menhir noroccidentales en contexto: conectividad y conexiones materiales durante el Bronce Tardío/Final

Este artículo analiza las estatuas-menhir noroccidentales que se distribuyen en el área comprendida entre los valles del río Duero y el río Miño, pero que descubrimientos recientes han extendido a regiones fuera de este área nuclear. Partiendo de tres aspectos claves para su interpretación (la cronología, su relación con el paisaje y su sentido iconográfico), se examinan las relaciones entre estas formas materiales y un paisaje socio-material de acción específico (las formas socio-materiales de interacción propias del Bronce final atlántico). Para ello, se tiene en cuenta diferentes conexiones materiales (presencia, encuentro, coexistencia, hibridación, etc.) que permiten contextualizar las estatuasmenhir dentro de un proceso histórico particular: la integración del noroeste de la península Ibérica en un contexto de relaciones mediterráneo-atlánticas, en la segunda mitad del II milenio a.C.

Characterisation of Large Disturbance Rotor Angle and Voltage Stability in Interconnected Power Networks with Distributed Wind Generation

Wind generation in highly interconnected power networks creates local and centralised stability issues based on their proximity to conventional synchronous generators and load centres. This paper examines the large disturbance stability issues (i.e. rotor angle and voltage stability) in power networks with geographically distributed wind resources in the context of a number of dispatch scenarios based on profiles of historical wind generation for a real power network. Stability issues have been analysed using novel stability indices developed from dynamic characteristics of wind generation. The results of this study show that localised stability issues worsen when significant penetration of both conventional and wind generation is present due to their non-complementary characteristics. In contrast, network stability improves when either high penetration of wind and synchronous generation is present in the network. Therefore, network regions can be clustered into two distinct stability groups (i.e. superior stability and inferior stability regions). Network stability improves when a voltage control strategy is implemented at wind farms, however both stability clusters remain unchanged irrespective of change in the control strategy. Moreover, this study has shown that the enhanced fault ride-through (FRT) strategy for wind farms can improve both voltage and rotor angle stability locally, but only a marginal improvement is evident in neighbouring regions.

Team Member’s Centrality, Cohesion, Conflict, and Performance in Multi-University Geographically Distributed Project Teams

This study examined team processes and outcomes among 12 multi-university distributed project teams from 11 universities during its early and late development stages over a 14-month project period. A longitudinal model of team interaction is presented and tested at the individual level to consider the extent to which both formal and informal network connections—measured as degree centrality—relate to changes in team members’ individual perceptions of cohesion and conflict in their teams, and their individual performance as a team member over time. The study showed a negative network centrality-cohesion relationship with significant temporal patterns, indicating that as team members perceive less degree centrality in distributed project teams, they report more team cohesion during the last four months of the project. We also found that changes in team cohesion from the first three months (i.e., early development stage) to the last four months (i.e., late development stage) of the project relate positively to changes in team member performance. Although degree centrality did not relate significantly to changes in team conflict over time, a strong inverse relationship was found between changes in team conflict and cohesion, suggesting that team conflict emphasizes a different but related aspect of how individuals view their experience with the team process. Changes in team conflict, however, did not relate to changes in team member performance. Ultimately, we showed that individuals, who are less central in the network and report higher levels of team cohesion, performed better in distributed teams over time.

Investigating Social Mobility: Cultural Frames and Cultural Frame Switching in First-Generation College Students

Thesis (Ph.D.)--University of Washington, 2016-08

Context-awareness privacy in data communications

Internet users consume online targeted advertising based on information collected about them and voluntarily share personal information in social networks. Sensor information and data from smart-phones is collected and used by applications, sometimes in unclear ways. As it happens today with smartphones, in the near future sensors will be shipped in all types of connected devices, enabling ubiquitous information gathering from the physical environment, enabling the vision of Ambient Intelligence. The value of gathered data, if not obvious, can be harnessed through data mining techniques and put to use by enabling personalized and tailored services as well as business intelligence practices, fueling the digital economy. However, the ever-expanding information gathering and use undermines the privacy conceptions of the past. Natural social practices of managing privacy in daily relations are overridden by socially-awkward communication tools, service providers struggle with security issues resulting in harmful data leaks, governments use mass surveillance techniques, the incentives of the digital economy threaten consumer privacy, and the advancement of consumergrade data-gathering technology enables new inter-personal abuses. A wide range of fields attempts to address technology-related privacy problems, however they vary immensely in terms of assumptions, scope and approach. Privacy of future use cases is typically handled vertically, instead of building upon previous work that can be re-contextualized, while current privacy problems are typically addressed per type in a more focused way. Because significant effort was required to make sense of the relations and structure of privacy-related work, this thesis attempts to transmit a structured view of it. It is multi-disciplinary - from cryptography to economics, including distributed systems and information theory - and addresses privacy issues of different natures. As existing work is framed and discussed, the contributions to the state-of-theart done in the scope of this thesis are presented. The contributions add to five distinct areas: 1) identity in distributed systems; 2) future context-aware services; 3) event-based context management; 4) low-latency information flow control; 5) high-dimensional dataset anonymity. Finally, having laid out such landscape of the privacy-preserving work, the current and future privacy challenges are discussed, considering not only technical but also socio-economic perspectives.

Dynamic load balancing of distributed memory parallel computational mechanics using unstructured meshes for multi-physical modelling

As the complexity of parallel applications increase, the performance limitations resulting from computational load imbalance become dominant. Mapping the problem space to the processors in a parallel machine in a manner that balances the workload of each processors will typically reduce the run-time. In many cases the computation time required for a given calculation cannot be predetermined even at run-time and so static partition of the problem returns poor performance. For problems in which the computational load across the discretisation is dynamic and inhomogeneous, for example multi-physics problems involving fluid and solid mechanics with phase changes, the workload for a static subdomain will change over the course of a computation and cannot be estimated beforehand. For such applications the mapping of loads to process is required to change dynamically, at run-time in order to maintain reasonable efficiency. The issue of dynamic load balancing are examined in the context of PHYSICA, a three dimensional unstructured mesh multi-physics continuum mechanics computational modelling code.

A visual system for distributed mosaics using an auvs fleet

The use of teams of Autonomous Underwater Vehicles for visual inspection tasks is a promising robotic field. The images captured by different robots can be also to aid in the localization/navigation of the fleet. In a previous work, a distributed localization system was presented based on the use of Augmented States Kalman Filter through the visual maps obtained by the fleet. In this context, this paper details a system for on-line construction of visual maps and its use to aid the localization and navigation of the robots. Different aspects related to the capture, treatment and construction of mosaics by fleets of robots are presented. The developed system can be executed on-line on different robotic platforms. The paper is concluded with a series of tests and analyses aiming at to system validation.

Towards Proactive Context-aware Computing and Systems

A primary goal of context-aware systems is delivering the right information at the right place and right time to users in order to enable them to make effective decisions and improve their quality of life. There are three key requirements for achieving this goal: determining what information is relevant, personalizing it based on the users’ context (location, preferences, behavioral history etc.), and delivering it to them in a timely manner without an explicit request from them. These requirements create a paradigm that we term as “Proactive Context-aware Computing”. Most of the existing context-aware systems fulfill only a subset of these requirements. Many of these systems focus only on personalization of the requested information based on users’ current context. Moreover, they are often designed for specific domains. In addition, most of the existing systems are reactive - the users request for some information and the system delivers it to them. These systems are not proactive i.e. they cannot anticipate users’ intent and behavior and act proactively without an explicit request from them. In order to overcome these limitations, we need to conduct a deeper analysis and enhance our understanding of context-aware systems that are generic, universal, proactive and applicable to a wide variety of domains. To support this dissertation, we explore several directions. Clearly the most significant sources of information about users today are smartphones. A large amount of users’ context can be acquired through them and they can be used as an effective means to deliver information to users. In addition, social media such as Facebook, Flickr and Foursquare provide a rich and powerful platform to mine users’ interests, preferences and behavioral history. We employ the ubiquity of smartphones and the wealth of information available from social media to address the challenge of building proactive context-aware systems. We have implemented and evaluated a few approaches, including some as part of the Rover framework, to achieve the paradigm of Proactive Context-aware Computing. Rover is a context-aware research platform which has been evolving for the last 6 years. Since location is one of the most important context for users, we have developed ‘Locus’, an indoor localization, tracking and navigation system for multi-story buildings. Other important dimensions of users’ context include the activities that they are engaged in. To this end, we have developed ‘SenseMe’, a system that leverages the smartphone and its multiple sensors in order to perform multidimensional context and activity recognition for users. As part of the ‘SenseMe’ project, we also conducted an exploratory study of privacy, trust, risks and other concerns of users with smart phone based personal sensing systems and applications. To determine what information would be relevant to users’ situations, we have developed ‘TellMe’ - a system that employs a new, flexible and scalable approach based on Natural Language Processing techniques to perform bootstrapped discovery and ranking of relevant information in context-aware systems. In order to personalize the relevant information, we have also developed an algorithm and system for mining a broad range of users’ preferences from their social network profiles and activities. For recommending new information to the users based on their past behavior and context history (such as visited locations, activities and time), we have developed a recommender system and approach for performing multi-dimensional collaborative recommendations using tensor factorization. For timely delivery of personalized and relevant information, it is essential to anticipate and predict users’ behavior. To this end, we have developed a unified infrastructure, within the Rover framework, and implemented several novel approaches and algorithms that employ various contextual features and state of the art machine learning techniques for building diverse behavioral models of users. Examples of generated models include classifying users’ semantic places and mobility states, predicting their availability for accepting calls on smartphones and inferring their device charging behavior. Finally, to enable proactivity in context-aware systems, we have also developed a planning framework based on HTN planning. Together, these works provide a major push in the direction of proactive context-aware computing.

A new distributed MIKEY mode to secure e-health applications.

Securing e-health applications in the context of Internet of Things (IoT) is challenging. Indeed, resources scarcity in such environment hinders the implementation of existing standard based protocols. Among these protocols, MIKEY (Multimedia Internet KEYing) aims at establishing security credentials between two communicating entities. However, the existing MIKEY modes fail to meet IoT specificities. In particular, the pre-shared key mode is energy efficient, but suffers from severe scalability issues. On the other hand, asymmetric modes such as the public key mode are scalable, but are highly resource consuming. To address this issue, we combine two previously proposed approaches to introduce a new hybrid MIKEY mode. Indeed, relying on a cooperative approach, a set of third parties is used to discharge the constrained nodes from heavy computational operations. Doing so, the pre-shared mode is used in the constrained part of the network, while the public key mode is used in the unconstrained part of the network. Preliminary results show that our proposed mode is energy preserving whereas its security properties are kept safe.

Dynamically managing sensed context data

This thesis reports on an investigation of the feasibility and usefulness of incorporating dynamic management facilities for managing sensed context data in a distributed contextaware mobile application. The investigation focuses on reducing the work required to integrate new sensed context streams in an existing context aware architecture. Current architectures require integration work for new streams and new contexts that are encountered. This means of operation is acceptable for current fixed architectures. However, as systems become more mobile the number of discoverable streams increases. Without the ability to discover and use these new streams the functionality of any given device will be limited to the streams that it knows how to decode. The integration of new streams requires that the sensed context data be understood by the current application. If the new source provides data of a type that an application currently requires then the new source should be connected to the application without any prior knowledge of the new source. If the type is similar and can be converted then this stream too should be appropriated by the application. Such applications are based on portable devices (phones, PDAs) for semi-autonomous services that use data from sensors connected to the devices, plus data exchanged with other such devices and remote servers. Such applications must handle input from a variety of sensors, refining the data locally and managing its communication from the device in volatile and unpredictable network conditions. The choice to focus on locally connected sensory input allows for the introduction of privacy and access controls. This local control can determine how the information is communicated to others. This investigation focuses on the evaluation of three approaches to sensor data management. The first system is characterised by its static management based on the pre-pended metadata. This was the reference system. Developed for a mobile system, the data was processed based on the attached metadata. The code that performed the processing was static. The second system was developed to move away from the static processing and introduce a greater freedom of handling for the data stream, this resulted in a heavy weight approach. The approach focused on pushing the processing of the data into a number of networked nodes rather than the monolithic design of the previous system. By creating a separate communication channel for the metadata it is possible to be more flexible with the amount and type of data transmitted. The final system pulled the benefits of the other systems together. By providing a small management class that would load a separate handler based on the incoming data, Dynamism was maximised whilst maintaining ease of code understanding. The three systems were then compared to highlight their ability to dynamically manage new sensed context. The evaluation took two approaches, the first is a quantitative analysis of the code to understand the complexity of the relative three systems. This was done by evaluating what changes to the system were involved for the new context. The second approach takes a qualitative view of the work required by the software engineer to reconfigure the systems to provide support for a new data stream. The evaluation highlights the various scenarios in which the three systems are most suited. There is always a trade-o↵ in the development of a system. The three approaches highlight this fact. The creation of a statically bound system can be quick to develop but may need to be completely re-written if the requirements move too far. Alternatively a highly dynamic system may be able to cope with new requirements but the developer time to create such a system may be greater than the creation of several simpler systems.

Household language, residential segregation and social mobility: continuity and change in eThekwini, South Africa

This article uses South African census data for 1996, 2001 and 2011 to explore the relationship between language and social mobility in the metropolitan region of eThekwini (including what was previously known as Durban). We focus particular attention on variables selected to shed light on residential segregation and social mobility, such as education level, income, race and in-migration. Data on adults at ward level (using 2011 ward boundaries) in eThekwini is used to develop a comparative spatial context for this analysis. Our main finding is that English appears in eThekwini to be the household language of the social elite as well as the language of upward mobility and empowerment.