13 resultados para HDFS bottleneck
em CentAUR: Central Archive University of Reading - UK
Resumo:
Three naming strategies are discussed that allow the processes of a distributed application to continue being addressed by their original logical name, along all the migrations they may be forced to undertake because of performance-improvement goals. A simple centralised solution is firstly discussed which showed a software bottleneck with the increase of the number of processes; other two solutions are considered that entail different communication schemes and different communication overheads for the naming protocol. All these strategies are based on the facility that each process is allowed to survive after migration, even in its original site, only to provide a forwarding service to those communications that used its obsolete address.
Resumo:
Recently, two approaches have been introduced that distribute the molecular fragment mining problem. The first approach applies a master/worker topology, the second approach, a completely distributed peer-to-peer system, solves the scalability problem due to the bottleneck at the master node. However, in many real world scenarios the participating computing nodes cannot communicate directly due to administrative policies such as security restrictions. Thus, potential computing power is not accessible to accelerate the mining run. To solve this shortcoming, this work introduces a hierarchical topology of computing resources, which distributes the management over several levels and adapts to the natural structure of those multi-domain architectures. The most important aspect is the load balancing scheme, which has been designed and optimized for the hierarchical structure. The approach allows dynamic aggregation of heterogenous computing resources and is applied to wide area network scenarios.
Resumo:
A full assessment of para-virtualization is important, because without knowledge about the various overheads, users can not understand whether using virtualization is a good idea or not. In this paper we are very interested in assessing the overheads of running various benchmarks on bare-‐metal, as well as on para-‐virtualization. The idea is to see what the overheads of para-‐ virtualization are, as well as looking at the overheads of turning on monitoring and logging. The knowledge from assessing various benchmarks on these different systems will help a range of users understand the use of virtualization systems. In this paper we assess the overheads of using Xen, VMware, KVM and Citrix, see Table 1. These different virtualization systems are used extensively by cloud-‐users. We are using various Netlib1 benchmarks, which have been developed by the University of Tennessee at Knoxville (UTK), and Oak Ridge National Laboratory (ORNL). In order to assess these virtualization systems, we run the benchmarks on bare-‐metal, then on the para-‐virtualization, and finally we turn on monitoring and logging. The later is important as users are interested in Service Level Agreements (SLAs) used by the Cloud providers, and the use of logging is a means of assessing the services bought and used from commercial providers. In this paper we assess the virtualization systems on three different systems. We use the Thamesblue supercomputer, the Hactar cluster and IBM JS20 blade server (see Table 2), which are all servers available at the University of Reading. A functional virtualization system is multi-‐layered and is driven by the privileged components. Virtualization systems can host multiple guest operating systems, which run on its own domain, and the system schedules virtual CPUs and memory within each Virtual Machines (VM) to make the best use of the available resources. The guest-‐operating system schedules each application accordingly. You can deploy virtualization as full virtualization or para-‐virtualization. Full virtualization provides a total abstraction of the underlying physical system and creates a new virtual system, where the guest operating systems can run. No modifications are needed in the guest OS or application, e.g. the guest OS or application is not aware of the virtualized environment and runs normally. Para-‐virualization requires user modification of the guest operating systems, which runs on the virtual machines, e.g. these guest operating systems are aware that they are running on a virtual machine, and provide near-‐native performance. You can deploy both para-‐virtualization and full virtualization across various virtualized systems. Para-‐virtualization is an OS-‐assisted virtualization; where some modifications are made in the guest operating system to enable better performance. In this kind of virtualization, the guest operating system is aware of the fact that it is running on the virtualized hardware and not on the bare hardware. In para-‐virtualization, the device drivers in the guest operating system coordinate the device drivers of host operating system and reduce the performance overheads. The use of para-‐virtualization [0] is intended to avoid the bottleneck associated with slow hardware interrupts that exist when full virtualization is employed. It has revealed [0] that para-‐ virtualization does not impose significant performance overhead in high performance computing, and this in turn this has implications for the use of cloud computing for hosting HPC applications. The “apparent” improvement in virtualization has led us to formulate the hypothesis that certain classes of HPC applications should be able to execute in a cloud environment, with minimal performance degradation. In order to support this hypothesis, first it is necessary to define exactly what is meant by a “class” of application, and secondly it will be necessary to observe application performance, both within a virtual machine and when executing on bare hardware. A further potential complication is associated with the need for Cloud service providers to support Service Level Agreements (SLA), so that system utilisation can be audited.
Resumo:
A two-sector Ramsey-type model of growth is developed to investigate the relationship between agricultural productivity and economy-wide growth. The framework takes into account the peculiarities of agriculture both in production ( reliance on a fixed natural resource base) and in consumption (life-sustaining role and low income elasticity of food demand). The transitional dynamics of the model establish that when preferences respect Engel's law, the level and growth rate of agricultural productivity influence the speed of capital accumulation. A calibration exercise shows that a small difference in agricultural productivity has drastic implications for the rate and pattern of growth of the economy. Hence, low agricultural productivity can form a bottleneck limiting growth, because high food prices result in a low saving rate.
Resumo:
Background: Patterns of mtDNA variation within a species reflect long-term population structure, but may also be influenced by maternally inherited endosymbionts, such as Wolbachia. These bacteria often alter host reproductive biology and can drive particular mtDNA haplotypes through populations. We investigated the impacts of Wolbachia infection and geography on mtDNA variation in the diamondback moth, a major global pest whose geographic distribution reflects both natural processes and transport via human agricultural activities. Results: The mtDNA phylogeny of 95 individuals sampled from 10 countries on four continents revealed two major clades. One contained only Wolbachia-infected individuals from Malaysia and Kenya, while the other contained only uninfected individuals, from all countries including Malaysia and Kenya. Within the uninfected group was a further clade containing all individuals from Australasia and displaying very limited sequence variation. In contrast, a biparental nuclear gene phylogeny did not have infected and uninfected clades, supporting the notion that maternally-inherited Wolbachia are responsible for the mtDNA pattern. Only about 5% (15/306) of our global sample of individuals was infected with the plutWBI isolate and even within infected local populations, many insects were uninfected. Comparisons of infected and uninfected isofemale lines revealed that plutWBI is associated with sex ratio distortion. Uninfected lines have a 1:1 sex ratio, while infected ones show a 2:1 female bias. Conclusion: The main correlate of mtDNA variation in P. xylostella is presence or absence of the plutWBI infection. This is associated with substantial sex ratio distortion and the underlying mechanisms deserve further study. In contrast, geographic origin is a poor predictor of moth mtDNA sequences, reflecting human activity in moving the insects around the globe. The exception is a clade of Australasian individuals, which may reflect a bottleneck during their recent introduction to this region.
High throughput, high resolution selection of polymorphic microsatellite loci for multiplex analysis
Resumo:
Background Large-scale genetic profiling, mapping and genetic association studies require access to a series of well-characterised and polymorphic microsatellite markers with distinct and broad allele ranges. Selection of complementary microsatellite markers with non-overlapping allele ranges has historically proved to be a bottleneck in the development of multiplex microsatellite assays. The characterisation process for each microsatellite locus can be laborious and costly given the need for numerous, locus-specific fluorescent primers. Results Here, we describe a simple and inexpensive approach to select useful microsatellite markers. The system is based on the pooling of multiple unlabelled PCR amplicons and their subsequent ligation into a standard cloning vector. A second round of amplification utilising generic labelled primers targeting the vector and unlabelled locus-specific primers targeting the microsatellite flanking region yield allelic profiles that are representative of all individuals contained within the pool. Suitability of various DNA pool sizes was then tested for this purpose. DNA template pools containing between 8 and 96 individuals were assessed for the determination of allele ranges of individual microsatellite markers across a broad population. This helped resolve the balance between using pools that are large enough to allow the detection of many alleles against the risk of including too many individuals in a pool such that rare alleles are over-diluted and so do not appear in the pooled microsatellite profile. Pools of DNA from 12 individuals allowed the reliable detection of all alleles present in the pool. Conclusion The use of generic vector-specific fluorescent primers and unlabelled locus-specific primers provides a high resolution, rapid and inexpensive approach for the selection of highly polymorphic microsatellite loci that possess non-overlapping allele ranges for use in large-scale multiplex assays.
Time-resolved gas-phase kinetic and quantum chemical studies of the reaction of silylene with oxygen
Resumo:
Time-resolved kinetic studies of the reaction of silylene, SiH2, generated by laser flash photolysis of phenylsilane, have been carried out to obtain rate constants for its bimolecular reaction with O-2. The reaction was studied in the gas phase over the pressure range 1-100 Torr in SF6 bath gas, at five temperatures in the range 297-600 K. The second order rate constants at 10 Torr were fitted to the Arrhenius equation: log(k/cm(3) molecule(-1) s(-1)) = (-11.08 +/- 0.04) + (1.57 +/- 0.32 kJ mol(-1))/RT ln10 The decrease in rate constant values with increasing temperature, although systematic is very small. The rate constants showed slight increases in value with pressure at each temperature, but this was scarcely beyond experimental uncertainty. From estimates of Lennard-Jones collision rates, this reaction is occurring at ca. 1 in 20 collisions, almost independent of pressure and temperature. Ab initio calculations at the G3 level backed further by multi-configurational (MC) SCF calculations, augmented by second order perturbation theory (MRMP2), support a mechanism in which the initial adduct, H2SiOO, formed in the triplet state (T), undergoes intersystem crossing to the more stable singlet state (S) prior to further low energy isomerisation processes leading, via a sequence of steps, ultimately to dissociation products of which the lowest energy pair are H2O + SiO. The decomposition of the intermediate cyclo-siladioxirane, via O-O bond fission, plays an important role in the overall process. The bottleneck for the overall process appears to be the T -> S process in H2SiOO. This process has a small spin orbit coupling matrix element, consistent with an estimate of its rate constant of 1 x 10(9) s(-1) obtained with the aid of RRKM theory. This interpretation preserves the idea that, as in its reactions in general, SiH2 initially reacts at the encounter rate with O-2. The low values for the secondary reaction barriers on the potential energy surface account for the lack of an observed pressure dependence. Some comparisons are drawn with the reactions of CH2 + O-2 and SiCl2 + O-2.
Resumo:
The past decade has witnessed explosive growth of mobile subscribers and services. With the purpose of providing better-swifter-cheaper services, radio network optimisation plays a crucial role but faces enormous challenges. The concept of Dynamic Network Optimisation (DNO), therefore, has been introduced to optimally and continuously adjust network configurations, in response to changes in network conditions and traffic. However, the realization of DNO has been seriously hindered by the bottleneck of optimisation speed performance. An advanced distributed parallel solution is presented in this paper, as to bridge the gap by accelerating the sophisticated proprietary network optimisation algorithm, while maintaining the optimisation quality and numerical consistency. The ariesoACP product from Arieso Ltd serves as the main platform for acceleration. This solution has been prototyped, implemented and tested. Real-project based results exhibit a high scalability and substantial acceleration at an average speed-up of 2.5, 4.9 and 6.1 on a distributed 5-core, 9-core and 16-core system, respectively. This significantly outperforms other parallel solutions such as multi-threading. Furthermore, augmented optimisation outcome, alongside high correctness and self-consistency, have also been fulfilled. Overall, this is a breakthrough towards the realization of DNO.
Resumo:
The increasing demand for cheaper-faster-better services anytime and anywhere has made radio network optimisation much more complex than ever before. In order to dynamically optimise the serving network, Dynamic Network Optimisation (DNO), is proposed as the ultimate solution and future trend. The realization of DNO, however, has been hindered by a significant bottleneck of the optimisation speed as the network complexity grows. This paper presents a multi-threaded parallel solution to accelerate complicated proprietary network optimisation algorithms, under a rigid condition of numerical consistency. ariesoACP product from Arieso Ltd serves as the platform for parallelisation. This parallel solution has been benchmarked and results exhibit a high scalability and a run-time reduction by 11% to 42% based on the technology, subscriber density and blocking rate of a given network in comparison with the original version. Further, it is highly essential that the parallel version produces equivalent optimisation quality in terms of identical optimisation outputs.
Resumo:
The ability to create accurate geometric models of neuronal morphology is important for understanding the role of shape in information processing. Despite a significant amount of research on automating neuron reconstructions from image stacks obtained via microscopy, in practice most data are still collected manually. This paper describes Neuromantic, an open source system for three dimensional digital tracing of neurites. Neuromantic reconstructions are comparable in quality to those of existing commercial and freeware systems while balancing speed and accuracy of manual reconstruction. The combination of semi-automatic tracing, intuitive editing, and ability of visualizing large image stacks on standard computing platforms provides a versatile tool that can help address the reconstructions availability bottleneck. Practical considerations for reducing the computational time and space requirements of the extended algorithm are also discussed.
Resumo:
Education, especially higher education, is considered vital for maintaining national and individual competitiveness in the global knowledge economy. Following the introduction of its “Free Education Policy” as early as 1947, Sri Lanka is now the best performer in basic education in the South Asian region, with a remarkable record in terms of high literacy rates and the achievement of universal primary education. However, access to tertiary education is a bottleneck, due to an acute shortage of university places. In an attempt to address this problem, the government of Sri Lanka has invested heavily in information and communications technologies (ICTs) for distance education. Although this has resulted in some improvement, the authors of this article identify several barriers which are still impeding successful participation for the majority of Sri Lankans wanting to study at tertiary level. These impediments include the lack of infrastructure/resources, low English language proficiency, weak digital literacy, poor quality of materials and insufficient provision of student support. In the hope that future implementations of ICT-enabled education programmes can avoid repeating the mistakes identified by their research in this Sri Lankan case, the authors conclude their paper with a list of suggested policy options.
Resumo:
With the emerging prevalence of smart phones and 4G LTE networks, the demand for faster-better-cheaper mobile services anytime and anywhere is ever growing. The Dynamic Network Optimization (DNO) concept emerged as a solution that optimally and continuously tunes the network settings, in response to varying network conditions and subscriber needs. Yet, the DNO realization is still at infancy, largely hindered by the bottleneck of the lengthy optimization runtime. This paper presents the design and prototype of a novel cloud based parallel solution that further enhances the scalability of our prior work on various parallel solutions that accelerate network optimization algorithms. The solution aims to satisfy the high performance required by DNO, preliminarily on a sub-hourly basis. The paper subsequently visualizes a design and a full cycle of a DNO system. A set of potential solutions to large network and real-time DNO are also proposed. Overall, this work creates a breakthrough towards the realization of DNO.
Resumo:
The Mobile Network Optimization (MNO) technologies have advanced at a tremendous pace in recent years. And the Dynamic Network Optimization (DNO) concept emerged years ago, aimed to continuously optimize the network in response to variations in network traffic and conditions. Yet, DNO development is still at its infancy, mainly hindered by a significant bottleneck of the lengthy optimization runtime. This paper identifies parallelism in greedy MNO algorithms and presents an advanced distributed parallel solution. The solution is designed, implemented and applied to real-life projects whose results yield a significant, highly scalable and nearly linear speedup up to 6.9 and 14.5 on distributed 8-core and 16-core systems respectively. Meanwhile, optimization outputs exhibit self-consistency and high precision compared to their sequential counterpart. This is a milestone in realizing the DNO. Further, the techniques may be applied to similar greedy optimization algorithm based applications.