Dependency bagging

In this paper, a new variant of Bagging named DepenBag is proposed. This algorithm obtains bootstrap samples at first. Then, it employs a causal discoverer to induce from each sample a dependency model expressed as a Directed Acyclic Graph (DAG). The attributes without connections to the class attribute in all the DAGs are then removed. Finally, a component learner is trained from each of the resulted samples to constitute the ensemble. Empirical study shows that DepenBag is effective in building ensembles of nearest neighbor classifiers.

Cause and effect in studies on unemployment, mental health and suicide: a meta-analytic and conceptual review

Background There are ongoing questions about whether unemployment has causal effects on suicide as this relationship may be confounded by past experiences of mental illness. The present review quantified the effects of adjustment for mental health on the relationship between unemployment and suicide. Findings were used to develop and interpret likely causal models of unemployment, mental health and suicide. Method A random-effects meta-analysis was conducted on five population-based cohort studies where temporal relationships could be clearly ascertained. Results Results of the meta-analysis showed that unemployment was associated with a significantly higher relative risk (RR) of suicide before adjustment for prior mental health [RR 1.58, 95% confidence interval (CI) 1.33–1.83]. After controlling for mental health, the RR of suicide following unemployment was reduced by approximately 37% (RR 1.15, 95% CI 1.00–1.30). Greater exposure to unemployment was associated with higher RR of suicide, and the pooled RR was higher for males than for females. Conclusions Plausible interpretations of likely pathways between unemployment and suicide are complex and difficult to validate given the poor delineation of associations over time and analytic rationale for confounder adjustment evident in the revised literature. Future research would be strengthened by explicit articulation of temporal relationships and causal assumptions. This would be complemented by longitudinal study designs suitable to assess potential confounders, mediators and effect modifiers influencing the relationship between unemployment and suicide.

Workflow scheduling on distributed systems

Growing evidence shows that in obtaining high performance, a well-managed time-constrained workflow scheduling is needed. Efficient workflow scheduling is critical for achieving high performance especially in heterogeneous computing system. However, it is a great challenge to improve performance and to optimize several objectives simultaneously. We propose a workflow scheduling algorithm that minimizes the makespan of the workflow application modeled by a Directed Acyclic Graph (DAG). The new proposed scheduling algorithm is named Multi Dependency Joint (MDJ) Algorithm. The performance of MDJ is compared with existing algorithms such as, Highest Level First with Estimated Time (HLFET), Modified Critical Path (MCP) and Earliest Time First (ETF). As a result, the experiments show that our proposed MDJ algorithm outperforms HLEFT, MCP, and EFT with a 7% lower overall completion time.

A parallel random forest algorithm for big data in a spark cloud computing environment

With the emergence of the big data age, the issue of how to obtain valuable knowledge from a dataset efficiently and accurately has attracted increasingly attention from both academia and industry. This paper presents a Parallel Random Forest (PRF) algorithm for big data on the Apache Spark platform. The PRF algorithm is optimized based on a hybrid approach combining data-parallel and task-parallel optimization. From the perspective of data-parallel optimization, a vertical data-partitioning method is performed to reduce the data communication cost effectively, and a data-multiplexing method is performed is performed to allow the training dataset to be reused and diminish the volume of data. From the perspective of task-parallel optimization, a dual parallel approach is carried out in the training process of RF, and a task Directed Acyclic Graph (DAG) is created according to the parallel training process of PRF and the dependence of the Resilient Distributed Datasets (RDD) objects. Then, different task schedulers are invoked for the tasks in the DAG. Moreover, to improve the algorithm's accuracy for large, high-dimensional, and noisy data, we perform a dimension-reduction approach in the training process and a weighted voting approach in the prediction process prior to parallelization. Extensive experimental results indicate the superiority and notable advantages of the PRF algorithm over the relevant algorithms implemented by Spark MLlib and other studies in terms of the classification accuracy, performance, and scalability.