Looking Inside The Black Box

Intensive family preservation services (IFPS), designed to stabilize at-risk families and avert out-of-home care, have been the focus of many randomized, experimental studies. The emphasis on "gold-standard" evaluation of IFPS has resulted in fewer "black box" studies that describe actual IFPS service patterns and the fidelity with which they adhere to IFPS program theory. Intervention research is important to the advancement of programs designed to protect the safety of children, improve family functioning, as well as prevent out-of-home placement. Employing a retrospective “clinical data-mining” (CDM) methodology, this exploratory study of Families First, an IFPS program, makes use of available information extracted from client records to describe interventions and service patterns provided over a two year period. This study uncovers actual IFPS service patterns, demonstrates IFPS program fidelity, as well as reveals the usefulness of CDM as a social work research methodology. These findings are particularly valuable for program planning and treatment, policy development and evidence-based practice research.

Analytical and numerical analysis of 40 Gbit/s regenerated transmissions using simple black box optical regenerator

A simplified (without phase modulator) scheme of a black box optical regenerator is proposed, where an appropriate nonlinear propagation is used to enhance regeneration. Applying semi-theoretical models the authors optimise and demonstrate feasibility of error-free long distance transmission at 40 Gbit/s.

Meaningful insights: explainability techniques for black-box models on tabular data

Artificial Intelligence (AI) and Machine Learning (ML) are novel data analysis techniques providing very accurate prediction results. They are widely adopted in a variety of industries to improve efficiency and decision-making, but they are also being used to develop intelligent systems. Their success grounds upon complex mathematical models, whose decisions and rationale are usually difficult to comprehend for human users to the point of being dubbed as black-boxes. This is particularly relevant in sensitive and highly regulated domains. To mitigate and possibly solve this issue, the Explainable AI (XAI) field became prominent in recent years. XAI consists of models and techniques to enable understanding of the intricated patterns discovered by black-box models. In this thesis, we consider model-agnostic XAI techniques, which can be applied to Tabular data, with a particular focus on the Credit Scoring domain. Special attention is dedicated to the LIME framework, for which we propose several modifications to the vanilla algorithm, in particular: a pair of complementary Stability Indices that accurately measure LIME stability, and the OptiLIME policy which helps the practitioner finding the proper balance among explanations' stability and reliability. We subsequently put forward GLEAMS a model-agnostic surrogate interpretable model which requires to be trained only once, while providing both Local and Global explanations of the black-box model. GLEAMS produces feature attributions and what-if scenarios, from both dataset and model perspective. Eventually, we argue that synthetic data are an emerging trend in AI, being more and more used to train complex models instead of original data. To be able to explain the outcomes of such models, we must guarantee that synthetic data are reliable enough to be able to translate their explanations to real-world individuals. To this end we propose DAISYnt, a suite of tests to measure synthetic tabular data quality and privacy.

Fail-Safe Testing of Web Applications

This dissertation introduces an approach to generate tests to test fail-safe behavior for web applications. We apply the approach to a commercial web application. We build models for both behavioral and mitigation requirements. We create mitigation tests from an existing functional black box test suite by determining failure type and points of failure in the test suite and weaving required mitigation based on weaving rules to generate a test suite that tests proper mitigation of failures. A genetic algorithm (GA) is used to determine points of failure and type of failure that needs to be tested. Mitigation test paths are woven into the behavioral test at the point of failure based on failure specific weaving rules. A simulator was developed to evaluate choice of parameters for the genetic algorithm. We showed how to tune the fitness function and performed tuning experiments for GA to determine what values to use for exploration weight and prospecting weight. We found that higher defect densities make prospecting and mining more successful, while lower mitigation defect densities need more exploration. We compare efficiency and effectiveness of the approach. First, the GA approach is compared to random selection. The results show that the GA performance was better than random selection and that the approach was robust when the search space increased. Second, we compare the GA against four coverage criteria. The results of comparison show that test requirements generated by a genetic algorithm (GA) are more efficient than three of the four coverage criteria for large search spaces. They are equally effective. For small search spaces, the genetic algorithm is less effective than three of the four coverage criteria. The fourth coverage criteria is too weak and unable to find all defects in almost all cases. We also present a large case study of a mortgage system at one of our industrial partners and show how we formalize the approach. We evaluate the use of a GA to create test requirements. The evaluation includes choice of initial population, multiplicity of runs and a discussion of the cost of evaluating fitness. Finally, we build a selective regression testing approach based on types of changes (add, delete, or modify) that could occur in the behavioral model, the fault model, the mitigation models, the weaving rules, and the state-event matrix. We provide a systematic method by showing the formalization steps for each type of change to the various models.

Testing for Seasonal Unit Roots when Residuals Contain Serial Correlations under HEGY Test Framework

This paper introduces a corrected test statistic for testing seasonal unit roots when residuals contain serial correlations, based on the HEGY test proposed by Hylleberg,Engle, Granger and Yoo (1990). The serial correlations in the residuals of test regressionare accommodated by making corrections to the commonly used HEGY t statistics. Theasymptotic distributions of the corrected t statistics are free from nuisance parameters.The size and power properties of the corrected statistics for quarterly and montly data are investigated. Based on our simulations, the corrected statistics for monthly data havemore power compared with the commonly used HEGY test statistics, but they also have size distortions when there are strong negative seasonal correlations in the residuals.

Testing motor fitness in karate

Background and Study Aim: Evaluation of sport skills test can be very useful tool for coach practice. The aim of the present paper was: (a) to evaluate the reliability and accuracy of the Specific Physical Fitness Tests (SPFT) (b) to review the results of karate athletes who represent different weight categories, and who are at different stages of schooling; (c) to establish grading criteria of physical fitness preparation. Material/Methods: The reseach was conducted among 219 Kyokushin karate players, whose profiles were presented as (chi) over bar +/- SD and their main characteristics were the following: age 26.8 +/- 4.67 (19-39) years, body mass 75.2 +/- 8.35 (50-97) kg and body height 176.4 +/- 5.67 (160-196) cm. The value of the BMI amounted to 24.1 +/- 2.17 (17.9-29.4) kg/m(2). All the subjects of the research had training experience of 10.5 +/- 3.71 (4-20) years and their degree of proficiency ranged from 4(th) kyu to 3(rd) dan. The physical fitness trials proposed by Story (1989) included: hip turning speed, speed punches, flexibility, rapid kicks, agility, and evasion actions. It was supplemented by a test of local strength endurance, composing a battery of the SPFT, which was implemented by first of the authors between 1991 and 2006. Results: SPFT is characterized by high reliability and it can be used to diagnose the physical fitness preparation and monitor the individual results of training. It discriminates accurately competitors with different sports level and it is characterized by very high accuracy, it is correlated with the test results of motor general physical fitness abilities and coordination abilities as well as it is connected with the somatic build of the athlete. The performance classification table was developed on the basis of our research. Discussion: Results obtained in SPFT were shortly discussed. Conclusions: The collected results of our research allowed us to come to, the conclusion: The table can be applied not only to assess karate fighters, but also adepts in taekwondo, kick-boxing, ju-jitsu, hapkido or other mixed martial arts.

A framework for specification-based testing

Test templates and a test template framework are introduced as useful concepts in specification-based testing. The framework can be defined using any model-based specification notation and used to derive tests from model-based specifications-in this paper, it is demonstrated using the Z notation. The framework formally defines test data sets and their relation to the operations in a specification and to other test data sets, providing structure to the testing process. Flexibility is preserved, so that many testing strategies can be used. Important application areas of the framework are discussed, including refinement of test data, regression testing, and test oracles.