Desenvolvimento de Frameworks de Testes Automáticos de Software

O objetivo deste trabalho é o desenvolvimento de frameworks de testes automáticos de software. Este tipo de testes normalmente está associado ao modelo evolucionário e às metodologias ágeis de desenvolvimento de software, enquanto que os testes manuais estão relacionados com o modelo em cascata e as metodologias tradicionais. Como tal foi efetuado um estudo comparativo sobre os tipos de metodologias e de testes existentes, para decidir quais os que melhor se adequavam ao projeto e dar resposta à questão "Será que realmente compensa realizar testes (automáticos)?". Finalizado o estudo foram desenvolvidas duas frameworks, a primeira para a implementação de testes funcionais e unitários sem dependências a ser utilizada pelos estagiários curriculares da LabOrders, e a segunda para a implementação de testes unitários com dependências externas de base de dados e serviços, a ser utilizada pelos funcionários da empresa. Nas últimas duas décadas as metodologias ágeis de desenvolvimento de software não pararam de evoluir, no entanto as ferramentas de automação não conseguiram acompanhar este progresso. Muitas áreas não são abrangidas pelos testes e por isso alguns têm de ser feitos manualmente. Posto isto foram criadas várias funcionalidades inovadoras para aumentar a cobertura dos testes e tornar as frameworks o mais intuitivas possível, nomeadamente: 1. Download automático de ficheiros através do Internet Explorer 9 (e versões mais recentes). 2. Análise do conteúdo de ficheiros .pdf (através dos testes). 3. Obtenção de elementos web e respetivos atributos através de código jQuery utilizando a API WebDriver com PHP bindings. 4. Exibição de mensagens de erro personalizadas quando não é possível encontrar um determinado elemento. As frameworks implementadas estão também preparadas para a criação de outros testes (de carga, integração, regressão) que possam vir a ser necessários no futuro. Foram testadas em contexto de trabalho pelos colaboradores e clientes da empresa onde foi realizado o projeto de mestrado e os resultados permitiram concluir que a adoção de uma metodologia de desenvolvimento de software com testes automáticos pode aumentar a produtividade, reduzir as falhas e potenciar o cumprimento de orçamentos e prazos dos projetos das organizações.

Caso de demonstração de uma framework para automatização do teste de APIs de aplicações SaaS

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Virtuaalisen oppimisympäristön funktionaalisen käyttöliittymätestauksen automatisointi

Tämän tutkimuksen kohteena on Turun yliopistossa kehitetyn www-pohjaisen ViLLE-oppimisjärjestelmän funktionaalisen käyttöliittymätestauksen automatisointityö. Tutkimusta varten olen kerännyt kattavasti aineistoa aihetta käsittelevästä yleisestä kirjallisuudesta ja artikkeleista sekä toteutuksen kannalta spesifistä tietoa tarjoavista Internet-lähteistä. Tutkimuksessa olen tehnyt myös pienehkön määrän testausalan asiantuntijahaastatteluja. Tutkimuksen empiirisessä osuudessa olen valinnut testaukseen käytettävän testaustyökalun sekä toteuttanut valitulla testaustyökalulla ViLLE-oppimisjärjestelmän testauksen automatisointityön soveltamalla käytäntöön tutkimuksen teoriaosuudessa esitettyä tietoa hyvistä käytänteistä funktionaalisen käyttöliittymätestauksen automatisoinnissa. Tutkimuksen toteutuksessa olen käyttänyt kvalitatiivista tutkimusmenetelmää. Tutkimuksen empiirisen osuuden pohjalta kerätyn havaintoaineiston perusteella olen selvittänyt vastaukset seuraaviin tutkimuksessa esitettyihin tutkimuskysymyksiin: • Miten käytetty testaustyökalu on valittu ja mitkä olivat valintaan vaikuttaneet tärkeimmät kriteerit? • Miten käytetty testaustyökalu soveltuu ViLLE-oppimisjärjestelmän funktionaalisen käyttöliittymätestauksen automatisointiin? • Millä eri tavoin käytäntöön viety hyvien testiautomaation laatimistapojen mukainen toteutus vaikuttaa nyt laadittuun testiautomaatioon? • Esiintyikö toteutetussa testiautomaatiossa tutkimuksen teoreettisessa viitekehyksessä kuvattuja funktionaalisen käyttöliittymätestauksen automatisoinnille tyypillisiä ongelmia ja miten ongelmat saatiin ratkaistua? Tutkimuksen tulokset osoittavat melko selvästi, että ViLLE-oppimisympäristön funktionaalisen käyttöliittymätestauksen automatisointityön toteutukseen valittu Vaadin TestBench -testaustyökalu, joka on valittu tutkimuksen alkuvaiheessa suoritetun evaluoinnin perusteella, soveltuu käyttötarkoitukseensa hyvin. Lisäksi pystyin luotettavasti havainnoimaan, että testiautomaation ylläpidon tarve sekä testien laatimiseen kuluva aika vähenevät merkittävästi, kun testit laaditaan heti alusta lähtien rakenteeltaan modulaariseksi sekä tietyin teknisin keinoin mahdollisimman vähän käyttöliittymän rakennetta huomioonottavaksi. Ongelmia testiautomaation laatimisessa voivat aiheuttaa käytetty työkalu itsessään, testattavan järjestelmän toteutus sekä testien suoritusympäristö. Huolimatta kirjallisuuskatsauksen perusteella tehdystä varautumisesta tyypillisiin testiautomaation laatimisessa esiintyviin ongelmiin, myös joitakin sellaisia ongelmia esiintyi, joihin en ollut osannut varautua. Mahdollisiin ongelmiin etukäteen varautuminen kuitenkin selvästi auttoi suurimpaan osaan testiautomaation laatimisessa esiintyneistä ongelmista.

Methodology for WSN communication technologies automated field tests

Wireless Sensor Networks(WSN) are networks of devices used to sense and act that applies wireless radios to communicate. To achieve a successful implementation of a wireless device it is necessary to take in consideration the existence of a wide variety of radios available, a large number of communication parameters (payload, duty cycle, etc.) and environmental conditions that may affect the device’s behaviour. However, to evaluate a specific radio towards a unique application it might be necessary to conduct trial experiments, with such a vast amount of devices, communication parameters and environmental conditions to take into consideration the number of trial cases generated can be surprisingly high. Thus, making trial experiments to achieve manual validation of wireless communication technologies becomes unsuitable due to the existence of a high number of trial cases on the field. To overcome this technological issue an automated test methodology was introduced, presenting the possibility to acquire data regarding the device’s behaviour when testing several technologies and parameters that care for a specific analysis. Therefore, this method advances the validation and analysis process of the wireless radios and allows the validation to be done without the need of specific and in depth knowledge about wireless devices.

Towards automated test and validation of SIP solutions

IP networks are currently the major communication infrastructure used by an increasing number of applications and heterogeneous services, including voice services. In this context, the Session Initiation Protocol (SIP) is a signaling protocol widely used for controlling multimedia communication sessions such as voice or video calls over IP networks, thus performing vital functions in an extensive set of public and enter- prise solutions. However, the SIP protocol dissemination also entails some challenges, such as the complexity associated with the testing/validation processes of IMS/SIP networks. As a consequence, manual IMS/SIP testing solutions are inherently costly and time consuming tasks, being crucial to develop automated approaches in this specific area. In this perspective, this article presents an experimental approach for automated testing/validation of SIP scenarios in IMS networks. For that purpose, an automation framework is proposed allowing to replicate the configuration of SIP equipment from the pro- duction network and submit such equipment to a battery of tests in the testing network. The proposed solution allows to drastically reduce the test and validation times when compared with traditional manual approaches, also allowing to enhance testing reliability and coverage. The automation framework comprises of some freely available tools which are conveniently integrated with other specific modules implemented within the context of this work. In order to illustrate the advantages of the proposed automated framework, a real case study taken from a PT Inovação customer is presented comparing the time required to perform a manual SIP testing approach with the one time required when using the proposed auto- mated framework. The presented results clearly corroborate the advantages of using the presented framework.

Automated systems in the identification and determination of methicillin resistance among coagulase negative staphylococci

Coagulase-negative staphylococci (CoNS) are an important cause of nosocomial bacteremia, specially in patients with indwelling devices or those submitted to invasive medical procedures. The identification of species and the accurate and rapid detection of methicillin resistance are directly dependent on the quality of the identification and susceptibility tests used, either manual or automated. The objective of this study was to evaluate the accuracy of two automated systems MicroScan and Vitek - in the identification of CoNS species and determination of susceptibility to methicillin, considering as gold standard the biochemical tests and the characterization of the mecA gene by polymerase chain reaction, respectively. MicroScan presented better results in the identification of CoNS species (accuracy of 96.8 vs 78.8%, respectively); isolates from the following species had no precise identification: Staphylococcus haemolyticus, S. simulans, and S. capitis. Both systems were similar in the characterization of methicillin resistance. The higher discrepancies for gene mec detection were observed among species other than S. epidermidis (S. hominis, S. saprophyticus, S. sciuri, S. haemolyticus, S. warneri, S. cohnii), and those with borderline MICs.

Laboratory detection of respiratory viruses by automated techniques.

Advances in clinical virology for detecting respiratory viruses have been focused on nucleic acids amplification techniques, which have converted in the reference method for the diagnosis of acute respiratory infections of viral aetiology. Improvements of current commercial molecular assays to reduce hands-on-time rely on two strategies, a stepwise automation (semi-automation) and the complete automation of the whole procedure. Contributions to the former strategy have been the use of automated nucleic acids extractors, multiplex PCR, real-time PCR and/or DNA arrays for detection of amplicons. Commercial fully-automated molecular systems are now available for the detection of respiratory viruses. Some of them could convert in point-of-care methods substituting antigen tests for detection of respiratory syncytial virus and influenza A and B viruses. This article describes laboratory methods for detection of respiratory viruses. A cost-effective and rational diagnostic algorithm is proposed, considering technical aspects of the available assays, infrastructure possibilities of each laboratory and clinic-epidemiologic factors of the infection.

Introduction of an automated system for the diagnosis and quantification of hepatitis B and hepatitis C viruses.

Hepatitis B virus (HBV) and Hepatitis C virus (HCV) infections pose major public health problems because of their prevalence worldwide. Consequently, screening for these infections is an important part of routine laboratory activity. Serological and molecular markers are key elements in diagnosis, prognosis and treatment monitoring for HBV and HCV infections. Today, automated chemiluminescence immunoassay (CLIA) analyzers are widely used for virological diagnosis, particularly in high-volume clinical laboratories. Molecular biology techniques are routinely used to detect and quantify viral genomes as well as to analyze their sequence; in order to determine their genotype and detect resistance to antiviral drugs. Real-time PCR, which provides high sensitivity and a broad dynamic range, has gradually replaced other signal and target amplification technologies for the quantification and detection of nucleic acid. The next-generation DNA sequencing techniques are still restricted to research laboratories.The serological and molecular marker methods available for HBV and HCV are discussed in this article, along with their utility and limitations for use in Chronic Hepatitis B (CHB) diagnosis and monitoring.

Evaluation of three automated systems for susceptibility testing of enterobacteria containing qnrB, qnrS, and/or aac(6')-Ib-cr.

The accuracy of the MicroScan WalkAway, BD Phoenix, and Vitek-2 systems for susceptibility testing of quinolones and aminoglycosides against 68 enterobacteria containing qnrB, qnrS, and/or aac(6 ')-Ib-cr was evaluated using reference microdilution. Overall, one very major error (0.09%), 6 major errors (0.52%), and 45 minor errors (3.89%) were noted.

The Effect Of Induced Intraocular Straylight (cataract Simulation) On Sensitivity Measures Of Standard Automated Perimetry, Pulsar Perimetry And Moorfields Motion Displacement Test

Purpose: To investigate the effect of incremental increases in intraocular straylight on threshold measurements made by three modern forms of perimetry: Standard Automated Perimetry (SAP) using Octopus (Dynamic, G-Pattern), Pulsar Perimetry (PP) (TOP, 66 points) and the Moorfields Motion Displacement Test (MDT) (WEBS, 32 points).Methods: Four healthy young observers were recruited (mean age 26yrs [25yrs, 28yrs]), refractive correction [+2 D, -4.25D]). Five white opacity filters (WOF), each scattering light by different amounts were used to create incremental increases in intraocular straylight (IS). Resultant IS values were measured with each WOF and at baseline (no WOF) for each subject using a C-Quant Straylight Meter (Oculus, Wetzlar, Germany). A 25 yr old has an IS value of ~0.85 log(s). An increase of 40% in IS to 1.2log(s) corresponds to the physiological value of a 70yr old. Each WOFs created an increase in IS between 10-150% from baseline, ranging from effects similar to normal aging to those found with considerable cataract. Each subject underwent 6 test sessions over a 2-week period; each session consisted of the 3 perimetric tests using one of the five WOFs and baseline (both instrument and filter were randomised).Results: The reduction in sensitivity from baseline was calculated. A two-way ANOVA on mean change in threshold (where subjects were treated as rows in the block and each increment in fog filters was treated as column) was used to examine the effect of incremental increases in straylight. Both SAP (p<0.001) and Pulsar (p<0.001) were significantly affected by increases in straylight. The MDT (p=0.35) remained comparatively robust to increases in straylight.Conclusions: The Moorfields MDT measurement of threshold is robust to effects of additional straylight as compared to SAP and PP.

Comparative resilience of clinical perimetric tests to induced levels of intraocular straylight.

PURPOSE: To investigate the effect of intraocular straylight (IOS) induced by white opacity filters (WOF) on threshold measurements for stimuli employed in three perimeters: standard automated perimetry (SAP), pulsar perimetry (PP) and the Moorfields motion displacement test (MDT).¦METHODS: Four healthy young (24-28 years old) observers were tested six times with each perimeter, each time with one of five different WOFs and once without, inducing various levels of IOS (from 10% to 200%). An increase in IOS was measured with a straylight meter. The change in sensitivity from baseline was normalized, allowing comparison of standardized (z) scores (change divided by the SD of normative values) for each instrument.¦RESULTS: SAP and PP thresholds were significantly affected (P < 0.001) by moderate to large increases in IOS (50%-200%). The drop in motion displacement (MD) from baseline with WOF 5, was approximately 5 dB, in both SAP and PP which represents a clinically significant loss; in contrast the change in MD with MDT was on average 1 minute of arc, which is not likely to indicate a clinically significant loss.¦CONCLUSIONS: The Moorfields MDT is more robust to the effects of additional straylight in comparison with SAP or PP.

Performance of an automated multiplex immunofluorescence assay for detection of Chlamydia trachomatis immunoglobulin G.

Chlamydia serology is indicated to investigate etiology of miscarriage, infertility, pelvic inflammatory disease, and ectopic pregnancy. Here, we assessed the reliability of a new automated-multiplex immunofluorescence assay (InoDiag test) to detect specific anti-C. trachomatis immunoglobulin G. Considering immunofluorescence assay (IF) as gold standard, InoDiag tests exhibited similar sensitivities (65.5%) but better specificities (95.1%-98%) than enzyme-linked immunosorbent assays (ELISAs). InoDiag tests demonstrated similar or lower cross-reactivity rates when compared to ELISA or IF.

Automated DNA extraction using the QIAsymphony platform: Estimation of DNA recovery from simulated forensic stains

The aim of this study was to evaluate the forensic protocol recently developed by Qiagen for the QIAsymphony automated DNA extraction platform. Samples containing low amounts of DNA were specifically considered, since they represent the majority of samples processed in our laboratory. The analysis of simulated blood and saliva traces showed that the highest DNA yields were obtained with the maximal elution volume available for the forensic protocol, that is 200 ml. Resulting DNA extracts were too diluted for successful DNA profiling and required a concentration. This additional step is time consuming and potentially increases inversion and contamination risks. The 200 ml DNA extracts were concentrated to 25 ml, and the DNA recovery estimated with real-time PCR as well as with the percentage of SGM Plus alleles detected. Results using our manual protocol, based on the QIAamp DNA mini kit, and the automated protocol were comparable. Further tests will be conducted to determine more precisely DNA recovery, contamination risk and PCR inhibitors removal, once a definitive procedure, allowing the concentration of DNA extracts from low yield samples, will be available for the QIAsymphony.

Multicenter study of a new fully automated HBsAg screening assay with enhanced sensitivity for the detection of HBV mutants.

In a multicenter study a new, fully automated Roche Diagnostics Elecsys HBsAg II screening assay with improved sensitivity to HBsAg mutant detection was compared to well-established HBsAg tests: AxSYM HBsAg V2 (Abbott), Architect HBsAg (Abbott), Advia Centaur HBsAg (Bayer) Enzygnost HBsAg 5.0 (Dade-Behring), and Vitros Eci HBsAg (Ortho). A total of 16 seroconversion panels, samples of 60 HBsAg native mutants, and 31 HBsAg recombinant mutants, dilution series of NIBSC and PEI standards, 156 HBV positive samples comprising genotypes A to G, 686 preselected HBsAg positive samples from different stages of infection, 3,593 samples from daily routine, and 6,360 unselected blood donations were tested to evaluate the analytical and clinical sensitivity, the detection of mutants, and the specificity of the new assay. Elecsys HBsAg II showed a statistically significant better sensitivity in seroconversion panels to the compared tests. Fifty-seven out of 60 native mutants and all recombinant mutants were found positive. Among 156 HBV samples with different genotypes and 696 preselected HBsAg positive samples Elecsys HBsAg II achieved a sensitivity of 100%. The lower detection limit for NIBSC standard was calculated to be 0.025 IU/ml and for the PEI standards ad and ay it was <0.001 and <0.005 U/ml, respectively. Within 2,724 daily routine specimens and 6.360 unselected blood donations Elecsys HBsAg II showed a specificity of 99.97 and 99.88%, respectively. In conclusion the new Elecsys HBsAg II shows a high sensitivity for the detection of all stages of HBV infection and HBsAg mutants paired together with a high specificity in blood donors, daily routine samples, and potentially interfering sera.

Comparison of five commercial serological tests for the detection of anti-Chlamydia trachomatis antibodies.

Screening for Chlamydia trachomatis-specific antibodies is valuable in investigating recurrent miscarriage, tubal infertility and extrauterine pregnancy. We compared here the performance of immunofluorescence (IF) to four other commercial tests in detecting IgG antibodies directed against C. trachomatis: two enzyme-linked immunosorbent assays (ELISAs) using the major outer membrane protein (MOMP) as the antigen, commercialised respectively by Medac and R-Biopharm (RB), one ELISA using the chlamydial heat shock protein 60 (cHSP60) as the antigen (Medac), as well as a new automated epifluorescence immunoassay (InoDiag). A total of 405 patients with (n = 251) and without (n = 154) miscarriages were tested by all five tests. The prevalence of C. trachomatis-specific IgG antibodies as determined by the IF, cHSP60-Medac, MOMP-Medac, MOMP-RB and InoDiag was 14.3, 23.2, 14.3, 11.9 and 26.2%, respectively. InoDiag exhibited the highest sensitivity, whereas MOMP-RB showed the best specificity. Cross-reactivity was observed with C. pneumoniae using IF, MOMP-RB and InoDiag, and Parachlamydia acanthamoebae using the cHSP60 ELISA test. No cross-reactivity was observed between C. trachomatis and the other Chlamydiales (Neochlamydia hartmannellae, Waddlia chondrophila and Simkania negevensis). Given its high sensitivity, the new automated epifluorescence immunoassay from InoDiag represents an interesting alternative. The MOMP-based ELISA of R-Biopharm should be preferred for large serological studies, given the high throughput of ELISA and its excellent specificity.