Overview of the ShARe/CLEF eHealth Evaluation Lab 2014

This paper reports on the 2nd ShARe/CLEFeHealth evaluation lab which continues our evaluation resource building activities for the medical domain. In this lab we focus on patients' information needs as opposed to the more common campaign focus of the specialised information needs of physicians and other healthcare workers. The usage scenario of the lab is to ease patients and next-of-kins' ease in understanding eHealth information, in particular clinical reports. The 1st ShARe/CLEFeHealth evaluation lab was held in 2013. This lab consisted of three tasks. Task 1 focused on named entity recognition and normalization of disorders; Task 2 on normalization of acronyms/abbreviations; and Task 3 on information retrieval to address questions patients may have when reading clinical reports. This year's lab introduces a new challenge in Task 1 on visual-interactive search and exploration of eHealth data. Its aim is to help patients (or their next-of-kin) in readability issues related to their hospital discharge documents and related information search on the Internet. Task 2 then continues the information extraction work of the 2013 lab, specifically focusing on disorder attribute identification and normalization from clinical text. Finally, this year's Task 3 further extends the 2013 information retrieval task, by cleaning the 2013 document collection and introducing a new query generation method and multilingual queries. De-identified clinical reports used by the three tasks were from US intensive care and originated from the MIMIC II database. Other text documents for Tasks 1 and 3 were from the Internet and originated from the Khresmoi project. Task 2 annotations originated from the ShARe annotations. For Tasks 1 and 3, new annotations, queries, and relevance assessments were created. 50, 79, and 91 people registered their interest in Tasks 1, 2, and 3, respectively. 24 unique teams participated with 1, 10, and 14 teams in Tasks 1, 2 and 3, respectively. The teams were from Africa, Asia, Canada, Europe, and North America. The Task 1 submission, reviewed by 5 expert peers, related to the task evaluation category of Effective use of interaction and targeted the needs of both expert and novice users. The best system had an Accuracy of 0.868 in Task 2a, an F1-score of 0.576 in Task 2b, and Precision at 10 (P@10) of 0.756 in Task 3. The results demonstrate the substantial community interest and capabilities of these systems in making clinical reports easier to understand for patients. The organisers have made data and tools available for future research and development.

An investigation into resistance practices at an SME consultancy

Purpose Prior research emphasizes that organizational founders have a good deal of influence in organizational development and, where ICTs are involved, a generic strategy is usually deployed by managers in order to deal with any resistance that might occur. Cognisant of this, we investigated the role played by a Managing Director of an SME consultancy in an ICT project associated with organizational development. Design/methodology/approach This study is based on an ethnography of a ICT related change management initiative which, theoretically, takes into account though from the social shaping of technology – speifically the idea that technologies in their broadest sense are subject to ongoing work beyond the design stage. Findings We argue that Markus’ Interaction Theory of resistance still has relevance today and we extend it by emphasizing the problem of homogenizing users and downplaying their ability to appropriate resistance strategies in situ. Research limitations/implications Our study is based upon one group of individual’s experiences. Further case studies of resistance success are required which further highlight how such this is achieved and why. Practical implications Those engaged with organisational development projects need to be better educated as to the reasons for resistance, particularly positive ones, and the methods by which this might take place. Originality/value This study conceptualises strategies for ‘overcoming’ resistance as managerial technologies. Conceptualising them in this way, shows the deployement of such technologies to be a complicated and active process where the audience for such things are involved in how they are received and appropriated to suite differing agendas.

IT consultants, salesmanship and the challenges of packaged software selection in SMEs

Purpose – This paper seeks to analyse the process of packaged software selection in a small organization, focussing particularly on the role of IT consultants as intermediaries in the process. Design/methodology/approach – This is based upon a longitudinal, qualitative field study concerning the adoption of a customer relationship management package in an SME management consultancy. Findings – The authors illustrate how the process of “salesmanship”, an activity directed by the vendor/consultant and focussed on the interests of senior management, marginalises user needs and ultimately secures the procurement of the software package. Research limitations/implications – Despite the best intentions the authors lose something of the rich detail of the lived experience of technology in presenting the case study as a linear narrative. Specifically, the authors have been unable to do justice to the complexity of the multifarious ways in which individual perceptions of the project were influenced and shaped by the opinions of others. Practical implications – Practitioners, particularly those from within SMEs, should be made aware of the ways in which external parties may have a vested interest in steering projects in a particular direction, which may not necessarily align with their own interests. Originality/value – This study highlights in detail the role of consultants and vendors in software selection processes, an area which has received minimal attention to date. Prior work in this area emphasises the necessary conditions for, and positive outcomes of, appointing external parties in an SME context, with only limited attention being paid to the potential problems such engagements may bring.