Duke Images LibGuide usability test - August 2016

Thomas Crichlow and Gordon Chadwick conducted a think-aloud usability study on August 1, 2016 in the Perkins Library at Duke University. The study tested users’ perceptions of a new LibGuides driven research guide for scholarly images by asking them to complete information seeking tasks and provide feedback.

UNDERSTANDING ESSAY RATING AS A SOCIALLY MEDIATED ACTIVITY: THE CASE OF A HIGH-STAKES ENGLISH TEST IN CHINA

Most essay rating research in language assessment has examined human raters’ essay rating as a cognitive process, thus overlooking or oversimplifying the interaction between raters and sociocultural contexts. Given that raters are social beings, their practices have social meanings and consequences. Hence it is important to situate essay rating within its sociocultural context for a more meaningful understanding. Drawing on Engeström’s (1987, 2001) cultural-historical activity theory (CHAT) framework with a sociocultural perspective, this study reconceptualized essay rating as a socially mediated activity with both cognitive (individual raters’ goal-directed decision-making actions) and social layers (raters’ collective object-oriented essay rating activity at related settings). In particular, this study explored raters’ essay rating at one provincial rating centre in China within the context of a high-stakes university entrance examination, the National Matriculation English Test (NMET). This study adopted a multiple-method multiple-perspective qualitative case study design. Think-aloud protocols, stimulated recalls, interviews, and documents served as the data sources. This investigation involved 25 participants at two settings (rating centre and high schools), including rating centre directors, team leaders, NMET essay raters who were high school teachers, and school principals and teaching colleagues of these essay raters. Data were analyzed using Strauss and Corbin’s (1990) open and axial coding techniques, and CHAT for data integration. The findings revealed the interaction between raters and the NMET sociocultural context. Such interaction can be understood through a surface structure (cognitive layer) and a deep structure (social layer) concerning how raters assessed NMET essays, where the surface structure reflected the “what” and the deep structure explained the “how” and “why” in raters’ decision-making. This study highlighted the roles of goals and rules in rater decision-making, rating tensions and raters’ solutions, and the relationship between essay rating and teaching. This study highlights the value of a sociocultural view to essay rating research, demonstrates CHAT as a sociocultural approach to investigate essay rating, and proposes a direction for future washback research on the effect of essay rating. This study also provides support for NMET rating practices that can potentially bring positive washback to English teaching in Chinese high schools.

Response process and test-retest reliability of the Context Assessment for Community Health tool in Vietnam

BACKGROUND: The recently developed Context Assessment for Community Health (COACH) tool aims to measure aspects of the local healthcare context perceived to influence knowledge translation in low- and middle-income countries. The tool measures eight dimensions (organizational resources, community engagement, monitoring services for action, sources of knowledge, commitment to work, work culture, leadership, and informal payment) through 49 items. OBJECTIVE: The study aimed to explore the understanding and stability of the COACH tool among health providers in Vietnam. DESIGNS: To investigate the response process, think-aloud interviews were undertaken with five community health workers, six nurses and midwives, and five physicians. Identified problems were classified according to Conrad and Blair's taxonomy and grouped according to an estimation of the magnitude of the problem's effect on the response data. Further, the stability of the tool was examined using a test-retest survey among 77 respondents. The reliability was analyzed for items (intraclass correlation coefficient (ICC) and percent agreement) and dimensions (ICC and Bland-Altman plots). RESULTS: In general, the think-aloud interviews revealed that the COACH tool was perceived as clear, well organized, and easy to answer. Most items were understood as intended. However, seven prominent problems in the items were identified and the content of three dimensions was perceived to be of a sensitive nature. In the test-retest survey, two-thirds of the items and seven of eight dimensions were found to have an ICC agreement ranging from moderate to substantial (0.5-0.7), demonstrating that the instrument has an acceptable level of stability. CONCLUSIONS: This study provides evidence that the Vietnamese translation of the COACH tool is generally perceived to be clear and easy to understand and has acceptable stability. There is, however, a need to rephrase and add generic examples to clarify some items and to further review items with low ICC.

The role of metalinguistic awareness and of L2 proficiency in positive lexical transfer from English (L2) to German (L3) by French-speaking Quebeckers

Il a été avancé que des apprenants expérimentés développeraient des niveaux élevés de conscience métalinguistique (MLA), ce qui leur faciliterait l’apprentissage de langues subséquentes (p.ex., Singleton & Aronin, 2007). De plus, des chercheurs dans le domaine de l’acquisition des langues tierces insistent sur les influences positives qu’exercent les langues précédemment apprises sur l’apprentissage formel d’une langue étrangère (p.ex., Cenoz & Gorter, 2015), et proposent de délaisser le regard traditionnel qui mettait l’accent sur l’interférence à l’origine des erreurs des apprenants pour opter pour une vision plus large et positive de l’interaction entre les langues. Il a été démontré que la similarité typologique ainsi que la compétence dans la langue source influence tous les types de transfert (p.ex., Ringbom, 1987, 2007). Cependant, le défi méthodologique de déterminer, à la fois l’usage pertinent d’une langue cible en tant que résultat d’une influence translinguistique (p.ex., Falk & Bardel, 2010) et d’établir le rôle crucial de la MLA dans l’activation consciente de mots ou de constructions reliés à travers différentes langues, demeure. La présente étude avait pour but de relever ce double défi en faisant appel à des protocoles oraux (TAPs) pour examiner le transfert positif de l’anglais (L2) vers l’allemand (L3) chez des Québécois francophones après cinq semaines d’enseignement formel de la L3. Les participants ont été soumis à une tâche de traduction développée aux fins de la présente étude. Les 42 items ont été sélectionnés sur la base de jugements de similarité et d’imagibilité ainsi que de fréquence des mots provenant d’une étude de cognats allemands-anglais (Friel & Kennison, 2001). Les participants devaient réfléchir à voix haute pendant qu’ils traduisaient des mots inconnus de l’allemand (L3) vers le français (L1). Le transfert positif a été opérationnalisé par des traductions correctes qui étaient basées sur un cognat anglais. La MLA a été mesurée par le biais du THAM (Test d’habiletés métalinguistiques) (Pinto & El Euch, 2015) ainsi que par l’analyse des TAPs. Les niveaux de compétence en anglais ont été établis sur la base du Michigan Test (Corrigan et al., 1979), tandis que les niveaux d’exposition ainsi que l’intérêt envers la langue et la culture allemandes ont été mesurés à l’aide d’un questionnaire. Une analyse fine des TAPs a révélé de la variabilité inter- et intra-individuelle dans l’activation consciente du vocabulaire en L2, tout en permettant l’identification de niveaux distincts de prise de conscience. Deux modèles indépendants de régressions logistiques ont permis d’identifier les deux dimensions de MLA comme prédicteurs de transfert positif. Le premier modèle, dans lequel le THAM était la mesure exclusive de MLA, a déterminé cette dimension réflexive comme principal prédicteur, suivie de la compétence en anglais, tandis qu’aucune des autres variables indépendantes pouvait prédire le transfert positif de l’anglais. Dans le second modèle, incluant le THAM ainsi que les TAPs comme mesures complémentaires de MLA, la dimension appliquée de MLA, telle que mesurée par les TAPs, était de loin le prédicteur principal, suivie de la dimension réflexive, telle que mesurée par le THAM, tandis que la compétence en anglais ne figurait plus parmi les facteurs ayant une influence significative sur la variable réponse. Bien que la verbalisation puisse avoir influencé la performance dans une certaine mesure, nos observations mettent en évidence la contribution précieuse de données introspectives comme complément aux résultats basés sur des caractéristiques purement linguistiques du transfert. Nos analyses soulignent la complexité des processus métalinguistiques et des stratégies individuelles, ce qui reflète une perspective dynamique du multilinguisme (p.ex., Jessner, 2008).

Cultural and linguistic context and lexical inferencing in English language learning: A case study

In an increasingly multilingual world, English language has kept a marked predominance as a global language. In many countries, English is the primary choice for foreign language learning. There is a long history of research in English language learning. The same applies for research in reading. A main interest since the 1970s has been the reading strategy defined as inferencing or guessing the meaning of unknown words from context. Inferencing has ben widely researched, however, the results and conclusions seem to be mixed. While some agree that inferencing is a useful strategy, others doubt its usefulness. Nevertheless, most of the research seem to agree that the cultural background affects comprehension and inferencing. While most of these studies have been done with texts and contexts created by the researches, little has been done using natural prose. The present study will attempt to further clarify the process of inferencing and the effects of the text’s cultural context and the linguistic background of the reader using a text that has not been created by the researcher. The participants of the study are 40 international students from Turku, Finland. Their linguistic background was obtained through a questionnaire and proved to be diverse. Think aloud protocols were performed to investigate their inferencing process and find connections between their inferences, comments, the text, and their linguistic background. The results show that: some inferences were made based on the participants’ world knowledge, experience, other languages, and English language knowledge; other inferences and comments were made based on the text, its use of language and vocabulary, and few cues provided by the author. The results from the present study and previous research seem to show that: 1) linguistic background is a source of information for inferencing but is not a major source; 2) the cultural context of the text affected the inferences made by the participants according to their closeness or distance from it.

Supporting self-management: development and evaluation of a digital resource for adults with asthma

The problem: Around 300 million people worldwide have asthma and prevalence is increasing. Support for optimal self-management can be effective in improving a range of outcomes and is cost effective, but is underutilised as a treatment strategy. Supporting optimum self-management using digital technology shows promise, but how best to do this is not clear. Aim: The purpose of this project was to explore the potential role of a digital intervention in promoting optimum self-management in adults with asthma. Methods: Following the MRC Guidance on the Development and Evaluation of Complex Interventions which advocates using theory, evidence, user testing and appropriate modelling and piloting, this project had 3 phases. Phase 1: Examination of the literature to inform phases 2 and 3, using systematic review methods and focussed literature searching. Phase 2: Developing the Living Well with Asthma website. A prototype (paper-based) version of the website was developed iteratively with input from a multidisciplinary expert panel, empirical evidence from the literature (from phase 1), and potential end users via focus groups (adults with asthma and practice nurses). Implementation and behaviour change theories informed this process. The paper-based designs were converted to the website through an iterative user centred process (think aloud studies with adults with asthma). Participants considered contents, layout, and navigation. Development was agile using feedback from the think aloud sessions immediately to inform design and subsequent think aloud sessions. Phase 3: A pilot randomised controlled trial over 12 weeks to evaluate the feasibility of a Phase 3 trial of Living Well with Asthma to support self-management. Primary outcomes were 1) recruitment & retention; 2) website use; 3) Asthma Control Questionnaire (ACQ) score change from baseline; 4) Mini Asthma Quality of Life (AQLQ) score change from baseline. Secondary outcomes were patient activation, adherence, lung function, fractional exhaled nitric oxide (FeNO), generic quality of life measure (EQ-5D), medication use, prescribing and health services contacts. Results: Phase1: Demonstrated that while digital interventions show promise, with some evidence of effectiveness in certain outcomes, participants were poorly characterised, telling us little about the reach of these interventions. The interventions themselves were poorly described making drawing definitive conclusions about what worked and what did not impossible. Phase 2: The literature indicated that important aspects to cover in any self-management intervention (digital or not) included: asthma action plans, regular health professional review, trigger avoidance, psychological functioning, self-monitoring, inhaler technique, and goal setting. The website asked users to aim to be symptom free. Key behaviours targeted to achieve this include: optimising medication use (including inhaler technique); attending primary care asthma reviews; using asthma action plans; increasing physical activity levels; and stopping smoking. The website had 11 sections, plus email reminders, which promoted these behaviours. Feedback during think aloud studies was mainly positive with most changes focussing on clarification of language, order of pages and usability issues mainly relating to navigation difficulties. Phase 3: To achieve our recruitment target 5383 potential participants were invited, leading to 51 participants randomised (25 to intervention group). Age range 16-78 years; 75% female; 28% from most deprived quintile. Nineteen (76%) of the intervention group used the website for an average of 23 minutes. Non-significant improvements in favour of the intervention group observed in the ACQ score (-0.36; 95% confidence interval: -0.96, 0.23; p=0.225), and mini-AQLQ scores (0.38; -0.13, 0.89; p=0.136). A significant improvement was observed in the activity limitation domain of the mini-AQLQ (0.60; 0.05 to 1.15; p = 0.034). Secondary outcomes showed increased patient activation and reduced reliance on reliever medication. There was no significant difference in the remaining secondary outcomes. There were no adverse events. Conclusion: Living Well with Asthma has been shown to be acceptable to potential end users, and has potential for effectiveness. This intervention merits further development, and subsequent evaluation in a Phase III full scale RCT.

A HUMAN-CENTERED APPROACH TO IMPROVING THE USER EXPERIENCE OF SOFTWARE UPDATES

Software updates are critical to the security of software systems and devices. Yet users often do not install them in a timely manner, leaving their devices open to security exploits. This research explored a re-design of automatic software updates on desktop and mobile devices to improve the uptake of updates through three studies. First using interviews, we studied users’ updating patterns and behaviors on desktop machines in a formative study. Second, we distilled these findings into the design of a low-fi prototype for desktops, and evaluated its efficacy for automating updates by means of a think-aloud study. Third, we investigated individual differences in update automation on Android devices using a large scale survey, and interviews. In this thesis, I present the findings of all three studies and provide evidence for how automatic updates can be better appropriated to fit users on both desktops and mobile devices. Additionally, I provide user interface design suggestions for software updates and outline recommendations for future work to improve the user experience of software updates.

Measuring the impact of illness perception on NHS audiology service usage in presbycusis patients – a feasibility study

Objective: Coping behaviour in adult hearing loss is still not well understood. Despite the high prevalence of hearing loss in those over 65, many people do not seek help for hearing loss. The common sense model of illness perceptions suggests that illness perceptions are a strong predictor of adapted coping behaviours, including help-seeking and take-up of treatments. This study aimed to determine the feasibility of using the brief illness perceptions questionnaire (bIPQ) to measure the impact of illness perception in predicting usage of NHS audiology services. Study design: Twenty-four volunteers were recruited from a standard NHS audiology outpatient clinic and illness perception was measured using the bIPQ. Two different recruitment strategies were explored and compared in terms of recruitment and retention rates. Comprehensibility of the questionnaire was assessed by Think Aloud Analysis in a subset of participants, while possible risks and burdens were monitored in structured telephone interviews. Results: The questionnaire is a comprehensive and quick tool to measure individual illness perception at minimal cost. We suggested minor adaptations of three questionnaire items to increase comprehension. Participants preferred to complete the questionnaire after their appointment at the clinic facilities rather than at home prior to their hearing assessment appointment. There were no identified risks or burdens to participants in this study. Conclusions: This approach met our criteria for feasibility. Understanding the impact of illness perception on patients’ coping behaviour in presbycusis could improve treatment outcomes and increase patient satisfaction, while promoting a more efficient and individualized audiology service.

UNDERSTANDING ESSAY RATING AS A SOCIALLY MEDIATED ACTIVITY: THE CASE OF A HIGH-STAKES ENGLISH TEST IN CHINA

Thesis (Ph.D, Education) -- Queen's University, 2016-09-22 22:05:24.246

The teaching of reading English in a foreign language in Libyan universities: methods and models

This study focuses on the learning and teaching of Reading in English as a Foreign Language (REFL), in Libya. The study draws on an action research process in which I sought to look critically at students and teachers of English as a Foreign Language (EFL) in Libya as they learned and taught REFL in four Libyan research sites. The Libyan EFL educational system is influenced by two main factors: the method of teaching the Holy-Quran and the long-time ban on teaching EFL by the former Libyan regime under Muammar Gaddafi. Both of these factors have affected the learning and teaching of REFL and I outline these contextual factors in the first chapter of the thesis. This investigation, and the exploration of the challenges that Libyan university students encounter in their REFL, is supported by attention to reading models. These models helped to provide an analytical framework and starting point for understanding the many processes involved in reading for meaning and in reading to satisfy teacher instructions. The theoretical framework I adopted was based, mainly and initially, on top-down, bottom-up, interactive and compensatory interactive models. I drew on these models with a view to understanding whether and how the processes of reading described in the models could be applied to the reading of EFL students and whether these models could help me to better understand what was going on in REFL. The diagnosis stage of the study provided initial data collected from four Libyan research sites with research tools including video-recorded classroom observations, semi-structured interviews with teachers before and after lesson observation, and think-aloud protocols (TAPs) with 24 students (six from each university) in which I examined their REFL reading behaviours and strategies. This stage indicated that the majority of students shared behaviours such as reading aloud, reading each word in the text, articulating the phonemes and syllables of words, or skipping words if they could not pronounce them. Overall this first stage indicated that alternative methods of teaching REFL were needed in order to encourage ‘reading for meaning’ that might be based on strategies related to eventual interactive reading models adapted for REFL. The second phase of this research project was an Intervention Phase involving two team-teaching sessions in one of the four stage one universities. In each session, I worked with the teacher of one group to introduce an alternative method of REFL. This method was based on teaching different reading strategies to encourage the students to work towards an eventual interactive way of reading for meaning. A focus group discussion and TAPs followed the lessons with six students in order to discuss the 'new' method. Next were two video-recorded classroom observations which were followed by an audio-recorded discussion with the teacher about these methods. Finally, I conducted a Skype interview with the class teacher at the end of the semester to discuss any changes he had made in his teaching or had observed in his students' reading with respect to reading behaviour strategies, and reactions and performance of the students as he continued to use the 'new' method. The results of the intervention stage indicate that the teacher, perhaps not surprisingly, can play an important role in adding to students’ knowledge and confidence and in improving their REFL strategies. For example, after the intervention stage, students began to think about the title, and to use their own background knowledge to comprehend the text. The students employed, also, linguistic strategies such as decoding and, above all, the students abandoned the behaviour of reading for pronunciation in favour of reading for meaning. Despite the apparent efficacy of the alternative method, there are, inevitably, limitations related to the small-scale nature of the study and the time I had available to conduct the research. There are challenges, too, related to the students’ first language, the idiosyncrasies of the English language, the teacher training and continuing professional development of teachers, and the continuing political instability of Libya. The students’ lack of vocabulary and their difficulties with grammatical functions such as phrasal and prepositional verbs, forms which do not exist in Arabic, mean that REFL will always be challenging. Given such constraints, the ‘new’ methods I trialled and propose for adoption can only go so far in addressing students’ difficulties in REFL. Overall, the study indicates that the Libyan educational system is underdeveloped and under resourced with respect to REFL. My data indicates that the teacher participants have received little to no professional developmental that could help them improve their teaching in REFL and skills in teaching EFL. These circumstances, along with the perennial problem of large but varying class sizes; student, teacher and assessment expectations; and limited and often poor quality resources, affect the way EFL students learn to read in English. Against this background, the thesis concludes by offering tentative conclusions; reflections on the study, including a discussion of its limitations, and possible recommendations designed to improve REFL learning and teaching in Libyan universities.

Universities, SMEs and innovation frameworks : think global, act local?

Universities are increasingly encouraged to take a leading role in economic development, particularly through innovation. Simultaneously, economic development policy itself is increasingly focused on small and medium-sized enterprises (SMEs), creating overlapping interactions in the roles of government policy, universities and SMEs and the processes of innovation creation and dissemination. This paper examines issues arising from these developments and relating to the key stakeholders (industry, government and universities in particular), the enabling mechanisms (network governance, relevant education, training and learning, and suitable structures), and local and cross-local links. The authors then use quantitative analysis of 450 SMEs in the UK to begin to evaluate the roles of universities and highlight areas for further theoretical development.

Higher Order Thinking through ICT: What do middle years teachers think really matters?

The middle years of schooling has emerged as an important focus in Australian education. Student disengagement and alienation, the negative effects of non-completion of the senior years of schooling and underachievement have raised concerns about the quality of education during the middle years. For many schools, reshaping the middle years has involved incorporating Information and Communication Technologies (ICT) to motivate students. However, simultaneously there is a need to ensure that programs are academically rigorous. There is little doubt that there are potential benefits to integrating ICT into programs for middle years’ students. However, little is known about how middle years’ teachers perceive higher order thinking, which is a component of academic rigour. This paper investigates the question of What are teachers’ perceptions of higher order thinking in an ICT environment? The study is underpinned by socio-cultural theory which is based on the belief that learning occurs through social interaction and that individuals are shaped by the social and cultural tools and instruments they engage with. This investigation used a collective case study design. Two methods were used for data collection. These methods are semi-structured interviews with individual teachers and a class and a focus group discussion with teachers. Findings indicate that teachers hold various perceptions of higher order thinking that lead to productive approaches to integrating ICT in middle years’ classrooms. The paper highlights that there may be a continuum of perceptions of higher order thinking with ICT. This continuum may inform professional developers who are guiding and supporting teachers to integrate ICT into middle years’ classrooms.