Increasing the rigour and trustworthiness of participatory evaluations : learnings from the field

Participatory evaluation and participatory action research (PAR) are increasingly used in community-based programs and initiatives and there is a growing acknowledgement of their value. These methodologies focus more on knowledge generated and constructed through lived experience than through social science (Vanderplaat 1995). The scientific ideal of objectivity is usually rejected in favour of a holistic approach that acknowledges and takes into account the diverse perspectives, values and interpretations of participants and evaluation professionals. However, evaluation rigour need not be lost in this approach. Increasing the rigour and trustworthiness of participatory evaluations and PAR increases the likelihood that results are seen as credible and are used to continually improve programs and policies.----- Drawing on learnings and critical reflections about the use of feminist and participatory forms of evaluation and PAR over a 10-year period, significant sources of rigour identified include:----- • participation and communication methods that develop relations of mutual trust and open communication----- • using multiple theories and methodologies, multiple sources of data, and multiple methods of data collection----- • ongoing meta-evaluation and critical reflection----- • critically assessing the intended and unintended impacts of evaluations, using relevant theoretical models----- • using rigorous data analysis and reporting processes----- • participant reviews of evaluation case studies, impact assessments and reports.

A time-variant medical data trustworthiness assessment model

Electronic Health Record (EHR) systems are being introduced to overcome the limitations associated with paper-based and isolated Electronic Medical Record (EMR) systems. This is accomplished by aggregating medical data and consolidating them in one digital repository. Though an EHR system provides obvious functional benefits, there is a growing concern about the privacy and reliability (trustworthiness) of Electronic Health Records. Security requirements such as confidentiality, integrity, and availability can be satisfied by traditional hard security mechanisms. However, measuring data trustworthiness from the perspective of data entry is an issue that cannot be solved with traditional mechanisms, especially since degrees of trust change over time. In this paper, we introduce a Time-variant Medical Data Trustworthiness (TMDT) assessment model to evaluate the trustworthiness of medical data by evaluating the trustworthiness of its sources, namely the healthcare organisation where the data was created and the medical practitioner who diagnosed the patient and authorised entry of this data into the patient’s medical record, with respect to a certain period of time. The result can then be used by the EHR system to manipulate health record metadata to alert medical practitioners relying on the information to possible reliability problems.

Ties that blind? How strong ties affect small business owner-managers’ perceived trustworthiness of their advisors

This research investigates how a strong personal relationship (strong tie) between a small business owner-manager and his professional or informal advisor affects the relationship between the advisor's recent performance and the owner-manager's perceptions of the advisor's trustworthiness in terms of ability, benevolence and integrity. A negative moderating effect could point to a 'tie that blinds': the owner-manager may be less critical in evaluating the advisor's perceived trustworthiness in light of their recent performance, because of the existing personal relationship. A conceptual model is constructed and examined with survey data comprising 153 young Finnish businesses. The results show that strong ties increase the owner-manager's perception of the advisor's integrity, disregarding their recent performance. For professional advisors, strong ties reduce the impact of recent performance in the owner-manager's evaluation of their ability. For informal advisors, a strong tie makes it more likely that their benevolence will be evaluated highly in light of their recent performance. While the results show that 'ties can blind' under certain circumstances, the limitations of the study raise the need for further research to specify these contextual factors and examine the causal link between the choice of advisor and business performance.

Interactions between organizational culture, trustworthiness, and mechanisms for inter-project knowledge sharing

This research used a multiple-case study approach to empirically investigate the complex relationship between factors influencing inter-project knowledge sharing—trustworthiness, organizational culture, and knowledge-sharing mechanisms. Adopting a competing values framework, we found evidence of patterns existing between the type of culture, on the project management unit level, and project managers’ perceptions of valuing trustworthy behaviors and the way they share knowledge, on the individual level. We also found evidence for mutually reinforcing the effect of trust and clan culture, which shape tacit knowledge-sharing behaviors.

Using qualitative mixed methods to study small health care organizations while maximising trustworthiness and authenticity

Background The primary health care sector delivers the majority of health care in western countries through small, community-based organizations. However, research into these healthcare organizations is limited by the time constraints and pressure facing them, and the concern by staff that research is peripheral to their work. We developed Q-RARA—Qualitative Rapid Appraisal, Rigorous Analysis—to study small, primary health care organizations in a way that is efficient, acceptable to participants and methodologically rigorous. Methods Q-RARA comprises a site visit, semi-structured interviews, structured and unstructured observations, photographs, floor plans, and social scanning data. Data were collected over the course of one day per site and the qualitative analysis was integrated and iterative. Results We found Q-RARA to be acceptable to participants and effective in collecting data on organizational function in multiple sites without disrupting the practice, while maintaining a balance between speed and trustworthiness. Conclusions The Q-RARA approach is capable of providing a richly textured, rigorous understanding of the processes of the primary care practice while also allowing researchers to develop an organizational perspective. For these reasons the approach is recommended for use in small-scale organizations both within and outside the primary health care sector.

the trustworthiness metric model of software process quality based-on life circle

Communication University of China; Wuhan University; James Madison University; Institute of Policy and Management, Chinese Academy of Sciences; IEEE Wuhan Section

process trustworthiness as a capability indicator for measuring and improving software trustworthiness

Chinese Acad Sci, ISCAS Lab Internet Software Technologies

a fuzzy-based method for evaluating the trustworthiness of software processes

University of Paderborn; Fraunhofer Inst. Exp. Softw. Eng. (IESE); Chinese Academy of Science (ISCAS)

Validity, trustworthiness and rigour:reasserting realism in qualitative research

Aim The aim of this paper is to challenge recent sceptical approaches to the possibility of validating qualitative research and to underline the benefits of adopting a realist approach to validity. Background In recent discussion about the methodological bases for qualitative research it has been argued that, because different methodologies take different approaches to validity, attempts to develop a common set of validation criteria are futile. On the basis of this sceptical view, a number of strategies for judging qualitative research have been proposed. These include suggestions that: it should be judged according to aesthetic or rhetorical criteria, rather than epistemological validity; responsibility for appraisal should move from researchers to readers; each methodology should be assessed individually according to its own merits. Discussion None of these suggestions provide a viable alternative to validity, defined as the extent to which research reflects accurately that to which it refers. Because the form of research does not determine its content, replacement of epistemology by aesthetics is unsustainable. Because research reports mediate between writer and reader, a one-sided approach to this relationship constitutes a false dichotomy. If we accept the criterion of practitioner confidence as a means of judging methodological approaches, this involves rejection of judgement according to a methodology�s own merits. Conclusion If qualitative research is actually about something, and if it is required to provide beneficial information, then a realist approach to validity holds out greatest promise.

Determining the Trustworthiness of New Electronic Contracts

Expressing contractual agreements electronically potentially allows agents to automatically perform functions surrounding contract use: establishment, fulfilment, renegotiation etc. For such automation to be used for real business concerns, there needs to be a high level of trust in the agent-based system. While there has been much research on simulating trust between agents, there are areas where such trust is harder to establish. In particular, contract proposals may come from parties that an agent has had no prior interaction with and, in competitive business-to-business environments, little reputation information may be available. In human practice, trust in a proposed contract is determined in part from the content of the proposal itself, and the similarity of the content to that of prior contracts, executed to varying degrees of success. In this paper, we argue that such analysis is also appropriate in automated systems, and to provide it we need systems to record salient details of prior contract use and algorithms for assessing proposals on their content. We use provenance technology to provide the former and detail algorithms for measuring contract success and similarity for the latter, applying them to an aerospace case study.