Studies on coordination of systems development process

This thesis examines coordination of systems development process in a contemporary software producing organization. The thesis consists of a series of empirical studies in which the actions, conceptions and artifacts of practitioners are analyzed using a theory-building case study research approach. The three phases of the thesis provide empirical observations on different aspects of systemsdevelopment. In the first phase is examined the role of architecture in coordination and cost estimation in multi-site environment. The second phase involves two studies on the evolving requirement understanding process and how to measure this process. The third phase summarizes the first two phases and concentrates on the role of methods and how practitioners work with them. All the phases provide evidence that current systems development method approaches are too naïve in looking at the complexity of the real world. In practice, development is influenced by opportunity and other contingent factors. The systems development processis not coordinated using phases and tasks defined in methods providing universal mechanism for managing this process like most of the method approaches assume.Instead, the studies suggest that managing systems development process happens through coordinating development activities using methods as tools. These studies contribute to the systems development methods by emphasizing the support of communication and collaboration between systems development participants. Methods should not describe the development activities and phases in a detail level, butshould include the higher level guidance for practitioners on how to act in different systems development environments.

Knowledge-elicitation and data-mining: fusing human and industrial plant information

Knowledge-elicitation is a common technique used to produce rules about the operation of a plant from the knowledge that is available from human expertise. Similarly, data-mining is becoming a popular technique to extract rules from the data available from the operation of a plant. In the work reported here knowledge was required to enable the supervisory control of an aluminium hot strip mill by the determination of mill set-points. A method was developed to fuse knowledge-elicitation and data-mining to incorporate the best aspects of each technique, whilst avoiding known problems. Utilisation of the knowledge was through an expert system, which determined schedules of set-points and provided information to human operators. The results show that the method proposed in this paper was effective in producing rules for the on-line control of a complex industrial process. (C) 2005 Elsevier Ltd. All rights reserved.

Fusing data-mining and knowledge elicitation for the supervisory control of an aluminium plate mill

Knowledge-elicitation is a common technique used to produce rules about the operation of a plant from the knowledge that is available from human expertise. Similarly, data-mining is becoming a popular technique to extract rules from the data available from the operation of a plant. In the work reported here knowledge was required to enable the supervisory control of an aluminium hot strip mill by the determination of mill set-points. A method was developed to fuse knowledge-elicitation and data-mining to incorporate the best aspects of each technique, whilst avoiding known problems. Utilisation of the knowledge was through an expert system, which determined schedules of set-points and provided information to human operators. The results show that the method proposed in this paper was effective in producing rules for the on-line control of a complex industrial process.

The road not taken: observing a process of issue selection and model conceptualization

This article describes a case study involving information technology managers and their new programmer recruitment policy, but the primary interest is methodological. The processes of issue generation and selection and model conceptualization are described. Early use of “magnetic hexagons” allowed the generation of a range of issues, most of which would not have emerged if system dynamics elicitation techniques had been employed. With the selection of a specific issue, flow diagraming was used to conceptualize a model, computer implementation and scenario generation following naturally. Observations are made on the processes of system dynamics modeling, particularly on the need to employ general techniques of knowledge elicitation in the early stages of interventions. It is proposed that flexible approaches should be used to generate, select, and study the issues, since these reduce any biasing of the elicitation toward system dynamics problems and also allow the participants to take up the most appropriate problem- structuring approach.

Application of hermeneutics to studying an experience mining process

In this paper we investigate an approach to eliciting practitioners’ problem-solving experience across an application domain. The approach is based on a well-known ‘pattern mining’ process which commonly results in a collection of sharable and reusable ‘design patterns’. While pattern mining has been recognised to work effectively in numerous domains, its main problem is the degree of technical proficiency that few domain practitioners are prepared to master. In our approach to pattern mining, patterns are induced indirectly from designers’ experience, as determined by analysing their past projects, the problems encountered and solutions applied in problem rectification. Through the cycles of hermeneutic revisions, the pattern mining process has been refined and ultimately its deficiencies addressed. The hermeneutic method used in the study has been clearly shown in the paper and illustrated with examples drawn from the multimedia domain. The resulting approach to experience elicitation provided opportunities for active participation of multimedia practitioners in capturing and sharing their design experience.

Elicitation of multivariate prior distributions: A nonparametric Bayesian approach

In the context of Bayesian statistical analysis, elicitation is the process of formulating a prior density f(.) about one or more uncertain quantities to represent a person's knowledge and beliefs. Several different methods of eliciting prior distributions for one unknown parameter have been proposed. However, there are relatively few methods for specifying a multivariate prior distribution and most are just applicable to specific classes of problems and/or based on restrictive conditions, such as independence of variables. Besides, many of these procedures require the elicitation of variances and correlations, and sometimes elicitation of hyperparameters which are difficult for experts to specify in practice. Garthwaite et al. (2005) discuss the different methods proposed in the literature and the difficulties of eliciting multivariate prior distributions. We describe a flexible method of eliciting multivariate prior distributions applicable to a wide class of practical problems. Our approach does not assume a parametric form for the unknown prior density f(.), instead we use nonparametric Bayesian inference, modelling f(.) by a Gaussian process prior distribution. The expert is then asked to specify certain summaries of his/her distribution, such as the mean, mode, marginal quantiles and a small number of joint probabilities. The analyst receives that information, treating it as a data set D with which to update his/her prior beliefs to obtain the posterior distribution for f(.). Theoretical properties of joint and marginal priors are derived and numerical illustrations to demonstrate our approach are given. (C) 2010 Elsevier B.V. All rights reserved.

Computational support for the process of software requirement specification

The software industry has become more and more concerned with the appropriate application of activities that composes requirement engineering as a way to improve the quality of its products. In order to support these activities, several computational tools have been available in the market, although it is still possible to find a lack of resources related to some activities. In this context, this paper proposes the inclusion of a module to aid in the requirements specification to a tool called Requirements Elicitation Support Tool. This module allows to specify requirements in accordance with IEEE 830 standard, thus contributing to the documentation of the requirements established for a software system, besides supporting the learning of concepts related to the requirements specification, which improves the skills of users of the tool. © 2012 IEEE.

Identifying Moderator Variables Through Requirements Elicitation Experiments Limitations

Interviews are the most widely used elicitation technique in Requirements Engineering (RE). Despite its importance, research in interviews is quite limited, in particular from an experimental perspective. We have performed a series of experiments exploring the relative effectiveness of structured and unstructured interviews. This line of research has been active in Information Systems in the past years, so that our experiments can be aggregated together with existing ones to obtain guidelines for practice. Experimental aggregation is a demanding task. It requires not only a large number of experiments, but also considering the influence of the existing moderators. However, in the current state of the practice in RE, those moderators are unknown. We believe that analyzing the threats to validity in interviewing experiments may give insight about how to improve further replications and the corresponding aggregations. It is likely that this strategy may be applied in other Software Engineering areas as well.

An interactive method of fuzzy probability elicitation in risk analysis

Expert knowledge is used to assign probabilities to events in many risk analysis models. However, experts sometimes find it hard to provide specific values for these probabilities, preferring to express vague or imprecise terms that are mapped using a previously defined fuzzy number scale. The rigidity of these scales generates bias in the probability elicitation process and does not allow experts to adequately express their probabilistic judgments. We present an interactive method for extracting a fuzzy number from experts that represents their probabilistic judgments for a given event, along with a quality measure of the probabilistic judgments, useful in a final information filtering and analysis sensitivity process.

Improving the use of visual interactive simulation as a knowledge elicitation tool

Knowledge elicitation is a well-known bottleneck in the production of knowledge-based systems (KBS). Past research has shown that visual interactive simulation (VIS) could effectively be used to elicit episodic knowledge that is appropriate for machine learning purposes, with a view to building a KBS. Nonetheless, the VIS-based elicitation process still has much room for improvement. Based in the Ford Dagenham Engine Assembly Plant, a research project is being undertaken to investigate the individual/joint effects of visual display level and mode of problem case generation on the elicitation process. This paper looks at the methodology employed and some issues that have been encountered to date. Copyright © 2007 Inderscience Enterprises Ltd.

Using a simulation model for knowledge elicitation and knowledge management

The work reported in this paper is part of a project simulating maintenance operations in an automotive engine production facility. The decisions made by the people in charge of these operations form a crucial element of this simulation. Eliciting this knowledge is problematic. One approach is to use the simulation model as part of the knowledge elicitation process. This paper reports on the experience so far with using a simulation model to support knowledge management in this way. Issues are discussed regarding the data available, the use of the model, and the elicitation process itself. © 2004 Elsevier B.V. All rights reserved.

Web based expert elicitation of uncertainties in environmental model inputs

When constructing and using environmental models, it is typical that many of the inputs to the models will not be known perfectly. In some cases, it will be possible to make observations, or occasionally physics-based uncertainty propagation, to ascertain the uncertainty on these inputs. However, such observations are often either not available or even possible, and another approach to characterising the uncertainty on the inputs must be sought. Even when observations are available, if the analysis is being carried out within a Bayesian framework then prior distributions will have to be specified. One option for gathering or at least estimating this information is to employ expert elicitation. Expert elicitation is well studied within statistics and psychology and involves the assessment of the beliefs of a group of experts about an uncertain quantity, (for example an input / parameter within a model), typically in terms of obtaining a probability distribution. One of the challenges in expert elicitation is to minimise the biases that might enter into the judgements made by the individual experts, and then to come to a consensus decision within the group of experts. Effort is made in the elicitation exercise to prevent biases clouding the judgements through well-devised questioning schemes. It is also important that, when reaching a consensus, the experts are exposed to the knowledge of the others in the group. Within the FP7 UncertWeb project (http://www.uncertweb.org/), there is a requirement to build a Webbased tool for expert elicitation. In this paper, we discuss some of the issues of building a Web-based elicitation system - both the technological aspects and the statistical and scientific issues. In particular, we demonstrate two tools: a Web-based system for the elicitation of continuous random variables and a system designed to elicit uncertainty about categorical random variables in the setting of landcover classification uncertainty. The first of these examples is a generic tool developed to elicit uncertainty about univariate continuous random variables. It is designed to be used within an application context and extends the existing SHELF method, adding a web interface and access to metadata. The tool is developed so that it can be readily integrated with environmental models exposed as web services. The second example was developed for the TREES-3 initiative which monitors tropical landcover change through ground-truthing at confluence points. It allows experts to validate the accuracy of automated landcover classifications using site-specific imagery and local knowledge. Experts may provide uncertainty information at various levels: from a general rating of their confidence in a site validation to a numerical ranking of the possible landcover types within a segment. A key challenge in the web based setting is the design of the user interface and the method of interacting between the problem owner and the problem experts. We show the workflow of the elicitation tool, and show how we can represent the final elicited distributions and confusion matrices using UncertML, ready for integration into uncertainty enabled workflows.We also show how the metadata associated with the elicitation exercise is captured and can be referenced from the elicited result, providing crucial lineage information and thus traceability in the decision making process.

Knowledge elicitation supported by simulation

The work reported in this paper is part of a project simulating maintenance operations in an automotive engine production facility. The decisions made by the people in charge of these operations form a crucial element of this simulation. Eliciting this knowledge is problematic. One approach is to use the simulation model as part of the knowledge elicitation process. This paper reports on the experience so far with using a simulation model to support knowledge management in this way. Issues are discussed regarding the data available, the use of the model, and the elicitation process itself.

TAKING THE PERSPECTIVE OF A SELLER AND A BUYER: IMPLICATIONS FOR PRICE ELICITATION AND PRICE FRAMING

This dissertation consists of two essays which investigate how assuming the role of a seller or a buyer affects valuations in a price elicitation task (essay I) and how different presentations of an equivalent price affect evaluations when a consumer plays the dual roles of a buyer and a seller in transactions involving trade-ins (essay II). Sellers’ willingness to accept (WTA) to give up a good is typically higher than buyers' willingness to pay (WTP) to obtain the good. Essay I proposes that valuation processes of sellers and buyers are guided by a motivational orientation of “getting the best.” For a seller (buyer) indicating WTA (WTP), getting the best implies receiving as much as possible to give up a specific good (giving up as little as possible to get the specific good). Results of six studies suggest that the WTA-WTP elicitation task activates different directional goals, leading to the WTA-WTP disparity. The different directional goals lead sellers and buyers to focus on different aspects and bias their cognitive reasoning and interpretation of information. By connecting the valuation process to the general motivation of getting the best, this research provides a unifying framework to explain the disparate interpretations of the WTA-WTP disparity. Many new purchases and replacement decisions involve consumers’ trading in their old products. In such transactions, the overall exchange may be priced either as separate transactions (partitioned) with price tags for the payment and the receipt or as a single net price (consolidated) which takes into account the value of the trade-in. Essay II examines whether consumers prefer a partitioned price versus a consolidated price presentation. The findings suggest that when consumers are trading in a product which has a low value relative to the price of a new product, they prefer a consolidated price. In contrast, when trading in a product which has high value, they prefer a partitioned price. The results suggest that consumers use the price of the new product as an anchor to evaluate the trade-in value, and the perception of the trade-in value influences the overall evaluation especially when the transaction is partitioned.