Project-based assessment for graduate coursework in physics

Project-based assessment, in the form of take-home exams, was trialed in an honours/masters level electromagnetic theory course. This assessment formed an integral part of the learning experience of the students, and students felt that this was effective method of learning.

Good practice guide in learning and teaching

This is the second edition of our Aston Business School (ABS) Good Practice Guide and the enthusiasm of the contributors appears undiminished. I am again reminded that I work with a group of very committed, dedicated and professional colleagues. Once again this publication is produced to celebrate and promote good teaching across the School and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. Contributors to this Guide were not chosen because they are the best teachers in the School, although they are undoubtedly all amongst my colleagues who are exponents of enthusiastic and inspiring approaches to learning. The Quality Unit approached these individuals because they declared on their Annual Module Reflection Forms that they were doing something interesting and worthwhile which they thought others might find useful. Amongst those reading the Guide I am sure that there are many other individuals who are trying to operate similar examples of good practice in their teaching, learning and assessment methods. I hope that this publication will provoke these people into providing comments and articles of their own and that these will form the basis of next year’s Guide. It may also provoke some people to try these methods in their own teaching. The themes of the articles this year can be divided into two groups. The first theme is the quest to help students to help themselves to learn via student-run tutorials, surprise tests and mock examinations linked with individual tutorials. The second theme is making learning come to life in exciting practical ways by, for example, hands-on workshops and simulations, story telling, rhetorical questioning and discussion groups. A common theme is one of enthusiasm, reflection and commitment on behalf of the lecturers concerned. None of the approaches discussed in this publication are low effort activities on the part of the facilitator, but this effort is regarded as worthwhile as a means of creating greater student engagement. As Biggs (2003)[1] says, in his similarly inspiring way, students learn more the less passive they are in their learning. (Ref). The articles in this publication bear witness of this and much more. Since last year Aston Business School has launched its Research Centre in Higher Education Learning and Management (HELM) which is another initiative to promote excellent learning and teaching. Even before this institution has become fully operational, at least one of the articles in this publication has seen the light of day in the research arena and at least two others are ripe for dissemination to a wider audience via journal publication. More news of our successes in this activity will appear in next year’s edition. May I thank the contributors for taking time out of their busy schedules to write the articles this summer, and to Julie Green who runs the ABS Quality Unit, for putting our diverse approaches into a coherent and publishable form and for chasing us when we have needed it! I would also like to thank Ann Morton and her colleagues in the Centre for Staff Development who have supported this publication. During the last year the Centre has further stimulated the learning and teaching life of the School (and the wider University) via their Learning and Teaching Week and sponsorship of Teaching Quality Enhancement Fund (TQEF) projects. Pedagogic excellence is in better health at Aston than ever before – long may this be because this is what life in HE should be about.

An action research study with low proficiency learners in Japan to promote the learning of vocabulary through collocations.

This action research (AR) study explores an alternative approach to vocabulary instruction for low-proficiency university students: a change from targeting individual words from the general service list (West, 1953) to targeting frequent verb + noun collocations. A review of the literature indicated a focus on collocations instead of individual words could potentially address the students’ productive challenges with targeted vocabulary. Over the course of four reflective cycles, this thesis addresses three main aspects of collocation instruction. First, it examines if the students believe studying collocations is more useful than studying individual lexical items. Second, the thesis investigates whether a focus on collocations will lead to improvements in spoken fluency. This is tested through a comparison of a pre-intervention spoken assessment task with the findings from the same task completed 15 weeks later, after the intervention. Third, the thesis explores different procedures for the instructing of collocations under the classroom constraints of a university teaching context. In the first of the four reflective cycles, data is collected which indicates that the students believe a focus on collocations is superior to only teaching individual lexical items, that in the students’ opinion their productive abilities with the targeted structures has improved, and that delexicalized verb collocations are problematic for low-proficiency students. Reflective cycle two produces evidence indicating that productive tasks are superior to receptive tasks for fluency development. In reflective cycle three, productively challenging classroom tasks are investigated further and the findings indicate that tasks with higher productive demands result in greater improvements in spoken fluency. The fourth reflective cycle uses a different type of collocation list: frequent adjective + noun collocations. Despite this change, the findings remain consistent in that certain types of collocations are problematic for low-proficiency language learners and that the evidence shows productive tasks are necessary to improve the students’ spoken ability.

Technological and organisational change in small- to medium-sized manufacturing companies:a learning organisation perspective

This paper examines the extent to which a learning organisation perspective is attainable in small- to medium-sized manufacturing companies. An audit tool is developed from the literature on organisational learning and recognised processes that lead towards becoming a learning organisation. The paper focuses on the application of the audit tool in three UK automotive component suppliers which are all experiencing pressures for change imposed by the major vehicle manufacturers. The main changes are concerned with tiering of the supply chain and substantial delegation of responsibilities to component suppliers including an increasing emphasis on innovation and continuous improvement. The companies presented in the paper are taken from a research project into the impact of changes in supply chain relationships on the operation of small- and medium-sized manufacturing firms in the West Midlands region of the UK. The ways in which the companies are responding to change are presented together with the results of a self-assessment using the developed audit tool. These results suggest that companies of this type tend to focus on change in those areas that involve least challenge to the established power and authority of management.

e-CAF: flexible marking schemes for electronic coursework assessment and feedback

Recent surveys reveal that many university students in the U.K. are not satisfied with the timeliness and usefulness of the feedback given by their tutors. Ensuring timeliness in marking can result in a reduction in the quality of feedback. Though suitable use of Information and Communication Technology should alleviate this problem, existing Virtual Learning Environments are inadequate to support detailed marking scheme creation and they provide little support for giving detailed feedback. This paper describes a unique new web-based tool called e-CAF for facilitating coursework assessment and feedback management directed by marking schemes. Using e-CAF, tutors can create or reuse detailed marking schemes efficiently without sacrificing the accuracy or thoroughness in marking. The flexibility in marking scheme design also makes it possible for tutors to modify a marking scheme during the marking process without having to reassess the students’ submissions. The resulting marking process will become more transparent to students.

Developments in assessment and teaching methodologies in modern languages in England and France in response to educational reform (1975-1985)

This comparative study considers the main causative factors for change in recent years in the teaching of modern languages in England and France and seeks to contribute, in a general sense, to the understanding of change in comparable institutions. In England by 1975 the teaching of modern languages in the comprehensive schools was seen to be inappropriate to the needs of children of the whole ability-range. A combination of the external factor of the Council of Europe initiative in devising a needs-based learning approach for adult learners, and the internal factor of teacher-based initiatives in developing a graded-objectives learning approach for the less-able, has reversed this situation to some extent. The study examines and evaluates this reversal, and, in addition, assesses teachers' attitudes towards, and understanding of, the changes involved. In France the imposition of `la reforme Haby' in 1977 and the creation of `le college unique' were the main external factors for change. The subsequent failure of the reform and the socialist government's support of decentralisation policies returning the initiative for renewal to schools are examined and evaluated, as are the internal factors for changes in language-teaching - `groupes de niveau' and the creation of `equipes pedagogiques'. In both countries changes in the function of examinations at 15/16 plus are examined. The final chapter compared the changes in both education systems.

Investigating Turkey's national innovation and learning system

The national systems of innovation (NIS) approach focuses on the patterns and the determinants of innovation processes from the perspective of nation-states. This paper reports on continuing work on the application of an NIS model to the development of technological capability in Turkey. Initial assessment of the literature shows that there are a number of alternative conceptualisations of NIS. An attempt by the Government to identify a NIS for Turkey shows the main actors in the system but does not pay sufficient attention to the processes of interactions between agents within the system. An operational model should be capable of representing these processes and interactions and assessing the strengths and weaknesses of the NIS. For industrialising countries, it is also necessary to incorporate learning mechanisms into the model. Further, there are different levels of innovation and capability in different sectors which the national perspective may not reflect. This paper is arranged into three sections. The first briefly explains the basics of the national innovation and learning system. Although there is no single accepted definition of NIS, alternative definitions reviewed share some common characteristics. In the second section, an NIS model is applied to Turkey in order to identify the elements, which characterise the country’s NIS. This section explains knowledge flow and defines the relations between the actors within the system. The final section draws on the “from imitation to innovation” model apparently so successful in East Asia and assesses its applicability to Turkey. In assessing Turkey’s NIS, the focus is on the automotive and textile sectors.

Evaluation of the quality and promotion of open learning materials

In the UK, Open Learning has been used in industrial training for at least the last decade. Trainers and Open Learning practitioners have been concerned about the quality of the products and services being delivered. The argument put forward in this thesis is that there is ambiguity amongst industrialists over the meanings of `Open Learning' and `Quality in Open Learning'. For clarity, a new definition of Open Learning is proposed which challenges the traditional learner-centred approach favoured by educationalists. It introduces the concept that there are benefits afforded to the trainer/employer/teacher as well as to the learner. This enables a focussed view of what quality in Open Learning really means. Having discussed these issues, a new quantitative method of evaluating Open Learning is proposed. This is based upon an assessment of the degree of compliance with which products meet Parts 1 & 2 of the Open Learning Code of Practice. The vehicle for these research studies has been a commercial contract commissioned by the Training Agency for the Engineering Industry Training Board (EITB) to examine the quality of Open Learning products supplied to the engineering industry. A major part of this research has been the application of the evaluation technique to a range of 67 Open Learning products (in eight subject areas). The findings were that good quality products can be found right across the price range - so can average and poor quality ones. The study also shows quite convincingly that there are good quality products to be found at less than 50. Finally the majority (24 out of 34) of the good quality products were text based.

An investigation of the factor-analytic approach to the determination of abilities involved in psychomotor learning

This research began with an attempt to solve a practical problem, namely, the prediction of the rate at which an operator will learn a task. From a review of the literature, communications with researchers in this area and the study of psychomotor learning in factories it was concluded that a more fundamental approach was required which included the development of a task taxonomy. This latter objective had been researched for over twenty years by E. A. Fleishman and his approach was adopted. Three studies were carried out to develop and extend Fleishman's approach to the industrial area. However, the results of these studies were not in accord with FIeishman's conclusions and suggested that a critical re-assessment was required of the arguments, methods and procedures used by Fleishman and his co-workers. It was concluded that Fleishman's findings were to some extent an artifact of the approximate methods and procedures which he used in the original factor analyses and that using the more modern computerised factor analytic methods a reliable ability taxonomy could be developed to describe the abilities involved in the learning of psychomotor tasks. The implications for a changing-task or changing-subject model were drawn and it was concluded that a changing task and subject model needs to be developed.

Ability assessment and job structure measures in occupational mobility

Recent and potential changes in technology have resulted in the anticipation of increases in the frequency of job changes. This has led manpower policy makers to investigate the feasibility of incorporating the employment skills of job groups in the general prediction of future job learning and performance with a view to the establishment of "job families" within which transfer might be considered reciprocally high. A structured job analysis instrument (the Position Analysis Questionnaire) is evaluated in terms of two distinct sets of scores; job dimensions and synthetically established attribute/trait profiles. Studies demonstrate that estimates of a job's structure/dimensions and requisite human attributes can be reliably established. Three alternative techniques of statistically assembling profiles of the requisite human attributes for jobs are found to have differential levels of reliability and differential degrees of validity in their estimation of the "actual" ability requirements of jobs. The utility of these two sets of job descriptors to serve as representations of the cognitive structure similarity of job groups is investigated in a study which simulates a job transfer situation. The central role of the index of similarity used to assess the relationship between "target" and "present" job is demonstrated. The relative extents to which job structure similarity and job attribute similariity are associated with positive transfer are investigated. The studies demonstrate that the dimensions of jobs, and more fruitfully their requisite human attributes can serve as bases to predict job transfer learning and performance. The nature of the index of similarity used to optimally formulate predictions of transfer is such that networks of jobs might be establishable to which current job incumbents could be expected to transfer positively. The derivation of "job families" with anticipated reciprocal transfer consequences is considered to be less appropriate.

A techno-economic assessment of the use of fast pyrolysis bio-oil from UK energy crops in the production of electricity and combined heat and power

This thesis investigates the cost of electricity generation using bio-oil produced by the fast pyrolysis of UK energy crops. The study covers cost from the farm to the generator’s terminals. The use of short rotation coppice willow and miscanthus as feedstocks was investigated. All costs and performance data have been taken from published papers, reports or web sites. Generation technologies are compared at scales where they have proved economic burning other fuels, rather than at a given size. A pyrolysis yield model was developed for a bubbling fluidised bed fast pyrolysis reactor from published data to predict bio-oil yields and pyrolysis plant energy demands. Generation using diesel engines, gas turbines in open and combined cycle (CCGT) operation and steam cycle plants was considered. The use of bio-oil storage to allow the pyrolysis and generation plants to operate independently of each other was investigated. The option of using diesel generators and open cycle gas turbines for combined heat and power was examined. The possible cost reductions that could be expected through learning if the technology is widely implemented were considered. It was found that none of the systems analysed would be viable without subsidy, but with the current Renewable Obligation Scheme CCGT plants in the 200 to 350 MWe range, super-critical coal fired boilers co-fired with bio-oil, and groups of diesel engine based CHP schemes supplied by a central pyrolysis plant would be viable. It was found that the cost would reduce with implementation and the planting of more energy crops but some subsidy would still be needed to make the plants viable.

Does reviewing lead to better learning and decision making? Answers from a randomized stock market experiment

Background - The literature is not univocal about the effects of Peer Review (PR) within the context of constructivist learning. Due to the predominant focus on using PR as an assessment tool, rather than a constructivist learning activity, and because most studies implicitly assume that the benefits of PR are limited to the reviewee, little is known about the effects upon students who are required to review their peers. Much of the theoretical debate in the literature is focused on explaining how and why constructivist learning is beneficial. At the same time these discussions are marked by an underlying presupposition of a causal relationship between reviewing and deep learning. Objectives - The purpose of the study is to investigate whether the writing of PR feedback causes students to benefit in terms of: perceived utility about statistics, actual use of statistics, better understanding of statistical concepts and associated methods, changed attitudes towards market risks, and outcomes of decisions that were made. Methods - We conducted a randomized experiment, assigning students randomly to receive PR or non–PR treatments and used two cohorts with a different time span. The paper discusses the experimental design and all the software components that we used to support the learning process: Reproducible Computing technology which allows students to reproduce or re–use statistical results from peers, Collaborative PR, and an AI–enhanced Stock Market Engine. Results - The results establish that the writing of PR feedback messages causes students to experience benefits in terms of Behavior, Non–Rote Learning, and Attitudes, provided the sequence of PR activities are maintained for a period that is sufficiently long.

Creating a culture for critical and situated technology use through effective learning design

The purpose of this concise paper is to propose, with evidence gathered through a systematic evaluation of an academic development programme in the UK, that training in the use of new and emerging learning technologies should be holistically embedded in every learning and training opportunity in learning, teaching and assessment in higher education, and not only as stand-alone modules or one-off opportunities. The future of learning in higher education cannot afford to allow Universities to disregard that digital literacy is an expected professional skill for their entire staff.

Can computer-based testing achieve quality and efficiency in assessment?

This paper reports findings of a two year study concerning the development and implementation of a general-purpose computer-based assessment (CBA) system at a UK University. Data gathering took place over a period of nineteen months, involving a number of formative and summative assessments. Approximately 1,000 students, drawn from undergraduate courses, were involved in the exercise. The techniques used in gathering data included questionnaires, observation, interviews and an analysis of student scores in both conventional examinations and computer-based assessments. Comparisons with conventional assessment methods suggest that the use of CBA techniques may improve the overall performance of students. However it is clear that the technique must not be seen as a "quick fix" for problems such as rising student numbers. If one accepts that current systems test only a relatively narrow range of skills, then the hasty implementation of CBA systems will result in a distorted and inaccurate view of student performance. In turn, this may serve to reduce the overall quality of courses and - ultimately - detract from the student learning experience. On the other hand, if one adopts a considered and methodical approach to computer-based assessment, positive benefits might include increased efficiency and quality, leading to improved student learning.