A text analytics evaluation of a first-year engineering project-based unit

In the undergraduate engineering program at Griffith University in Australia, the unit 1006ENG Design and Professional Skills aims to provide an introduction to engineering design and professional practice through a project-based learning (PBL) approach to problem solving. It provides students with an experience of PBL in the first-year of their programme. The unit comprises an underpinning lecture series, design work including group project activities, an individual computer-aided drawing exercise/s and an oral presentation. Griffith University employs a ‘Student Experience of Course’ (SEC) online survey as part of its student evaluation of teaching, quality improvement and staff performance management processes. As well as numerical response scale items, it includes the following two questions inviting open-ended text responses from students: i) What did you find particularly good about this course? and ii) How could this course be improved? The collection of textual data in in student surveys is commonplace, due to the rich descriptions of respondent experiences they can provide at relatively low cost. However, historically these data have been underutilised because they are time consuming to analyse manually, and there has been a lack of automated tools to exploit such data efficiently. Text analytics approaches offer analysis methods that result in visual representations of comment data that highlight key individual themes in these data and the relationships between those themes. We present a text analytics-based evaluation of the SEC open-ended comments received in the first two years of offer of the PBL unit 1006ENG. We discuss the results obtained in detail. The method developed and documented here is a practical and useful approach to analysing/visualising open-ended comment data that could be applied by others with similar comment data sets.