Enhancing and assessing group and team learning in architecture and related design contexts

EXECUTIVE SUMMARYTeamwork skills are essential in the design industry where practitioners negotiate often-conflicting design options in multi-disciplinary teams. Indeed, many of the bodies that accredit design courses explicitly list teamwork skills as essential attributes of design graduates e.g., the Australian Institute of Architects (AIA), Royal Institute of British Architects (RIBA), the National Council of Architectural Registration Boards (NCARB) of the United States and the Institution of Engineers, Australia (IEAust). In addition to the need to meet the demands of the accrediting bodies, there are many reasons for the ubiquitous use of teamwork assignments in design schools. For instance, teamwork learning is seen as being representative of work in practice where design is nearly always a collaborative activity. Learning and teaching in teamwork contexts in design education are not without particular challenges. In particular, two broad issues have been identified: first, many students leave academia without having been taught the knowledge and skills of how to design in teams; second, teaching, assessment and assignment design need to be better informed by a clear understanding of what leads to effective teamwork and the learning of teamwork skills. In recognition of the lack of a structured approach to integrating teamwork learning into the curricula of design programs, this project set out to answer three primary research questions: • How do we teach teamwork skills in the context of design? • How do we assess teamwork skills?• How do design students best learn teamwork skills?In addition, four more specific questions were investigated:1. Is there a common range of learning objectives for group-and-team-work in architecture and related design disciplines that will enable the teaching of consistent and measurable outcomes?2. Do group and team formation methods, learning styles and team-role preferences impact students’ academic and course satisfaction outcomes?3. What combinations of group-and-team formation methods, teaching and assessment models significantly improve learning outcomes?4. For design students across different disciplines with different learning styles and cultural origins, are there significant differences in performance, student satisfaction (as measured through questionnaires and unit evaluations), group-and-team working abilities and student participation?To elucidate these questions, a design-based research methodology was followed comprising an iterative series of enquiries: (a) A literature review was completed to investigate: what constitutes effective teamwork, what contributes to effectiveness in teams, what leads to positive design outcomes for teams, and what leads to effective learning in teams. The review encompassed a range of contexts: from work-teams in corporate settings, to professional design teams, to education outside of and within the design disciplines. The review informed a theoretical framework for understanding what factors impact the effectiveness of student design teams. (b) The validity of this multi-factorial Framework of Effectiveness in Student Design Teams was tested via surveys of educators’ teaching practices and attitudes, and of students’ learning experiences. 638 students and 68 teachers completed surveys: two pilot surveys for participants at the four partner institutions, which then informed two national surveys completed by participants from the majority of design schools across Australia. (c) The data collected provided evidence for 22 teamwork factors impacting team effectiveness in student design teams. Pedagogic responses and strategies to these 22 teamwork factors were devised, tested and refined via case studies, focus groups and workshops. (d) In addition, 35 educators from a wide range of design schools and disciplines across Australia attended two National Teaching Symposiums. The first symposium investigated the wider conceptualisation of teamwork within the design disciplines, and the second focused on curriculum level approaches to structuring the teaching of teamwork skills identified in the Framework.The Framework of Effectiveness in Student Design Teams identifies 22 factors impacting effective teamwork, along with teaching responses and strategies that design educators might use to better support student learning. The teamwork factors and teaching strategies are categorised according to three groups of input (Task Characteristics, Individual Level Factors and Team Level Factors), two groups of processes (Teaching Practice & Support Structures and Team Processes), and three categories of output (Task Performance, Teamwork Skills, and Attitudinal Outcomes). Eight of the 22 teamwork factors directly relate to the skills that need to be developed in students, one factor relates to design outputs, and the other thirteen factors inform pedagogies that can be designed for better learning outcomes. In Table 10 of Section 4, we outline which of the 22 teamwork factors pertain to each of five stakeholder groups (curriculum leaders, teachers, students, employers and the professional bodies); thus establishing who will make best use the information and recommendations we make. In the body of this report we summarise the 22 teamwork factors and teaching strategies informed by the Framework of Effectiveness in Student Design Teams, and give succinct recommendations arising from them. This material is covered in depth by the project outputs. For instance, the teaching and assessment strategies will be expanded upon in a projected book on Teaching Teamwork in Design. The strategies are also elucidated by examples of good practice presented in our case studies, and by Manuals on Teamwork for Teachers and Students. Moreover, the project website (teamwork-in-design.com index.html=""> visited by representatives of stakeholder groups in Australia and Canada), is seeding a burgeoning community of practice that promises dissemination, critical evaluation and the subsequent refinement of our materials, tools, strategies and recommendations. The following three primary outputs have been produced by the project in answer to the primary research questions:1. A theoretical Framework of Effectiveness in Student Design Teams;2. Manuals on Teamwork for Teachers and Students (available from the website);3. Case studies of good/innovative practices in teaching and assessing teamwork in design;In addition, five secondary outputs/outcomes have been produced that provide more nuanced responses:4. Detailed recommendations for the professional accrediting bodies and curriculum leaders;5. Online survey data (from over 700 participants), plus Team Effectiveness Scale to determine the factors influencing effective learning and successful outputs for student design teams;6. A community of practice in policy, programs, practice and dialogue;7. A detailed book proposal (with sample chapter), submitted to prospective publishers, on Teaching Teamwork in Design; 8. An annotated bibliography (accessed via the project website) on learning, teaching and assessing teamwork.The project has already had an international impact. As well as papers presented in Canada and New Zealand, the surveys were participated in by six Canadian schools of architecture, whose teaching leaders also provided early feedback on the project aims and objectives during visits made to them by the project leader. In addition, design schools in Vancouver, Canada, and San Diego in the USA have already utilised the Teacher’s Manual, and in February 2014 the project findings were discussed at Tel Aviv University in a forum focusing on the challenges for sustainability in architectural education.teamwork-in-design.com>

The role of arts education in advancing leadership life skills for young people

This paper contends that the arts provide a foundation for the intensifying effort that leadership, creative aptitude and expertise are making to existing and emerging professions. Participation in arts-based (school and/or community) programs 'have proven to be educational, developmentally rich, and cost-effective ways to provide students the skills they need to be productive participants in today's economy' (Psilos, 2002, p. 2). In particular, this paper explores the relationship between leadership development in young people through their engagement with arts education experiences, specifically the capacity of the arts to develop the generic skills of communication, team work, problem-solving and creative interpretation-skills considered essential for productive participation in today's economy and skills that augment leadership potential.

Touch it, feel it and experience it: developing professional IS skills using interview-style experiential simulations

The IS education field has made increasing use of computerised experiential simulations, but few attempts have been made to create an authentic learning environment that combines and balances elements of video-based computer simulation with real-life learning activities. This paper explores the design principles used to develop a CD-ROM simulation where learners use interviewing skills to elicit system requirements from simulated employees in an authentic context. The employees are videoed actors who converse with each other and with learners within a dynamic interaction model. The paper also describes how we combined this simulation with other teaching approaches such as in-class discussions, student team work, formal presentations, etc.

Teamwork and assessment: a critique

Australian universities commonly cite "teamwork" as a graduate attribute and an integral part of their set of flexible and transferable skills suitable for different types of employment. This paper raises concerns as to whether teamwork as experienced in the university education system can claim to achieve the imparting of team skills. Pedagogically, learning through team work may be a useful educational tool however when the concept of 'teamwork' is explored a number of inconsistencies arise in the tertiary environment. The outcomes derived from team work do not generally satisfy the criteria for effective and efficient teamwork. However, it is suggested that to utilise teamwork as a learning tool for a graduate attribute the assessment criteria should take into consideration how the team worked together--the team dynamics.

Online self and peer assessment in large, multi-campus, multi-cohort contexts

The increasing use of team assignments within higher education is well documented. The driving forces behind this include desires to facilitate reflective and collaborative learning, to develop generic teamwork skills for graduate employment and to reduce the grading workloads of faculty staff. Students however consistently report dissatisfaction when the assessment of team assignments produces a common grade for all team members. Self-andpeer-assessment (SAPA) is presented as a fair, valid and reliable method of producing information about ongoing team processes. This information can provide ongoing feedback to team members and rich formative data to instructors attempting to assess the team process and students’ teamwork skills. This data can also enable individualised summative assessment in dysfunctional teams or situations of uneven team member contributions. Whilst manual SAPA protocols can work effectively for smaller classes, computer-assisted SAPA offers a solution to the problems of large classes. This paper reports on the early stages of an online SAPA tool, originally developed for small classes of architecture students, adapted for use by very large business communication classes comprising up to 1000 students in a semester. This large unit is delivered on four Australian campuses as well as off-campus and in off-shore mode, by up to fourteen instructors at any one time. The paper documents how three researchers from very different backgrounds worked to create their own research team, implement a pilot study, and adapt the online tool, whilst adhering to comparability of assessment constraints and maintaining integrity of research design.

The student experience of working in teams online

The research reported here is part of a project undertaken at a large Australian university in late 2005. The overall aim of the project was to identify the characteristics of student learning in an online environment. A university-wide student survey was conducted to ascertain student views of online learning and also of online teaching as part of the project. In the survey students were asked questions about their experiences of team work in online environments. The student perceptions of teams and team work are the focus of this paper.

The findings from the survey indicated that students appreciated the opportunity of working in diverse teams. They supported the view that their opinions were valued in such teams. Student views of how teaching should be conducted in units with online teams were also expressed. Concerns about team interactions, technological barriers and communication and cultural issues were also raised. The implications for teaching with online student teams are presented and discussed.

The use of Web 2.0 Technologies to promote higher order thinking skills

During 2007 several independent Victorian secondary schools participated in a study exploring the ways in which the use of learning technologies can support the development of higher order thinking skills for students. This paper focuses on the use of Information and Communications Technologies (ICT) including Web 2.0 technologies for promoting effective teaching and learning in science. A case study methodology was used to describe how individual teachers used ICT and Web 2.0 in their settings. Data included interviews (focus group and individual), questionnaires, monitoring of teacher and student use of smart tools, analysis of curriculum documents and delivery methods and of student work samples. The evaluation used an interpretive methodology to investigate five research areas: Higher-order thinking, Metacognitive awareness, Team work/collaboration, Affect towards school/learning and Ownership of learning. Three cases are reported on in this paper. Each describes how student engagement and learning increased and how teachers’ attitudes and skills developed. Examples of student and teacher blogs are provided to illustrate how such technologies encourage students and teachers to look beyond text science.

Examining how teachers use web 2.0 technologies in Science lessons to promote higher order thinking in teaching science

During 2007 several independent Victorian secondary schools participated in a study exploring the ways in which the use of learning technologies can support the development of higher order thinking skills for students. This paper focuses on the use of Information and Communications Technologies (ICT) including Web 2.0 technologies for promoting effective teaching and learning in science. A case study methodology was used to describe how individual teachers used ICT and Web 2.0 in their settings. Data included interviews (focus group and individual), questionnaires, monitoring of teacher and student use of smart tools, analysis of curriculum documents and delivery methods and of student work samples. The evaluation used an interpretive methodology to investigate five research areas'. Higher-order thinking, Metacognitive awareness, Team work/collaboration, Affect towards school/learning and Ownership of learning. Three cases are reported on in this paper. Each describes how student engagement and learning increased and how teachers' attitudes and skills developed. Examples of student and teacher blogs are provided to illustrate how such technologies encourage  students and teachers to look beyond text science.

Generic skills : do capstone courses deliver?

Generic skills are increasingly the focus of universities worldwide and are often developed in professional practice courses. This paper presents qualitative findings from students regarding their perceptions of the generic skills they developed during a capstone course in an Information Systems program. The study found that the capstone course improved their collaborative team-work, presentation skills and ability to apply skills/knowledge to new situations. The paper also demonstrates that students’ perceptions of generic skills were more closely tied to the discipline than university-wide generic skills. This lends support for generic skills policy/practice to be driven bottom-up rather than top-down.

Student and staff perceptions of learning and teaching online, 2005 data

The dataset consists of data gathered from Deakin University staff and students.

Staff-derived data consists of qualitative data relating to advantages and disadvantages of teaching online; manifestation of cultural diversity in online learning environments; strategies to accommodate cultural diversity online; and using online environments to support cultural diversity

Student-derived data consists of quantitative and qualitative data relating to student perceptions of online learning; student demographics; student expectations of their university experience; students' approach to learning and online learning; perceptions of online learning and online team work; and perceptions of student and teacher roles at university.

The social side of social media : technology or collaboration?

This paper discusses how course design may draw upon social media in order to teach students appropriate skills for a network society in the context of team-work based learning. The emphasis is not upon web 2.0 and social media as inherently suited to providing educational solutions, but upon the ways in which they can be adapted by course designers within the framework of explicit learning objectives. More specifically, we provide a case study of how the use of social media in a blended or wholly-online learning environment provides affordances for team-based collaborative learning, especially when incorporated within a course design that encourages independent, self-directed and authentic learning. This paper argues we need to assess the social aspects of social media, rather than upon the technological, that is, avoid the fetishisation of 'apps,' through the creation of assessment that alternately foregrounds a critical appraisal of web 2.0 technologies and places onus upon the students to develop, with guidance, teamwork skills and processes. We provide an example of how it is possible to integrate web 2.0 technologies into their learning processes and assessment, in order to teach about the realities of collaborating with others in small teams in a work environment increasingly mediated by the Internet. In order to achieve these learning outcomes, course design needs to balance scaffolding with the need to place the imperative for learning specific content and skills upon the students, the latter through the provision of assessment outcomes and resources that the students need to work towards together.

Undergraduate nursing students' performance in recognising and responding to sudden patient deterioration in high psychological fidelity simulated environments: an Australian multi-centre study

Objectives
This paper reports the quantitative findings of the first phase of a larger program of ongoing research: Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends (FIRST2ACTTM). It specifically aims to identify the characteristics that may predict primary outcome measures of clinical performance, teamwork and situation awareness in the management of deteriorating patients.

Design
Mixed-method multi-centre study.

Setting
High fidelity simulated acute clinical environment in three Australian universities.

Participants
A convenience sample of 97 final year nursing students enrolled in an undergraduate Bachelor of Nursing or combined Bachelor of Nursing degree were included in the study.

Method
In groups of three, participants proceeded through three phases: (i) pre-briefing and completion of a multi-choice question test, (ii) three video-recorded simulated clinical scenarios where actors substituted real patients with deteriorating conditions, and (iii) post-scenario debriefing. Clinical performance, teamwork and situation awareness were evaluated, using a validated standard checklist (OSCE), Team Emergency Assessment Measure (TEAM) score sheet and Situation Awareness Global Assessment Technique (SAGAT). A Modified Angoff technique was used to establish cut points for clinical performance.

Results
Student teams engaged in 97 simulation experiences across the three scenarios and achieved a level of clinical performance consistent with the experts' identified pass level point in only 9 (1%) of the simulation experiences. Knowledge was significantly associated with overall teamwork (p = .034), overall situation awareness (p = .05) and clinical performance in two of the three scenarios (p = .032 cardiac and p = .006 shock). Situation awareness scores of scenario team leaders were low overall, with an average total score of 41%.

Conclusions
Final year undergraduate nursing students may have difficulty recognising and responding appropriately to patient deterioration. Improving pre-requisite knowledge, rehearsal of first response and team management strategies need to be a key component of undergraduate nursing students' education and ought to specifically address clinical performance, teamwork and situation awareness.

Leadership and teamwork in medical emergencies: performance of nursing students and registered nurses in simulated patient scenarios

Aims and objectives: To examine nursing students' and registered nurses' teamwork skills whilst managing simulated deteriorating patients. Background: Studies continue to show the lack of timely recognition of patient deterioration. Management of deteriorating patients can be influenced by education and experience. Design: Mixed methods study conducted in two universities and a rural hospital in Victoria, and one university in Queensland, Australia. Methods: Three simulation scenarios (chest pain, hypovolaemic shock and respiratory distress) were completed in teams of three by 97 nursing students and 44 registered nurses, equating to a total of 32 student and 15 registered nurse teams. Data were obtained from (1) Objective Structured Clinical Examination rating to assess performance; (2) Team Emergency Assessment Measure scores to assess teamwork; (3) simulation video footage; (4) reflective interview during participants' review of video footage. Qualitative thematic analysis of video and interview data was undertaken. Results: Objective structured clinical examination performance was similar across registered nurses and students (mean 54% and 49%); however, Team Emergency Assessment Measure scores differed significantly between the two groups (57% vs 38%, t = 6·841, p < 0·01). In both groups, there was a correlation between technical (Objective Structured Clinical Examination) and nontechnical (Team Emergency Assessment Measure) scores for the respiratory distress scenario (student teams: r = 0·530, p = 0·004, registered nurse teams r = 0·903, p < 0·01) and hypovolaemia scenario (student teams: r = 0·534, p = 0·02, registered nurse teams: r = 0·535, p = 0·049). Themes generated from the analysis of the combined quantitative and qualitative data were as follows: (1) leadership and followership behaviours; (2) help-seeking behaviours; (3) reliance on previous experience; (4) fixation on a single detail; and (5) team support. Conclusions: There is scope to improve leadership, team work and task management skills for registered nurses and nursing students. Simulation appears to be beneficial in enabling less experienced staff to assess their teamwork skills. Relevance to clinical practice: There is a need to encourage less experienced staff to become leaders and for all staff to develop improved teamwork skills for medical emergencies. © 2014 John Wiley & Sons Ltd.

Interactive exploration of data with visual metaphors

Visual Analytics (VA) is an approach to data analysis by means of visual manipulation of data representation, which relies on innate human abilities of perception and cognition. Even though current visual toolkits in the Business Analytics (BA) domain have improved the effectiveness of data exploration, analysis and reporting, their features are often not intuitive, and can be confusing and difficult to use. Moreover, visualizations generated from these toolkits are mostly accessible to specialist users. Thus, there is a need for analytic environments that support data exploration, interpretation and communication of insight that do not add to the cognitive load of the analyst and their non-technical clients. In this conceptual paper, we explore the potential of primary metaphors, which arise out of human lived and sensory-motor experiences, in the design of immersive visual analytics environments. Primary metaphors provide ideas for representation of time, space, quantity, similarity, actions and team work. Using examples developed in our own work, we also explain how to combine such metaphors to create complex and cognitively acceptable visual metaphors, such as 3D data terrains that approximate our intuition of reality and create opportunities for data to be viewed, navigated, explored, touched, changed, discussed, reported and described to others, individually or collaboratively.