Assessment: What Drives Innovation?

Nancy Falchikov

The University of Edinburgh & Napier University, Edinburgh
n­.falchikov@napier.ac.uk

Karen Thomson

Napier University, Edinburgh
k.thomson@napier.ac.uk

Introduction

Because all academics have their own understandings of what is meant by 'assessment', we should first define what we mean by the term. As Boud and Falchikov (2005: 6 of 8) have argued, 'It is unfortunate that in English we have a single term—assessment—that denotes radically different things, which has been positioned, probably for ever, as a sign of the dominant one.' In spite of several decades of innovation, for many, assessment is still seen as concerning measurement by professional assessors. Boud and Falchikov assert that a new discourse is required that positions assessment as an act of asserting control over one's own learning rather than one of subjugation to others. They have also championed the view that assessment needs to stimulate the development of learning for the longer term (Boud and Falchikov, 2004, 2005; Falchikov and Boud, 2007). We would also argue that assessment practices should benefit the learner directly, both at the time of the assessment event and in future learning. Assessment needs to support learning in general and be driven by the learner, to foster the attributes we expect of graduates and help learners prepare for a lifetime of learning.

However, as we have indicated, we are not so naïve as to think that this represents the state of higher education assessment today. Much current assessment practice seems to benefit others as much as it benefits learners. For example, Biggs and Tang (2007) suggest bureaucratic convenience and ease of procedures for those in a teaching role as two determinants of assessment patterns. It has long been recognised that the traditional grading system can be antagonistic to the goals of education (Burke, 1969). Traditional 'assessment by measurement' (Serafini, 2000) relies heavily on a very limited number of techniques such as unseen examinations and essay-type continuous assessments which have been found to be inadequate for the development of many desirable graduate characteristics such as critical thinking or creativity (e.g. Elton and Johnson, 2002). Traditional methods are, thus, not good at laying foundations for lifetime learning or preparation for work (Falchikov, 2005, chapter 2). However, not all innovations in assessment, even recent ones, are associated with beneficial outcomes. For example, multiple choice questionnaire (MCQ) type assessment is frequently associated with the use of technology. As we shall see below, while we found some papers in which the use of technology enhanced learning, the use of multiple choice questions featured in a 'preponderance of articles' (Hounsell et al., 2007: 53). MCQs are consistent with the measurement model of assessment (Taylor, 1994) where learning is seen as being quantifiable. Students focus on low level cognitive processes. The simple accumulation of marks is what is important, not how the score is comprised, with the ideas contained in one item being of the same 'value' as any other item. Maximising marks is seen as the important thing, not necessarily the overall structure of what is learned (Biggs and Tang, 2007) and this is consistent with a superficial treatment of subject matter and thinking as opposed to using extended abstract or synthesis as a tool for learning.

None the less, innovations in assessment, particularly those which involve students, have been found to give rise to many and varied benefits to learners. Not only have improvements to cognitive and meta-cognitive competencies (e.g. Dochy et al., 1999) and improvements to skills and performance (e.g. Falchikov, 2005) been observed, but social competencies and 'affective dispositions' (Birenbaum, 1996: 4) have improved as a result of innovative assessment procedures.

Assessment and the Demands of Practice

Before we can address the question of what drives innovation in assessment we need to consider how and to what extent assessment meets the demands of practice, and further what these demands might be. In the real world, assessment can be driven by external events and philosophies, such as the current endorsement of e-learning and assessment in the UK and elsewhere which can be associated with widening participation, expansion of student numbers and internationalisation or globalisation. For example, The Higher Education Funding Council strategic plan (2007) sets out the agenda for widening participation in higher education and in the face of this rapidly expanding and diverse student population, learning technology can be seen as a way of enabling flexibility in the teaching, learning and assessment environment to encompass this change. Inglis, Ling and Joosten (1999) in Australia examined the shift towards the use of e-learning in the context of globalisation. E-learning can cross time and space limitations and they suggested that tightly controlled e-learning is criticised as being assessment driven. However, this is not necessarily negative if the 'assessment is appropriate to the learning needs and learning outcomes' Inglis et al (1999: 98). Littlejohn and Pegler (2007) argue that the motives which lead to innovation are not always pedagogical but political, or concerned with funding. They further suggest that cost, quality and student expectations, in addition to widening participation, may play a part in shaping the learning, teaching and assessment environment. As higher education teaching becomes 'professionalised' by the requirement to join bodies such as the UK's Higher Education Association (HEA), some teachers may see innovation as a means of both fulfilling the formal requirements of membership and as a way of enhancing their chances of promotion.

The reasons why practitioners innovate in one particular area of assessment (student involvement) was explored systematically by Falchikov (2005). In an analysis of reasons given by teachers and researchers, she noted changing patterns over the course of the decades from the 1950s to 2000. Initially, desires to calculate and investigate reliability and validity constituted the rationale for involving students. However, a growing emphasis on benefits to learners was noted, such that, by the 1980s and 90s, benefits to students constituted the most frequently cited reason for innovation. In addition, some examples of what might be deemed to be external drivers were also noted. For example, some authors were citing increased workload and time constraints as reasons for innovation. Others referred to demands made by educational or professional bodies or external examiners. Falchikov's survey pointed to a change in the early years of the 2000s, associated with the development in the use of technology to support student involvement. Once again, practitioners seemed to be concerned with questions of reliability and validity. Falchikov predicted that, as researchers and teachers became reassured that their innovations in assessment delivered as much in terms of reliability and validity as traditional methods, we should see a 'flowering of reasons for implementing such studies' (Falchikov, 2005: 111).

In attempting to shed light on some of these issues, we shall base our arguments on data collected as part of the INNOVAS project (Innovative Assessment across the disciplines, 2007) which was carried out for the UK's Higher Education Academy (Hounsell et al, 2007). The project took the form of an analytical cross disciplinary review of the literature on innovative assessment in higher education including an analysis of publications retrieved from a search of the literature from the ten years between 1996 and 2006 by theme and discipline/subject area (based on the twenty four subject areas represented in the Academy's network of subject teaching centres). We took an inclusive and pragmatic approach to the issue of what exactly constitutes 'innovation' (e.g. as used in Hounsell et al., 2007).

Unlike Falchikov's work which focused on one specific assessment innovation, in the INNOVAS project, part of the thematic critique of the articles took the form of examining innovations of six different types (see also Thomson and Falchikov, 2007). These were

• Presentations and other non- written assessments
  
  
• Portfolios and other non conventional writing assignments
  
  
• Assessment of groups and collaboration
  
  
• Involving students in assessment
  
  
• Use of new technology in assessment
  
  
• Guidance and feedback - how and when
  
  

Within each of these thematic critique areas research questions were applied which sought to illuminate:

• the main drivers of, or stimuli to the innovations
  
  
• the main conceptual/empirical/practical points of reference in the innovation
  
  
• the main implications for assessment practices, within and beyond the discipline concerned and, where appropriate, for assessment procedures and policies.

More broadly, the analytical review aimed to evaluate the robustness and fitness for purpose of the literature on innovative assessment (for example in supporting and advancing assessment practices, policies and research) and ultimately to consider how the value of both practice and its reporting may be enhanced.

Thus, the results of the analytical review will enable us to see what the demands are on current innovative assessment in the UK. They will also enable us to evaluate Falchikov's prediction concerning a blossoming of reasons for assessment innovation in the field of the use of technology.

Results of the INNOVAS Survey: What Drives Assessment Innovation?

The drivers in the use of presentations and other non- written assessments were seen to be related to transferable skills or positive attributes in the students, for example improving communication skills, or in group presentations facilitating group working skills. The motivational factors involved in students' presentations and the link between interactive questioning and a deep approach to learning were noted as drivers. Student involvement in both setting and monitoring assessment criteria was suggested as a way of democratising assessment. The development of graduate attributes as endorsed by external political agencies such as the Quality Assurance Agency or by employers was also cited as a driver.

The use of portfolios and other non conventional writing assignments was process driven and made reference to student centred principles of lifelong learning, active learning and reflection. The Higher Education Academy's generic centre assessment guides and briefings featured as a driver in a number of papers as did the use of portfolios being prompted by the professional body. 

In several of the articles involving assessment of groups and collaboration the driver is an aspiration to encourage students to see the learning environment as a collaborative one rather than a competitive one. In some instances the consideration is to develop students' learning through embracing new perspectives and engendering abilities to understand, analyse and interpret. A further driver for this innovation is often a perceived need to produce graduates who have transferable skills, including the ability to work in teams and collaborate, as these skills are considered to be valued by prospective employers. In some cases the students are taking vocational or professional courses where group work is seen as not just desirable but necessary for their career and that assessment practices at university match working practice. In other cases, the driver may be the departmental or institutional context: the need to cope with increasing numbers of students, or students from more diverse backgrounds including from overseas, and to enable students to benefit from the experiences of those of other cultures. Group work may also be seen as a way of helping students make the transition to higher education by encouraging socialisation and the discussion of what it means to study in a particular discipline. 

The drivers to involving students in assessment were seen to be empowering learners, encouraging attention and dialogue, the enhancement of learning and/or performance and the improvement of affect and an understanding of assessment and its criteria. Other drivers included the expansion of higher education, the agenda of specific named bodies and the needs of the global economy or learning society. The presence of online learning environments and technological advances were also seen to drive some innovations.

The use of new technology in assessment was often driven by a review of established practice or the need to make use of new online learning environments or technology. Although much of the assessment which was reported involved multiple choice type questions there was evidence of a driver of enriching learning through technology used to facilitate networking and collaborative assessment. A major driver of technological innovation was cited as external funding mainly from government initiatives.

The drivers to innovation in feedback were disparate but often involved the management of ever increasing student cohorts and the desire by academics to provide feedback of sufficient quality and quantity which students would engage with in the midst of constrained resources. The perception of widespread student dissatisfaction and the desire to capitalise on opportunities opened up by developments in information and communication technology were also cited.

Summary

In essence we could divide drivers into two basic types- those which are related directly to the student learning experience (referred to as 'internal') and those which are clearly driven by external forces ('external') (see table 1).

Table 1 Examples of 'internal' and 'external' drivers to innovative assessment

Internal drivers included democratising assessment, developing lifelong learning, active learning, reflection, socialisation of students into a discipline where students 'learn the discourse' associated with the particular subject they are studying (Northedge 2003). The other benefits to students frequently cited as drivers included empowering learners, encouraging attention, creating partnerships, fostering proactive strategies in teachers and students, reflection, enhancement of learning, performance, personal development/ autonomy, improvement of affect, student understanding of assessment and student understanding of criteria.

As seen above, there were also abundant examples of external drivers to assessment within the INNOVAS project database. These included assessments developed in response to the demands of current educational practice or specific named bodies (e.g. governmental or professional), the recent expansion of higher education, the needs of the global economy, needs of the learning society or the work of others. Much involvement of technology in assessment seemed to be associated with the existence of external funding (e.g. TLTP; HEFC).

Not infrequently the two types of driver – internal and external - were linked. For example, innovative approaches to the assessment of presentations were designed in response to the teacher's perceived need for students to develop communication skills, but the need itself was sometimes identified by external agencies such as the QAA or employers (Hounsell et al., 2007).

Thus, a simple answer to the question we posed in the title of the paper is that there are many and varied drivers. Did we find differences between the different themes? The answer to this question is both 'Yes' and 'No', in that, while some drivers were theme-specific, many were common to several themes. In terms of Falchikov's (2005) prediction regarding an increase in variety of reasons for innovative assessment initiatives involving the use of technology, the INNOVAS survey again showed the wide diversity observed in the 1980s and 90s.

Limitations of the project and its findings

The brief of the INNOVAS project was to focus on the UK literature. In practice, the team took UK authorship and/or a focus on innovative assessment in UK higher education settings as definitive criteria, thus we can make no claims to wider generalisability of our findings. However, we were influenced by other non-UK literature on innovative assessment in higher education in designing and implementing our review. In particular, two Australian authors, Boud (e.g. 1986; 1995) and Sadler (1989; 1998), and two compendia of changing practices (Nightingale et al., 1996, James et al. 2002) were influential. In addition, we also found helpful important publications originating in South East Asia, continental Europe and the US (e.g. Carless et al., 2006; Segers, Dochy and Cascallar, 2003; Wilson and Scalise, 2006).

Within the review of the literature there were a number of constraints and limitations on the project team which are acknowledged here. For example, while every attempt was made to identify and locate all relevant articles, the short timescale of the project (September 2006 to March 2007) may have militated against achievement of this aim. In particular, 'the core higher education journals were given relatively (though not exclusively) greater priority in the search-and-review strategy, which have contributed in some degree to the predominance of journal articles in the database' (Hounsell et al, 2007: 66).

In addition, as argued in Hounsell et al (2007), the survey was not of the incidence of innovative assessment in UK higher education, but its documentation in the literature.

Discussion

We opened this paper with a discussion of some new views of assessment. The variety of themes from the INNOVAS literature suggests that practitioners are reporting moving beyond the traditional conceptions of assessment. This may be consistent with the view that innovation in assessment is a response to the demands of practice, whether these are external or internal. To what extent does the notion of assessment to support long-term learning feature in our findings? While explicit references to this are sparse, the implications are that the effects of innovations are likely to persist beyond the immediate assessment episode within formal education.

The INNOVAS project mined a very rich vein of innovative assessment practice. It seems reasonable to assume that these documented examples make up a minority of on-going work in this area, and that innovation is even more widely spread than the database indicates.

We have illustrated that, within the INNOVAS database, there are two main types of driver to innovation: internal which involve student-centred considerations of improving both learning and assessment itself, and external which act as not only drivers but magnets to innovation (c.f. Dunn et al, 2004). Overall, the survey indicated that: 'Recurring stimuli included a desire to recraft assessment practices the better to reflect contemporary mass higher education with its large and diverse cohorts of students and pressures on resources.' (Hounsell et al, 2007: 10). External pressures may also provide guidelines or recommendations for assessment practice. Commercial and professional bodies state what they require in graduates they might employ. Funding possibilities stimulate uptake of some innovative practices. To totally misquote the 1989 film, 'Field of Dreams', 'If they fund it, they will innovate'.

Did the survey illuminate Falchikov's (2005) speculation regarding increased variability to drivers of the use of technology? To some extent, it did. The use of technology to facilitate collaborative and active learning was found amongst the most recent articles in the dataset. Given this finding, it seems very possible that the trend to increased variety will continue.

Can the professional practice of academics with respect to assessment be seen as the balancing of internal and external drivers, where 'professional practice' takes place not only in the context of the individual, but also of subject groups and institutional policy makers all working within the framework of the educational mission statement of the organization? The aims of these parties are not always necessarily consonant. This can leave an individual practitioner with the problem of managing dilemmas, prioritising and balancing competing demands. Innovation in assessment may be a pragmatic way in which practitioners resolve some of these potential conflicts and assessment does 'double duty' supporting student learning while addressing political, professional or institutional policy demands. If the individual academic is to provide a bridge between the two main types of drivers – student benefits and external pressures – then the question arises as to how they may be supported in this.

Acknowledgements

The authors would like to acknowledge the contributions made by the rest of the project team: Sheena Blair, Dai Hounsell, Jenny Hounsell, Mark Huxham and Monika Klampfleitner.

References

Biggs, J. & Tang, C. (2007). Teaching for Quality Learning at University (3rd Edition). Berks: SRHE & OUP.

Birenbaum, M. (1996). 'Assessment 2000: towards a pluralistic approach to assessment', in M. Birenbaum & F.J.R.C. Dochy (Eds.) (1996) Alternatives in assessment of achievements, learning processes and prior knowledge, Boston, Dordrecht and London: Kluwer Academic Publishers.

Boud, D. (1986). Implementing Student Self-Assessment. HERDSA Green Guides, no. 5. Kensington NSW: Higher Education Research and Development Society of Australasia.

Boud, D. (1995) Enhancing learning through self assessment, London and Philadelphia: Kogan Page.

Boud, D. & Falchikov, N. (2006). Aligning assessment with long-term learning, Assessment and Evaluation in Higher Education, 31, 4: 399—413.

Boud, D. & Falchikov, N. (2005). Redesigning assessment for learning beyond higher education, in Research and Development in Higher Education, 28. A. Brew & C. Asmar, (Eds.), Sydney: HERDSA, 34—41.

Burke, R.J. (1969). Some preliminary data on the use of self-evaluations and peer-ratings in assigning university course grades, Journal of Educational Research, 62, 10: 444—448.

Carless, D.; Joughin, G.; Liu, N.-F. et al, eds. (2006). How Assessment Supports Learning: Learning-Orientated Assessment in Action. Hong Kong: Hong Kong University Press. http://www.hkupress.org/asp/bookinfo.asp?PD NUM=9622098231

Dochy, F.; Segers, M. & Sluijsmans, D. (1999) The use of self-, peer and co-assessment in higher education: a review, Studies in Higher Education, 24, 3: 331—349.

Dunn, L.; Morgan, C.; O'Reilly, M. & Parry, S. (2004) The student assessment handbook: new directions in traditional and online assessment, Oxon: RoutledgeFalmer.

Falchikov, N. (2005). Improving assessment through student involvement: practical solutions for aiding learning in higher and further education, London: RoutledgeFalmer.

Falchikov, N. & Boud, D. (2007). 'The role of assessment in preparing for lifelong learning: problems and challenges', in A. Havnes & L. McDowell, (Eds) Balancing dilemmas in assessment and learning in contemporary education.

Hannan, A. & Silver, H. (2000). Innovating in Higher Education: Teaching, Learning and Institutional Cultures. Buckingham: SRHE & Open UP.

Havnes, A. & McDowell, L. (Eds.) (2007). Balancing dilemmas in assessment and learning in contemporary education, Routledge: USA.

Higher Education Funding Council strategic plan (2007).

http://www.hefce.ac.uk/pubs/hefce/2007 (last accessed 15.2.08)

Hounsell, D.; Falchikov, N.; Hounsell, J.; Klampfleitner, M.; Huxham, M.; Thomson, K. & Blair, S. (2007). Innovative assessment across the disciplines: an analytical review of the literature, Higher Education Academy.

http://www.heacademy.ac.uk/assets/York/documents/ourwork/research/Innovative_assessment_LR.pdf (last accessed 22.2.8).

Inglis, A.; Ling, P. & Joosten, V. (1999). Delivering Digitally:Managing the Transition to the Knowledge Media. London, KoganPage.

Littlejohn, A. & Pegler, C. (2007). Preparing for Blended e-Learning. Oxon: Routledge.

Nightingale, P.; Te Wiata, I.; Toohey, S.; Ryan, G.; Hughes, C. & Magin, D. (1996). Assessing learning in universities, Sydney: University of New South Wales Press.

Northedge, A. (2003). Enabling participation in academic discourse. Teaching in Higher Education 8(2), 169-180.

Sadler, D.R. (1989). Formative assessment and the design of instructional systems, Instructional Science, 18, 119-144. 

Sadler, D.R. (1998). Formative assessment: revisiting the territory, Assessment in Education, 5, 1: 77-84.

Segers, M.; Dochy, F. & Cascallar, E., eds (2003). Optimising New Methods of Assessment: In Search of Qualities and Standards. Dordrecht: Kluwer.

Serafini, F. (2000). Three paradigms of assessment: measurement, procedure, and enquiry, The Reading Teacher, Dec., 54, i4, 384- (accessed electronically, 11.9.1).

Taylor, C. (1994). Assessment for measurement or standards: the peril and promise of large- scale assessment reform. American Educational Research Journal, 31: 231-62.

Thomson, K. & Falchikov, N. (2007). An Analytical View of Innovation in Assessment and its Relationship to Practice. In HERDSA Proceedings of the Annual HERDSA Conference 2007: Enhancing Higher Education Theory and Scholarship. 8-11 July 2007, Adelaide Australia. ISSN 1441001X. ISBN 0908557728

Wilson, M. & Scalise, K. 2006). Assessment to improve learning in higher education: the BEAR assessment system. Higher Education, 52, 4: 635-663.