back to contents page

Introduction

In Australia, there is a growing public perception that academic standards at universities are slipping. Research reported that plagiarism, as well as cheating in general, was increasing in universities (James & McInnes, 2001; Maslen, 2003; O'Connor, 2002; Zobel & Hamilton, 2002). This is often fuelled by media exposure of declining attention by our seats of higher learning to issues of academic rigour in assessment. The media frenzy over the issues of plagiarism at Curtin University in Western Australia (Malatesta, 2001); Monash University (Madden, 2002a, 2002b) and RMIT (Hunt, 2003) in Victoria and the University of Newcastle (Davis, 2003) in New South Wales, have pressured universities to publicly react - commonly in a punitive manner. Staff have been sacked, external investigations instigated and independent commissions of inquiry held in order to allay fears of lack of academic integrity. Many universities have overhauled curricula and assessment practices, amend plagiarism or academic misconduct policies and overtly introduced technology in order to reduce plagiarism. Not only have studies overseas (McCabe, 2003) indicated that technology increases the chances of detecting plagiarism, but recent Australian studies support the advantages of anti-plagiarism software - particularly Turnitin.

Helen Marsden's (2001; 2005) research into dishonest academic behaviours at the Australian National University found that extremely high rates of plagiarism existed for students with low levels of academic self-esteem. In 2002, Steve O'Connor conducted an Australia-wide study investigating the extent to which university students plagiarised. He employed Turnitin to measure the amount of plagiarism occurring and reported that 70 per cent of universities participating in the study were affected. He concluded that Turnitin was able to detect plagiarism in a large number of cases. Against the backdrop of universities seeking effective, cost-effective and timely detection of plagiarism, Turnitin has been adopted, even lauded as a solution to forms of academic misdemeanour. This paper provides a small-scale study exploring the effectiveness of the software from a user's point of view as, to date, little empirical data has been obtained about teacher-users' perceptions of Turnitin.

The Study Design and Methodology

The Turnitin anti-plagiarism software package was developed by Dr. John Barrie of the University of Berkeley, California, and is used by many universities in Australia, and indeed in 51 countries around the world (iParadigms, 2004). In the wake of other Australian universities adopting the software, South-Coast University (SCU) decided to trial the software during 2004 with 2,000 students and seven teachers. Participants in the trial were located on all four campuses of SCU and the seven teachers taught in undergraduate programs from five different faculties: Arts, Business, Law, Education and Science & Technology. The aim of the study was to explore teachers' perceptions of the usefulness and applicability of Turnitin in tertiary classrooms. In addition, staff were asked to comment as to how Turnitin might be used as an educative tool. The purpose of the research was to inform the University Academic Board whether staff considered Turnitin would be a worth while investment of university funding, and if a licence was purchased, whether staff would find it easy to access and therefore utilise the product.

A qualitative case study approach using interviews and observation of staff was adopted. Three interviews with staff were designed: an initial interview to gauge teachers' conceptions of what the software would do; a medial interview after software training and a final interview in which teachers uploaded assignments to Turnitin and checked for text matches. Observation of teachers' use of the software and questioning about application and usability were undertaken at this stage. Observation notes were taken by a research assistant while the primary researcher asked questions. Interviews were transcribed and N*Vivo, a qualitative package useful for coding data, was used in order to organise data by theme.

Teachers' Perceptions of Turnitin at Initial Interviews

At initial interviews, which were conducted before teachers had undergone Turnitin training, most teachers expected the software would identify plagiarism. Their expectations were that in some way, the software would highlight passages of text and identify those passages as plagiarised. They expected they would need very little input in identification of plagiarism. Six teachers at initial interview favoured the introduction of the Turnitin software. The majority considered that the software would probably detect plagiarism more efficiently and consistently than they could. They also believed that the online detection tool would be quicker and could search more widely than each individual teacher. This, they considered, would save time, as at present they used a battery of search engines, such as Google, Yahoo and Dogpile. Their comments are captured in the response of Glen, a psychology teacher from the Faculty of Arts who said:

This is going to be great. It should save me hours because it will trawl through thousands of pages of material on the databases and find spots where text has been lifted. I hope it will be a quick and relatively thorough means of finding out where students plagiarise and highlight those spots for me. It takes me so long at the moment and even though I can spend a couple of hours checking suspect papers, I'm never really convinced that I've got everyone. This could be the answer. (Glen, interview, May, 2004).

Comments such as 'It'll be fantastic, the software will find where students' have plagiarised from and then they can be dealt with' (Kate, Faculty of Law, interview June 2004) summarises the common perception of six of the seven teachers involved in the trial. One teacher from the Faculty of Education, however, expressed reservations about using the software. He was concerned about the way in which such a tool could be used to punish rather than help students overcome issues with plagiarism. Barry said:

You know, I'm worried about the application of this software. I hope we as a university don't invest in it. I think it makes it all too easy to rely on technology as empirical evidence of plagiarism and this will allow staff not to check out with students what's going on with their writing. It offers an easy solution to academics to point the finger at students and in a way there's an abrogation or our responsibility to teach the students not to plagiarise in the first place. I think it will be used as a stick rather than a carrot. (Barry, interview, April 2004).

Generally, staff perceptions were that the software would identify plagiarism and take that responsibility away from staff. Of course, this is a slightly na•ve expectation, as the software can identify matching tracts of text but cannot, of course, make the decision whether the text-match constitutes plagiarism or not. This is because some of the text-matches that occur are correctly referenced, as the software does not distinguish between text that is referenced and text that is not. The software only identifies matching sets of text. The decision as to whether the text-match constitutes plagiarism or not must be made by individual teachers. It is assumed that when teachers decide a student has breached the plagiarism regulations of the institution that the teachers have relied on those regulations as stipulated by their own university.

Teacher Perceptions After Turnitin Training

Medial interviews were conducted after teachers had been trained in the use of Turnitin's anti-plagiarism software. Of the seven teachers, six were a little disappointed that the software did not discount text that was correctly referenced. Comments are aptly summarised by Deb from the Faculty of Business and Law who said:

I had thought that the text-match would pick up text that was not correctly referenced. It hadn't occurred to me that ALL text that matched would be identified. Really, in terms of effort, I have to now go through and see whether the text that has been identified as 'match' has been correctly cited or referenced by the student or not. If it has, then there's no question of plagiarism. I didn't expect this, as I said, so it will take a bit more time than I thought initially. Also, the lists of references are identified as 'match' because they've been used in the body of the assignment. Again, that's fairly automatic checking but it'll take time that I didn't think I'd need to spend. Hopefully the software can be improved so that things like direct quotations can be recognised and discounted as potential plagiarism. (emphasis in the original, Deb, interview, August 2004).

On the other hand Will, from the same faculty, said he had expected this to be the case. He was not surprised that the software detected text-matches but did not distinguish between correctly referenced and non-referenced text. Will said he had used other anti-plagiarism software in the past and this was a common problem. The advantage of Turnitin software, as teachers perceived it, is that the degree of text match is identified by colour and highlighted in-text. Will thought this feature was highly desirable. He said:

This is of huge benefit. Generally, I'm only going to check the text that comes up anywhere from 100% match [red] to 33% match [yellow]. I don't tend to worry about the blue and green as they're so insignificant that there's not likely to be a problem with that degree of match. The colours mean I can decide without opening individual documents, which ones I'm going to check more closely, just by looking at the colour-coding percentage of match. It's wonderful! (my insertions, Will, interview).

The overall perceptions of the teaching staff were that the software would have done more of the 'hard yakka' (Kate) for them. They commented that although the software detected text-matching and identified it by a colour scheme that was easily read, the time-consuming task for staff would be trawling through the identified text-matches. Teachers still needed to ascertain whether within the identified text-matches the student had used direct quotations, lists of references or footnotes.

Teachers stated that this was as time-consuming as the previous manual methods they employed. As Kate said, she felt overwhelmed by the workload and felt it was a question of 'choosing whether it's the devil or the deep blue sea' in terms of effective plagiarism detection. Six of the seven teachers considered that the software would increase their workload and they were not sure about the benefits to individual students' learning.

Final Perceptions of Turnitin by Teachers

At final interview, teachers were asked about the usefulness and usability of the software. Most teachers were reasonably happy with the usability of the software. They considered that instructions for use were clear and uploading documents to the Turnitin site was quite manageable. One teacher, Will, felt that the software needed to be more compatible with SCU's internal software system. Will taught a large number of off-campus classes where work, discussion and interaction took place online. Assessment tasks were also submitted and marked online. He considered it would be more useful to be able to take students' assignments from the WebCT platform that SCU uses and upload straight into Turnitin. This could not be done at the time of this trial. Will described that the number of steps needed to transfer the data from one place on the SCU system to Turnitinom.com as 'clunky', meaning it was laborious and time-consuming for the teacher. At the time of writing this paper, iParadigms (the company responsible for marketing Turnitin software) is investigating the improvements to interfaces with different Web platforms.

Teachers expressed different reactions to Turnitin's speed of delivery of results. Some teachers considered the speed at which papers were returned (within minutes) to be most efficient. Others were not impressed as their papers took up to one hour to be returned. One teacher, Deb, from the Faculty of Business and Law, said that she had experienced problems with the system and several times had not been able to log on to check her students' work. From our project there appeared to be no pattern with speed of processing from Turnitin's central server. There may, however, be an increased chance of delay with a greater number of papers being sent for checking. For example, Nola only sent two papers to Turnitin to be checked and they were returned within one minute and she was delighted with the efficiency of the service. Deb sent 68 papers and they were returned one hour later. She considered this to be a lengthy delay. Glen sent 168 papers to the site and they were returned in one and a half hours. He said this was quite acceptable. Therefore, it appears that degree of satisfaction with the speed of checking may depend on the individual teacher's expectation of software efficiency.

Overall teachers said that the software was useful to identify assignments where plagiarism was a problem. Teachers considered the colour-coding system of identifying text match by quantity, and presenting it as a total percentage of the whole work, assisted them deciding whether to check the assignment further. Most staff, after checking through some papers with the full range of colours, decided that checking only yellow, orange and red cases was worthwhile. They found that in most cases, where red coding indicated 100 per cent text match existed (i.e. the entire assignment) that the assignment had been sent to the site twice. This means the assignments, when checked against each other, will be exactly the same, therefore red will appear as the coding for both pieces of work. A 100% match could be the result of human error, or in the case of one teacher - Craig - it was an assignment that contained an OECD report that had simply been 'cut and pasted' into the assignment answer.

The student had only added a brief introduction. Craig said he was suspicious of the assignment initially when he had begun marking it and decided to send it to the Turnitin site for checking. He was most impressed with the results. The check came back within minutes and clearly identified the source of the OECD paper, the date and other information. Craig said:

This is great because I intend to use this program to catch similar 'cheats'. How stupid do they think I am? Fancy just downloading a whole OECD report and expecting me to swallow that it's their own work. It was just cut and pasted from the OECD site...the whole thing! (Craig, Faculty of Business, interview 14 September, 2004).

Issues of Academic Integrity - How Can Turnitin be Used to 'Educate' Rather Than Punish?

Teachers were asked to consider how they might use Turnitin in their different faculties and disciplines to inform students about plagiarism. This use of Turnitin as an educative tool was proposed as an additional device to the myriad of specific practices different faculties implemented to reduce plagiarism. It is not the aim of this paper to discuss the broader measures taken to reduce plagiarism through curricula reforms or student support services, but to focus on the ways in which Turnitin can add to existent learning services. Four staff members were concerned that teachers could, and would view the software as a 'stick not a carrot' - a purely punitive tool. Three members of staff felt that the tool was primarily useful as an anti-plagiarism detection device and should be used as such. They said that providing students were given 'due notice' that the software was used at SCU, the university had discharged its responsibilities. The teachers felt that where students were caught for plagiarism and punished, that would be the educative value of the anti-plagiarism software, as students would be unlikely to re-offend.

Four teachers considered that Turnitin should be used in a more 'positive educative manner' (Barry). There were two main suggestions that are currently being developed into practical learning activities for students:

• A series of online materials to be made available, in interactive form, detailing how students could check their own work through Turnitin before submission to staff. This be integrated with specific 'academic integrity' materials designed to help students understand issues of plagiarism and collusion.
  
  
• Subject specific requirements for adherence to citation conventions be introduced into the Support Tutorial program.
  
  

Online Assistance

Staff felt that online materials needed to be made available to all students concerning Turnitin. The following are being developed:

• A clear, one page instruction guide for students about accessing the Turnitin site. Although Turnitin does provide guidelines for submission to the Web site, the document is long and quite complex. It needs to be broken down into smaller, easy to follow steps for students, particularly for those whose first language is not English.
  
  
• An 'easy to understand' definition of plagiarism and also collusion, with hotlinks to dictionary definitions of words and also the university policy. Staff felt that the university policy definition was unwieldy for students and the nuances of meaning were difficult to grasp for second language students. Although the policy has been recently revamped, staff felt that is was still exceptionally difficult for first year undergraduate students to internalise.
  
  
• A series of subject specific scenarios to be developed, following Carroll's (2002:41-2) model, for students to apply their understandings of academic integrity issues to practical situations. Explanations of various solutions are to be provided at the end of each exercise with 'voices' of different students' reactions to the scenarios being included. The aim of these exercises is to engage students in scenarios based on real-life experiences of plagiarism and encourage them to think about student plagiarism from different perspectives.
  
  
• A series of exercises in academic writing skills using introductory textbook material be offered for all compulsory first year units of study. Sections of original text will be provided that relate to past assignment topics in the subject. Students will then read the text and write an online summary, using correct, faculty-specific citation conventions. Online samples answers will be provided. Teachers believe that this will be of greater benefit than the 'generic' students support materials currently available.
  
  
• Hotlinks will be provided for students to various students support services available for academic writing assistance, academic writing counselling and generic plagiarism online workshops.
  
  

Teachers explained that students needed the flexibility of online access to this type of resource and they considered it needed to be broader than just a 'how to access Turnitin'. As Glen said, 'Students need to link the concept of intellectual honesty to checking their work. They need to do more than just run their assignments through the Turnitin site to see whether they'll be picked up or not. It's much deeper than that'. These teachers considered that Turnitin could form part of the raft of existent support services for students. They were keen that Turnitin be positioned in such a way that it was not an easy tool to 'catch cybercheats' but marketed by the university as a device to students to learn how to engage with textual attribution conventions.

Tutorial Support Program

Staff also believed that linking the physical checking for plagiarism through Turnitin could be fleshed out in the subject specific Tutorial Support program, currently available for students. These optional classes are taken by specialist staff in each compulsory first year subject and the aim is to help students grasp key ideas but also develop academic skills needed for the tertiary undergraduate journey. Staff were unclear about how this could be achieved in the Tutorial Support program, but thought that it was another avenue to make the information and skills available for on-campus students. These suggestions are currently being implemented as a pilot scheme in SCU's faculty of Business.

All names are pseudonyms to ensure anonymity for participants

Conclusion

This study highlighted the benefits of using plagiarism detection software, such as Turnitin. Whilst some felt that it was beneficial in the detection of text-matching, most had reservations about how time-saving it would be. Others feared it could be used as a means of punishing students under the policy and that the underlying issue of plagiarism remains unresolved. Staff considered that Turnitin could perform a useful function in heightening awareness of plagiarism as an issue of academic integrity, where subject-specific exercises were developed and students invited to submit their own work through Turnitin. Staff considered understanding the process of submission to the site needed to be specifically taught, as site navigation was not intuitive. The findings of this study are being implemented with the aim of assisting students in both on-campus and remote access locations to better understand and cope with the issue of plagiarism in academic writing.

References

Carroll, J. (2002). A handbook for deterring plagiarism in higher education. Oxford: Oxford Centre for Staff and Learning Development.

Hunt, Elissa. (2003). Uni cheat gets off. Herald Sun, Wednesday 4 August, pp. 1-2.

iParadigms. (2004). iParadigms: Digital solutions for a new era in information. Retrieved 31 December 2004,
from http://www.iparadigms.com

James, Richard & McInnes, Craig. (2001). Standards oil tertiary debate. 'The Australian', Wednesday August 15, p.28.

Madden, James. (2002a). Academic Thievery won't wash. 'The Australian', Wednesday 10 July, pp. 6-7.

Madden, James. (2002b). Plagiarism returns to haunt Vice-Chancellor. 'The Australian', Thursday 26 June, p.4.

Malatesta, Grace. (2001). Cheat case not put to rest. 'The Australian', Wednesday 23 May, p.44.

Marsden, Helen. (2001). Who cheats at University? Unpublished Honours paper in Applied Psychology, University of Canberra, Canberra.

Marsden, H.; Carroll, M. & Neill, J. (2005). Who cheats at University? A self-report study of dishonest academic behaviours in a sample of Australian university students. Australian Journal of Psychology, 57 (1), 1-10.

Maslen, Geoff. (2003). Dirty marks. The Bulletin, Tuesday June 10, pp.18-22.

McCabe, D. (2003). Promoting academic integrity - A US / Canadian perspective. Educational Integrity: Plagiarism and other perplexities conference proceedings, (pp. 3-12). Adelaide, Australia, 21-22 November, 2003.

O'Connor, Steve. (2002). Electronic plagiarism and its impact on educational quality. Paper presented at the CAVAL electronic plagiarism conference, (6 October), Melbourne, Australia

Zobel, Justin & Hamilton, Margaret. (2002). Managing student plagiarism in large academic departments. Australian Universities Review, 45(2), 23-30.

back to contents page