|
Integrating Information Literacy Into Curriculum Assessment Practice: An Informatics Case StudyAnnette Meldrum and Holly TootellUniversity of Wollongongameldrum@uow.edu.auAbstract
Introduction to assessment practiceAs we strive toward the goal of helping students acquire highly developed graduate attributes we need to create assessment tools and strategies that access, define or measure the extent of these learning outcomes. Assessment is in itself a teaching method (Nulty, 2003). Assessment of information literacy promotes learning, provides a yardstick to measure teaching performance and outcomes, highlights learning gaps, promotes the need for in depth structure for learning, can provide opportunities for student and teacher discourse on the topic of information literacy needs or difficulties and provides a foundation for teaching evaluation and certification (Lupton, 2004). This article explores the extent of our effectiveness in these areas. James and McInnis (2001) identified a number of factors that influence student assessment in Australian higher education, including the prominence of generic skills and the changing nature of the students themselves. The University of Wollongong Review of Assessment Practices and Processes Report (RAPP) (UOW, 2002) found that increased class sizes, increased internationalisation and increasingly diverse student entry backgrounds is necessitating a rethink of teaching and assessment practices. These phenomena are all reflected in the makeup of this case study group. IACT201 Information Technology and Citizens' Rights, is a compulsory subject offered by the School of Information Technology and Computer Science at the University of Wollongong. The subject caters for approximately 300 students at both undergraduate and postgraduate levels. We attempted to integrate an information literacy structure where learning is embedded within the discipline specific subject curriculum promoting the 'interdependence of information, technology and academic literacy' (Hart, McCarthy & Peacock, 2003). This was of major importance as many of these students come from diverse backgrounds. They included many international students and for some postgraduate students, this subject was their first experience of study in Australia.
Information Literacy at the University of WollongongThe common theme for both the integration and assessment of Information Literacy learning is that there needs to be close collaboration between all educators (Candy, Crebert & O'Leary, 1994; Bruce, 1995; Iannuzzi, 1998; Wright & McGurk (2000); Lupton, 2004; Colvin & Keene, 2004; Thaxton, Faccioli & Mosby, 2004). In this instance, collaboration has involved, the Subject Coordinator, Tutors, Faculty Librarian and Lecturers from the Learning Development Centre. Collaboration works well when there is acknowledgement of the academic's in-depth subject knowledge, the librarian's information literacy competencies and their combined teaching experience and knowledge of the problems students' experience. The Council of Australian University Librarians (CAUL) (2001) defines information literacy as the ability to define, locate, access, evaluate and use information in an ethical and socially responsible manner as part of a lifelong learning strategy. Tertiary literacy is defined in the University of Wollongong's Tertiary Literacy Policy and Procedures as consisting of academic literacy, information literacy, computer literacy, statistical literacy and professional practices. These literacies in turn refer to the detailed levels of knowledge and skills that lead to the development of the University of Wollongong Graduate Attributes. The integration of tertiary literacy skills into curricula is a priority of the University's Learning and Teaching Strategic Plan 1997-2005 and is seen as a collaboration between faculty academics, educational development lectures, learning development lecturers, librarians, career practitioners and administrators. The University Library is recognised for its groundbreaking work in the field of information literacy in Australia (Lipu, 2004, p.50). It was one of the first to endorse an Information Literacy Policy and develop and implement the Information Literacy Introductory Program (ILIP), a compulsory, non-credit point information literacy subject for all undergraduate students (Wright & McGurk, 2000). Faculty Librarians work in close collaboration with academic staff to integrate information literacy into their subjects by providing information literacy instruction and support. Fostering information literacy is seen as the Library's central role.
Integrating Assessment and Information Literacy in IACT201The large class size and increasing diversity of students in IACT201 necessitated helping students make the critical connection between understanding content and developing academic skills (Bruce & Candy, 1995). Information Technology students tend to consider themselves, highly computer literate but fail to recognise that they are information illiterate and assume that they can intuitively discover how to use the systems unassisted (Emmons & Martin, 2002; Flaspohler, 2003). Until 2003, information literacy activities had been presented separate to the subject material. In theory this appeared to be a useful mechanism for students to separate content from skill acquisition, however information literacy activities were regarded by students as unimportant. Student feedback in 2002 suggested that the information literacy activities be integrated into the subject in a more 'invisible' manner. In order to address these issues, a workbook model was adopted to bring together the integration of information literacy skills and subject content which included; lecture material, tutorial activities that comprised of readings, class exercises and information literacy activities. This provided an integrated model which gave students greater flow and direction as they worked through the subject. This holistic approach was welcomed by students whose feedback included;
"The implementation of the tutorial workbook was a welcome addition. It added a more organised and comprehensive structure" Information literacy activities focused on three major areas: research and evaluation of research sources with fair and accurate referencing; critical thinking and essay writing. Students learnt not just to cut and paste but to critique, analyse, evaluate and modify their writing to continually develop their skills and knowledge. Students identified the value of this approach in feedback;
"Good skills-development exercises for academic research and writing" The changed assessment was documented as a best practice example of Graduate Attribute 5 "ability to logically analyse issues, consider different options and view points and implement decisions". This formed part of a project funded by a 2003 Collaborative Grant from the University of Wollongong Educational Strategies Development Fund, 'Developing the Attributes of a Wollongong Graduate' (http://www.uow.edu.au/about/teaching/attributes/informatics/attribute5.html).
AssessmentAssessment should help students to learn and be consistent with subject objectives which reflect an emphasis on building information literacy skills (CSHE, 2002a; Flaspohler, 2003). Assessment and subject content particularly in an Informatics subject need to reflect changing technologies and world views to be relevant to today's students and their diverse backgrounds (UOW, 2002). In 2002, the first phase of a subject review was undertaken to align assessment and information literacy components. In consultation with Learning Development Lecturers and the Faculty Librarian, information literacy activities were integrated into the assessment tasks. This was further expanded in 2004 to include lecture content, assessment quizzes and comprehensive tutorial activities that involve student interaction with information literacy activities. This variety of assessment tasks enabled the assessment of learning at varying levels and promoted a wider experience of both generic and discipline-specific skills (Lupton, 2004). Markless, Streatfield and Baker (1992) and Wright & McGurk (2000) believe that students are more convinced of the value of information literacy skills when they are articulated in the classroom. All information literacy presentations were delivered in the context of the topic being studied and focused on the approaching assignment question. Summative assessment where marks contribute to final grades (Nulty, 2003) helped to provide a focus for the student and gave credence to the information literacy components. See the essay marking criteria in Appendix 1. Library presentations within lectures were scheduled to strategically align with assignment preparation dates as shown in Figure 1. Time of need instruction is one of the principles of successful information literacy development (Nimon, 2001). These lectures were audio taped for delivery to the South Coast Campus students. Copies of the Power Point presentations were loaded onto the University's Online Learning platform, WebCT. The Shoalhaven Campus Librarian offered the local students short library tutorials mirroring the main campus lecture presentations.
Figure 1: Information Literacy Structure in Curriculum Assessment Practice - IACT201 Library services, especially online guides to finding information, generic library workshops, online tutorials, database guides and the Information Technology Resources by Subject webpage were presented. URL's to the resources shown were included in the student's workbook. Veldof & Beavers (2001) extol the usefulness of online tutorials to support faculty in integrating information literacy into their curricula. Multiple choice quizzes and activities were developed collaboratively to reinforce the training and develop familiarity with the resources. Bruce & Candy (1995), Crebert & O'Leary (1995); Nimon (2001) and Flaspohler (2003) all cite the need for repeated opportunities to reinforce learning. Library activities included; writing annotated bibliographies using academic journal articles from the resources demonstrated and the inclusion of printouts showing search strategies. These small assessment tasks also attracted marks and were due the following week. A challenge in IACT201 is the setting of assessment tasks that take into account:
The assessment tasks were designed to synthesise major concepts and skills learned throughout the subject. The assessment criteria for the essays was designed to encourage the students to practice their information literacy skills and be rewarded for their efforts (Wallace et al., 1999, p.139). To meet the challenges of assessment for the large class, clear assessment criteria were communicated to students and tutors in the form of Assessment Sheets (see Appendix 1) together with the key to ratings, clearly setting out what was being assessed and how marks would be assigned. The importance of this process is outlined in the Centre for the Study of Higher Education report on Assessing Large Classes (CSHE, 2002b). For each assessment task, students knew the criteria they would be assessed on prior to commencing the work. Formative evaluation was provided by the tutors on the assessment sheet. In the lecture, summary statistics were shown to allow students to gauge their performance across the enrolment. Students also participated in guided reflection activities in tutorials giving them the opportunity to reflect on this formative evaluation and identify areas for improvement. This type of formative assessment provides valuable feedback to enable students to progress and learn through experience (Nulty, 2003). For most students this was an opportunity to see the importance of correct referencing, writing structure and flow of ideas. By the time students submitted the second essay they had participated in further reflection sessions where they brought skeleton essays to class in order to test ideas and further develop arguments from group feedback. This reinforcement and improved familiarisation with information literacy skills was reflected in the higher average mark in the second essay. Students were more confident in their approach to the task and displayed increased competency in their ability to find suitable information. Feedback confirms the increased confidence of students;
"The workbook was good - it taught us some of the things that we're already supposed to know even though no-one else does it"
Analysis of results
Figure 2: Comparison of essay results The literature universally acclaims the inherent difficulties of assessing improvements in information literacy however Barclay (cited in Flaspohler, 2003) maintains that 'some hard evaluation data, even if the data may be less than perfect, are better than either no data at all or soft data obtained from anecdotal observation and surveys of student satisfaction'. Figure 2 represents the spread of marks between essay 1 and 2 in 2004. The average mark for the essays improved from 68% in essay 1 to 73% in essay 2 for the undergraduate intake. The average mark improved from 58% to 67.5% for the postgraduate student group. This seems to indicate that students from non-English speaking backgrounds especially benefited from the information literacy component of the subject. This group mostly comprised newly arrived international students. The subject content was designed to help these students adjust to the new expectations of higher education practice in Australia and possibly unfamiliar assessment practices as detailed in the Centre for the Study of Higher Education report (CSHE, 2002c). The explicit marking criteria and practice exercises enabled students to develop familiarity before summative assessment submission. Students showed an attempt to use scholarly journals in essay 1, but most restricted their searching to one online database, with most support for the essay arguments coming from one source. The formative evaluation focused on these issues. It also became a target issue for essay 2 where a marked improvement was seen. Figure 3 shows a trend to a higher average overall grade in the subject. This figure consolidates the results for both the undergraduate and postgraduate groups. In 2003 32% of students received a Distinction or High Distinction. This increased to 46% in 2004. We believe that this is a reflection of the success of integrating information literacy into the student learning experience.
Figure 3: Final Grade Comparison
ConclusionThe integration of Information Literacy instruction into the IACT201 program appears to have made a significant improvement to the grades of both undergraduate and postgraduate student groups and ensured progressive development of increasingly sophisticated information skills (Bruce & Candy, 1995). The case study results support information literacy principles that collaboration of library and academic staff, the time of need integration of information literacy training into the curriculum and linking it with the assessment process ensures successful outcomes (Colvin & Keene, 2004). This experience has provided us with a wider perspective and the benefits of collaborative energy (Thaxton, Faccioli & Mosby, 2004, p.185). It is hoped that ongoing collaboration will enable the adoption and integration of information literacy across the whole faculty so that skills can continue to be developed to higher levels as students move through the course (Wallace et al, 1999 p.137). Just as the quality framework focuses on the need to constantly review and improve all processes, we have already identified areas to target in 2005. We will strengthen the relationship between information literacy and assessment in lecture material, associated assessment tasks and the assessment criteria. User feedback is essential (Veldof & Beavers 2001) as a measure of student satisfaction as well as success. Fox, Richter and White (1996) believe that evaluation is best if focused on both formal measurement and informal feedback. From feedback received, we believe that the integration of assessment and information literacy into IACT201 curriculum assessment has been successful.
"[There was] excellent inclusion of relevant...and effective workbook exercises"
"The workbook I think was a good idea. It clearly helps students understand basic writing formats. The work we do on essay and report writing...is priceless"
ReferencesBarclay, D. (1993). Evaluating library instruction: doing the best you can with what you have. RQ, 33(2) pp.195-202. ISSN: 0033-7072 cited in Flaspohler, M.R. (2003). Information literacy program assessment: one small college takes the big plunge. Reference Services Review, 31(2), pp. 129-140. ISSN: 0090-7324. Bruce, C. (1995). Information literacy: A framework for higher education. The Australian Library Journal, 44(3), pp.158-170. ISSN: 0004-9670
Bruce, C. & Candy, P. (1995). Developing Information Literate Graduates: Prompts for good practice. Queensland University of Technology, Brisbane.
Candy, P, Crebert, G & O'Leary, J. (1994). Developing lifelong learners through undergraduate education. National Board of Employment, Education and Training Commissioned Report No.28, Canberra, AGPS.
Centre for the Study of Higher Education, (CSHE). (2002a). The fundamentals of effective assessment: Twelve principles. In Assessing Learning in Australian Universities: Ideas, Strategies and Resources for Quality in Student Assessment.
Centre for the Study of Higher Education, (CSHE). (2002b). Assessing large classes. In Assessing Learning in Australian Universities: Ideas, Strategies and Resources for Quality in Student Assessment.
Centre for the Study of Higher Education, (CSHE). (2002c). Assessing students unfamiliar with assessment practices in Australian higher education. In Assessing Learning in Australian Universities: Ideas, Strategies and Resources for Quality in Student Assessment.
Council of Australian University Librarians (CAUL).(2001). Information literacy standards. Canberra: Council of Australian Information Literacy Standards. Colvin, J & Keene, J. (2004). Supporting undergraduate learning through the collaborative promotion of e-journals by library and academic departments. Information Research, 9(2), paper 173. ISSN: 1368-1613 [Available online at http://InformationR.net/ir/9-2/paper173.html ] Davis, P.M. (2003). Effect of the web on undergraduate citation behaviour: Guiding student scholarship in a networked age. portal: Libraries and the Academy, 3(1) pp. 41-51. ISSN: 1531-2542 Emmons, M & Martin, W. (2002). Engaging conversation: Evaluating the contribution of library instruction to the quality of student research. College and Research Libraries, 63(6) pp.545-560. ISSN: 0010-0870. Fox, L.M, Richter, J.M & White, N.E. (1996). A multidimensional evaluation of a nursing information-literacy program. The Bulletin of the Medical Library Association, 84 (1996), pp.182-190. ISSN: 0025-7338 Flaspohler, M.R. (2003). Information literacy program assessment: one small college takes the big plunge. Reference Services Review, 31(2), pp. 129-140. ISSN: 0090-7324. Hart, G, McCarthy, J & Peacock, J. (2003). Double strength, mazimum gain: Optimising student learning via collaborative partnerships @ QUT. Library Review, 52(9), pp.433-437. ISSN: 0024-2535 Iannuzzi, P. (1998). Faculty development and information literacy: Establishing campus partnerships. Reference Services Review, 26(3/4) pp. 97-116. ISSN: 0090-7324.
James, R. & McInnis, C. (2001). Strategically re-positioning student assessment: A discussion paper on the assessment of student learning in universities, Melbourne: Centre for the Study of Higher Education.
Lipu, S. (2004). A flying start for our future teachers: A comprehensive information literacy program for pre-service education students at the University of Wollongong, Australia. In D.M, Shinew and S. Walter (Eds), Information Literacy Instruction for Educators: Professional Knowledge for an Information Age. ISBN:07890-2073-4.
Lupton, M. (2004). Curriculum alignment and assessment of information literacy learning. In A. Bundy (Eds), Australian and New Zealand Information Literacy Framework: Principles standards and practice, 2nd Edn. Australian and New Zealand Institute for Information Literacy: Adelaide, pp.25-28.
Markless, S, Streatfield, D & Baker, L. (1992). Cultivating information skills in further education: Eleven case-studies. British Library Board, London. Nimon, M. (2001). The role of academic libraries in the development of the information literate student: The interface between librarian, academic and other stakeholders. Australian Academic and Research Libraries, 32(1) pp. 43-52. ISSN: 0004-8623. Nulty, D.D. (2003). Assessing student learning: Particularly information literacy and other generic skills. Invited Symposium address presented at the 2nd ANZIIL Symposium Waikato University, Hamilton, New Zealand, 4-5 December 2003. Thaxton, L, Faccioli, M.B & Mosby, A.P. (2004). Leveraging collaboration for information literacy in psychology. Reference Services Review, 32 (2) pp.185-189. ISSN: 0090-7324. University of Wollongong, (UOW). (2002). Review of Assessment Practices and Processes Report: Report to Academic Senate (RAPP). University of Wollongong: Wollongong.
University of Wollongong, (UOW). (2004). Teaching strategies for graduate attributes.
Veldof J.R & Beavers, K. (2001). Going mental: Tackling mental models for the online library tutorial at the University of Minnesota Libraries. Research Strategies 18(1) pp.3-20. ISSN: 0734-3310 Wallace, M.C & Shorten, A & Crookes, P.A & McGurk, C & Brewer, C. (1999). Integrating information literacies into an undergraduate nursing programme. Nurse Education Today, 19 pp.136-141. ISSN: 0260-6917. Wright, L & McGurk, C. (2000). Integrating information literacy: University of Wollongong experience. In C. Bruce and P. Candy, Information literacy around the world: advances in programs and research, Centre for Information Studies, Charles Sturt University, Wagga Wagga. ISSN: 1443-4334.
Appendix 1
|
