References and publications
Project Information | Logistics | Effective Rubric Characteristics | Co-construction Process | Using Co-constructed Rubrics | Rubrics Examples | Research Methods | Recommendations for Practice | References & Publications | Model of Collaborative Rubric Construction | Workshop

Project publications

The research team has published, or is in the process of publishing, a total of six refereed outputs from the project, including four peer-reviewed journal articles and two refereed conference papers.

1) Published refereed conference paper (December 2017): Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics?

Informed by a literature review and advice from assessment experts, a set of 37 characteristics of effective rubrics were identified and published in the project’s first publication: Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? This paper was presented at the Higher Education Research and Development Society of Australasia (HERDSA) Conference in June 2017 and published in the conference proceedings in December 2017.

The full text of this paper is available here: HERDSA 2017 Conference Proceedings: Williams et al.

Williams, A., Northcote, M., Morton, J. K., & Seddon, J. (2017). Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? In R. G. Walker & S. B. Bedford (Eds.), Research and Development in Higher Education: Curriculum Transformation. Refereed papers from the 40th HERDSA Annual International Conference (Vol. 40, pp. 423-433). Hammondville, NSW, Australia: Higher Education Research and Development Society of Australasia, Inc.

2) Conference paper (presented June 2018) and refereed journal article (submitted June 2018): Student collaboration in the co-construction of assessment rubrics

The paper includes a discussion of the results collected from the co-construction research conducted by one of the cohorts that took part in the study. The rubric construction team included first year students and one academic teaching staff member in a creative non-fiction writing subject. They worked collaboratively to construct and then use the assessment rubric. The co-construction processes reported in this paper, undertaken during the same semester, enabled students and their lecturer to simultaneously learn about the role of a rubric. This presentation was an ideal opportunity to share practical reflections from this project with other academics working in this discipline. Specifically, through working with the students, practical implications emerged about how discussions of subjectivity around the assessment of writing can be counterbalanced by concrete language and aesthetic requirements through the use of co-constructed rubric tools.

This paper was presented in June 2018 at the Great Writing International Creative Writing (IGW) 2018 Conference. The full paper is currently under review by editors of the journal, New Writing: The International Journal for the Practice and Theory of Creative Writing.

Joseph, S., Rickett, C., Northcote, M., & Christian, B. (2018). Student collaboration in the co-construction of assessment rubrics. Paper presented at the Great Writing International Creative Writing (IGW) Conference, Imperial College, London.

3) Refereed conference paper (submitted September 2018): A plan for the co-construction and collaborative use of rubrics for student learning

The paper outlines a set of processes adopted to co-construct rubrics within six different contexts through collaboration with students and lecturers. The paper reports on how a mixed methods approach was utilised to gather data from both the lecturer-participants and student-participants in the study, using questionnaires, interviews and focus groups. Findings were presented in terms of student and lecturer views of rubrics and rubric co-construction, and the paper concludes with recommendations for practical approaches to rubric co-construction and future research directions. Student participants largely felt that engaging in the co-construction process broadened their understanding of not just rubrics themselves, but also the whole assessment process. This paper reported on student views about how rubric co-construction made the assessment process fairer, simpler, and less subjective. While academic staff reported feeling optimistic about the benefits of co-construction, they were less optimistic about the administrative and organisational problems they foresaw. These problems included time constraints for developing subject material, ensuring that all involved students had a good understanding of the process and what was required of them, managing unrealistic student expectations of the process and its outcomes, and the possibility of conflicting opinions between students and lecturers. Academic staff participants saw three main benefits to students in the co-construction process:

1) an improvement in student learning and understanding of assessment tasks and rubric use;

2) an increase in ownership of the learning process by students; and

3) the co-construction process had allowed students to gain insight into the role of the lecturer, to understand the complexity of designing assessments, and to appreciate the time involved in creating a meaningful learning experience.

This journal article was submitted in September 2018 to the Assessment and Evaluation in Higher Education journal and is currently under review.

Kilgour, P., Williams, A., Kilgour, A., & Northcote, M. (2018). A plan for the co-construction and collaborative use of rubrics for student learning. Assessment and Evaluation in Higher Education.

4) Conference paper (due to be presented November 2018) and refereed journal article (to be submitted in December 2018): Peripheries and praxis: the effect of rubric co-construction on student perceptions of their learning

Responding to a gap in rubric construction practice, this paper discusses the research project where students moved from rubric user to the centre of collaborative design, drawing on data collected from a team of rubric co-constructors from one Sydney university campus – first year students and an academic in a creative non-fiction writing subject.

This conference paper has been submitted, accepted and due for presentation at the Australasian Association of Writing Programs Conference in Perth in November 2018; the full paper will be submitted for publication as a refereed journal article to TEXT: Journal of Writing and Writing Courses.

Rickett, C., Joseph, S., Northcote, M., Christian, B., & Seddon, J. (2018). Peripheries and praxis: the effect of rubric co-construction on student perceptions of their learning. Paper to be presented at the Australasian Association of Writing Programs Conference, Perth, Western Australia.

5) Published refereed conference paper (due to be submitted February 2019): Model of Collaborative Rubric Construction and Use

This paper is currently being prepared by the project team. The focus of this paper will be the practical recommendations that emerged from the project which can be applied across multi-disciplinary contexts. The paper will specifically provide an answer to the fourth research question of the project: What practical, research-informed recommendations can be used to engage students and academic staff in the collaborative process of designing and using assessment rubrics to promote learning?

This refereed conference paper is due to be submitted in February 2019 and presented in July 2019 at the HERDSA 2019 conference, Next Generation Higher Education: Challenges, Changes and Opportunities, in Auckland, New Zealand. This paper will report on a summary of the project. The authors will be Maria Northcote, Peter Kilgour, Carolyn Rickett, Wendy Jackson and John Seddon.

6) Peer-reviewed journal article (due to be submitted in February 2019): Teachers’ and students’ perceptions of student learning as a result of rubric co-construction

This article will report on how students engaged with their lecturers as a result of the rubric design processes used in the project. The article will specifically provide an answer to the third research question of the project: What effect does the co-construction and use of rubrics have on students’ and lecturers’ perceptions of student learning? The authors of this article will be Wendy Jackson, Tony Williams, Bev Christian and Andrew Kilgour. The article is due for submission to the journal, International Journal for Students as Partners, in February 2019.

Key references

The following references are related to assessment, rubrics, rubric-creation, assessment design, feedback, assessment criteria, grading and marking. Most of these references have been drawn from higher education contexts and represent multiple disciplinary contexts.

 

Allen, D., & Tanner, K. (2006). Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE-Life Sciences Education, 5(3), 197-203.

Andrade, H. L. (2005). Teaching with rubrics: The good, the bad, and the ugly. College Teaching, 53(1), 27-31. doi: 10.3200/CTCH.53.1.27-31.

Andrade, H. L., Andrade, H. G., & Wang, X. (2008). Putting rubrics to the test: The effect of a model, criteria generation, and rubric-referenced self-assessment on elementary school students' writing. Educational Measurement: Issues and Practice, 27(2), 3-13.

Andrade, H. L., & Ying, D. (2005). Student perspectives on rubric-referenced assessment. Practical Assessment. Research and Evaluation, 10(3), 1-11.

Becker, A. (2016). Student-generated scoring rubrics: Examining their formative value for improving ESL students’ writing performance. Assessing Writing, 29, 15-24.

Bevan, R., Badge, J., Cann, A., Willmott, C., & Scott, J. (2008). Seeing eye-to-eye? Staff and student views on feedback. Bioscience Education, 12(1), 1-15.

Bharuthram, S. (2015). Lecturers’ perceptions: The value of assessment rubrics for informing teaching practice and curriculum review and development. Africa Education Review, 12(3), 415-428. doi: 10.1080/18146627.2015.1110907

Boud, D., & Associates. (2010). Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council.

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-712.

Brown, S. (2004-2005). Assessment for learning. Learning and Teaching in Higher Education, 1(1), 81-89.

Burke, D. (2009). Strategies for using feedback students bring to higher education. Assessment & Evaluation in Higher Education, 34(1), 41-50.

Crampton, A., & Ragusa, A. T. (2015). Exploring the role of technology in fostering sense of belonging in students studying by distance. Sydney: Office for Learning and Teaching, Department of Education and Training.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Dawson, P. (2015). Assessment rubrics: Towards clearer and more replicable design, research and practice. Assessment & Evaluation In Higher Education, 1-14. doi: 10.1080/02602938.2015.1111294

Hafner, J., & Hafner, P. (2003). Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer-group rating. International Journal of Science Education, 25(12), 1509-1528. doi: 10.1080/0950069022000038268

Kemmis, S., & McTaggart, R. (2005). Participatory action research: Communicative action and the public sphere. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage book of qualitative research (3rd ed., pp. 559-604). Thousand Oaks, CA: Sage.

Hager, P. (2004). The competence affair, or why vocational education and training urgently needs a new understanding of learning. Journal of Vocational Education & Training, 56(3), 409-433. doi:10.1080/13636820400200262

James, R., Krause, K., & Jennings, C. (2010). The first-year experience in Australian universities: Findings from 1994 to 2009. Melbourne: Centre for the Study of Higher Education, The University of Melbourne.

Jones, L., Allen, B., Dunn, P., & Brooker, L. (2016). Demystifying the rubric: a five-step pedagogy to improve student understanding and utilisation of marking criteria. Higher Education Research & Development, 1-14. doi: 10.1080/07294360.2016.1177000

Jönsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63-76.

Jönsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment & Evaluation in Higher Education, 39(7), 840-852.

Jönsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144.

Keeney, S., Hasson, F., & McKenna, H. (2006). Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of advanced nursing, 53(2), 205-212.

Li, J., & Lindsey, P. (2015). Understanding variations between student and teacher application of rubrics. Assessing Writing, 26, 67-79.

Panadero, E., & Jönsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129-144.

Panadero, E., & Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2), 133-148. doi: 10.1080/0969594x.2013.877872

Powell, C. (2003). The Delphi technique: myths and realities. Journal of Advanced Nursing, 41(4), 376-382.

Macintyre, R., & Macdonald, J. R. (2011). 'Remote from what?' Perspectives of distance learning students in remote rural areas of Scotland. The International Review of Research in Open and Distance Learning, 12(4), 1-16.

Maor, D., & Taylor, P. C. (2000, 2-4 February). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. Paper presented at the Flexible futures in tertiary teaching: Proceedings of the 9th Annual Teaching Learning Forum, Curtin University of Technology.

Orrell, J., Cooper, L., & Bowden, M. (2010). Work integrated learning: A guide to effective practice: Routledge.

Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). Thousand Oaks, California: SAGE Publications, Inc.

Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41,376-382.

Reddy, Y. M., & Andrade, H. L. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435-448. doi: 10.1080/02 602930902862859

Robson, C., & McCartan, K. (2016). Real world research (4th ed.). Chichester, West Sussex, United Kingdom: John Wiley & Sons.

Sadler, D. R. (2007). Perils in the Meticulous Specification of Goals and Assessment Criteria. Assessment in Education: Principles, Policy and Practice, 14(3), 387-392.

Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159-179.

Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1.

Stevens, D. D., & Levi, A. J. (2011). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Virginia: Stylus Publishing.

Strayhorn, T. L. (2012). College students' sense of belonging: A key to educational success for all students. New York: Routledge.

Sundeen, T. H. (2014). Instructional rubrics: Effects of presentation options on writing quality. Assessing Writing, 21, 74-88.

Taylor, K., & Marienau, C. (1997). Constructive‐development Theory as a Framework for Assessment in Higher Education. Assessment & Evaluation in Higher Education, 22(2), 233-243. doi:10.1080/0260293970220211

Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14.

Williams, A., Northcote, M., Morton, J. K., & Seddon, J. (2017). Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? In R. G. Walker & S. B. Bedford (Eds.), Research and Development in Higher Education: Curriculum Transformation. Refereed papers from the 40th HERDSA Annual International Conference (Vol. 40, pp. 423-433). Hammondville, NSW, Australia: Higher Education Research and Development Society of Australasia, Inc.

Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student learning. The Journal of Effective Teaching, 7(1), 3-14.

Yucel, R., Bird, F. L., Young, J., & Blanksby, T. (2014). The road to self-assessment: exemplar marking before peer review develops first-year students’ capacity to judge the quality of a scientific report. Assessment & Evaluation in Higher Education, 39(8), 971-986.



NEXT


Support for the production of this website has been provided by the Australian Government Department of Education and Training. The views expressed in this report do not necessarily reflect the views of the Australian Government Department of Education and Training.  
 Creative Commons licence
Unless otherwise noted, content on this site is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License

© 2018 Avondale College Ltd trading as Avondale College of Higher Education
CRICOS Provider No. 02731D ABN: 53 108 186 401  Terms of Use | Sitemap