Project publications

The research team has published, or is in the process of publishing, the following peer-reviewed journal articles and refereed conference papers.

1) Published refereed conference paper (December 2017): Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics?

Informed by a literature review and advice from assessment experts, a set of 37 characteristics of effective rubrics were identified and published in the project’s first publication: Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? This paper was presented at the Higher Education Research and Development Society of Australasia (HERDSA) Conference in June 2017 and published in the conference proceedings in December 2017.

The full text of this paper is available here: HERDSA 2017 Conference Proceedings: Williams et al.

Williams, A., Northcote, M., Morton, J. K., & Seddon, J. (2017). Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? In R. G. Walker & S. B. Bedford (Eds.), Research and Development in Higher Education: Curriculum Transformation. Refereed papers from the 40th HERDSA Annual International Conference (Vol. 40, pp. 423-433). Hammondville, NSW, Australia: Higher Education Research and Development Society of Australasia, Inc.

2) Conference paper (presented June 2018) and refereed journal article (published in 2019): ‘Who are you to judge my writing?’: Student collaboration in the co-construction of assessment rubrics

The paper includes a discussion of the results collected from the co-construction research conducted by one of the cohorts that took part in the study. The rubric construction team included first year students and one academic teaching staff member in a creative non-fiction writing subject. They worked collaboratively to construct and then use the assessment rubric. The co-construction processes reported in this paper, undertaken during the same semester, enabled students and their lecturer to simultaneously learn about the role of a rubric. This presentation was an ideal opportunity to share practical reflections from this project with other academics working in this discipline. Specifically, through working with the students, practical implications emerged about how discussions of subjectivity around the assessment of writing can be counterbalanced by concrete language and aesthetic requirements through the use of co-constructed rubric tools.

This paper was presented in June 2018 at the Great Writing International Creative Writing (IGW) 2018 Conference.

The full version of this paper is available as a journal article in the journal, New Writing: The International Journal for the Practice and Theory of Creative Writing.

Joseph, S., Rickett, C., Northcote, M., & Christian, B. (2019). ‘Who are you to judge my writing?’: Student collaboration in the co-construction of assessment rubrics. New Writing: The International Journal for the Practice and Theory of Creative Writing 17(1), 1-19. doi:10.1080/14790726.2019.1566368

3) Refereed journal article: A plan for the co-construction and collaborative use of rubrics for student learning

The paper outlines a set of processes adopted to co-construct rubrics within six different contexts through collaboration with students and lecturers. The paper reports on how a mixed methods approach was utilised to gather data from both the lecturer-participants and student-participants in the study, using questionnaires, interviews and focus groups. Findings were presented in terms of student and lecturer views of rubrics and rubric co-construction, and the paper concludes with recommendations for practical approaches to rubric co-construction and future research directions. Student participants largely felt that engaging in the co-construction process broadened their understanding of not just rubrics themselves, but also the whole assessment process. This paper reported on student views about how rubric co-construction made the assessment process fairer, simpler, and less subjective. While academic staff reported feeling optimistic about the benefits of co-construction, they were less optimistic about the administrative and organisational problems they foresaw. These problems included time constraints for developing subject material, ensuring that all involved students had a good understanding of the process and what was required of them, managing unrealistic student expectations of the process and its outcomes, and the possibility of conflicting opinions between students and lecturers. Academic staff participants saw three main benefits to students in the co-construction process:

1) an improvement in student learning and understanding of assessment tasks and rubric use;

2) an increase in ownership of the learning process by students; and

3) the co-construction process had allowed students to gain insight into the role of the lecturer, to understand the complexity of designing assessments, and to appreciate the time involved in creating a meaningful learning experience.

This journal article was published in 2020 in the Assessment and Evaluation in Higher Education journal:

Kilgour, P., Northcote, M., Williams, A., & Kilgour, A. (2020). A plan for the co-construction and collaborative use of rubrics for student learning. Assessment and Evaluation in Higher Education, 45(1), 140-153. doi:10.1080/02602938.2019.1614523

4) Conference paper (presented November 2018) and refereed journal article (published in 2019): Peripheries and praxis: The effect of rubric co-construction on student perceptions of their learning

Responding to a gap in rubric construction practice, this paper discusses the research project where students moved from rubric user to the centre of collaborative design, drawing on data collected from a team of rubric co-constructors from one Sydney university campus – first year students and an academic in a creative non-fiction writing subject.

This conference paper was presented at the Australasian Association of Writing Programs Conference in Perth in November 2018.

The full version of this paper is available as a journal article in the journal, TEXT - Journal of Writing and Writing Courses:

Rickett, C., Joseph, S., Northcote, M., Christian, B. J., & Seddon, J. (2019). Peripheries and praxis: The effect of rubric co-construction on student perceptions of their learning. TEXT - Journal of Writing and Writing Courses - Special Issue, 57, 1-16. doi:10.52086/001c.23589

5) Refereed journal article: Sharing the construction of assessment rubrics with students: A model for collaborative rubric construction

This article was published in 2021 in the journal, Journal of University Teaching & Learning Practice:

Morton, J., Northcote, M., Kilgour, P., & Jackson, W. (2021). Sharing the construction of assessment rubrics with students: A model for collaborative rubric construction. Journal of University Teaching & Learning Practice, 18(4), 9-13. Retrieved from https://ro.uow.edu.au/jutlp/vol18/iss4/9

6) Peer-reviewed journal article (due to be submitted in 2021): Teachers’ and students’ perceptions of student learning as a result of rubric co-construction

This article will report on how students engaged with their lecturers as a result of the rubric design processes used in the project. The article will specifically provide an answer to the third research question of the project: What effect does the co-construction and use of rubrics have on students’ and lecturers’ perceptions of student learning? The authors of this article are Wendy Jackson, Bev Christian and Andrew Kilgour. The article is due for submission to the journal, International Journal for Students as Partners, in 2021.

7) Peer-reviewed journal article (due to be submitted in 2021): Rubric co-construction in medical and allied health education: Lecturers’ and students’ perceptions

This article will report on the concept of rubric co-construction from the perspectives of both educators and students across two different higher education institutions, and from several different health disciplines, including both undergraduate and postgraduate cohorts. The authors of this article are Andrew Kilgour, Jason Morton, Linda Cloete, Sonja Dawson and Maria Northcote. The article is due for submission to the journal, Advances in Health Sciences Education, in 2021.

8) Final project report

The final project report was published in 2019

Northcote, M., Williams, A., Christian, B., Jackson, W., Joseph, S., Kilgour, A., . . . Seddon, J. (2019). Owning the Rubric: Student engagement in rubric design and use: Project ID16-5374 (Final Report). Retrieved from Canberra, Australia: https://ltr.edu.au/resources/ID16-5374_NorthcoteandWilliams_FinalReport_2019.pdf

Key references

The following references are related to assessment, rubrics, rubric-creation, assessment design, feedback, assessment criteria, grading and marking. Most of these references have been drawn from higher education contexts and represent multiple disciplinary contexts.

Allen, D., & Tanner, K. (2006). Rubrics: tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE-Life Sciences Education, 5(3), 197-203.

Andrade, H. L. (2005). Teaching with rubrics: The good, the bad, and the ugly. College Teaching, 53(1), 27-31. doi: 10.3200/CTCH.53.1.27-31.

Andrade, H. L., Andrade, H. G., & Wang, X. (2008). Putting rubrics to the test: The effect of a model, criteria generation, and rubric-referenced self-assessment on elementary school students' writing. Educational Measurement: Issues and Practice, 27(2), 3-13.

Andrade, H. L., & Ying, D. (2005). Student perspectives on rubric-referenced assessment. Practical Assessment. Research and Evaluation, 10(3), 1-11.

Becker, A. (2016). Student-generated scoring rubrics: Examining their formative value for improving ESL students’ writing performance. Assessing Writing, 29, 15-24.

Bevan, R., Badge, J., Cann, A., Willmott, C., & Scott, J. (2008). Seeing eye-to-eye? Staff and student views on feedback. Bioscience Education, 12(1), 1-15.

Bharuthram, S. (2015). Lecturers’ perceptions: The value of assessment rubrics for informing teaching practice and curriculum review and development. Africa Education Review, 12(3), 415-428. doi: 10.1080/18146627.2015.1110907

Boud, D., & Associates. (2010). Assessment 2020: Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council.

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698-712.

Brown, S. (2004-2005). Assessment for learning. Learning and Teaching in Higher Education, 1(1), 81-89.

Burke, D. (2009). Strategies for using feedback students bring to higher education. Assessment & Evaluation in Higher Education, 34(1), 41-50.

Crampton, A., & Ragusa, A. T. (2015). Exploring the role of technology in fostering sense of belonging in students studying by distance. Sydney: Office for Learning and Teaching, Department of Education and Training.

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

Dawson, P. (2015). Assessment rubrics: Towards clearer and more replicable design, research and practice. Assessment & Evaluation In Higher Education, 1-14. doi: 10.1080/02602938.2015.1111294

Hafner, J., & Hafner, P. (2003). Quantitative analysis of the rubric as an assessment tool: an empirical study of student peer-group rating. International Journal of Science Education, 25(12), 1509-1528. doi: 10.1080/0950069022000038268

Kemmis, S., & McTaggart, R. (2005). Participatory action research: Communicative action and the public sphere. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage book of qualitative research (3rd ed., pp. 559-604). Thousand Oaks, CA: Sage.

Hager, P. (2004). The competence affair, or why vocational education and training urgently needs a new understanding of learning. Journal of Vocational Education & Training, 56(3), 409-433. doi:10.1080/13636820400200262

James, R., Krause, K., & Jennings, C. (2010). The first-year experience in Australian universities: Findings from 1994 to 2009. Melbourne: Centre for the Study of Higher Education, The University of Melbourne.

Jones, L., Allen, B., Dunn, P., & Brooker, L. (2016). Demystifying the rubric: a five-step pedagogy to improve student understanding and utilisation of marking criteria. Higher Education Research & Development, 1-14. doi: 10.1080/07294360.2016.1177000

Jönsson, A. (2013). Facilitating productive use of feedback in higher education. Active Learning in Higher Education, 14(1), 63-76.

Jönsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment & Evaluation in Higher Education, 39(7), 840-852.

Jönsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144.

Keeney, S., Hasson, F., & McKenna, H. (2006). Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of advanced nursing, 53(2), 205-212.

Li, J., & Lindsey, P. (2015). Understanding variations between student and teacher application of rubrics. Assessing Writing, 26, 67-79.

Panadero, E., & Jönsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129-144.

Panadero, E., & Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2), 133-148. doi: 10.1080/0969594x.2013.877872

Powell, C. (2003). The Delphi technique: myths and realities. Journal of Advanced Nursing, 41(4), 376-382.

Macintyre, R., & Macdonald, J. R. (2011). 'Remote from what?' Perspectives of distance learning students in remote rural areas of Scotland. The International Review of Research in Open and Distance Learning, 12(4), 1-16.

Maor, D., & Taylor, P. C. (2000, 2-4 February). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. Paper presented at the Flexible futures in tertiary teaching: Proceedings of the 9th Annual Teaching Learning Forum, Curtin University of Technology.

Orrell, J., Cooper, L., & Bowden, M. (2010). Work integrated learning: A guide to effective practice: Routledge.

Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). Thousand Oaks, California: SAGE Publications, Inc.

Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41,376-382.

Reddy, Y. M., & Andrade, H. L. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435-448. doi: 10.1080/02 602930902862859

Robson, C., & McCartan, K. (2016). Real world research (4th ed.). Chichester, West Sussex, United Kingdom: John Wiley & Sons.

Sadler, D. R. (2007). Perils in the Meticulous Specification of Goals and Assessment Criteria. Assessment in Education: Principles, Policy and Practice, 14(3), 387-392.

Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading. Assessment & Evaluation in Higher Education, 34(2), 159-179.

Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education, 6, 1.

Stevens, D. D., & Levi, A. J. (2011). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Virginia: Stylus Publishing.

Strayhorn, T. L. (2012). College students' sense of belonging: A key to educational success for all students. New York: Routledge.

Sundeen, T. H. (2014). Instructional rubrics: Effects of presentation options on writing quality. Assessing Writing, 21, 74-88.

Taylor, K., & Marienau, C. (1997). Constructive‐development Theory as a Framework for Assessment in Higher Education. Assessment & Evaluation in Higher Education, 22(2), 233-243. doi:10.1080/0260293970220211

Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3-14.

Williams, A., Northcote, M., Morton, J. K., & Seddon, J. (2017). Towards engaging students in curriculum transformation: What are the effective characteristics of rubrics? In R. G. Walker & S. B. Bedford (Eds.), Research and Development in Higher Education: Curriculum Transformation. Refereed papers from the 40th HERDSA Annual International Conference (Vol. 40, pp. 423-433). Hammondville, NSW, Australia: Higher Education Research and Development Society of Australasia, Inc.

Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student learning. The Journal of Effective Teaching, 7(1), 3-14.

Yucel, R., Bird, F. L., Young, J., & Blanksby, T. (2014). The road to self-assessment: exemplar marking before peer review develops first-year students’ capacity to judge the quality of a scientific report. Assessment & Evaluation in Higher Education, 39(8), 971-986.