Menu
Log in

 

Login

Log in


Evaluating an Interface: Metacognitive Approaches to AI and Tutoring

Jessica Craig, Utah Valley University

abstract

Given the growing advertisements of AI as a viable replacement for human tutors, demonstrating the differences between AI and human tutoring emphasizes the importance of humanity in tutorials and shows tutors productive methods of engaging with AI. This discussion-based lesson experiments with using AI as a tutor throughout the writing process. Using tutorial scenarios and follow-up prompts, trainers engage with a generative AI to test its limitations and capabilities as a tutor. After the experiments, tutors and trainers critically respond to the AI program’s outputs by comparing it to their responses and experiences as tutors. Aiming to help tutors feel comfortable navigating AI during tutorials, the critical thinking exercises in this lesson can also supplement tutors’ skills in textual evaluation. By emphasizing the importance of redirection and human connection during tutorials, this lesson helps tutors to appropriately intervene when using AI, which can be passed on through tutor-writer conversations.

KEYWORDS

Writing center theory/philosophy

In-person tutoring

Synchronous online tutoring

Written feedback/response

Instructional feedback/approaches

Tutor ethos/persona

Writing process

Research

LESSON context

As generative AI becomes part of the everyday in work and educational spaces, writers are asked more frequently to use it productively and efficiently, despite having little to no instruction on using AI. One solution for the lack of education in writing with AI is to practice metacognition, which may aid writers in effectively collaborating with and using generative AI (Sidra & Mason, 2024). However, in a landmark paper that questions the risk of using it without critical awareness, Bender et al. (2021) warn of interpreting meaning in generative AI outputs when “there is none” (p. 611). By emphasizing the importance of AI as facilitating rather than replacing learning (Atchley et al., 2024), tutors can “[promote] metacognition and rhetorical awareness” when working with AI alongside writers (McGovern, 2022).

Previous research suggests that metacognition and critical thinking are more effective in experiential learning environments (Duchatelet et al., 2023; Richmond et al., 2017), which has motivated the experimental and participatory form of this lesson. The primary purpose of this lesson is to instruct tutors on evaluating generative AI so they can later guide writers in utilizing it in a productive learning environment. Other learning outcomes include the wider application of metacognition to other written works and identifying the limitations and strengths of using generative AI.

Atchley, P., Pannell, H., Wofford, K., Hopkins, M., & Atchley, R. A. (2024). Human and AI collaboration in the higher education environment: Opportunities and concerns. Cognitive Research Principles and Implications, 9(1). http://dx.doi.org/10.1186/s41235-024-00547-9

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big?🦜. FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–623. https://doi.org/10.1145/3442188.3445922

Duchatelet, D., Cornelissen, F., & Volman, M. (2023). Features of experiential learning environments in relation to generic learning outcomes in higher education: A scoping review. Journal of Experiential Education, 47(3), 400–423. https://doi.org/10.1177/10538259231211537

McGovern, B. T. (2022). The writing center’s role in disciplinary writing development: Enhancing discourse community knowledge through metacognitive dialogue. Praxis: A Writing Center Journal, 19(2). https://www.praxisuwc.com/192-mcgovern

Richmond, A. S., Bacca, A. M., Becknell, J. S., & Coyle, R. P. (2017). Teaching metacognition experientially: A focus on higher versus lower level learning. Teaching of Psychology, 44(4), 298–305. https://doi.org/10.1177/0098628317727633

Sidra, S., & Mason, C. (2024). Reconceptualizing AI literacy: The importance of metacognitive thinking in an artificial intelligence (AI)-enabled workforce. 2024 IEEE Conference on Artificial Intelligence (CAI), 1181–1186. https://ieeexplore.ieee.org/document/10605569

FORMAT TYPE

In-person
Online synchronous
Hybrid
Case study or scenario
Demo or observation
Experiment
Discussion
Reflection

TIMING & OCCURRENCE

Lesson Time: 60 minutes or less

Prep Time: 30 minutes or less

Occurrence: ongoing or one-time training

Timeline: midsemester

Certification Alignment: This training is intended to be adaptable, which allows it to align with some CRLA requirements, depending on what the trainer chooses to focus on. The base training fits especially well with the “Role of the Tutor” topic but could easily be linked to others, such as “Critical Thinking Skills” and “Active Learning Strategies.”

TUTOR AUDIENCE 

Experienced tutors

Tutor administrators

High school student tutors

Undergraduate student tutors

Graduate student tutors

Faculty/professional tutors

In-person tutors

Synchronous online tutors

Asynchronous online tutors

MATERIALS NEEDED



CLICK HERE FOR COMPLETE LESSON PLAN MATERIALS 



AUTHOR INFORMATION

Jessica Craig, Utah Valley University
https://orcid.org/0009-0006-7980-2269

Jessica Craig is a recent graduate with a bachelor’s in English from Utah Valley University in Orem, UT. She has worked as a writing tutor and served as editor-in-chief for an undergraduate journal, and her research interests include digital media, narrative, and nineteenth-century literature. She has previously presented at the Interdisciplinary Nineteenth-Century Studies conference and Utah College Learning Center Association conference about contemporary technology and education. Co-authored with Dr. Ashley Nadeau, her forthcoming article in Nineteenth-Century Contexts explores this intersection through audiobook use in the undergraduate classroom. Her future research will focus on interactive, collaborative narratives, and AI.


Follow our activities

© Wild Apricot teachers association. 

Powered by Wild Apricot Membership Software