Reflection is a key component of experiential learning at the d.school, and developing reflective skills is an important learning outcome in itself.
In d.school classes and workshops, learners from Stanford and beyond reflect at different moments during an activity—not just at the end—because reflection helps them notice their first-hand experience and contrast it with that of their peers and with theories they might pull from different disciplines. Leticia Britos Cavagnaro has created an AI tool to help people get better at reflection.
This tool is part of a broader question of how technologies can transform the ways we teach and learn—a tool for human-AI collaboration.
As AI tools are increasingly being used as part of coursework—whether by the initiative of students who use tools like ChatGPT, or introduced by teachers—how can we help students develop healthy and productive interactions with AI?
While the use of technology to enhance human capabilities is not new, the broad availability of AI agents that can interact in ways that mimic human-human interactions creates new opportunities but also new risks. Beyond giving students clarity about expectations related to ownership of content and academic integrity, it is important that educators and students openly discuss the potential, limitations, and implications of human-AI collaboration.
As an experimenter with learning futures, I’m always looking for ways to improve the experiences I create for my students, and to share what works with other educators. I designed Riff, a chatbot powered by generative AI to assist students in incorporating reflection in their learning process, and as a tool that educators can use to embed reflection in all kinds of learning experiences.
For many years, I’ve been trying different approaches to help students develop reflection skills and habits within my courses and, more recently, in asynchronous, self-directed ways. For that, I’ve used rule-based conversational agents (aka chatbots) to guide students through design and reflection exercises. (Many of these exercises also appear in my book Experiments in Reflection).
The recent advent and wide accessibility of powerful AI technologies based in Large Language Models (LLM) allowed me to use conversational agents and build on my previous reflection experiments to create a tool that dynamically generates questions starting with the learner’s input.
While chatbots powered by generative AI are typically used to answer questions and perform tasks—you may be familiar with ChatGPT, for instance—Riff turns the tables and focuses on asking learners thought-provoking questions about their experiences. It can be used by students following a class activity—replacing or augmenting a traditional reflection paper or journaling entry—or to create an opportunity for individual thinking before engaging in a group discussion or activity.
Through conversation, Riff might ask the learner to:
- Elaborate and be more specific in describing what they noticed from their experience.
- Contrast the current experience with other past or hypothetical experiences.
- Go beyond the first thing they noticed.
- Think about the future they want to shape and how their present actions and attitudes could change accordingly.
Riff aims to deliver distinct value for students and for teachers:
- By asking a series of questions, Riff helps students practice and improve their ability for introspective questioning to process and make sense of their experiences; and,
- By giving access to the conversations with students (with their knowledge), Riff helps teachers make the thinking of their students visible and gain insights that can be used to improve their teaching practice.
As part of a teacher-in-the-loop approach, Riff’s platform allows teachers to create custom versions of the bot in which the initial prompt is crafted around the specific activity or concept they want students to reflect on. Equally important, teachers can see all the individual Riff-student conversations, and use an AI copilot to analyze all conversations for a given session, which allows them to identify patterns and salient themes.
But before students start using Riff — or any other AI tool — teachers should introduce the tool to students and establish rules of engagement so that they understand why we are inviting them to use the tool, as well as its limitations and the potential risks involved. This is an example of transparent pedagogy.
Here is the gist of what I share with my students when introducing Riff:
Introduction (the why): Reflection helps you learn. For this learning experience you will be using a reflection assistant called Riff.
Riff is a chatbot powered by generative Artificial Intelligence(*) that has been designed to ask questions in response to what you share with it.
Let’s review some rules of engagement with this AI tool towards ensuring that using it is beneficial to you and your learning.
(*) Generative AI chatbots like Riff use Large Language Models (LLMs), which are a type of algorithm that can generate text. LLMs are trained by feeding them vast amounts of data –including books and web pages– so that they learn to identify patterns and establish connections between words. GPT, Claude, and Gemini are some examples of LLMs.
Rule # 1: Understand what this AI tool can and cannot do.
You may be familiar with other generative AI applications like ChatGPT. While ChatGPT is a general purpose tool, Riff has been optimized to ask you reflective questions in response to what you share with it.
Riff is not intended to provide:
Factual information. Generative AI is known to produce inaccurate facts—this is known as ‘hallucinations.’
Advice. Giving advice requires an understanding of the person’s circumstances and goals that is beyond the capabilities of this tool.
If it provides either of those, don’t take them at face value, and consult with your teacher.
Rule # 2: Be mindful of how your data is being used.
What you write will be visible to me [the teacher who invited you to use Riff] (with your name, if Riff asked for it). Think of it as the equivalent of a reflection paper that you submit as coursework. Your conversation with Riff, quotes extracted from it, or a summary, may be shared with your classmates (without your name) to contribute to the collective learning of the class. The content of your conversations will not be used to train the large language models (LLM) that power Riff.
Rule # 3: You are in charge!
If Riff asks you something you don’t want to answer or you feel isn’t relevant to your reflection, simply tell it to move on to something else. Never share something you are not comfortable sharing. If a question sounds repetitive to you, let Riff know so it can take the conversation in a more productive direction.
As an artificial agent, Riff does not get tired and will keep asking you questions—the mission it was designed for– until you want to stop. You can decide when your Riff-assisted reflection has reached the point of diminishing returns and stop. You may also set a timer for yourself (5–10 min is a good time to try), or aim at a number of turns of conversation (8 turns is a good target).
Rule # 4: Give Riff the context and details it needs to be useful to you.
Riff might have some information about what happened during the activity you’ve been asked to reflect on, but will only know about your experience through what you share with it.
Consequently, the more specific and detailed your answers to Riff’s questions are, the better the conversation will flow and it will be more helpful to you (this is analogous to a conversation with another human who does not know you well).
Rule # n: This set of rules is not exhaustive.
The capabilities of genAI are rapidly evolving and, as such, some of the rules above may become obsolete, and new rules may be needed. For instance, it is likely that in a near future Riff will be able to “remember” details about you that you have shared in past conversations, and it may also be able to access and refer to previous activities in the course or even world events. When this happens, you will be made aware of the change.
As you experience what it’s like to interact with Riff, let’s co-create any new rules that would help you feel safe and productive. Reach out to me [your teacher] with any questions, concerns and ideas.
While I have only recently started to conduct formal experiments, my hypothesis—informed by anecdotal information provided by many educators using Riff—is that, all other things being equal (time, context), these personalized and conversational nudges result in a deeper reflection than a response to a single reflection prompt.
My hope is that, by using Riff, learners will be able to reflect deeper when using it and it will develop their ability to reflect on their own, with or without the aid of Riff or other humans.
Riff for Educators
You can access a slide deck with an outline of these rules to incorporate in your materials here (it will prompt you to create a copy so you can adapt as needed).
As made explicit by the last rule (#n) if we invite our students to co-create these rules of engagement, we are not only facilitating their productive use of a specific tool; we are communicating to them something far more important: as educational institutions grapple with the advances in AI and the consequences — intended and unintended — for teaching and learning, we can’t afford not to include the voice and ingenuity of students as the protagonists and co-designers of their education.
If you are an educator working with 13+ year old learners and want to try Riff, please submit a request for early access to Riff’s educator platform here.
Credits
Leticia Britos Cavagnaro