Reflecting on Reflection

This article presents three novel ways of enhancing student reflection on assignments for better concept mapping and improved learning outcomes in the economics classroom. All three approaches have one key feature in common: they ask the student to reflect upon why the instructor assigned the task. Their responses show that doing so helps students (i) connect deeper with the assigned task, (ii) connect key lessons from the task with previous and ongoing course topics to obtain a deeper conceptual understanding of how the topics are interrelated, and (iii) uncover self-guided discoveries on effective studying strategies facilitating enhanced retention and performance.

I would prepare differently by actually taking notes in class.I pay attention well in class but take very little notes, and since we have gotten our test grades back I have started taking detailed notes and have come to the realization that this is very helpful (surprise!).
-Excerpt from Student Response on "Exam Reflection" Activity (Activity 3, see below)

Introduction
As instructors of economics, we strive to promote a desire in our students to learn and succeed.While it is apparent that our students want to learn, it is also evident that they do not always have a realistic understanding of what it takes to learn, as the quote above demonstrates.Even though many instructors may have a set of general learning-related lessons intended to facilitate better course learning outcomes-remedial math and graphing lessons or lessons on good-writing practices-such one-size-solutions rarely fit all.Many, if not all, of us walk into our classes realizing some students will benefit from reminders about areas of triangles before the lesson on consumer surplus, for instance.But how many of us realize some students would benefit more from reminders about taking notes in the first place?Students, on the other hand, learn best when they are the agents of their own learning.Facilitating such agency requires finding innovative and effective ways to engage and motivate students, and to make economics understandable, relevant, and enjoyable.One approach that can help to achieve these objectives is the use of reflective writing assignments.Recent scholarship on economics pedagogy notes, among other benefits, that reflective writing helps students assess their understanding and shortcomings, promotes subsequent learning, and fosters professional growth (ElGhouty, 2020;Euler & Kühner, 2017: Olmsted & Ruediger, 2013).This paper adds to this emerging literature by offering three easily adopted specific assignments to facilitate student reflection on assignments to improve learning outcomes.At the core of all three assignments is an intentional design to help students discover the unique ways that works for them.
Several studies have examined and validated the effectiveness of reflective assignments in economics pedagogy.ElGhouty (2020) describes how his teaching based on two learning theories affected his own learning and that of his students.He also reflects on his teaching practices and assessment techniques and concludes with some recommendations.He suggests assigning additional questions that require students to provide their own solutions, using more open-ended questions, having students work on assignments in groups, preparing lesson plans to facilitate lecture development, and enhancing instructor competence through reflecting on student performances on online quizzes and discussions.Euler and Kühner (2017) find that problem-based assignments elicit learning activities that develop ethical and reflective capabilities in students in economics courses but only to a limited extent.It is not apparent why, although the results of their study provide some guidance on adjusting assignments.Kassens (2014) describes how Twitter was employed to complement traditional lectures in a small macroeconomics principles course and concludes that tweets help students to express their thoughts concisely and develop reflection and writing skills.Olmsted and Ruediger (2013) examine how reflective writing assignments in economics classes improve exam performances of students.Brewer and Jozefowicz (2006) address two informal writing assignments implemented in introductory economics classes-short reflection papers and short journal entries-designed to help students appreciate the relevance of economics in their daily lives.They discuss pedagogical benefits to students, and benefits and costs to faculty.reflection on assignments.Murillo-Llorente et al. (2021) examine the effects of the reflective practice journal among nursing students and find that it facilitates learning, self-criticism, acquisition of values, and responsibility in the distribution of available resources.Szenes and Tilakaratna (2020) demonstrate how successful students in social work and business engage with subjective meanings in reflective writing, and how these students progress from a basic understanding of problematic situations to demonstrating disciplinary knowledge and values in their fields of study.Boutet et al. (2017) describe the implementation of a reflection exercise in an undergraduate psychology course and conclude that reflection writing promotes the development of life-long learning skills among students.Tsingos-Lucas, et al. (2017) find that possessing good reflective-writing skills is associated with improved academic performance in undergraduate pharmacy students.Lauer (2013) suggests that students who engage in reflective assessment produce stronger visual designs.Koliba (2004) uses an assessment tool to evaluate reflection assignments in the context of teaching public affairs courses, draws references to experiential learning theories, and explores implications for educating reflective practitioners.
In summary, pedagogical literature in economics and other disciplines underscores various benefits of reflective assignments.The activities we propose deliver benefits of reflection, but, in contrast to most grading-intensive writing-based assignments, also have the attractive feature of imposing minimal costs in terms of instructor time.

The Activities
This section describes the three activities assigned.After introducing each activity, we comment on its various key components.

A. Activity 1: Demand-Supply in Action
In this activity, students listen to two podcast episodes and respond to the subsequent prompts.In the first podcast, students learn about unique strategies adopted by the Trader Joe's grocery chain.In the second, they learn why taxi medallions (licenses) in New York City were, for the longest time, one of the most lucrative investments, and why they are not anymore.
Both parts of this activity involve students thinking through marketplace interactions between buyers, sellers, and especially how presence and absence of rival sellers lead to different market outcomes.In introductory courses, this activity is best suited as a bridge between the phase of introducing the demand-supply-equilibrium model and the phase of introducing advanced applications demonstrating impacts of simultaneous changes in demand and supply.While the class discussion about theories intended to capture buyer and seller behavior at first may seem abstract, asking students to apply these ideas to understand real-world situations is an effective way to help them take these concepts beyond the classroom.However, instead of informing the students about the objectives of the activity during class lectures, the final question of the assignment asks them to reflect on the assignment, to help them draw these conclusions.Instead of telling students why learning the demand-supply model is both important and useful, the last question inspires them to discover these insights on their own through reflection. 1 For instance, in an upper-division course on Industrial Organization, we used this assignment as the first prelecture activity of the semester.The follow-up lecture first derived and discussed implications of the optimal pricing rule that connects markups to the elasticity of demand.The second half of the lecture tied Trader Joe's strategies discussed in the first podcast to determinants of elasticity of demand and demonstrated how each strategy was a way to make demand more inelastic.This, in turn, would increase the markup, as the optimal pricing rule shows.The lecture then addressed the case of taxi medallions discussed in the second podcast to motivate a brief discussion on entry barriers, pricing, and competition, and concluded by summarizing the associated fundamental questions that the course will address.
This activity assigns a recent article, Chakraborti and Roberts (2020), as reading and asks students to respond to five follow-up questions.The activity makes students confront a puzzle that often comes up following discussions on price controls: if price controls have such terribly inefficient impacts, why are they still prevalent?
After having worked through traditional in-class examples in rental markets, labor markets, or simplified abstractions, students are well primed for modern-day examples.Furthermore, if the course has an empirical component that introduces the scientific method of inquiry, this task is an excellent way to connect various threads developed in the course.
To help students prepare to read, the preamble to the assignment offers three suggestions.First, students are asked to note how the scientific approach-the development and testing of a hypothesis-looms in the background of the paper.Second, the preamble reminds students that the paper's methodology applies certain statistical procedures to identify causality rather than correlation.To help students navigate the methodological technicalities, the preamble tells them not to get bogged down by the unnecessary details but to stay anchored to the idea of identifying causality.Finally, the preamble asks students to note how the hypothesis is entirely developed from basic principles of economics.There are no elaborate mathematical abstractions, just the familiar basics from the demand-supply model.
The first four questions ensure that students do indeed read and reflect upon the paper in some depth.The first two questions ask about the research question and its significance.The third question asks about methodology and the rationale for a particular measure.Just by reading the paper, students can find answers to these three questions.The fourth question, however, asks them to confront the motivating puzzle one more time, now that they have a deeper understanding of the issues involved.
Though the purpose of each element of this design is clear from the instructor's perspective, it may not necessarily be so for the students.To help them connect all the themes running through within the assignment, as well as to help them connect threads across various concepts and discussions during the semester, the final reflection question then asks them to reflect on the purpose of the reading assignment itself.As students think through the purpose of the reading, they start relating its various parts to the motivating discussion, and to earlier examples.They also start appreciating the significance of the demand-supply framework and the role of natural experiments in teasing out causality from correlation.
Activity 2 is well suited for any standard introductory economics course that accords at least one lecture to the topic of price controls.The activity is especially relevant if the same course also touches upon the scientific method of inquiry, or empiricism, at any length. 2The most effective timing for the assignment would be immediately following the lecture on price controls.The students will just have worked through-some even struggled through-the challenges of graphical expositions of shortages and their welfare costs.To see the graph's explanatory abilities in action and applied to a real-world problem that they lived through, affirms the significance of the analysis and the benefits of learning these tools well.

C. Activity 3: Exam 1 Reflection
Often, a key challenge is providing useful feedback on exam performance.While students, and perhaps many instructors as well, approach exams simply as a means of testing student understanding, useful feedback can help develop a better understanding not just of the concepts assessed, but also better preparation and test-taking strategies.By asking students to reflect upon their performance, Activity 3 checks all these boxes.Activity 3 is well suited for any course-introductory, intermediate, or advanced-as long as exams make for a significant component in assessment.This activity is best assigned immediately after the first exam, though the general framework can be applied to any graded assignment, once the answer key is made available.The preamble starts by setting the context right.It reminds students that reviewing the test and their performance is a powerful learning opportunity to improve their economic thinking.The last line in the preamble highlights the significance of the all-important "why." That is, better understanding and performance results from why a student did well on certain questions and not on others, not just superficially noting which questions they answered correctly and which ones they did not.
The first step in the activity requires students to simply compare their answers to those on the answer key, and mark which ones they got right and which ones they did not.
In the second step, the activity asks students to retry the ones they answered incorrectly one more time to see if they get it right this time.This step helps students gauge whether the incorrect response was an accident, like reading the question wrong, missing a term, or an error stemming from incomplete comprehension of an underlying principle as would be revealed by the suggested comparison with the response in the answer key.Further, this step also allows students to observe whether their comprehension has improved as would be indicated by getting the answers right this time around.
The third step asks students to repeat this process for all questions: now they work through the ones they answered correctly as well, to ensure that was not an accident, that they do indeed have the comprehension down.
The fourth step asks students to reflect on the responses they answered incorrectly on the exam and isolate which underlying principles they did not understand well enough.
Finally, the fifth step asks students to write a short reflective narrative that helps them connect everything they learned in the process of completing these tasks.They are asked to identify the concepts they need more work on and justify why these concepts need more work; they are asked to rethink their test-preparation strategies, given what they've learned in the process of completing these tasks; and they are asked to summarize the key lesson into a bitesized nugget.Finally, they are asked to reflect upon why the instructor assigned this task to them.
Standard economics instruction recognizes the importance of providing students the opportunity to carefully consider their performance on exams and learn from their mistakes.Offering that opportunity to students remains the fundamental purpose behind the standard practice of returning graded exams.But if countless instructors already engage in the standard exam grading/return protocol, how is Activity 3 unique?In two ways.
First, Activity 3 offers students not just the opportunity, but also explicit incentives, for careful self-guided reviewing of their performance.Undoubtedly, our top students often have the intrinsic motivation and drive to do so on their own, but most students do not.Our experience suggests that many students remain unaware about the benefits of such a practice.And other students-ironically, most often the ones who would benefit the most from a careful review-simply lack the motivation to engage in such an endeavor on their own.Activity 3 incentivizes this group of students, students who would otherwise not conduct a careful review of their performance, to do so.
Second, the specific questions in Activity 3 require students to dig deeper and go beyond just noting corrections.These questions guide students to think through two aspects integral to future improvement: why their preparation process resulted in mistakes they made, and how that process can be upgraded to avoid these mistakes in the future.The standard exam grading/return protocol only uncovers what mistakes students made.Activity 3, by contrast, uncovers why they made the mistake, and how they can do better.

D. Adaptability
We conclude this section with two observations for instructors considering implementing these activities.First, instructors should note that the unique reflective component in each activity-the last question asking students to consider the purpose of the activity-is the essential feature.Every other component is replaceable.For instance, the podcast episodes in Activity 1 can be easily replaced by other podcast episodes or videos the instructor prefers, and the paper assigned in Activity 2 can be easily replaced by other articles.Activity 3 can be easily applied to all graded problem sets, not just the first exam.Because of this flexibility, with some minor adjustments, variations of these activities can be adopted in any economics course and any existing assignment instructors already implement can incorporate the benefits of reflection.The only key ingredient required is the add-on graded question at the very end that asks "Why do you think I assigned you this activity?"Second, and especially relevant for larger class sizes, instructors can easily keep the extra time required for grading these activities down to a minimum.Skimming the responses to the first few questions can be enough to verify that students did indeed listen to the podcast or read the paper.Awarding full credit for all answers that provide this evidence ensures the right incentives for students and low time costs for instructors.Similarly, skimming the responses to the open-ended questions, including the reflection, and grading these for completion only, can keep grading time to a bare minimum.

A. Preliminaries
Table 1 summarizes the relevant details about courses which incorporated these activities.As the table makes clear, we have adopted all three activities in a variety of settings.We have adopted them in introductory and upper division courses, across a wide range of class sizes, offered in-person as well as online.All introductory courses in the table adopted all three activities.The upper-division Industrial Organization course adopted Activities 1 and 3 only.Each course listed in Table 1 had between six to eight equally weighted homework assignments in total, with homework counting for 20 -40 percent of the total course grade.For each of the activities adopted, the instructor implemented the activity as a standalone homework assignment.For instance, in The Economic Way of Thinking-an introductory economics course geared towards non-majors-eight homework assignments together count for 20 percent of the total course grade.These include five regular problem sets, and the three specific activities presented here.

B. Performance
Student responses to all three activities indicate that the assignment met its goal for the large majority of students.We summarize the key findings from student responses for each activity below.
Activity 1 delivers three noteworthy outcomes.First, every single student response noted the activity helped them connect the course concepts of demand-supply to real-world applications, and the applications assisted in helping them appreciate the significance of the model.Second, some students noted being challenged by the fixed supply of taxi medallions: what does a supply curve look like in such a case?The noted challenge of incorporating this fixed supply into the standard demand-supply model motivates a short follow-up discussion in class that students are now primed for, having noted the challenge in the example.Third, some students noted that the activity challenged them and made them realize they needed to review the demand-supply model more in order to successfully complete the assignment.This self-realization from incomplete understanding, to reviewing more to understand, and finally to successful application indicates the strength of this activity in inspiring students to become agents of their own learning.
Student responses to Activity 2 indicate success in three key dimensions.First, students observe the power of the demand-supply model in explaining an unintended consequence of a well-intentioned policy that they themselves have lived through.Second, they appreciate learning about natural experiments in teasing out causality over correlation.Finally, in their reflections on why the activity was assigned, students emphasize being able to map out a thread across several course topics covered by going through this exercise.As best practices in pedagogy note, such self-motivated concept mapping is key to instigating deeper, richer learning (e.g., Wiggins & McTighe, 2005).
Student responses to Activity 3 indicate three key successes as well.First, most students report discovering that most of their errors indicate something missing from their understanding of the core concepts required for the correct response.By recasting the problem as one of a misunderstanding of the core concept, rather than errors of not understanding the question, students start appreciating the significance of learning the core concepts rather than memorizing solutions to example problems.Second, students discuss specific strategies and techniques they can adopt to avoid the mistakes they made, which then becomes a selfdiscovered template for future success.Finally, many students note despite being demotivated initially by their exam grades, reviewing their exams in this step-by-step fashion, helped them formulate a clear action plan to do better in the future.Moreover, this self-formulated concrete action plan gave them hope that they can do better.In their reflection responses, the large majority of students expressed their gratitude to the instructor for assigning this activity, which helped them realize what was missing so that they could start doing better.
Student responses to the reflection question in Activity 3 also provide clear evidence that the students find this incentivized self-guided exam reflection more effective than the traditional protocol of returning graded exams.For instance, consider the following excerpts from student responses: • "While I would typically look at the answer key, see what I got wrong, and move on, through this assignment, I am thinking about what I need to do for the next test, so I do not make the same mistakes." • "Through this exercise, I have learned how important it is to actually analyze my incorrect answers, and not just look at the correct answers.I think I often just quickly review my tests, instead of attempting to understand how I could improve.To change how I will prepare for the next test, I think what would benefit me the most would be going through the previous homework assignments.By reading over those questions, as well as the ones we go over in class, I would not only analyze why certain answers are correct, but also why the wrong answers are incorrect." • "I think the most important lesson I've learned was to draw the graphs.Honestly, I don't really like drawing them, but I need to get used to drawing them if I want to do better on these exams." • "If you had not assigned this reflection, I most likely would not have reviewed my mistakes and analyzed how to improve for next time.This was a great exercise for diving into my brain and analyzing how I made mistakes." • "I think you assigned this task to help us in understanding that 1) it is okay to make mistakes, and 2) that learning from those mistakes and using them to improve in the future is very important.These kinds of skills will most definitely help in college, but a lot of this translates over to life.In our future careers and also day to day life interactions we will make mistakes, but we have to learn to realize that we make mistakes and use those mistakes to learn from them."

C. Challenges and Modifications
The first few rounds of implementing these activities revealed a few unexpected challenges.We conclude this section by discussing these challenges and the modifications to the activities that helped address these challenges.
The first challenge resulted from an oversight about the point distribution across each component of the activities.The initial versions only indicated the "total points available" for the entire assignment; a part-by-part breakup was missing.Without specific grades assigned to each part, students were free to interpret the relative importance of each part and write their responses accordingly.As a result, some submissions clearly accorded much less weight to the reflective portion of the assignment than the other parts, which undermined the fundamental purpose of the assignment.Future versions of the assignments addressed this concern by attaching the specific points to each component of the activities as displayed in Figures 1,  2, and 3. Since this update, student responses have been much less skewed.The reflection component has received at least as much attention as the other parts from most students.This improvement suggests that just this small change of making component-specific points salient is effective in communicating the relative importance of the reflection component to students.
The second challenge resulted from an oversight in Activity 3, this time about one particular group of students: Exceptional performers-students who did well on the exam and were satisfied with their performances.The initial version of the activity was entirely focused on students whose performances indicated sufficient scope of improvement.But this focus meant that top students with perfect, or near perfect, scores would have very little to say about how they could improve.All such students responded with variants of "[m]y plan is to keep doing what I did.I don't know what else to write." To address this oversight and make the top performers feel more included, future versions of the assignment made two changes.First, the following sentence was added under part (c): "If you were satisfied with your performance, what strategies helped you be successful, and what would your advice be for others who weren't as successful?"Second, with the students' permission, best responses offering effective strategies for peers who want to do better were shared with the rest of the class.Since implementing these changes, the responses from top performers have indicated careful reflection and contained several useful tips for their peers seeking concrete steps to improve.But beyond that, we have found the step of sharing responses with the rest of the class improves general collegiality and fosters a greater collaborative spirit among more students within the classroom.

Summary and Conclusion
This paper has presented three innovative approaches to enrich student reflection in assignments and improve student learning in economics classes.Each of these techniques requires students to consider why the instructor assigned the task.Our assessment of student responses suggests that all three activities met their goals.
Two key limitations, however, restrict us from making formal causal claims.First, since these activities were not randomly assigned, we do not have a suitable counterfactual baseline to compare these results against.Second, since these activities were graded and student names were identifiable, some students may have responded in accordance with what they believed would help them obtain better grades, rather than in accordance with their true feelings.
Based on our own past experiences in the classroom, and the responses obtained, we conclude that if the students' own words are to be believed, the reflective dimension helped students map out connections between topics and chart out plans for doing better in the course.Instead of the usual groans and complaints about difficult exams or assignments, these activities generated expressions of gratitude from students for helping them find joy in learning economics and assisting them in discovering self-guided paths to doing better in the course.

Table 1 :
Summary of Courses in Which Reflection Activities Were Adopted