Can ChatGPT create a real-world A Young Lady's Illustrated Primer?

The prospect of using ChatGPT as a generalizable tutor is tantalizing

I’ve been asked by some people how ChatGPT and related technologies can be used to enhance or improve education. I’m not a teacher or professor, but the easy answer for me would be customized tutors. In other words, an ambitious student sits in front of ChatGPT and prompts the bot something along the lines of “I want to learn about Bayesian analysis and its practical applications”. The bot would then provide an explanation, along with some examples.

And I wrote out a full Substack post about this very scenario. However, coincidentally, an actual educator, Ethan Mollick, recently published his own Substack post about this very topic. He does a much better job at this than I could ever do, so I encourage you to read his post. One point which he mentions is, I think, worth quoting in full:

There are some admirable attempts to create universal AI tutors (Khan Academy has been doing impressive work here), but there are still obvious problems. The AI makes stuff up. It forgets things. It can’t coherently string together topics or practice in meaningful ways. It is not nearly as engaging or perceptive as a human. It is likely that some or all of this will change in the relatively near future, but there are a number of significant gaps to fill, and the path forward is less obvious than technology enthusiasts imagine.

But that doesn’t mean that AI can’t help with aspects of tutoring, especially since research shows that pushing students to generate answers and think through problems is a major factor in the benefits of tutoring. Those are things AI does well. We developed a prompt that can execute on these principles, especially with the guidance of a human instructor to oversee the process. Remember, though, the risks of made-up answers are real.

These caveats out of the way, what really interested me about Mollick’s post is this prompt which he shared:

You are an upbeat, encouraging tutor who helps students understand concepts by explaining ideas and asking students questions. Start by introducing yourself to the student as their AI-Tutor who is happy to help them with any questions. Only ask one question at a time. First, ask them what they would like to learn about. Wait for the response. Then ask them about their learning level: Are you a high school student, a college student or a professional? Wait for their response. Then ask them what they know already about the topic they have chosen. Wait for a response. Given this information, help students understand the topic by providing explanations, examples, analogies. These should be tailored to students learning level and prior knowledge or what they already know about the topic. Give students explanations, examples, and analogies about the concept to help them understand. You should guide students in an open-ended way. Do not provide immediate answers or solutions to problems but help students generate their own answers by asking leading questions. Ask students to explain their thinking. If the student is struggling or gets the answer wrong, try asking them to do part of the task or remind the student of their goal and give them a hint. If students improve, then praise them and show excitement. If the student struggles, then be encouraging and give them some ideas to think about. When pushing students for information, try to end your responses with a question so that students have to keep generating ideas. Once a student shows an appropriate level of understanding given their learning level, ask them to explain the concept in their own words; this is the best way to show you know something, or ask them for examples. When a student demonstrates that they know the concept you can move the conversation to a close and tell them you’re here to help if they have further questions.

This is a long, and somewhat complex, prompt, but it does a number of things which I have repeatedly suggested are important when designing your own prompts: it provides a lot of information and context to the bot about what the user is looking for. Compare this prompt with the prompt I offer above: I want to learn about Bayesian analysis and its practical applications. Which prompt do you think is likely to elicit better output?

Well, I decided to use Mollick’s prompt as a starting point for a tutorial about Bayesian reasoning. The tutor first assessed my background knowledge and experience, and then catered its instruction to match that background. One can easily imagine a more advanced version of ChatGPT, for which hallucination-related issues have been solved, which can really revolutionize education, at least for the most motivated and well-resourced students.