Building a simple tutor with ChatGPT

Can I use Python to build a simple tutor to help me understand matrix manipulation?

Dec 19, 2022

It seems obvious to me that one of the things that will be most affected by ChatGPT is education. ChatGPT and its future versions promise to upend much of traditional education by commoditizing instruction.

What I mean by this is relatively straightforward. It’s pretty simple, in principle, to tell ChatGPT “I want to learn about discounted cash flow analysis. What can you tell me?” Here’s its output:

Discounted cash flow (DCF) analysis is a method used to estimate the value of an investment or asset based on its future cash flows. It involves forecasting the future cash flows that an investment is expected to generate, and then discounting those cash flows back to the present using a discount rate. The resulting present value is then used to determine the investment's intrinsic value.
Here's the general formula for discounted cash flow analysis:
Intrinsic value = SUM(CF1/(1+r)^1 + CF2/(1+r)^2 + ... + CFn/(1+r)^n)
Where:
CF1, CF2, ..., CFn are the expected future cash flows
r is the discount rate
n is the number of periods over which the cash flows are expected to occur
The discount rate is a measure of the time value of money, which reflects the fact that a dollar received in the future is worth less than a dollar received today. It represents the opportunity cost of capital, or the return that an investor could expect to earn by investing in an alternative asset with similar risk.
To use discounted cash flow analysis, you will need to make estimates of the expected future cash flows for the investment, as well as the appropriate discount rate to use. These estimates will be based on a variety of factors, including the investment's expected growth rate, the level of risk involved, and the current economic environment.
There are a number of different applications for discounted cash flow analysis, including valuing businesses, stocks, real estate, and other investments. It is a widely used tool in financial analysis and can be an important factor in making investment decisions.

Now, I know enough about discounted cash flow analysis to understand that this is a pretty good overview. You can imagine a future, more advanced and more reliable version of ChatGPT, call it ChatGPT 2.0 or 3.0, in which a user can just assume that it is able to accurately answer a question about discounted cash flow analysis, or pretty much any other subject someone wants to learn.

Here’s Samuel Hammond expounding upon this idea:

education seems expecially exposed to disruption by near-term large language models, starting at the youngest ages. Parents are told to read and talk to their kids from infancy, helping prime their language faculty and build some basic vocabulary. So why not have a plush chatbot teddy bear that converses with the child 24-7? It could even teach the infant a second language through repetition and immersion, becoming their very best amigo.
For older kids, mastery based learning is one of the most effective interventions in the education literature. The only constraint is human labor. So why not create an adaptive AI tutor? Soon a precocious homeschooler will bootup their web browser and meet an animated Ms. Frizzle, ready to pick up where they left off. She’ll help break down problems, keep the student motivated, and summon visual aids to a virtual blackboard. The language model could be fed a variety of metrics, from the timbre of the student’s voice to their latest test scores, optimizing its approach to maximize learning outcomes. “I see you got a B on the grammar test. Let’s go back over verbs and adverbs one last time.”

So can I build such a tutor today? I’ve been learning Python, and have used ChatGPT to write a lot of different little scripts for me. What if I wanted to write a script that manipulates matrices? And, even better, what if I wanted this tutor-script to provide a natural language explanation of how a particular result was generated? You can imagine the dynamic between a student and a human tutor: the tutor performs a mathematical operation, and explains, in natural language to her student, how the operation works. Can that entire process—a mathematical operation and a natural language explanation of it—be instantiated in Python by ChatGPT? It certainly seems within its abilities.

I put together a Notion doc with my results, which you can review here. You’ll note that my first prompt did not generate the code I expected. Here’s my first prompt:

Can you write a python script for me that performs addition, subtraction, or multiplication of matrices? Let the use define the size of the matrices, and the operation to be performed (addition, subtraction, or multiplication). Output the result to a text file, and include in the output a natural language explanation of how the operation was performed.

ChatGPT did not interpret this correctly. Or, more accurately, it did not interpret my prompt in the way that I expected it to. Here’s the code it generated from this prompt:

# Import the required modules
import numpy as np
import sys

# Define the operation to be performed
operation = input("Enter the operation to be performed (addition, subtraction, or multiplication): ")

# Read the matrices from the input file
try:
    with open("matrices.txt", "r") as f:
        lines = f.readlines()
        matrix1 = np.array([[int(x) for x in line.split()] for line in lines[:int(lines[0])]])
        matrix2 = np.array([[int(x) for x in line.split()] for line in lines[int(lines[0]):]])
except:
    print("Error reading input file")
    sys.exit()

# Perform the specified operation
if operation == "addition":
    result = matrix1 + matrix2
    operation_str = "addition"
elif operation == "subtraction":
    result = matrix1 - matrix2
    operation_str = "subtraction"
elif operation == "multiplication":
    result = matrix1.dot(matrix2)
    operation_str = "multiplication"
else:
    print("Invalid operation")
    sys.exit()

# Output the result to the output file
try:
    with open("result.txt", "w") as f:
        f.write("Result of matrix {}:\n".format(operation_str))
        f.write(str(result))
        f.write("\n\nExplanation:\n")
        if operation == "addition":
            f.write("The result of the matrix addition is obtained by adding the corresponding entries of the two matrices.\n")
        elif operation == "subtraction":
            f.write("The result of the matrix subtraction is obtained by subtracting the corresponding entries of the two matrices.\n")
        elif operation == "multiplication":
            f.write("The result of the matrix multiplication is obtained by multiplying the entries of the first matrix by the corresponding entries in the second matrix and summing the results.\n")
except:
    print("Error writing output file")
    sys.exit()

print("Operation completed successfully")

If you don’t know Python, it won’t be apparent to you what is wrong with this code. But we have a clue in the text that ChatGPT prepended to the code:

Certainly! Here is a Python script that allows the user to define the size of two matrices and the operation to be performed (addition, subtraction, or multiplication). The script reads the matrices from a text file, performs the specified operation, and outputs the result to another text file, along with a natural language explanation of how the operation was performed:

Now, this is clearly wrong: the intent behind my prompt was to allow the user to enter information about the matrices, and have the script output the results and a natural language explanation of the results, to a text file. But ChatGPT, for some reason, understood the prompt to mean reading from a text file, not outputting data to a text file.

So, I have to iterate. Let’s try a different prompt:

Thanks, but this doesn't quite do what I asked. I want the script to prompt the user for three things: (1) the operation to be performed on the matrices (addition, subtraction, or multiplication); (2) the dimension of the matrices; (3) the values for each matrix. Then, I want the script to perform the desired operation, and output the result to a TXT file. That TXT file should also include a natural language explanation of how the specified operation was performed.

This is a much more structured prompt. It reads less like natural language and more like, well, computer code. It is very logical and coherent, but it does not reflect how humans speak to one another. It is stilted speech. And yet, it generated the results I expected. This is, in short, prompt engineering.

Here’s the output from this prompt:

# Import the required modules
import numpy as np
import sys

# Prompt the user for the operation to be performed
operation = input("Enter the operation to be performed (addition, subtraction, or multiplication): ")

# Prompt the user for the dimensions of the matrices
rows1 = int(input("Enter the number of rows in the first matrix: "))
cols1 = int(input("Enter the number of columns in the first matrix: "))
rows2 = int(input("Enter the number of rows in the second matrix: "))
cols2 = int(input("Enter the number of columns in the second matrix: "))

# Check that the matrices are compatible for the specified operation
if operation == "addition" and (rows1 != rows2 or cols1 != cols2):
    print("Error: Matrices must have the same dimensions for addition")
    sys.exit()
elif operation == "subtraction" and (rows1 != rows2 or cols1 != cols2):
    print("Error: Matrices must have the same dimensions for subtraction")
    sys.exit()
elif operation == "multiplication" and cols1 != rows2:
    print("Error: Number of columns in first matrix must match number of rows in second matrix for multiplication")
    sys.exit()

# Prompt the user for the values of the matrices
matrix1 = np.empty((rows1, cols1))
print("Enter the values for the first matrix:")
for i in range(rows1):
    matrix1[i,:] = [int(x) for x in input().split()]

matrix2 = np.empty((rows2, cols2))
print("Enter the values for the second matrix:")
for i in range(rows2):
    matrix2[i,:] = [int(x) for x in input().split()]

# Perform the specified operation
if operation == "addition":
    result = matrix1 + matrix2
    operation_str = "addition"
elif operation == "subtraction":
    result = matrix1 - matrix2
    operation_str = "subtraction"
elif operation == "multiplication":
    result = matrix1.dot(matrix2)
    operation_str = "multiplication"
else:
    print("Invalid operation")
    sys.exit()

# Output the result to the output file
try:
    with open("result.txt", "w") as f:
        f.write("Result of matrix {}:\n".format(operation_str))
        f.write(str(result))
        f.write("\n\nExplanation:\n")
        if operation == "addition":
            f.write("The result of the matrix addition is obtained by adding the corresponding entries of the two matrices.\n")
        elif operation == "subtraction":
            f.write("The result of the matrix subtraction is obtained by subtracting the corresponding entries of the two matrices.\n")
        elif operation == "multiplication":
            f.write("The result of the matrix multiplication is obtained by multiplying the entries of the first matrix by the corresponding entries in the second matrix and summing the results.\n")
except:

The curious thing about this output is that everything after except: seems to have gone missing! I assume this is a limitation of the length of output that you can currently generate from ChatGPT.

However, we can synthesize the two scripts. If you look at the first output, the last five lines of code read:

except:
    print("Error writing output file")
    sys.exit()

print("Operation completed successfully")

And so there you have it. This tutor’s natural language explanation of how matrix operations work is rather terse. I had hoped for a less terse, and more expansive, explanation of how addition, subtraction, or multiplication of matrices work. But this is nonetheless impressive. And, given the directional arrows of progress, one can see how AI tutors will become more robust over time.

Buy the Rumor; Sell the News

Discussion about this post