Robust prompt engineering is a valuable skill

Once you figure out how to make ChatGPT work for you, the results are impressive

I see a lot of people complain that ChatGPT hallucinates or gives them wrong information. I think part of the problem is that people’s expectations don’t match reality. People see hyperbolic claims about ChatGPT having passed the Turing Test, or AGI being imminent, and they assume that they can interact with ChatGPT like they can with their friends.

But the problem is this: the language we use with other people is often imprecise, elliptical, metaphorical, nuanced, abstract, or vague. And yet, in spite of all these defects, our interlocutors usually are able to infer context and meaning from out inept speech!

ChatGPT conveys the sense that it can do this, too. But if a prompt is not well-structured and logical, ChatGPT may well give you a response that is unhelpful or simply wrong.

Let me make this more concrete. A couple of days ago, I had the idea to have ChatGPT write a Python script which would prompt the user for some input, construct a loan amortization table based on the input, and save it as an Excel file. You can imagine a person speaking to her assistant at a bank: “I need you to build a loan amortization table. Principal is $1,000, rate of 2%, and period of 60.” The assistant would dutifully fire up Excel and build out the loan amortization table.

So this is the prompt I used:

write a python script that builds a loan amortization table and saves the output to excel. Ask the user for input for the following variables: loan amount, number of payment periods, interest rates. Make sure to include the variables in the Excel spreadsheet.

It generated this script:

import openpyxl

# Ask the user for input
loan_amount = float(input("Enter the loan amount: "))
num_periods = int(input("Enter the number of payment periods: "))
interest_rate = float(input("Enter the interest rate: "))

# Calculate the monthly payment
monthly_rate = interest_rate / 12
monthly_payment = loan_amount * (monthly_rate / (1 - (1 + monthly_rate) ** (-num_periods)))

# Create a new Excel workbook
wb = openpyxl.Workbook()
sheet = wb.active

# Add the input variables to the first row
sheet['A1'] = "Loan Amount"
sheet['B1'] = "Number of Payment Periods"
sheet['C1'] = "Interest Rate"
sheet['A2'] = loan_amount
sheet['B2'] = num_periods
sheet['C2'] = interest_rate

# Add the headers for the amortization table
sheet['A4'] = "Payment Number"
sheet['B4'] = "Payment Amount"
sheet['C4'] = "Principal"
sheet['D4'] = "Interest"
sheet['E4'] = "Remaining Balance"

# Initialize the remaining balance
remaining_balance = loan_amount

# Build the amortization table
for i in range(1, num_periods + 1):
    # Calculate the interest and principal for this payment
    interest = remaining_balance * monthly_rate
    principal = monthly_payment - interest

    # Update the remaining balance
    remaining_balance -= principal

    # Write the payment information to the spreadsheet
    sheet.cell(row=i+4, column=1).value = i
    sheet.cell(row=i+4, column=2).value = monthly_payment
    sheet.cell(row=i+4, column=3).value = principal
    sheet.cell(row=i+4, column=4).value = interest
    sheet.cell(row=i+4, column=5).value = remaining_balance

# Save the workbook
wb.save("loan_amortization.xlsx")
print("Loan amortization table saved to loan_amortization.xlsx")

Here is the loan amortization table it built (the screen shot is cut off after the 24th payment period):

In other words, the prompt worked flawlessly. It generated a Python script which prompted the user to enter three variables. The Python script took those three variables and iteratively calculated a loan balance for 60 payment periods. The script saved the table to Excel.

So, it’s not the case that ChatGPT is useless. But it is the case that prompts have to be carefully designed to generate the desired output.