How to fix “openai.error.RateLimitError: The server had an error”

If you’re a regular user of the OpenAI GPT API, you might have recently faced a very annoying issue, causing your Python script to fail 👇

openai.error.RateLimitError: The server had an error while processing your request. Sorry about that!

There’s no rational reason explaining this “RateLimitError” (except of course if you’re actually flooding the OpenAI server with tons of requests).

The error is probably caused by the intense load on OpenAI’s servers caused by the impressive popularity of their services (ChatGPT, Dall-E and the GPT-3 API).

If you have a Python script which iterates through a FOR / WHILE LOOP or if your request to the OpenAI API is part of a much longer script, you don’t want the script to exit for an unexplainable “openai.error”.

Since you don’t pay for failed attempts to connect to the API, you should include smart retries in your script.

Here’s how I do it, with a WHILE LOOP and a nested TRY / EXCEPT rule.

qu_attempts = 1

while qu_attempts <= 5:

  try: 

    answer = question(prompt,topic)
    answer_text = answer[0]
    answer_usage = answer[1]
    usage += answer_usage
    qu_attempts = 6

  except:

    print(f"openai error, attempt {qu_attempts}")
    qu_attempts += 1
    time.sleep(5)

Basically, as long as the # of attempts is lower than 5, the script will try to get the answer from the API.

Note: in my custom function (which I import from another Python file), I pass two variables in the question sent to GPT, the prompt and the topic attached to the prompt. You can of course pass everything in a single variable.

I also return two pieces of data form my function: the text answer and the usage in tokens, which I store to calculate the overall cost of running script. I wrote a detailed article about this feature.

If the script doesn’t connect to the API, I print out a dynamic error message and I wait 5 seconds before a retry (don’t forget to import time at the top of your script).

If the script successfully gets the answer, I increment the qu_attempts variable to 6 to exit the while loop. You could of course set a higher amount of retries (and try a shorter pause).

FYI, I’ve just used the script on a series of 64 consecutive queries in a for loop and in most cases the script needed 2 or 3 attempts to get a successful response.

👉 This while loop with try/except rules can be part of a wider while loop or for loop if you’re iterating through data rows.

Of course this solution is just a temporary way to mitigate an issue which should be properly fixed by OpenAI to deliver the service deserved by its premium clients.