How to fix “openai.error.RateLimitError: The server had an error”

If you’re a regular user of the OpenAI GPT API, you might have recently faced a very annoying issue, causing your Python script to fail 👇

openai.error.RateLimitError: The server had an error while processing your request. Sorry about that!

or maybe this version, which I’ve seen more often over the last few weeks:

AttributeError: 'RateLimitError' object has no attribute 'retry_after'

There’s no rational reason explaining this “RateLimitError” (except of course if you’re actually flooding the OpenAI server with tons of requests).

The error is probably caused by the intense load on OpenAI’s servers caused by the impressive popularity of their services (ChatGPT, Dall-E and the GPT-3.5 / GPT4 API).

If you have a Python script which iterates through a FOR / WHILE LOOP or if your request to the OpenAI API is part of a much longer script, you don’t want the script to exit for an unexplainable “openai.error”.

Since you don’t pay for failed attempts to connect to the API, you should include smart retries in your script.

A few days ago (May 2023), I started encountering another type of error using the OpenAI API (Bad Gateway).

An error occurred: Bad gateway. {"error":{"code":502,"message":"Bad gateway.","param":null,"type":"cf_bad_gateway"}} 502 {'error': {'code': 502, 'message': 'Bad gateway.', 'param': None, 'type': 'cf_bad_gateway'}} {'Date': 'Thu, 18 May 2023 08:07:36 GMT', 'Content-Type': 'application/json', 'Content-Length': '84', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'same-origin', 'Cache-Control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'Server': 'cloudflare', 'CF-RAY': '---', 'alt-svc': 'h3=":443"; ma=86400, h3-29=":443"; ma=86400'}

You may also encounter the following connection error when trying to connect to the OpenAI API

An error occurred: ('Connection aborted.', OSError(22, 'Invalid argument'))

The best way to handle this situation is to call the API with a TRY / EXCEPT rule for each type of error.

I added 4 different rules to handle the exceptions:

• one re: to the RateLimitError
• one related to a more general API Error
• one related to the connection error
• one related to the TimeOut error

See for instance how I structure my call to the chat completion API.

def chat(prompt):

   openai.api_key = "YOUR API KEY"

   try:

      response = openai.ChatCompletion.create(
      model = "gpt-3.5-turbo",
      messages = [{"role": 'system', "content": f'{prompt}'}]
      )

      answer = response["choices"][0]["message"]["content"]
      usage = response["usage"]["total_tokens"]

      return answer, usage


    except openai.error.RateLimitError as e:
      retry_time = e.retry_after if hasattr(e, 'retry_after') else 30
      print(f"Rate limit exceeded. Retrying in {retry_time} seconds...")
      time.sleep(retry_time)
      return chat(prompt)

   except openai.error.APIError as e:
      retry_time = e.retry_after if hasattr(e, 'retry_after') else 30
      print(f"API error occurred. Retrying in {retry_time} seconds...")
      time.sleep(retry_time)
      return chat(prompt)

   except openai.error.ServiceUnavailableError as e:
      retry_time = 10  # Adjust the retry time as needed
      print(f"Service is unavailable. Retrying in {retry_time} seconds...")
      time.sleep(retry_time)
      return chat(prompt)

   except openai.error.Timeout as e:
      retry_time = 10  # Adjust the retry time as needed
      print(f"Request timed out: {e}. Retrying in {retry_time} seconds...")
      time.sleep(retry_time)
      return chat(prompt)

   except OSError as e:
      if isinstance(e, tuple) and len(e) == 2 and isinstance(e[1], OSError):
         retry_time = 10  # Adjust the retry time as needed
         print(f"Connection error occurred: {e}. Retrying in {retry_time} seconds...")
         time.sleep(retry_time)
         return chat(prompt)
      else:
         retry_time = 10  # Adjust the retry time as needed
         print(f"Connection error occurred: {e}. Retrying in {retry_time} seconds...")
         time.sleep(retry_time)
         raise e

As you can see, if I encounter an error, I wait for up to 30 seconds before a retry which basically re-attempts to call the API.

The beauty of it is that those error-handling solutions were advised to me by… ChatGPT itself.

Of course this solution is just a temporary way to mitigate an issue which should be properly fixed by OpenAI to deliver the service deserved by its premium clients.