exceptions.FirebaseError does not catch BrokenPipeError

[bitard-life]

OS: Debian 10 (buster)
Python: Python 3.7.3
Library: firebase-admin 4.4.0
Product: messaging

Error:

Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "main.py", line 597, in thread_live
    response = messaging.send_multicast( message )
  File "/usr/local/lib/python3.7/dist-packages/firebase_admin/messaging.py", line 167, in send_multicast
    return _get_messaging_service(app).send_all(messages, dry_run)
  File "/usr/local/lib/python3.7/dist-packages/firebase_admin/messaging.py", line 390, in send_all
    batch.execute()
  File "/usr/local/lib/python3.7/dist-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/googleapiclient/http.py", line 1540, in execute
    self._execute(http, self._order, self._requests)
  File "/usr/local/lib/python3.7/dist-packages/googleapiclient/http.py", line 1470, in _execute
    self._batch_uri, method="POST", body=body, headers=headers
  File "/usr/local/lib/python3.7/dist-packages/google_auth_httplib2.py", line 201, in request
    uri, method, body=body, headers=request_headers, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
  File "/usr/local/lib/python3.7/dist-packages/httplib2/__init__.py", line 1651, in _request
    conn, request_uri, method, body, headers
  File "/usr/local/lib/python3.7/dist-packages/httplib2/__init__.py", line 1558, in _conn_request
    conn.request(method, request_uri, body, headers)
  File "/usr/lib/python3.7/http/client.py", line 1244, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1065, in _send_output
    self.send(chunk)
  File "/usr/lib/python3.7/http/client.py", line 987, in send
    self.sock.sendall(data)
  File "/usr/lib/python3.7/ssl.py", line 1015, in sendall
    v = self.send(byte_view[count:])
  File "/usr/lib/python3.7/ssl.py", line 984, in send
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

Steps to reproduce:

The script was sending push notifications for a long time and suddenly an error appeared.

Code:

while True:
   try:
     response = messaging.send_multicast( message )
   except firebase_admin.exceptions.FirebaseError as ex:
     print( ex.message )

   sleep( 60 )

[hiranya911]

Given that your error handler is simply logging the error message, you can just catch the parent Exception type to catch all possible errors. At least that would be a reasonable temporary workaround.

The problem here is that googleapiclient can actually raise a wide range of exception types, but we only handle HttpError. We can try to improve that in the future.

[GabrieleCalarota]

Sorry to reopen this, but I'm getting quite a few BrokenPipeError as long as ReadTimeout when sending_multicast notifications from server-side. Why are these errors thrown and what is a possible workaround for this?

[sedrakk]

Getting same error occasionally when trying to multicast a message and except doesn't work, nothing the text doesn't get printed:

    try:
        response = messaging.send_multicast(message)
    except Exception as error:
        print(f'Something went wrong: {error}')

[mcookhome]

I also am seeing this issue with send_multicast

[JennyCGT]

I also have this same issue, when I am using send_multicast. I am using it in Django +Celery hosted in Elastic Beanstalk.

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 450, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 731, in protected_call
return self.run(*args, **kwargs)
File "/code/apps/notifications_api/notification_task.py", line 78, in message_notifications
sent =send_multicast(multo)
File "/usr/local/lib/python3.7/site-packages/firebase_admin/messaging.py", line 167, in send_multicast
return _get_messaging_service(app).send_all(messages, dry_run)
File "/usr/local/lib/python3.7/site-packages/firebase_admin/messaging.py", line 390, in send_all
batch.execute()
File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 1560, in execute
self._execute(http, self._order, self._requests)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 1490, in _execute
self._batch_uri, method="POST", body=body, headers=headers
File "/usr/local/lib/python3.7/site-packages/google_auth_httplib2.py", line 225, in request
**kwargs
File "/usr/local/lib/python3.7/site-packages/httplib2/init.py", line 1994, in request
cachekey,
File "/usr/local/lib/python3.7/site-packages/httplib2/init.py", line 1651, in _request
conn, request_uri, method, body, headers
File "/usr/local/lib/python3.7/site-packages/httplib2/init.py", line 1558, in _conn_request
conn.request(method, request_uri, body, headers)
File "/usr/local/lib/python3.7/http/client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.7/http/client.py", line 1065, in _send_output
self.send(chunk)
File "/usr/local/lib/python3.7/http/client.py", line 987, in send
self.sock.sendall(data)
File "/usr/local/lib/python3.7/ssl.py", line 1034, in sendall
v = self.send(byte_view[count:])
File "/usr/local/lib/python3.7/ssl.py", line 1003, in send
return self._sslobj.write(data)
socket.timeout: The write operation timed out

[abel-castro]

I am also having this issue using send_multicast and send_all in my Django + Celery setup.
My solution was to try/catch the BrokenPipe errors, get the firebase app again and retry the send_* command.

For that I created a retry_send_all and a retry_send_multicast:

from firebase_admin import messaging

def retry_send_all(messages, firebase_app, **kwargs):
    try:
        return messaging.send_all(messages, app=firebase_app, **kwargs)
    except BrokenPipeError:
        firebase_app = get_firebase_app()
        return messaging.send_multicast(message, app=firebase_app, **kwargs)

def retry_send_multicast(message, firebase_app, **kwargs):
    try:
        return messaging.send_multicast(message, app=firebase_app, **kwargs)
    except BrokenPipeError:
        firebase_app = get_firebase_app()
        return messaging.send_multicast(message, app=firebase_app, **kwargs)