Gunicorn worker terminated due to signal 9

Not sure what’s going on, unable to invite new users now via SMTP even though the settings for SMTP are correct. When I make SMTP “console” in the .env file it allows me to add existing users. I tried turning off firewall and same issue. With SMTP enabled it causes an error. In the browser console I get a 502 Bad Gateway nginx/1.19.10 when sending the POST request.

Please see the logs:

taiga-docker-taiga-gateway-1          | 172.18.0.1 - - [05/Mar/2024:18:52:35 +0000] "POST /api/v1/memberships/bulk_create HTTP/1.0" 502 158 "https://taiga.website.com/project/project-name/admin/memberships" "Mozilla/5.0 (Windows NT 10.0; rv:123.0) Gecko/20100101 Firefox/123.0" "198.44.122.31"
taiga-docker-taiga-back-1             | [2024-03-05 18:52:36 +0000] [1] [WARNING] Worker with pid 27 was terminated due to signal 9
taiga-docker-taiga-back-1             | [2024-03-05 18:52:36 +0000] [47] [INFO] Booting worker with pid: 47

This is my host machine nginx conf that is proxy passing to port 9000:

server {
      server_name taiga.website.com;

      location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_redirect off;
        proxy_pass http://localhost:9000/;
      }

      # Events
      location /events {
        proxy_pass http://localhost:9000/events;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_connect_timeout 7d;
        proxy_send_timeout 7d;
        proxy_read_timeout 7d;
      }

      # TLS: Configure your TLS following the best practices inside your company
      # Logs and other configurations


    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/taiga.website.com/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/taiga.website.com/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
  server {
    if ($host = taiga.website.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


      server_name taiga.website.com;
    listen 80;
    return 404; # managed by Certbot


}

Here’s the latest container info, a docker pull was run just today so everything is up to date.

CONTAINER ID   IMAGE                            COMMAND                  CREATED          STATUS                   PORTS                                                                  NAMES
5fa1525a80d9   nginx:1.19-alpine                "/docker-entrypoint.…"   15 minutes ago   Up 3 minutes             0.0.0.0:9000->80/tcp, :::9000->80/tcp                                  taiga-docker-taiga-gateway-1
55dfdc5594e4   taigaio/taiga-back:latest        "./docker/entrypoint…"   15 minutes ago   Up 3 minutes             8000/tcp                                                               taiga-docker-taiga-back-1
8cb930d8bf31   taigaio/taiga-back:latest        "/taiga-back/docker/…"   15 minutes ago   Up 3 minutes             8000/tcp                                                               taiga-docker-taiga-async-1
786705e6dc30   postgres:12.3                    "docker-entrypoint.s…"   15 minutes ago   Up 3 minutes (healthy)   5432/tcp                                                               taiga-docker-taiga-db-1
37dc77e12cb8   rabbitmq:3.8-management-alpine   "docker-entrypoint.s…"   15 minutes ago   Up 3 minutes             4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   taiga-docker-taiga-events-rabbitmq-1
39e4902e2189   taigaio/taiga-front:latest       "/docker-entrypoint.…"   15 minutes ago   Up 3 minutes             80/tcp                                                                 taiga-docker-taiga-front-1
4fd4fa4f8a98   taigaio/taiga-protected:latest   "./docker/entrypoint…"   15 minutes ago   Up 3 minutes             8003/tcp                                                               taiga-docker-taiga-protected-1
32667475ad79   rabbitmq:3.8-management-alpine   "docker-entrypoint.s…"   15 minutes ago   Up 3 minutes             4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 15691-15692/tcp, 25672/tcp   taiga-docker-taiga-async-rabbitmq-1

Thoughts so far: something is wrong with my nginx conf but not sure what. This may be causing the gunicorn workers to timeout. SMTP for email used to work but stopped.

Hi @87cb5fm0

I don’t think the problem is in nginx. I think it is when trying to send the email via SMTP that the problem appears. Can you check the taiga-aync logs? Maybe taiga can’t connect with the SMTP server.

Best regards

@david.barragan here’s the logs for the taiga async container after trying to invite a new member. It doesn’t show anything about bulk_create

[2024-03-11 18:37:20,343: INFO/MainProcess] Connected to amqp://amusement8223:**@taiga-async-rabbitmq:5672/taiga
[2024-03-11 18:37:20,401: INFO/MainProcess] mingle: searching for neighbors
[2024-03-11 18:37:21,484: INFO/MainProcess] mingle: all alone
[2024-03-11 18:37:34,700: INFO/MainProcess] Task taiga.timeline.service.push_to_timelines[162946cf-41ca-40e4-bbee-ccc89bad2c46] received
[2024-03-11 18:37:40,031: INFO/MainProcess] Task taiga.timeline.service.push_to_timelines[e3db0155-2a7f-4f0c-bf47-d7f8424dea03] received
[2024-03-11 18:37:47,426: INFO/MainProcess] Task taiga.timeline.service.push_to_timelines[63dbddee-90d6-4ce6-9e48-82bc27854db2] received
[2024-03-11 18:37:49,894: INFO/MainProcess] Task taiga.timeline.service.push_to_timelines[c5d2305a-fa62-4e9d-88ee-d05a13407779] received
[2024-03-11 18:38:32,713: INFO/MainProcess] Task taiga.timeline.service.push_to_timelines[17914068-9843-4c52-a9d8-310e8ca47415] received
[2024-03-11 18:38:56,868: INFO/Beat] Scheduler: Sending due task send-bulk-emails (taiga.projects.notifications.tasks.send_bulk_email)
[2024-03-11 18:38:56,925: INFO/MainProcess] Task taiga.projects.notifications.tasks.send_bulk_email[7fcf7ed7-a831-4406-941e-69986ee3ac4b] received

But when I check the response in dev tools to the API post it shows this:

Here’s what my .env looks like, I confirmed my smtp login works, I replaced with dummy data here, and have tried multiple accounts that work from different providers.

# Taiga's SMTP settings - Variables to send Taiga's emails to the users
EMAIL_BACKEND=smtp  # use an SMTP server or display the emails in the console (either "smtp" or "console")
EMAIL_HOST=mail.domain.org  # SMTP server address
EMAIL_PORT=465   # default SMTP port
EMAIL_HOST_USER=taiga@sub.domain.com  # user to connect the SMTP server
EMAIL_HOST_PASSWORD=strongpassword
EMAIL_DEFAULT_FROM=taiga@sub.domain.com  # default email address for the automated emails
# EMAIL_USE_TLS/EMAIL_USE_SSL are mutually exclusive (only set one of those to True)
EMAIL_USE_TLS=True  # use TLS (secure) connection with the SMTP server
EMAIL_USE_SSL=False  # use implicit TLS (secure) connection with the SMTP server

I hope this information is not useful to understand what is happening. There is something wrong with your SMTP settings and taiga throw an error every time it try to connect with the smtp server to send an email. It could be something related with EMAIL_USE_TLS and EMAIL_USE_SSL.

There’s a command to send tests emails that maybe can help us to see if there is any error in the connection process between taiga and the smtp server

./taiga-manage.sh test_emails your@email.com

Here’s the error that output after running that command:
@david.barragan

ERROR:2024-03-11 21:59:57,931: Error sending email notifications
Traceback (most recent call last):
  File "/taiga-back/taiga/projects/notifications/services.py", line 372, in send_sync_notifications
    email.send(user.email, context, headers=headers)
  File "/opt/venv/lib/python3.11/site-packages/djmail/template_mail.py", line 118, in send
    return email.send()
           ^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/core/mail/message.py", line 284, in send
    return self.get_connection(fail_silently).send_messages([self])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 102, in send_messages
    new_conn_created = self.open()
                       ^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/core/mail/backends/smtp.py", line 62, in open
    self.connection = self.connection_class(self.host, self.port, **connection_params)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/smtplib.py", line 343, in connect
    (code, msg) = self.getreply()
                  ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/smtplib.py", line 405, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed

So there is an error with the SMTP settings here. Maybe you should try to change the values for EMAIL_USE_TLS and EMAIL_USE_SSL. You can try with

EMAIL_USE_TLS=False
EMAIL_USE_SSL= True 

This is what the documentation sais about these two parameters:

MAIL_USE_TLS

Default: False

Whether to use a TLS (secure) connection when talking to the SMTP server. This is used for explicit TLS connections, generally on port 587. If you are experiencing hanging connections, see the implicit TLS setting EMAIL_USE_SSL.

EMAIL_USE_SSL

Default: False

Whether to use an implicit TLS (secure) connection when talking to the SMTP server. In most email documentation this type of TLS connection is referred to as SSL. It is generally used on port 465. If you are experiencing problems, see the explicit TLS setting EMAIL_USE_TLS.

Note that EMAIL_USE_TLS/EMAIL_USE_SSL are mutually exclusive, so only set one of those settings to True.

@david.barragan that worked changing EMAIL_USE_SSL = True

Thank you!

Great \o/

You are welcome.