crashLoopBackOff Error with Taiga-Back Deployment in Kubernetes (RKE2/Rancher)

Troubleshooting
1

Hello,

I’m experiencing a persistent issue with deploying Taiga in a Kubernetes environment, specifically with the Taiga-Back service entering a CrashLoopBackOff state. I’ve successfully deployed other components (Taiga-Front, Taiga-Protected, Taiga-Events, PostgreSQL, and RabbitMQ) and they are all running fine. However, the Taiga-Back service continually crashes despite numerous attempts to resolve the issue.

Steps Taken:

  1. Generated Docker images using a custom Docker Compose setup and pushed them to our private Harbor registry.
  2. Configured and deployed all services in RKE2/Rancher, ensuring all environment variables were correctly set up, including the crucial SECRET_KEY.
  3. Ensured all dependent services like PostgreSQL and RabbitMQ are operational and accessible.

Error Log from Taiga-Back Pod:

kubectl logs taiga-back-755f9745d8-zt54f -n taiga-hml
Executing pending migrations
Traceback (most recent call last):
  File "/taiga-back/manage.py", line 17, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 395, in execute
    django.setup()
  File "/opt/venv/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/opt/venv/lib/python3.11/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/apps/config.py", line 224, in create
    import_module(entry)
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/taiga-back/taiga/base/api/__init__.py", line 43, in <module>
    from .viewsets import ModelListViewSet
  File "/taiga-back/taiga/base/api/viewsets.py", line 38, in <module>
    from . import views
  File "/taiga-back/taiga/base/api/views.py", line 124, in <module>
    class APIView(View):
  File "/taiga-back/taiga/base/api/views.py", line 128, in APIView
    authentication_classes = api_settings.DEFAULT_AUTHENTICATION_CLASSES
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/taiga-back/taiga/base/api/settings.py", line 230, in __getattr__
    val = perform_import(val, attr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/taiga-back/taiga/base/api/settings.py", line 182, in perform_import
    return [import_from_string(item, setting_name) for item in val]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/taiga-back/taiga/base/api/settings.py", line 182, in <listcomp>
    return [import_from_string(item, setting_name) for item in val]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/taiga-back/taiga/base/api/settings.py", line 194, in import_from_string
    module = importlib.import_module(module_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/taiga-back/taiga/auth/authentication.py", line 37, in <module>
    from .settings import api_settings
  File "/taiga-back/taiga/auth/settings.py", line 270, in <module>
    'SIGNING_KEY': settings.SECRET_KEY,
                   ^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/conf/__init__.py", line 90, in __getattr__
    raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.")
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty.

This error persists even though the SECRET_KEY is explicitly defined in the deployment YAML. Here is a snippet of the environment variables from the deployment manifest:

  • name: SECRET_KEY
    value: “verysecretkey12345@”
  • name: POSTGRES_DB
    value: “taiga”
  • name: POSTGRES_USER
    value: “taigauser”

I’ve checked for typos and formatting errors in the YAML and environment variable definitions but haven’t identified any issues that could lead to this behavior.

apiVersion: apps/v1
kind: Deployment
...
spec:
  containers:
    - env:
        - name: SECRET_KEY
          value: "verysecretkey12345@"
        - name: POSTGRES_HOST
          value: "taiga-db"
...

Could someone help diagnose the potential misconfiguration or suggest any additional steps I might take to resolve this issue? Any insights or similar experiences would be greatly appreciated.

Thank you!