Create project failed

Troubleshooting Self hosted Taiga
1

I have recently set up Taiga on a self hosted kubernetes cluster and am having a few snags getting things working.

The immediate problem is

POST http://192.168.1.179:8000/api/v1/projects net::ERR_FAILED 500 (Internal Server Error)

Other api calls are working.

When I check the logs for the backend I see no 500 error logged, so the message on the front end which reads

Something happened and the Taiga has captured the error to be able to work on it.

Leaves me wondering where Taiga has captured the error. If I could find the error log I guess I could stand a chance debugging.

Also I note that I need to run 2 rabbit-mq instances, yet running them in the same pod gives port clashes and an error that basically says “Rabbit is already running” when it tries to create the second rabbit-mq instance.

So my question is as follows:

Where is the 500 log stored?
Will the fact that I am only running one rabbit-mq instance (for events) cause this error?

2

Hi there,

We are not familiar at all with Kubernetes, and it is not a supported way to deploy Taiga.

That said, the log should either be on taiga-back, or in your ingress/reverse proxy.

Having only one rabbitmq instance could very well be the cause of the error, not sure how could you solve that from the Kubernetes side, but you can try setting the same host for both async and events rabbitmq using the environment variables, and only create one rabbitmq.

Hope this helps!

3

Thank you for your response. In a similar vain to your suggestion to try using one rabbit instance (with the emphasis on the word try :wink: ) can I ask what would be the minimum set of containers to run a very basic self hosted version of Taiga? To better define “minimum” I am not interested in smtp, public registration or 3rd party integrations. I just want to use the kanban features as a single user. Because I have a simple kubernetes cluster for my other apps (I am a startup/side hustle running this on a fleet of raspberry PIs) I would prefer to integrate this into the cluster.

4

So I set up a pod with the following services -

taiga-back - port 8000
taiga-async
taiga-gateway - port 9000 (exposed on 192.168.1.179)
taiga-events - port 8888
taiga-front - port 80 (exposed on 192.168.1.179)
rabbitmq - port 5762 (exposed on 192.168.1.179)

When I open the front end I still get an error

“Something has happened and the Taiga has captured the error to be able to work on it”

If I then look in the developer tools (F12) in the browser I can see a number of errors

Leaving the CORS errors to one side as I think I can troubleshoot those on my own, the events container seems to be having some problems.

In the browser I get the following error when the front end is trying to access the service

WebSocket connection to 'ws://192.168.1.179:9000/events' failed: app.js:3322

The offending line of code at app.js:3322 reads

this.ws = new this.win.WebSocket(url);

and putting a breakpoint on this line and refreshing the page I can see the url is

ws://192.168.1.179:9000/events

Starting Taiga events

> taigaio-events@6.8.0 start:production
> NODE_ENV=production node ./src/index.js

{"message":"WS server is started","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:02:27"}
{"message":"Liveness / Readiness server was started in http://localhost:3023/healthz","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:02:27"}
{"message":"ws-connection-open: 58ab1275-843d-47fa-a54e-faa57b7d163a","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:03:42"}
{"message":"auth: {\"cmd\":\"auth\",\"data\":{\"token\":\"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNzE5MDM3NTA5LCJqdGkiOiJiOTAxNzU2OTczZTg0MDc4OWY1YTlhNmI4MDU3ZmVjYiIsInVzZXJfaWQiOjV9.BBLMYwl8Cufnr3kQrfwzStElr1r6RVeYeCMxvexZ9tM\",\"sessionId\":\"5f191fbc8a06f6765d089da2caf654b1978507f6\"}}","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:03:43"}
{"message":"subscribe: {\"cmd\":\"subscribe\",\"routing_key\":\"notifications\"}","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:03:43"}
{"message":"subscribe: {\"cmd\":\"subscribe\",\"routing_key\":\"web_notifications.5\"}","level":"info","service":"taiga-events","timestamp":"2024-06-21 07:03:43"}
{"message":"Unhandled Rejection: \"Error: Expected ConnectionOpenOk; got <ConnectionClose channel:0>\"","level":"error","service":"taiga-events","timestamp":"2024-06-21 07:03:43"}

I am not able to proceed with troubleshooting this further without some guidance. Please can you help?

5

So this specific error “Expected ConnectionOpenOk…” is caused by the taiga-events container not being able to connect to rabbitmq. To make life easier I opened port 15672 on the loadbalance so I could see the rabbit-mq dashboard.

Then I noticed the vhost for rabbit was set to /taiga and not taiga. Getting the vhost set correctly and then setting the environment variable RABBITMQ_URL to “amqp://rabbit:rabbit@localhost:5672/taiga” for the taiga-events container did the trick. NB I am using localhost here because I am trying to run everything in the same pod (with only one instance of rabbitmq)

Then I had rather overzealously added CORS headers to my gateway config by adding

add_header Access-Control-Allow-Origin *;

to each location block. This resulted in an error coming back that the CORS had 2 entries “,”, so my thinking is the back end is also adding the CORS header.

Anywho, removing this did the trick and the CORS errors are silenced (in the most part).

I am able to visit the new project page without error then when I fill it out and hit the button to create a new project I get this

Access to XMLHttpRequest at 'http://192.168.1.179:9000/api/v1/projects' from origin 'http://192.168.1.179' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

So I shelled into the gateway container and curled the upstream server on

curl localhost:8000/api/v1/projects

and I get a json response back with an HTTP 200.

So it looks to me like the backend for some reason has a CORS problem, albeit an intermittent one.

On the POST to this endpoint in the gateway container I get this

[22/Jun/2024:11:07:44 +0000] "POST /api/v1/projects HTTP/1.1" 500 141 "http://192.168.1.179/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36" "-"

So you can see the taiga-back container is returning a 500.

But checking the logs for taiga-back I see no 500 error

Starting Taiga API...
[2024-06-22 11:06:45 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-06-22 11:06:45 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2024-06-22 11:06:45 +0000] [1] [INFO] Using worker: sync
[2024-06-22 11:06:45 +0000] [33] [INFO] Booting worker with pid: 33
[2024-06-22 11:06:45 +0000] [34] [INFO] Booting worker with pid: 34
[2024-06-22 11:06:45 +0000] [35] [INFO] Booting worker with pid: 35

So again I am at a loss as to how to troubleshoot this as it has effectively swallowed this error and not logged what went wrong.

Please help!

6

Additionally I shelled into the gateway container to attempt to send the POST request on there to see if I could see what was happening.

curl -X POST localhost:8000/api/v1/projects -H "Content-Type: application/json" -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiZXhwIjoxNzE
5MTM4ODQ2LCJqdGkiOiIzMjJhMmYzMDMzMDQ0ZTcwOWNkZmNmYzEyYjAwZDJkZSIsInVzZXJfaWQiOjV9.telbPTVVBI_nK4rdB1UX_MobvB0AugyZRvtoTVTGHqg" -d '{ "is_private": true, "creation_template": 2, "name": "my-project", "description": "My Project" }'

Response?

<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>

Logs on taiga-back?

[2024-06-22 11:06:45 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2024-06-22 11:06:45 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2024-06-22 11:06:45 +0000] [1] [INFO] Using worker: sync
[2024-06-22 11:06:45 +0000] [33] [INFO] Booting worker with pid: 33
[2024-06-22 11:06:45 +0000] [34] [INFO] Booting worker with pid: 34
[2024-06-22 11:06:45 +0000] [35] [INFO] Booting worker with pid: 35

Again, where is this 500 error being logged?

7

We were facing quite a lot of issues / similar issues as well. Even after ignoring 5xx errors, it was just simply impossible to use (roles could not be created).

Its a very promising project, but felt buggy / incomplete. Hoping it will make a comeback though still :muscle: