@phi1ipp Fixed it! It was my server config. Just for future reference for anyone that might have the same problem.
Changed the settings to this instead:
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://192.168.43.31:5000;