Ensuring that your Supervisor subprocesses can run your Python applications properly behind your http proxy on Ubuntu Server 14.0.4

I had been using Supervisor to run my Python application for quite a while on a Ubuntu Server 14.0.4 box.

There was this OAuth feature that I had implemented on my Python Flask application to allow my users to sign in with their social account.

After completing the OAuth feature and ensuring that it worked fine on my development environment, I deployed the feature on my Ubuntu Server 14.0.4 instance.

However, my Python application encountered a HTTP request timeout error when it attempted to contact the OAuth server to authenticate my user login.

It turned out that there was no HTTP and HTTPS proxy settings available for my Python application to use when it tried to contact the OAuth server which is sitting somewhere in the Internet.

This post documents three ways which I had considered for propagating HTTP and HTTPS proxy settings to my Python application via the http_proxy and https_proxy environment variables.

Applying HTTP and HTTPS proxy settings via Supervisor init script

Since the Supervisor daemon will propagate the environment variables that it owns to the subprocesses that it manages, one way to export the http_proxy and https_proxy environment variables will be to include them in Supervisor's default init script.

To apply HTTP and HTTPS proxy settings via Supervisor's init script in my Ubuntu Server 14.0.4, I would open /etc/default/supervisor with my favourite editor:

sudo nano /etc/default/supervisor

and paste the following codes in /etc/default/supervisor:

export http_proxy='http://username:password@proxy-host:proxy-port'
export https_proxy='https://username:password@proxy-host:proxy-port'

After I save my changes, I will then restart my supervisor daemon to get it to take the HTTP and HTTPS proxy settings:

sudo service supervisor restart

Once Supervisor restarts successfully, the http_proxy and https_proxy environment variables will be visible to all its subprocesses, including my Python application.

Applying HTTP and HTTPS proxy settings via the environment global inside the subprocess configuration file

If I only want to include the HTTP and HTTPS proxy settings for a few applications, I can set the http_proxy and https_proxy variables through the environment global inside the respective subprocess configuration files.

Taking the same configuration script for ensuring that my Python Flask application releases binded port(s) during a supervisor restart as an example, I will include my http_proxy and https_proxy variables with the following configuration file:

[program:techcoil-flask-app]
directory=/opt/techcoil/flask-app
command=/bin/bash -E -c ./start.sh
autostart=true
autorestart=true
stopsignal=INT
stopasgroup=true
killasgroup=true
environment=http_proxy='http://username:password@proxy-host:proxy-port',https_proxy='https://username:password@proxy-host:proxy-port'

After saving the changes made to the configuration files of applications that I intend to configure HTTP and HTTPS proxy settings with, I will then restart the supervisor daemon to spawn the subprocesses with the new HTTP and HTTPS proxy settings:

sudo service supervisor restart

Applying HTTP and HTTPS proxy settings via a customized shell script that starts my Python application

A third way that I can use for exporting the http_proxy and https_proxy variables will be to place the export commands inside of the shell script that starts my Python application.

Since, I had already instructed my Supervisor daemon to use /bin/bash to run start.sh, I can open up start.sh:

sudo nano /opt/techcoil/flask-app/start.sh

and change its contents to:

export http_proxy='http://username:password@proxy-host:proxy-port'
export https_proxy='https://username:password@proxy-host:proxy-port'

python app.py

Similar to the previous case, I will then restart the supervisor daemon to spawn the subprocesses with the shell scripts that include the new HTTP and HTTPS proxy settings:

sudo service supervisor restart

About Clivant

Clivant a.k.a Chai Heng enjoys composing software and building systems to serve people. He owns techcoil.com and hopes that whatever he had written and built so far had benefited people. All views expressed belongs to him and are not representative of the company that he works/worked for.