{"id":495,"date":"2016-05-15T22:20:20","date_gmt":"2016-05-15T14:20:20","guid":{"rendered":"https:\/\/www.techcoil.com\/blog\/?p=495"},"modified":"2018-09-04T23:41:33","modified_gmt":"2018-09-04T15:41:33","slug":"ensuring-that-your-supervisor-subprocesses-can-run-your-python-applications-properly-behind-your-http-proxy-on-ubuntu-server-14-0-4","status":"publish","type":"post","link":"https:\/\/www.techcoil.com\/blog\/ensuring-that-your-supervisor-subprocesses-can-run-your-python-applications-properly-behind-your-http-proxy-on-ubuntu-server-14-0-4\/","title":{"rendered":"Ensuring that your Supervisor subprocesses can run your Python applications properly behind your http proxy on Ubuntu Server 14.0.4"},"content":{"rendered":"

I had been using Supervisor<\/a> to run my Python application for quite a while on a Ubuntu Server 14.0.4 box.<\/p>\n

There was this OAuth feature that I had implemented on my Python Flask application to allow my users to sign in with their social account.<\/p>\n

After completing the OAuth feature and ensuring that it worked fine on my development environment, I deployed the feature on my Ubuntu Server 14.0.4 instance.<\/p>\n

However, my Python application encountered a HTTP request timeout error when it attempted to contact the OAuth server to authenticate my user login.<\/p>\n

It turned out that there was no HTTP and HTTPS proxy settings available for my Python application to use when it tried to contact the OAuth server which is sitting somewhere in the Internet.<\/p>\n

This post documents three ways which I had considered for propagating HTTP and HTTPS proxy settings to my Python application via the http_proxy<\/code> and https_proxy<\/code> environment variables.<\/p>\n

Applying HTTP and HTTPS proxy settings via Supervisor init script<\/h2>\n

Since the Supervisor daemon will propagate the environment variables that it owns to the subprocesses that it manages, one way to export the http_proxy<\/code> and https_proxy<\/code> environment variables will be to include them in Supervisor's default init script.<\/p>\n

To apply HTTP and HTTPS proxy settings via Supervisor's init script in my Ubuntu Server 14.0.4, I would open \/etc\/default\/supervisor<\/code> with my favourite editor:<\/p>\n

\r\nsudo nano \/etc\/default\/supervisor\r\n<\/pre>\n
and paste the following codes in \/etc\/default\/supervisor<\/code>:<\/p>\n
\r\nexport http_proxy='http:\/\/username:password@proxy-host:proxy-port'\r\nexport https_proxy='https:\/\/username:password@proxy-host:proxy-port'\r\n<\/pre>\n
After I save my changes, I will then restart my supervisor daemon to get it to take the HTTP and HTTPS proxy settings:<\/p>\n
\r\nsudo service supervisor restart\r\n<\/pre>\n
Once Supervisor restarts successfully, the http_proxy<\/code> and https_proxy<\/code> environment variables will be visible to all its subprocesses, including my Python application.<\/p>\n
Applying HTTP and HTTPS proxy settings via the environment<\/code> global inside the subprocess configuration file<\/h2>\n
If I only want to include the HTTP and HTTPS proxy settings for a few applications, I can set the http_proxy<\/code> and https_proxy<\/code> variables through the environment<\/code> global inside the respective subprocess configuration files.<\/p>\n
Taking the same configuration script for ensuring that my Python Flask application releases binded port(s) during a supervisor restart<\/a> as an example, I will include my http_proxy<\/code> and https_proxy<\/code> variables with the following configuration file:<\/p>\n
\r\n[program:techcoil-flask-app]\r\ndirectory=\/opt\/techcoil\/flask-app\r\ncommand=\/bin\/bash -E -c .\/start.sh\r\nautostart=true\r\nautorestart=true\r\nstopsignal=INT\r\nstopasgroup=true\r\nkillasgroup=true\r\nenvironment=http_proxy='http:\/\/username:password@proxy-host:proxy-port',https_proxy='https:\/\/username:password@proxy-host:proxy-port'\r\n<\/pre>\n
After saving the changes made to the configuration files of applications that I intend to configure HTTP and HTTPS proxy settings with, I will then restart the supervisor daemon to spawn the subprocesses with the new HTTP and HTTPS proxy settings:<\/p>\n
\r\nsudo service supervisor restart\r\n<\/pre>\n
Applying HTTP and HTTPS proxy settings via a customized shell script that starts my Python application<\/h2>\n
A third way that I can use for exporting the http_proxy<\/code> and https_proxy<\/code> variables will be to place the export commands inside of the shell script that starts my Python application.<\/p>\n
Since, I had already instructed my Supervisor daemon to use \/bin\/bash<\/code> to run start.sh<\/code>, I can open up start.sh<\/code>:<\/p>\n
\r\nsudo nano \/opt\/techcoil\/flask-app\/start.sh\r\n<\/pre>\n
and change its contents to:<\/p>\n
\r\nexport http_proxy='http:\/\/username:password@proxy-host:proxy-port'\r\nexport https_proxy='https:\/\/username:password@proxy-host:proxy-port'\r\n\r\npython app.py\r\n<\/pre>\n
Similar to the previous case, I will then restart the supervisor daemon to spawn the subprocesses with the shell scripts that include the new HTTP and HTTPS proxy settings:<\/p>\n
\r\nsudo service supervisor restart\r\n<\/pre>\n\n