How to host your Python 3 Flask MVP with Supervisor on Ubuntu Server 16.04
Due to its minimalistic design, the Python Flask framework is ideal for building the web server layer of minimal viable products (MVP) to validate customers' needs. However, development work is just one part of the user validation efforts. To ensure that our customer can access our Flask MVP and provide feedback as and when they are available, we will need to get it running with as a server daemon.
Supervisor is a convenient tool for running applications as a server daemon.
This post documents the steps that I took to host a Python 3 Flask MVP with Supervisor on an Ubuntu Server 16.04 instance.
Creating a simple Python 3 Flask application as a proof of concept
To proof that it is possible to host a Python 3 Flask application with Supervisor on a Ubuntu Server 16.04 instance, I first created a "Hello World" application with minimal Python 3 dependencies.
Project structure
This "Hello World" application consists of a folder and three files within the folder:
mvpapp |--requirements.txt |--start.sh |--app.py
The requirements.txt
The requirements.txt is for specifying the Python 3 dependencies that our Python 3 Flask application is going to need. Since this is a minimalistic application, the requirements.txt contains only the following line:
Flask==0.12.2
Shell script to start the Python 3 Flask Application: start.sh
The shell script is intended for the Supervisor daemon to start the "Hello World" Python 3 Flask application. The shell script contains the following content:
source mvpappenv/bin/activate python app.py deactivate
The mvpappenv folder contains a virtual environment that we will created later with virtualenv. In this shell script, we first get into the virtual environment by sourcing the activate script that resides in mvpappenv/bin folder. We then use the python binary to run the Python 3 Flask Application. The deactivate command will be run when the Python 3 Flask application got terminated by whatever reason. In such a case, the executor of the shell script will get out of the virtual environment.
The Python 3 Flask source code: app.py
To test drive the approach to hosting a Python 3 Flask MVP with Supervisor on a Ubuntu Server 16.04 instance, we can simply create a Python 3 Flask application that will run a development server with a simple endpoint that returns a simple string as a response to HTTP GET requests:
from flask import Flask
app = Flask(__name__)
@app.route('/')
def mvp_hello():
return 'Hello there, I am a MVP!'
app.run(host='0.0.0.0', port=80)
Copying the project files over to the server instance
Once we had provisioned an instance of Ubuntu Server 16.04 for our minimal viable product, we can then copy over the project folder from our machine to the Ubuntu Server 16.04 instance. Assuming that we got an instance of Ubuntu Server 16.04 running at 55.12.32.45 and we are at the folder that contains the mvpapp folder, we will run the following shell command to perform the copy:
scp -r mvpapp root@55.12.32.45:/var/flaskapp/
Logging into the Ubuntu Server 16.04 instance through ssh
After we get the mvpapp folder into the Ubuntu Server 16.04 instance, we can use ssh to get into the Ubuntu Server 16.04 instance to perform the necessary configurations:
ssh root@55.12.32.45
Installing Python 3 Package Manager (pip3)
If you do not have the Python 3 Package Manager in your Ubuntu Server 16.04 instance, you can install it via the following command:
sudo apt-get install python3-pip
Installing virtualenv
If you do not have virtualenv in your Ubuntu Server 16.04 instance, you can install it via the following command:
sudo apt-get install virtualenv
Installing Supervisor
To install Supervisor, we can run the following command:
sudo apt-get install supervisor -y && sudo supervisord
After the command completes, Supervisor will be installed and started on the Ubuntu Server 16.04 instance.
Preparing the virtual environment for running our Python 3 Flask MVP
After installing Supervisor, we can then proceed to prepare the virtual environment that our Python 3 Flask MVP will need. To do so, we run the following commands:
cd /var/flaskapp/mvpapp virtualenv -p python3 mvpappenv source mvpappenv/bin/activate pip install -r requirements.txt deactivate
The above commands does the following:
- Get into the project folder of our MVP app.
- Create a virtual environment,
mvpappenvwith python3 as the python executable. - Get into the virtual environment specified by
mvpappenv. - Install the Python dependencies that are described in
requirements.txt. - Get out of the virtual environment specified by
mvpappenv.
Making start.sh executable
In order for start.sh to be executable by the Supervisor daemon, we run the following command:
sudo chmod +x /var/flaskapp/mvpapp/start.sh
Creating a Supervisor configuration file to run an instance of our Python 3 Flask application
By default, the Supervisor daemon will look inside the /etc/supervisor/conf.d folder to look for instructions to get applications running. To get the Supervisor daemon to run our Python 3 Flask MVP as its subprocess, we first point nano to a configuration file in /etc/supervisor/conf.d:
sudo nano /etc/supervisor/conf.d/mvpapp.conf
And paste in the following contents:
[program:mvp-flask-app] directory=/var/flaskapp/mvpapp command=/bin/bash -E -c ./start.sh autostart=true autorestart=true stopsignal=INT stopasgroup=true killasgroup=true
This set of configurations will ensure that our Python 3 Flask application releases binded port(s) during a supervisor restart.
After we had saved the configuration file, we can then run the following command to get the Supervisor daemon to take our program configurations:
sudo systemctl restart supervisor.service
With that, the Supervisor daemon will run our Python 3 Flask MVP as a daemon. We can use supervisorctl to check whether our Python 3 Flask MVP is running successfully. Running the following command gets us into the supervisor control panel:
sudo supervisorctl
And in the supervisor control panel, we should see something similar to the following output:
mvp-flask-app RUNNING pid 15146, uptime 0:18:29 supervisor>
Inspecting the logs generated by our Python 3 Flask MVP
We can check the logs generated by our Python 3 Flask MVP inside of the supervisor control panel. Since the Flask development server writes to standard error, we can check the logs by typing the following command inside of the supervisor control panel:
tail mvp-flask-app stderr
This command will produce the following output:
* Running on http://0.0.0.0:80/ (Press CTRL+C to quit)
Alternatively, we can inspect the logs at the default location where Supervisor daemon writes to. Since we did not specify the location where the logs for our Python 3 Flask MVP will write to inside mvpapp.conf, the Supervisor daemon will place the logs at the /var/log/supervisor folder. The naming of the logs will be similar to the following:
mvp-flask-app-stderr---supervisor-NXcYnI.logfor output to standard error.mvp-flask-app-stdout---supervisor-U6uIKG.logfor output to standard out.
Whenever we restart the Supervisor daemon, the log files will be deleted and new ones with different names will be generated.
Securing Python 3 Flask endpoints with https
If our Python 3 Flask MVP will be dealing with user input, it would be wise to secure the endpoints with https. To do so, we can follow the steps in this section.
Mapping a domain name to the public IP address of the Ubuntu Server instance running our Python 3 Flask MVP
Let's Encrypt is an amazing certificate authority who wants to give free encryption for everyone. With Let's Encrypt, we can secure our minimal viable products with SSL/TLS certificates that are certified by a certificate authority. In order to use Let's Encrypt to create free SSL/TLS certificates, we would need to map a domain name to the IP address of our Python3 Flask MVP. For this example, let's assume that I had created an A record to map mvp1.techcoil.com to 55.12.32.45.
Installing CertBot
To get our free SSL/TLS certificate from Let's Encrypt, we will need to use a ACME client. Since Let's Encrypt recommended the use of CertBot at this point of writing, we shall install CertBot on our Ubuntu Server 16.04 instance.
Following the instructions from CertBot, we will run the following commands to install CertBot on our Ubuntu Server 16.04 instance:
sudo apt-get update sudo apt-get install software-properties-common -y sudo add-apt-repository ppa:certbot/certbot sudo apt-get update sudo apt-get install certbot -y
Modifications needed at our Python 3 Flask MVP to demonstrate control over our domain
After we got CertBot installed on our Ubuntu Server 16.04 instance, we will proceed to modify our Python 3 Flask MVP to demonstrate control over our domain.
The Let's Encrypt server will look for some random file written by the ACME client at the /.well-known uri path of our domain in order to be convinced that we owned the domain that we wish to secure with https.
Hence, we will need to create an endpoint that serves static files over the /.well-known uri. The location where it reads the static files will be where CertBot writes the random file while participating in the ACME protocol.
Referencing the article on how to serve static files with Python 3 + Flask, we will modify our app.py file to contain the following codes:
import os
from flask import Flask
from flask import send_from_directory
app = Flask(__name__)
well_known_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), '.well-known')
@app.route('/')
def mvp_hello():
return 'Hello there, I am an MVP!'
@app.route('/.well-known/<path:path>')
def serve_file_for_acme(path):
print(path)
return send_from_directory(well_known_dir, path)
app.run(host='0.0.0.0', port=80)
After we had saved the changes that we made to app.py, we then restart Supervisor to make Python 3 run the our new codes:
sudo systemctl restart supervisor.service
Creating an SSL Certificate for our Python 3 Flask MVP
To create the SSL certificate for our Python 3 Flask MVP, we then run the following command:
sudo certbot certonly --webroot -w /var/flaskapp/mvpapp -d mvp1.techcoil.com
This will cause the CertBot client to create the create the ACME challenges inside the /var/flaskapp/mvpapp folder in the process of creating a SSL certificate for the mvp1.techcoil.com domain.
At the first run, the CertBot client will ask for an email address for urgent renewal and security notices, agreement to terms of services and whether we are willing to share your email address with the Electronic Frontier Foundation.
Once we had provided answers to those questions, CertBot will create the SSL artefacts inside the /etc/letsencrypt/live/mvp1.techcoil.com folder. Inside /etc/letsencrypt/live/mvp1.techcoil.com, we will also find cert.pem, chain.pem, fullchain.pem and privkey.pem. The files inside /etc/letsencrypt/live/mvp1.techcoil.com folder are symbolic links for facilitating certificate renewal without changing our server side configurations.
Creating another Python 3 Flask application for serving https traffic
After the CertBot client had created the files needed for serving https traffic, we can then proceed to create another Python 3 Flask application that speaks https. To do so, we first add the following files in the mvpapp folder:
- https-app.py
- start-https-app.sh
The Python 3 Flask source code for serving https: https-app.py
import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def mvp_hello():
return 'Hello there, I am an MVP served through https!'
app.run(host='0.0.0.0',
port=443,
ssl_context = ('/etc/letsencrypt/live/mvp1.techcoil.com/cert.pem',
'/etc/letsencrypt/live/mvp1.techcoil.com/privkey.pem'))
Similar to app.py, we create a route that returns a simple string as a response to HTTP GET requests. The difference is that we include a tuple that contains the path to the certificate and private key that CertBot had created for us earlier via the ssl_context parameter of the app.run function.
The shell script to start the Python 3 Flask application that serves https traffic: start-https-app.sh
source mvpappenv/bin/activate python https-app.py deactivate
This shell script is similar to start.sh, except that it runs https-app.py.
In order for start-ssl.sh to be executable by the Supervisor daemon. We make it executable by running the following command:
sudo chmod +x /var/flaskapp/mvpapp/start-https-app.sh
Creating a new Supervisor configuration file to run an instance of our Python 3 Flask application that speaks https
Finally we create a new Supervisor configuration file to run an instance of our Python 3 Flask application that speaks https:
[program:mvp-flask-app-with-https] directory=/var/flaskapp/mvpapp command=/bin/bash -E -c ./start-https-app.sh autostart=true autorestart=true stopsignal=INT stopasgroup=true killasgroup=true
As with the case of the first Python 3 Flask application that we had created earlier, we will save the this new Supervisor configurations file at the /etc/supervisor/conf.d folder. Let's assume that we name this file as mvpapp-with-https.conf.
Getting Supervisor daemon to start the Python 3 Flask application that serves https traffic
With the files in place, we will then run the following command to get the Supervisor daemon to start the Python 3 Flask application that serves https traffic:
sudo systemctl restart supervisor.service
Once the Supervisor daemon had restarted successfully, we can then use the supervisorctl command to check the status of the Python 3 Flask application that serves https traffic:
sudo supervisorctl
In the supervisor control panel, we will see something similar to the following output:
mvp-flask-app RUNNING pid 15146, uptime 1:25:12 mvp-flask-app-with-https RUNNING pid 16346, uptime 0:18:29 supervisor>
We can then check whether mvp-flask-app-with-https was started successfully by issuing the following command in the supervisor control panel:
tail mvp-flask-app-with-https stderr
Which will produce the following output:
* Running on http://0.0.0.0:443/ (Press CTRL+C to quit)
Share
Tweet
Share
Save
