How to download a file via HTTP POST and HTTP GET with Python 3 requests library

When you are building a HTTP client with Python 3, you could be coding it to upload a file to a HTTP server or download a file from a HTTP server.

Previously, we discussed how to upload a file and some data through HTTP multipart in Python 3 using the requests library. In this post, let's see how we can download a file via HTTP POST and HTTP GET.

Downloading a file from a HTTP server endpoint via HTTP GET

Generally, downloading a file from a HTTP server endpoint via HTTP GET consists of the following steps:

1. Construct the HTTP GET request to send to the HTTP server.
2. Send the HTTP request and receive the HTTP Response from the HTTP server.
3. Save the contents of the file from HTTP Response to a local file.

Python 3 function that downloads a file from a HTTP server endpoint via HTTP GET

Given these points, we can create a Python 3 function that downloads a file from a HTTP server endpoint via HTTP GET:

import requests

def download_file_from_server_endpoint(server_endpoint, local_file_path):

    # Send HTTP GET request to server and attempt to receive a response
    response = requests.get(server_endpoint)
    
    # If the HTTP GET request can be served
    if response.status_code == 200:
        
        # Write the file contents in the response to a file specified by local_file_path
        with open(local_file_path, 'wb') as local_file:
            for chunk in response.iter_content(chunk_size=128):
                local_file.write(chunk)

As shown above, the download_file_from_server_endpoint function expects a server endpoint and a local file path as input. After the function is called, it first uses requests.get to get a Response object.

When the status code in the HTTP response is a 200 OK, a file handler is created to write binary data to the path specified by local_file_path.

Inside the with statement, data is read from the HTTP response in 128-byte chunks and written to local_file.

Using download_file_from_server_endpoint to download a file from a HTTP server endpoint via HTTP GET

After we had defined the download_file_from_server_endpoint function, we can then use it to download a file from a HTTP server endpoint via HTTP GET. For example, if we want to download Techcoil's logo, we can call the download_file_from_server_endpoint function as follows:

download_file_from_server_endpoint('https://www.techcoil.com/ph/img/logo.png', 'logo.png')

After the function call completes, you should find Techcoil's logo in the same directory as the Python script if your computer is connected to the Internet.

Downloading a file HTTP from a HTTP server endpoint via HTTP POST

There could be cases where the client need to supply some information to the HTTP server in order to download a file. In such a situation, HTTP POST is a more appropriate HTTP method to use for downloading the file.

As with HTTP GET, downloading of a file from the HTTP server via HTTP POST consists of the following steps:

1. Construct the HTTP POST request to send to the HTTP server.
2. Send the HTTP request and receive the HTTP Response from the HTTP server.
3. Save the contents of the file from HTTP Response to a local file.

Python 3 function that downloads a file from a HTTP server endpoint via HTTP POST

Given these points, we can create a Python 3 function that downloads a file from a HTTP server endpoint via HTTP POST:

def download_file_from_server_endpoint(server_endpoint, local_file_path, data_dict):

    # Send HTTP GET request to server and attempt to receive a response
    response = requests.post(url=server_endpoint, data=data_dict)

    # If the HTTP GET request can be served
    if response.status_code == 200:

        # Write the file contents in the response to a file specified by local_file_path
        with open(local_file_path, 'wb') as local_file:
            for chunk in response.iter_content(chunk_size=128):
                local_file.write(chunk)

As shown above, the download_file_from_server_endpoint function now accepts a server endpoint, a local file path, and a data dictionary.

In this case, we had used requests.post instead of requests.get to get a Response object. In addition, we had supplied server_endpoint as the url and data_dict as the POST variables for requests.post to construct and send the HTTP request to the server for a HTTP response.

After we had gotten the Response object, we save the file contents in the response just like the function that downloads the file via HTTP GET.

Using download_file_from_server_endpoint to download a file from a HTTP server endpoint via HTTP POST

In order to visualise how download_file_from_server_endpoint works, we can reuse the server backend from the proof of concept that we can use jQuery to push a dynamically generated file to the web browser based on the user's input.

Given that, we can create the following Python 3 code to use download_file_from_server_endpoint to download a file from a HTTP server endpoint via HTTP POST:

download_file_from_server_endpoint('https://www.techcoil.com/process/proof-of-concepts/userNameAndLuckyNumberTextFileGeneration'
                                   , 'number.txt', {'visitorName': 'John Doe', 'luckyNumber': 1234})

After the function call completes, you should find number.txt in the same directory as the python script if your computer is connected to the Internet. In addition, number.txt should contain the following content:

Hi John Doe. 
You have drawn the number 1234.

• Share
• Tweet
• Share
• Save