{"id":2032,"date":"2020-05-15T19:16:46","date_gmt":"2020-05-15T11:16:46","guid":{"rendered":"https:\/\/www.techcoil.com\/blog\/?p=2032"},"modified":"2020-05-15T14:11:08","modified_gmt":"2020-05-15T06:11:08","slug":"how-to-use-threading-condition-to-wait-for-several-flask-apscheduler-one-off-jobs-to-complete-execution-in-your-python-3-application","status":"publish","type":"post","link":"https:\/\/www.techcoil.com\/blog\/how-to-use-threading-condition-to-wait-for-several-flask-apscheduler-one-off-jobs-to-complete-execution-in-your-python-3-application\/","title":{"rendered":"How to use threading.Condition to wait for several Flask-APScheduler one-off jobs to complete execution in your Python 3 application"},"content":{"rendered":"

Previously, I discussed how to use Flask-APScheduler in your Python 3 Flask application to run multiple tasks in parallel, from a single HTTP request<\/a>.<\/p>\n

When we run jobs as discussed in that post, jobs are ran once by the underlying ApScheduler<\/a> instance. In addition, our Flask<\/a> endpoint return the HTTP response<\/a> back to the HTTP client<\/a> as soon as the jobs are scheduled. <\/p>\n

If we do not want the HTTP client to know the outcome of the jobs within that HTTP call, then we are good. But what if we want to include any errors that the jobs encounter in the same HTTP response? <\/p>\n

In such a situation, we will need a mechanism to wait for the one-off jobs to complete execution before<\/strong> returning that response.<\/p>\n

Given that in mind, this post shows how we can use threading.Condition<\/code> to wait for several Flask-APScheduler<\/code> one-off jobs to complete execution.<\/p>\n

What is threading.Condition<\/code> used for?<\/h2>\n

In case you are wondering, an instance of threading.Condition<\/code><\/a> can help us synchronize activities between different threads. Once we have an instance of threading.Condition<\/code>, we can use its wait<\/code><\/a>, notify<\/code><\/a> and notify_all<\/code><\/a> methods to synchronize activities between different threads. <\/p>\n

If we want the thread holding the lock to release the lock and hold further execution, then we call the wait<\/code> method. When we want a waiting thread to resume operation, we call the notify<\/code> method. If we want all waiting threads to resume operations, then we call the notify_all<\/code> method.<\/p>\n

Creating the JobsSynchronizer<\/code> class to help the Flask endpoint wait for jobs to complete execution before returning a HTTP response<\/h2>\n

Since threading.Condition<\/code> provides us with the ability to synchronize activities between different threads, we can build a helper class to serve our needs:<\/p>\n

\r\nimport threading\r\n\r\nclass JobsSynchronizer:\r\n\r\n    def __init__(self, num_tasks_to_complete):\r\n        self.condition = threading.Condition()\r\n        self.current_completed = 0\r\n        self.status_list = []\r\n        self.num_tasks_to_complete = num_tasks_to_complete\r\n\r\n    def notify_task_completion(self, status_to_report=None):\r\n        with (self.condition):\r\n            self.current_completed = self.current_completed + 1\r\n            if status_to_report is not None:\r\n                self.status_list.append(status_to_report)\r\n            # Notify waiting thread\r\n            if self.current_completed == self.num_tasks_to_complete:\r\n                self.condition.notify()\r\n\r\n    def wait_for_tasks_to_be_completed(self):\r\n        with(self.condition):\r\n            self.condition.wait()\r\n\r\n    def get_status_list(self):\r\n        return self.status_list\r\n<\/pre>\n
Before we define the JobsSynchronizer<\/code> class, we import the threading<\/code><\/a> module into our script. When we do so, we can create an instance of threading.Condition<\/code> later.<\/p>\n
__init__<\/code> method<\/h3>\n
Next, we define a constructor for the JobsSynchronizer<\/code> class which takes a total number of jobs to wait for, num_tasks_to_complete<\/code>, as input. <\/p>\n
When an instance of JobsSynchronizer<\/code> is created, we perform the following actions inside the default constructor:<\/p>\n