{"id":165,"date":"2018-10-20T17:15:37","date_gmt":"2018-10-20T09:15:37","guid":{"rendered":"https:\/\/www.techcoil.com\/blog\/?p=165"},"modified":"2018-10-20T17:28:32","modified_gmt":"2018-10-20T09:28:32","slug":"how-to-use-flask-apscheduler-in-your-python-3-flask-application-to-run-multiple-tasks-in-parallel-from-a-single-http-request","status":"publish","type":"post","link":"https:\/\/www.techcoil.com\/blog\/how-to-use-flask-apscheduler-in-your-python-3-flask-application-to-run-multiple-tasks-in-parallel-from-a-single-http-request\/","title":{"rendered":"How to use Flask-APScheduler in your Python 3 Flask application to run multiple tasks in parallel, from a single HTTP request"},"content":{"rendered":"<p>When you build an API endpoint that serves <a href=\"https:\/\/www.techcoil.com\/glossary\/http-request\/\" rel=\"noopener\" target=\"_blank\">HTTP requests<\/a> to work on long-running tasks, consider using a scheduler. Instead of holding up a <a href=\"https:\/\/www.techcoil.com\/glossary\/http-client\/\" rel=\"noopener\" target=\"_blank\">HTTP client<\/a> until a task is completed, you can return an identifier for the client to query the task status later. In the meantime, your <a href=\"https:\/\/www.techcoil.com\/glossary\/http-server\/\" rel=\"noopener\" target=\"_blank\">HTTP server<\/a> can offload the task to a scheduler which will complete it and update the status.<\/p>\n<p>When you are building your HTTP server with Python 3 <a href=\"http:\/\/flask.pocoo.org\/\" rel=\"noopener\" target=\"_blank\">Flask<\/a>, <a href=\"https:\/\/github.com\/viniciuschiele\/flask-apscheduler\" rel=\"noopener\" target=\"_blank\">Flask-APScheduler<\/a> gives you the facilities to schedule tasks to be executed in the background. <\/p>\n<p>In this post, we look at how we can get <code>Flask-APScheduler<\/code> in your Python 3 Flask application to run multiple tasks in parallel, from a single HTTP request.<\/p>\n<h2>Installing Flask-APScheduler<\/h2>\n<p>In order to use <code>Flask-APScheduler<\/code>, we will need to install it into our Python environment:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\npip install Flask-APScheduler\r\n<\/pre>\n<h2>Flask-APScheduler built-in trigger types<\/h2>\n<p>Since <code>Flask-APScheduler<\/code> is based on <a href=\"https:\/\/apscheduler.readthedocs.io\/en\/latest\/\" rel=\"noopener\" target=\"_blank\"><code>APScheduler<\/code><\/a>, it comes with three built-in trigger types:<\/p>\n<ul>\n<li><strong>date<\/strong>: use when you want to run the job just once at a certain point of time<\/li>\n<li><strong>interval<\/strong>: use when you want to run the job at fixed intervals of time<\/li>\n<li><strong>cron<\/strong>: use when you want to run the job periodically at certain time(s) of day<\/li>\n<\/ul>\n<p>As shown above, the first trigger type is what we will need for run multiple tasks in parallel from a single HTTP request.<\/p>\n<h2>Example Python 3 Flask application that run multiple tasks in parallel, from a single HTTP request<\/h2>\n<p>In order to see the effects of using <code>Flask-APScheduler<\/code>, let's build a simple Flask application:<\/p>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nfrom flask import Flask\r\nfrom flask_apscheduler import APScheduler\r\n\r\nimport time\r\n\r\napp = Flask(__name__)\r\nscheduler = APScheduler()\r\nscheduler.init_app(app)\r\nscheduler.start()\r\n\r\n@app.route('\/')\r\ndef welcome():\r\n    return 'Welcome to flask_apscheduler demo', 200\r\n\r\n@app.route('\/run-tasks')\r\ndef run_tasks():\r\n    for i in range(10):\r\n        app.apscheduler.add_job(func=scheduled_task, trigger='date', args=&#x5B;i], id='j'+str(i))\r\n\r\n    return 'Scheduled several long running tasks.', 200\r\n\r\ndef scheduled_task(task_id):\r\n    for i in range(10):\r\n        time.sleep(1)\r\n        print('Task {} running iteration {}'.format(task_id, i))\r\n        \r\napp.run(host='0.0.0.0', port=12345)\r\n<\/pre>\n<p>As an illustration for using <code>Flask-APScheduler<\/code>, we create the above script that will run a web server at port <strong>12345<\/strong>. <\/p>\n<h3>Initializing Flask and APScheduler<\/h3>\n<p>After importing the dependencies that are needed, we create a <code>Flask<\/code> object and a <code>APScheduler<\/code> object. Once we had created these two objects, we use <code>scheduler.init_app(app)<\/code> to associate our <code>APScheduler<\/code> object with our Flask object. <\/p>\n<h3>Starting the APScheduler object<\/h3>\n<p>When we had associated the APScheduler object with our Flask object, we then start the <code>APScheduler<\/code> object running at the background. Given that, we can then add tasks to the <code>APScheduler<\/code> object to run our tasks later.<\/p>\n<h3>Scheduling jobs for APScheduler inside the run_tasks function<\/h3>\n<p>Next, we define two functions and decorate them with <code>@app.route<\/code>. Given that, our web server will be able to serve HTTP GET requests designated to <strong>\/<\/strong> and <strong>\/run-tasks<\/strong>.<\/p>\n<p>When a HTTP request is received at <strong>\/run-tasks<\/strong>, <code>run_tasks<\/code> will be run. In this case, we add 10 jobs that will run <code>scheduled_task<\/code> via <code>app.apscheduler.add_job<\/code> and the following keyword arguments:<\/p>\n<ul>\n<li><code>func=scheduled_task<\/code>: the function to run afterwards is <code>scheduled_task<\/code>.<\/li>\n<li><code>trigger='date'<\/code>: an indication that we want to run the task immediately afterwards, since we did not supply an input for <code>run_date<\/code>.<\/li>\n<li><code>args=[i]<\/code>: a list of arguments to pass to <code>scheduled_task<\/code> when <code>APScheduler<\/code> runs it.<\/li>\n<li><code>id='j'+str(i)<\/code>: an identifier for the job. When another job with the same identifier is added, it will be ignored by default. This is a good feature when we want to avoid running duplicate work while there is a same task that had not been completed yet.<\/li>\n<\/ul>\n<h3>Simulating long running tasks in <code>scheduled_task<\/code><\/h3>\n<p>When <code>scheduled_task<\/code> is run by <code>APScheduler<\/code>, we simply print 10 statements that are spaced with 1 second delays.<\/p>\n<h3>Starting the web server<\/h3>\n<p>Finally, at the end of the script, we start our web server through <code>app.run(host='0.0.0.0', port=12345)<\/code>.<\/p>\n<h2>Observations from running the code<\/h2>\n<p>When you run the Python script, a web server will listen at port <strong>12345<\/strong>. After that, you can then run the following command to initiate a HTTP request to run the 10 tasks:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\ncurl localhost:12345\/run-tasks\r\n<\/pre>\n<p>Once you run the command, you will notice that a <a href=\"https:\/\/www.techcoil.com\/glossary\/http-response\/\" rel=\"noopener\" target=\"_blank\">HTTP response<\/a> is immediately returned to you. When you look at the terminal that you use for starting the script, you will notice that 10 statements are printed every second. In addition to that, at each second interval, the statements are not printed in any particular order. <\/p>\n<p>A sample run of the program is as follows:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\n * Serving Flask app &quot;run_app&quot; (lazy loading)\r\n * Environment: production\r\n   WARNING: Do not use the development server in a production environment.\r\n   Use a production WSGI server instead.\r\n * Debug mode: off\r\n * Running on http:\/\/0.0.0.0:12345\/ (Press CTRL+C to quit)\r\n127.0.0.1 - - &#x5B;20\/Oct\/2018 17:11:01] &quot;GET \/run-tasks HTTP\/1.1&quot; 200 -\r\nTask 0 running iteration 0\r\nTask 1 running iteration 0\r\nTask 2 running iteration 0\r\nTask 3 running iteration 0\r\nTask 5 running iteration 0\r\nTask 6 running iteration 0\r\nTask 8 running iteration 0\r\nTask 9 running iteration 0\r\nTask 7 running iteration 0\r\nTask 4 running iteration 0\r\nTask 0 running iteration 1\r\nTask 1 running iteration 1\r\nTask 3 running iteration 1\r\nTask 2 running iteration 1\r\nTask 5 running iteration 1\r\nTask 6 running iteration 1\r\nTask 9 running iteration 1\r\nTask 4 running iteration 1\r\nTask 8 running iteration 1\r\nTask 7 running iteration 1\r\nTask 3 running iteration 2\r\nTask 0 running iteration 2\r\nTask 1 running iteration 2\r\nTask 2 running iteration 2\r\nTask 6 running iteration 2\r\nTask 9 running iteration 2\r\nTask 5 running iteration 2\r\nTask 4 running iteration 2\r\nTask 8 running iteration 2\r\nTask 7 running iteration 2\r\nTask 0 running iteration 3\r\nTask 2 running iteration 3\r\nTask 1 running iteration 3\r\nTask 3 running iteration 3\r\nTask 9 running iteration 3\r\nTask 6 running iteration 3\r\nTask 7 running iteration 3\r\nTask 8 running iteration 3\r\nTask 4 running iteration 3\r\nTask 5 running iteration 3\r\nTask 2 running iteration 4\r\nTask 1 running iteration 4\r\nTask 0 running iteration 4\r\nTask 3 running iteration 4\r\nTask 9 running iteration 4\r\nTask 8 running iteration 4\r\nTask 7 running iteration 4\r\nTask 4 running iteration 4\r\nTask 6 running iteration 4\r\nTask 5 running iteration 4\r\nTask 1 running iteration 5\r\nTask 3 running iteration 5\r\nTask 2 running iteration 5\r\nTask 0 running iteration 5\r\nTask 8 running iteration 5\r\nTask 9 running iteration 5\r\nTask 4 running iteration 5\r\nTask 7 running iteration 5\r\nTask 6 running iteration 5\r\nTask 5 running iteration 5\r\nTask 1 running iteration 6\r\nTask 3 running iteration 6\r\nTask 0 running iteration 6\r\nTask 2 running iteration 6\r\nTask 8 running iteration 6\r\nTask 9 running iteration 6\r\nTask 4 running iteration 6\r\nTask 7 running iteration 6\r\nTask 6 running iteration 6\r\nTask 5 running iteration 6\r\nTask 1 running iteration 7\r\nTask 0 running iteration 7\r\nTask 2 running iteration 7\r\nTask 3 running iteration 7\r\nTask 8 running iteration 7\r\nTask 9 running iteration 7\r\nTask 4 running iteration 7\r\nTask 7 running iteration 7\r\nTask 6 running iteration 7\r\nTask 5 running iteration 7\r\nTask 1 running iteration 8\r\nTask 0 running iteration 8\r\nTask 3 running iteration 8\r\nTask 2 running iteration 8\r\nTask 8 running iteration 8\r\nTask 4 running iteration 8\r\nTask 9 running iteration 8\r\nTask 7 running iteration 8\r\nTask 6 running iteration 8\r\nTask 5 running iteration 8\r\nTask 1 running iteration 9\r\nTask 0 running iteration 9\r\nTask 2 running iteration 9\r\nTask 3 running iteration 9\r\nTask 8 running iteration 9\r\nTask 4 running iteration 9\r\nTask 9 running iteration 9\r\nTask 6 running iteration 9\r\nTask 5 running iteration 9\r\nTask 7 running iteration 9\r\n<\/pre>\n\n      <ul id=\"social-sharing-buttons-list\">\n        <li class=\"facebook\">\n          <a href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=https%3A%2F%2Fwp.me%2Fp245TQ-2F\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n            <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Facebook.png\" alt=\"Facebook icon\"> Share\n          <\/a>\n        <\/li>\n        <li class=\"twitter\">\n          <a href=\"https:\/\/twitter.com\/intent\/tweet?text=&url=https%3A%2F%2Fwp.me%2Fp245TQ-2F&via=Techcoil_com\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Twitter.png\" alt=\"Twitter icon\"> Tweet\n          <\/a>\n        <\/li>\n        <li class=\"linkedin\">\n          <a href=\"https:\/\/www.linkedin.com\/shareArticle?mini=1&title=&url=https%3A%2F%2Fwp.me%2Fp245TQ-2F&source=https:\/\/www.techcoil.com\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/linkedin.png\" alt=\"Linkedin icon\"> Share\n          <\/a>\n        <\/li>\n        <li class=\"pinterest\">\n          <a href=\"https:\/\/pinterest.com\/pin\/create\/button\/?url=https%3A%2F%2Fwww.techcoil.com%2Fblog%2Fwp-json%2Fwp%2Fv2%2Fposts%2F165&description=\" class=\"pin-it-button\" target=\"_blank\" role=\"button\" rel=\"nofollow\" count-layout=\"horizontal\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Pinterest.png\" alt=\"Pinterest icon\"> Save\n          <\/a>\n        <\/li>\n      <\/ul>\n    ","protected":false},"excerpt":{"rendered":"<p>When you build an API endpoint that serves <a href=\"https:\/\/www.techcoil.com\/glossary\/http-request\/\" rel=\"noopener\" target=\"_blank\">HTTP requests<\/a> to work on long-running tasks, consider using a scheduler. Instead of holding up a <a href=\"https:\/\/www.techcoil.com\/glossary\/http-client\/\" rel=\"noopener\" target=\"_blank\">HTTP client<\/a> until a task is completed, you can return an identifier for the client to query the task status later. In the meantime, your <a href=\"https:\/\/www.techcoil.com\/glossary\/http-server\/\" rel=\"noopener\" target=\"_blank\">HTTP server<\/a> can offload the task to a scheduler which will complete it and update the status.<\/p>\n<p>When you are building your HTTP server with Python 3 <a href=\"http:\/\/flask.pocoo.org\/\" rel=\"noopener\" target=\"_blank\">Flask<\/a>, <a href=\"https:\/\/github.com\/viniciuschiele\/flask-apscheduler\" rel=\"noopener\" target=\"_blank\">Flask-APScheduler<\/a> gives you the facilities to schedule tasks to be executed in the background. <\/p>\n<p>In this post, we look at how we can get <code>Flask-APScheduler<\/code> to run multiple tasks in parallel, from a single HTTP request.<\/p>\n","protected":false},"author":1,"featured_media":1244,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"footnotes":""},"categories":[375],"tags":[584,583,226,233],"jetpack_featured_media_url":"https:\/\/www.techcoil.com\/blog\/wp-content\/uploads\/Python-Logo.gif","jetpack_shortlink":"https:\/\/wp.me\/p245TQ-2F","jetpack-related-posts":[],"jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts\/165"}],"collection":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/comments?post=165"}],"version-history":[{"count":0,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts\/165\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/media\/1244"}],"wp:attachment":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/media?parent=165"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/categories?post=165"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/tags?post=165"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}