{"id":465,"date":"2016-03-13T13:14:00","date_gmt":"2016-03-13T05:14:00","guid":{"rendered":"https:\/\/www.techcoil.com\/blog\/?p=465"},"modified":"2018-09-04T22:54:43","modified_gmt":"2018-09-04T14:54:43","slug":"how-to-dynamically-loop-through-lines-of-text-in-a-text-file-from-a-shell-script","status":"publish","type":"post","link":"https:\/\/www.techcoil.com\/blog\/how-to-dynamically-loop-through-lines-of-text-in-a-text-file-from-a-shell-script\/","title":{"rendered":"How to loop through lines of text in a text file from a shell script dynamically"},"content":{"rendered":"<p>There are times when we need to dynamically run shell commands based on the items listed in a text file. <\/p>\n<p>The shell script that I ran in my bash shell does the following:<br \/>\nfor each line separated by newline and not preceded by a '#' character, print the line to console. <\/p>\n<p>In order to build the shell script, I would need the following code pieces:<\/p>\n<ul>\n<li>A way to remove\/delete empty lines or leading and trailing whitespace in a text file <\/li>\n<li>A way to loop through the text file, line by line<\/li>\n<li>A way to check if the first character of the string <\/li>\n<\/ul>\n<p>This post documents what I had ran in my bash shell (in Ubuntu 14.0.4) to loop through items separated by newlines in a text file. <\/p>\n<h3>1. Using the sed command to remove\/delete empty lines or leading and trailing whitespace in a text file<\/h3>\n<p>The text files that my shell script process could include empty lines or leading and trailing whitespace that probably makes the files readable to the author. However, lines that are empty or are strings that contain only leading and trailing whitespace, could affect the commands that my shell script would run against. <\/p>\n<p>To solve this problem, we can use the <code>sed<\/code> command to remove\/delete all unnecessary whitespace from the text file:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nsed '\/^&#x5B; \\t\\r\\n]*$\/d' textFile.txt\r\n<\/pre>\n<p>In this code, I had passed two input parameters to the <code>sed<\/code> command. <\/p>\n<p>The first input consists of a regular expression and the corresponding action to perform if any line matches the regular expression. <code>\/^[ \\t\\r\\n]*$\/<\/code> is the regular expression that matches every string that contains either \\t, \\r or \\n consecutively. The <code>d<\/code> tells <code>sed<\/code> to delete every string that matches the regular expression.<\/p>\n<p>The second input is the path to my text file.<\/p>\n<h3>2. Looping through the contents in the file, line by line.<\/h3>\n<p>We can loop through the contents of file with the following shell script code:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nwhile read line; do\r\n    # do something with the 'line' variable\r\ndone \r\n<\/pre>\n<h3>3. Checking if the first character of a line is a character that is not a '#' character<\/h3>\n<p>To check if the first character of a line is not a '#', I can use the following shell script code:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nif &#x5B;&#x5B; ${line:0:1} != &quot;#&quot; ]]; then\r\n    # The first character of $line will not contain the '#' character\r\nfi\r\n<\/pre>\n<h3>Putting all parts together: the shell script code to loop through lines of text in a text file<\/h3>\n<p>Putting the main code elements together, my shell script looks like the following:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nsed '\/^&#x5B; \\t\\r\\n]*$\/d' textFile.txt | while read line; do\r\n    if &#x5B;&#x5B; ${line:0:1} != &quot;#&quot; ]]; then\r\n\t  echo &quot;$line&quot;\r\n    fi\r\ndone \r\n<\/pre>\n<p>I directed the output of the <code>sed<\/code> command to a while loop that reads content line by line. Inside the while loop, I check for the first character of a line that is not a '#' character. If the first letter of that line is not a '#' character, I use the <code>echo<\/code> command to print out the contents of the line.<\/p>\n\n      <ul id=\"social-sharing-buttons-list\">\n        <li class=\"facebook\">\n          <a href=\"https:\/\/www.facebook.com\/sharer\/sharer.php?u=https%3A%2F%2Fwp.me%2Fp245TQ-7v\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n            <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Facebook.png\" alt=\"Facebook icon\"> Share\n          <\/a>\n        <\/li>\n        <li class=\"twitter\">\n          <a href=\"https:\/\/twitter.com\/intent\/tweet?text=&url=https%3A%2F%2Fwp.me%2Fp245TQ-7v&via=Techcoil_com\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Twitter.png\" alt=\"Twitter icon\"> Tweet\n          <\/a>\n        <\/li>\n        <li class=\"linkedin\">\n          <a href=\"https:\/\/www.linkedin.com\/shareArticle?mini=1&title=&url=https%3A%2F%2Fwp.me%2Fp245TQ-7v&source=https:\/\/www.techcoil.com\" target=\"_blank\" role=\"button\" rel=\"nofollow\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/linkedin.png\" alt=\"Linkedin icon\"> Share\n          <\/a>\n        <\/li>\n        <li class=\"pinterest\">\n          <a href=\"https:\/\/pinterest.com\/pin\/create\/button\/?url=https%3A%2F%2Fwww.techcoil.com%2Fblog%2Fwp-json%2Fwp%2Fv2%2Fposts%2F465&description=\" class=\"pin-it-button\" target=\"_blank\" role=\"button\" rel=\"nofollow\" count-layout=\"horizontal\">\n          <img decoding=\"async\" src=\"\/ph\/img\/3rd-party\/social-icons\/Pinterest.png\" alt=\"Pinterest icon\"> Save\n          <\/a>\n        <\/li>\n      <\/ul>\n    ","protected":false},"excerpt":{"rendered":"<p>There are times when we need to dynamically run shell commands based on the items listed in a text file. <\/p>\n<p>The shell script that I ran in my bash shell does the following:<br \/>\nfor each line separated by newline and not preceded by a &#8216;#&#8217; character, print the line to console. <\/p>\n<p>In order to build the shell script, I would need the following code pieces:<\/p>\n<ul>\n<li>A way to remove\/delete empty lines or leading and trailing whitespace in a text file <\/li>\n<li>A way to loop through the text file, line by line<\/li>\n<li>A way to check if the first character of the string <\/li>\n<\/ul>\n<p>This post documents what I had ran in my bash shell (in Ubuntu 14.0.4) to loop through items separated by newlines in a text file. <\/p>\n","protected":false},"author":1,"featured_media":1242,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"footnotes":""},"categories":[375],"tags":[230,195],"jetpack_featured_media_url":"https:\/\/www.techcoil.com\/blog\/wp-content\/uploads\/Terminal-Shell-Logo.gif","jetpack_shortlink":"https:\/\/wp.me\/p245TQ-7v","jetpack-related-posts":[],"jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts\/465"}],"collection":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/comments?post=465"}],"version-history":[{"count":0,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/posts\/465\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/media\/1242"}],"wp:attachment":[{"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/media?parent=465"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/categories?post=465"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.techcoil.com\/blog\/wp-json\/wp\/v2\/tags?post=465"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}