{"id":272,"date":"2019-07-05T08:58:33","date_gmt":"2019-07-04T20:58:33","guid":{"rendered":"http:\/\/www.herbert.org.nz\/?p=272"},"modified":"2019-07-05T08:59:29","modified_gmt":"2019-07-04T20:59:29","slug":"ansible-and-the-lookup-file-module-removes-line-endings-from-file","status":"publish","type":"post","link":"https:\/\/www.herbert.org.nz\/?p=272","title":{"rendered":"Ansible and the lookup file module removes line endings from file"},"content":{"rendered":"<p>The <a href=\"https:\/\/docs.ansible.com\/ansible\/latest\/plugins\/lookup\/file.html\">lookup<\/a>\u00a0file module is useful for reading in local files (not remote ones) and saving the contents into a variable. One gotcha that caught me recently was the default action to run <a href=\"https:\/\/docs.python.org\/3\/library\/stdtypes.html#str.rstrip\">rstrip()<\/a>\u00a0which according to the documentation removes whitespace from the end of the looked up file. What it doesn&#8217;t mention is that is also removes the trailing newline &#8216;\\n&#8217; character from the file. Normally this isn&#8217;t a problem except I was using it to read in a ssl certificate and send it to a <a href=\"https:\/\/github.com\/fluent\/fluentd\">fluentd<\/a> server which then wouldn&#8217;t complete it&#8217;s startup and listen on ports &#8211; it just hung. Checking the certificate with openssl showed that at least openssl thought the certificate file was valid, and so did ruby (tested with\u00a0OpenSSL::PKey::RSA.new() and File.read()). Turns out <a href=\"https:\/\/github.com\/fluent\/fluentd\">fluentd<\/a> specifically checked for the <a href=\"https:\/\/github.com\/fluent\/fluentd\/issues\/2458\">newline character at the end of the ssl certificate file<\/a>.<\/p>\n<p>So until we upgrade to the very latest version of fluentd with this fix, we need to run the lookup module with the rstrip=False parameter like so:<\/p>\n<blockquote>\n<pre><span class=\"nv\">lookup<\/span><span class=\"o\">(<\/span><span class=\"s1\">'file'<\/span><span class=\"o\">,<\/span> <span class=\"s1\">'\/etc\/foo.txt', rstrip=False<\/span><span class=\"o\">)<\/span><\/pre>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>The lookup\u00a0file module is useful for reading in local files (not remote ones) and saving the contents into a variable. One gotcha that caught me recently was the default action to run rstrip()\u00a0which according to the documentation removes whitespace from &hellip; <a href=\"https:\/\/www.herbert.org.nz\/?p=272\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[14],"tags":[],"class_list":["post-272","post","type-post","status-publish","format-standard","hentry","category-ansible"],"_links":{"self":[{"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/posts\/272","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=272"}],"version-history":[{"count":1,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/posts\/272\/revisions"}],"predecessor-version":[{"id":273,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=\/wp\/v2\/posts\/272\/revisions\/273"}],"wp:attachment":[{"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=272"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=272"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.herbert.org.nz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=272"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}