{"id":113,"date":"2016-06-22T11:52:04","date_gmt":"2016-06-22T15:52:04","guid":{"rendered":"http:\/\/williamkennerly.com\/blog\/?p=113"},"modified":"2016-06-26T23:19:18","modified_gmt":"2016-06-27T03:19:18","slug":"python-dealing-with-incomplete-files","status":"publish","type":"post","link":"http:\/\/williamkennerly.com\/blog\/python-dealing-with-incomplete-files\/","title":{"rendered":"Python: Error Handling"},"content":{"rendered":"<p>One part of my updates to my Python script has been to manage what the script does in the case of incomplete output files. I have developed ways of circumventing a couple of issues, which include missing or different nstates values and incomplete files.<\/p>\n<p>To get the number of states, I use a function to find the route line, and then I use a regular expression to extract the nstates value. The function that I use to find the route line can be seen in <a href=\"http:\/\/williamkennerly.com\/blog\/python-single-point-geometry-energy-script-update\/\">a previous post<\/a>.<\/p>\n<p>I use a simple regular expression to find the nstates value from the route line: <code>nstatesRegex = re.compile(r\"nstates=(\\d*)\")<\/code><\/p>\n<p>An example of a route line, which the <code>find_parameters<\/code> function extracts, is as follows: # td=(nstates=6) cam-b3lyp\/6-31+g(d) geom=connectivity<\/p>\n<p>I use <code>nstatesRegex<\/code> to get the nstates value from that line, but sometimes the nstates value does not appear in the route line. My script does not work properly if the nstates is not specified in the route line of an output file. For now, my code just prints a message specifying any files that do not include an nstates value in the route line.<\/p>\n<p>If an nstates value is different from the other nstates values, I need to compensate for missing values in the list that contains all the relevant data. I first loop through all the files to get the <code>largest_nstates<\/code> value. In the second loop of all the files in which I append the relevant data to the <code>master_results<\/code> list, I do the following after I append the energy values:<\/p>\n<p><code>for x in xrange(nstates, largest_nstates):<br \/>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EE_lst.append(\" \")<br \/>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;abs_EE_lst.append(\" \")<br \/>\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;osc_lst.append(\" \")<\/code><\/p>\n<p>The empty spaces account for empty cells, where the output file expects energy values. I extend my <code>master_results<\/code> list with those three lists. The empty spaces help each of these lists be the same length as the largest lists I find in the directory, so that the lists are a uniform length and can be iterated through more easily.<\/p>\n<p>Sometimes a user may have incomplete output files in the directory. An easy way to verify that an output file is complete is to check if it has a job time. This method is imperfect, since I have recently encountered an error in Gaussian itself of an incomplete file that nonetheless included its job time up until it crashed. Checking the job time will at least ensure that all files have run to completion, whether or not they failed in Gaussian. On the latest version, my script fails when it tries to read an incomplete file, but it will print a message that tells the user which files did not have job times, so that the user knows which files to remove from the directory.<\/p>\n<p>There are several updates that I can make on this script to help it deal with incomplete files. I could have it skip incomplete files (such as those without job times) instead of just printing a message about which files are incomplete, so that the script still reads all the complete files. Although I have made progress in dealing with files that have different numbers of states, I could set the default to 3 states (if that is indeed the default), when nstates is not specified in the route line. I could also have my script check for Gaussian error messages and inform the user of the error message and the pertinent file. These updates are meant to help the user more quickly identify problems and spend less time looking at the script and various output files to determine what went wrong in a failed run.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One part of my updates to my Python script has been to manage what the script does in the case of incomplete output files. I have developed ways of circumventing a couple of issues, which include missing or different nstates &hellip; <a href=\"http:\/\/williamkennerly.com\/blog\/python-dealing-with-incomplete-files\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[13,14],"class_list":["post-113","post","type-post","status-publish","format-standard","hentry","category-programming","tag-gaussian","tag-python"],"_links":{"self":[{"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/posts\/113","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/comments?post=113"}],"version-history":[{"count":10,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/posts\/113\/revisions"}],"predecessor-version":[{"id":165,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/posts\/113\/revisions\/165"}],"wp:attachment":[{"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/media?parent=113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/categories?post=113"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/williamkennerly.com\/blog\/wp-json\/wp\/v2\/tags?post=113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}