0

New to Python. Im able to create script fail alerts for R files because the cmd files generate .Rout files(in this case LocationToBlockGroup.Rout) My python script-

import pandas as pd
import smtplib
from email.message import EmailMessage
import glob
import os
import shutil

df = pd.read_fwf(r'LocationToBlockGroup.Rout', header=None)
end_str = '#--- END ---'
cols_to_check = ["0"]


def email_alert(subject, body, to):
    msg = EmailMessage()
    msg.set_content(body)
    msg['subject'] = subject
    msg['to'] = to
    user = "DataScienceScriptAlerts@xx.com"
    msg['from'] = user
    server = smtplib.SMTP("smtprelay.corp.xx.com", 25)
    server.starttls()
    server.send_message(msg)
    server.quit()


src = r'C:/R'
dest = r'C:/R/Failed Scripts'
if __name__ == '__main__':
    for col in cols_to_check:
        if not df[0].str.contains(end_str).any():
            body = "The LocationToBlockGroup.R script in xx had errors on the last execution " + col + "."
            print(body)
            email_alert("LocationToBlockGrp failure alert", body, "htvldba@xx.com")
        if not df[0].str.contains(end_str).any():
            for file_path in glob.glob(os.path.join(src, '*.Rout'), recursive=True):
                new_path = os.path.join(dest, os.path.basename(file_path))
                shutil.copy(file_path, new_path) 

But how do I do this for python scripts that do not generate .Rout files?is there a python equivalent or something I need to add in the .bat script itself? I just want that if the batch file fails for a scheduled python script, it should send an email alert. This is achieved as I stated above for R files but not sure how to replicate for python files.Am i over engineering this and i can just add a simple line in my .bat file that can do the same?

Park
  • 1,924
  • 1
  • 15
  • 19
  • We can't know how those batch files are being run. In general, if you run them from Python, check the return code from `subprocess`. If you are running them in a BAT file, examine the CMD concept `errorlevel`. – tripleee May 20 '22 at 04:22
  • Tangentially, you don't need to use raw strings when your strings don't contain any backslashes. Windows helpfully allows you to use a forward slash instead of a backslash precisely so you can avoid having to backslash all backslashes or use raw strings etc. – tripleee May 20 '22 at 04:22
  • Also, code you put in `if __name__ == '__main__’:` should be trivial; the purpose of this boilerplate is to allow you to `import` the code, which you will not want to do anyway if the logic you need is not available via `import`. See also https://stackoverflow.com/a/69778466/874188 – tripleee May 20 '22 at 04:24
  • "C:\Program Files\R\R-4.0.4\bin\x64\R.exe" R CMD BATCH LocationToBlockGroup.R – Jay Janardhan May 20 '22 at 04:34
  • thats the content of the original .bat file – Jay Janardhan May 20 '22 at 04:34
  • @tripleee what can I do so that the python file also gives an ouptut file similar to .Rout so i can detect if the script ran successfully by recognizing '--END--' – Jay Janardhan May 20 '22 at 04:37
  • That's the wrong way to do it. Again, examine the `errorlevel` in the script which runs Python. Every properly written program sets its exit status to indicate whether it succeeded or not in a simple machine-readable fashion. In very brief, if `errorlevel` is 0 it succeeded, and any other value meant it failed. – tripleee May 20 '22 at 04:39
  • But we are still missing the full picture here. Which Python script where is being run how exactly? – tripleee May 20 '22 at 04:39
  • Any python script. For example lets say I have a script called Cosmos.py. This script is run from a .bat file via task scheduler using - call C:\ProgramData\Anaconda3\Scripts\activate.bat C:\ProgramData\Anaconda3\python.exe "C:\Python\Cosmos.py" . I just want that if this batch file doesnt run succesfully, I send an email alert – Jay Janardhan May 20 '22 at 04:42
  • @tripleee acc to errorlevel docs, i need to add - IF %ERRORLEVEL% NEQ 0 ( send email alert ) to myscript. But i do not know how to do that within a .bat file. how do i access the smtplib,pandas,etc from the .bat file – Jay Janardhan May 20 '22 at 04:52

1 Answers1

0

Put the email sending in a separate Python script if you want to access it from multiple flows in your batch file.

Something like this cmd file:

python script.py 
if errorlevel 1 (
  python sendemail.py "python script.py failed"
)

r script.R
if errorlevel 1 (
  python rfail.py
)

exit 0

where sendemail.py contains the email sending code, something like

from email.message import EmailMessage
import smtplib

def email_alert(subject, body, to):
    msg = EmailMessage()
    msg.set_content(body)
    msg['subject'] = subject
    msg['to'] = to
    user = "DataScienceScriptAlerts@xx.com"
    msg['from'] = user
    with smtplib.SMTP("smtprelay.corp.xx.com", 25) as server:
        server.starttls()
        server.send_message(msg)
        server.quit()

def main():
    import sys
    email_alert("htvldba@xx.com", "(no body)", sys.argv[1])

if __name__ == "__main__":
    main()

Of course, there are simpler ways to send email, but if you are on Windows, this might be one of the simpler within easy reach.

Then from rfail.py just import that code.

import pandas as pd
import glob
import os
import shutil

from sendemail import email_alert


def main():
    end_str = '#--- END ---'
    cols_to_check = ["0"]
    src = 'C:/R'
    dest = 'C:/R/Failed Scripts'

    df = pd.read_fwf(r'LocationToBlockGroup.Rout', header=None)

    for col in cols_to_check:
        if not df[0].str.contains(end_str).any():
            body = "The LocationToBlockGroup.R script in xx had errors on the last execution " + col + "."
            print(body)
            email_alert("LocationToBlockGrp failure alert",body,"htvldba@xx.com")
        if not df[0].str.contains(end_str).any():
                for file_path in glob.glob(os.path.join(src,'*.Rout'), recursive=True):
                    new_path = os.path.join(dest, os.path.basename(file_path))
                    shutil.copy(file_path, new_path) 

if __name__ == "__main__":
    main()

You will need to make sure the directory which contains sendemail.py is on your PYTHONPATH if you need to import it from a script which isn't being run from the same directory.

tripleee
  • 158,107
  • 27
  • 234
  • 292
  • Batch files are extremely blunt instruments; if Windows is not central to your use case, perhaps think about migrating to a less frustrating platform. – tripleee May 20 '22 at 07:34