Check filename format using regular expression in oozie

Question

I want to check whether a file with a specific name exists or not in the given HDFS location. My file name is something like this: TEST_20190930.csv

I did some searches on the internet, found that fs:exists can help us to check this.

I want to check in my coordinator oozie if the file exists with the mentioned name (Test_20190930) where 20190903 is a date of receiving the file on the HDFS.

do you know the exact absolute path of the file? or do you want to consider any file with that name? — Horatiu Jeflea, Sep 30 '19 at 18:56
I know the exact absolute path: /dev/datalake/app/raw/TEST_20190930.csv — HamidOvic, Oct 01 '19 at 08:43
have you tried https://stackoverflow.com/questions/39033693/how-to-check-whether-the-file-exist-in-hdfs-location-using-oozie — Horatiu Jeflea, Oct 01 '19 at 11:42

score 0 · Answer 1 · answered Oct 03 '19 at 09:28

You can use a unix script to check the file pattern as it provides more flexibility and options and call this script in oozie workflow

workflow can be something like that

<shell>
<exec> script.sh </exec>
</shell>
    <ok to="[NODE-NAME]"/>
    <error to="[NODE-NAME]"/>

while script.sh can be

 if hdfs dfs -test -e $HDFS_PATH; then
    echo "[$HDFS_PATH] exists on HDFS"
    hdfs dfs -ls $HDFS_PATH
exit 0
    else
echo "File not Found"
exit 1
fi

Check filename format using regular expression in oozie

1 Answers1