Problems with integrating Python script in a Bash script Git pre-receive hook

Question

We have a pre-receive hook script in a bash shell script that enforces commits with messages having the right ticket number in a pre-defined format. We also have a separate python script that can be called in the pre-receive hook bash script to call PMD to statically analyze source code in a branch that developers push to the target remote branch.

Prior to now, we have been using the pre-receive bash script for validating our development process with respect to our Gitlab server issue tickets.

I have recently attempted to call the python script in the bash script in such a way that our ticketing and milestone validations are done before calling up PMD to expose source code that does not meet the standard. This did not work in that order, so I decided to call the python script to run PMD before the ticket validations. This also resulted in only the python script running alone without calling the rest of the bash script code.

I will appreciate any help. Please kindly find the source code of the pre-receive bash script for when I tried calling python/PMD first. Kindly point me to how I can call the python last after the ticket validations.

#!/bin/bash
#
# pre-receive hook for Commit Check
#
COMPANY_EMAIL="mycorp.org"

readonly PROGNAME=$(basename $0)
readonly PROGDIR=$(readlink -m $(dirname $0))
IS_MERGE=0

check_single_commit()
{
  COMMIT_CHECK_STATUS=1
    echo "Repo >> $REPOSITORY_BASENAME"

    if [[ "$COMMIT_MESSAGE" == "Merge branch"* ]] || [[ "$COMMIT_MESSAGE" == "WIP MI"* ]]; then
      COMMIT_CHECK_STATUS=0
      IS_MERGE=1
    else
    workFlowResult=`java -jar -Dspring.config.location=/home/gitlab/gitlab_custom_hooks/application.properties /home/gitlab/gitlab_custom_hooks/gitlab-tool.jar -prercv "$COMMIT_AUTHOR" "$COMMIT_MESSAGE" "$REPOSITORY_BASENAME"`
    echo "COMMIT_AUTHOR=$COMMIT_AUTHOR, COMMIT_MESSAGE=$COMMIT_MESSAGE, REPOSITORY_BASE=$REPOSITORY_BASENAME"

      echo " >>>>>>>>>>>>>>>>> $workFlowResult >>>>>>>>>>>>>>>>>" >&2
    if [[ "$workFlowResult" == *"PRE_RECEIVE_OK"* ]]; then
      echo " >>>>>>>>>>>>>>>>> $workFlowResult >>>>>>>>>>>>>>>>>" >&2
      COMMIT_CHECK_STATUS=0
    fi

    fi
}

check_all_commits()
{
  REVISIONS=$(git rev-list $OLD_REVISION..$NEW_REVISION)
  IFS='\n' read -ra LIST_OF_REVISIONS <<< "$REVISIONS"
if [ $(git rev-parse --is-bare-repository) = true ]
then
    REPOSITORY_BASENAME=$(basename "$PWD")
else
    REPOSITORY_BASENAME=$(basename $(readlink -nf "$PWD"/..))
fi
    echo REPOSITORY_BASENAME is $REPOSITORY_BASENAME
    REPOSITORY_BASENAME=$(basename "$PWD")
    REPOSITORY_BASENAME=${REPOSITORY_BASENAME%.git}

  for rid in "${!LIST_OF_REVISIONS[@]}"; do
    REVISION=${LIST_OF_REVISIONS[rid]}
    COMMIT_MESSAGE=$(git cat-file commit $REVISION | sed '1,/^$/d')
    COMMIT_AUTHOR=$(git cat-file commit $REVISION | grep committer | sed 's/^.* \([^@ ]\+@[^ ]\+\) \?.*$/\1/' | sed 's/<//' | sed 's/>//' | sed 's/@$COMPANY_EMAIL//')
    check_single_commit

    if [ "$COMMIT_CHECK_STATUS" != "0" ]; then
      echo "Commit validation failed for commit $REVISION" >&2
      exit 1
    fi

  done
}



echo "Checking the coding style, coding standards and look out for potential issues"

if python /home/gitlab/gitlab_custom_hooks/pmd-policy.py; then
   echo "PMD_CHECK_OK"
   PMD_CHECK_STATUS=0

  # Get custom commit message format
    while read OLD_REVISION NEW_REVISION REFNAME ; do
        check_all_commits
    done



    exit 0

else

  echo "PMD_CHECK_FAILED: Review your Code and fix the issues"
  exit 1

fi

Below is the code for the python script that calls PMD last commit to a branch at the Gitlab server.

#!/usr/bin/env python

import subprocess
import sys
import tempfile
import shutil
import os
import errno


pmd = '/home/gitlab/gitlab_custom_hooks/pmd-bin-5.5.4/bin/run.sh'

# implementing check_output for python < 2.7
if not hasattr(subprocess, 'check_output'):
    def check_output(*popenargs, **kwargs):
        if 'stdout' in kwargs:
            raise ValueError('stdout argument not allowed, it will be overridden.')
        process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
        output, unused_err = process.communicate()
        retcode = process.poll()
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
            er = subprocess.CalledProcessError(retcode, cmd)
            er.output = output
            raise er
        return output
    subprocess.check_output = check_output


# helper for calling executables
def call(*args, **kwargs):
    return subprocess.check_output(*args, **kwargs).strip()


# helper for calling git
def call_git(cmd, *args, **kwargs):
    return call(['git'] + cmd, *args, **kwargs)


# get all new commits from stdin
def get_commits():
    commits = {}
    for line in sys.stdin:
        old, new, ref = line.strip().split(' ')
        if old == '0000000000000000000000000000000000000000':
            old = '4b825dc642cb6eb9a060e54bf8d69288fbee4904'

        if ref not in commits:
            commits[ref] = []
        commits[ref].append({
            'old': old,
            'new': new,
            'files': get_changed_files(old, new)
            })

    return commits


# get a list of changed files between to commits
def get_changed_files(old, new):
    return call_git(['diff', '--name-only', old, new]).split('\n')


# get filemode, object type (blob,tree,commit), hash for the given file at the
# given commit
def get_change_type(commit, filename):
    return call_git(['ls-tree', commit, filename]).split('\t')[0].split(' ')


commits = get_commits()

# use the latest file commit only
print "Cleaning up file list..."

files = {}
count = 0
for ref, data in commits.iteritems():
    files[ref] = {}
    for commit in data:
        for filename in commit['files']:
            if not filename.lower().endswith('.java'): continue
            files[ref][filename] = get_change_type(commit['new'], filename)
    count += len(files[ref])

print "%d Files to check in %d branches" % (count, len(files))

# create temporary dir and save a copy of the new files
tempdir = tempfile.mkdtemp('git_hook')
for ref, files in files.iteritems():
    for filename, data in files.iteritems():
        dname = os.path.dirname(filename)
        bname = os.path.basename(filename)
        try:
            os.makedirs(os.path.join(tempdir, dname))
        except OSError, exc:
            if exc.errno == errno.EEXIST:  # directory exists already
                pass
            else:
                raise

        with open(os.path.join(tempdir, dname, bname), 'w') as fp:
            fp.write(call_git(['cat-file', data[1], data[2]]))

try:
    # call checkstyle and/or pmd and print output
    print call([pmd, 'pmd', '-d', tempdir, '-f', 'text', '-R', 'rulesets/java/basic.xml,rulesets/java/unusedcode.xml,rulesets/java/imports.xml,rulesets/java/strings.xml,rulesets/java/braces.xml,rulesets/java/clone.xml,rulesets/java/design.xml,rulesets/java/clone.xml,rulesets/java/finalizers.xml,rulesets/java/junit.xml,rulesets/java/migrating.xml,rulesets/java/optimizations.xml,rulesets/java/strictexception.xml,rulesets/java/sunsecure.xml,rulesets/java/typeresolution.xml'])
    print "SUCCESS"
    sys.exit(0)
except subprocess.CalledProcessError, ex:
    print ex.output  # print checkstyle and/or pmd messages
    exit(1)
finally:
    # remove temporary directory
    shutil.rmtree(tempdir)