0

On a remote lab system, I have a shell script that initiates a reverse SSH tunnel to my main jumpbox that gets run with a cron job every 5 minutes. If the SSH tunnel is up nothing happens, if it is down it initiates it.

#!/bin/bash
createTunnel() {
/usr/bin/ssh -N -R :2222:localhost:22 username@jumpbox.example.com
if [[ $? -eq 0 ]]; then
 echo Tunnel to jumpbox created successfully
else
 echo An error occurred creating a tunnel to jumpbox. RC was $?
fi
}
/bin/pidof ssh
if [[ $? -ne 0 ]]; then
  echo Creating new tunnel connection
  createTunnel
fi

This has been extremely reliable to ensure my access to the remote machine if it gets rebooted or as my jumpbox IP changes. However, I recently added a second SSH tunnel to this system and had a situation where one of the two tunnels went down and was never re-established. It appears that since there was one tunnel up, the pidof output still returned with a PID so the script never ran "createTunnel". Since I have two SSH tunnels, the pidof output shows both PID's:

$ /bin/pidof ssh
28281 28247

How can I adjust my script to determine if only one of the tunnels is down?

Andrew Plas
  • 3
  • 2
    You may be reinventing the wheel, which in your case is autossh -f ... that never exits. It's not clear whether the two connections are to the same endpoint username@jumpbox.example.com. If so, a single ssh (or autossh) can create both tunnels, just use additional -R or -L. – Kamil Maciorowski Jan 02 '18 at 06:05

1 Answers1

2

My thoughts:

  1. Pid files. I think this is a general approach.
  2. XY problem, use autossh.

(Some other answers may elaborate these ideas.)

  1. There are two scripts, right? (not exactly complying with DRY). Give them different names. Your script waits for its ssh to exit. The additional instance, instead of checking pid(s) of ssh, should check pid(s) of its own name. Exactly one pid two pids mean no previous script is still running:

    [ $(pidof -x scriptname | wc -w) -eq 2 ] && createTunnel

    Why two? Because $() starts a subshell that also counts. Quite dirty, oh well.

  2. Somewhat outside-the-box, I think. Create two symlinks with unique names, like:

    ln -s /usr/bin/ssh ssh-foo
ln -s /usr/bin/ssh ssh-bar

    Let the first script run ssh-foo, let the second run ssh-bar. Their pidof invocations should target ssh-foo or ssh-bar respectively as well. This way they won't mix. As a bonus, additional ssh (possibly run for a completely different reason in the future) won't affect them.

And finally:

  1. You don't have to check anything.

    /usr/bin/ssh -o ExitOnForwardFailure=yes -N -R :2222:localhost:22 username@jumpbox.example.com

    If the tunnel already exists, port forwarding will fail for sure and ssh will exit. I guess you can run this line directly from a crontab, no script is required.

Kamil Maciorowski
  • 74,068
  • Excellent, thank you! Using "ExitOnForwardFailure=yes" in the command I am initiating the SSH tunnels from by crontab directly. The only thing I added was ">> /dev/null 2>&1" to prevent the user mailbox from being filled up with the cron failures. /usr/bin/ssh -o ExitOnForwardFailure=yes -N -R :2222:localhost:22 username@jumpbox.example.com >> /dev/null 2>&1 – Andrew Plas Jan 05 '18 at 00:58