Often in needed to run some jobs remotely, for example, tests on remote test environment. I used different custom "daemon"-like scripts for this, monitoring the jobs/tests, but another and simple solution for Linux is to use screen tool. It supports sessions, detaching, so is good for such kind of tasks. Here is the script which runs multiple jobs on remote host (setted via $RMHOST env. var). Its syntax is:
-cls
-run JOB-NUMBER 'JOB-COMMAND'
-runall < FILE-WITH-COMMANDS
-submit < FILE-WITH-COMMANDS
Used env. vars: \$USER (login), \$RMHOST (hostname)
so to run it you must:
- set
$RMHOSTenv. var - create file with commands which will be ran remotely: each command is on own line and can have any
$variables for substitution: they will be substituted locally - then call script as
V1=a V2=b script -submit < created_file. This will substituteV1andV2increated_fileand send resulting file over scp to remote host$RMHOST(you must have configured public-key access to host), then will send itself and call itself remotely with command-runall.
After it you can go to host, call screen -r to see your created session, to switch between windows (each job is ran on separate window). You will have local $LOG file with decscription of running commands and remote log file per job with info about what and when was run (job-<DATE>-<JOB_NUMBER>.log). Yes, each job has number/index related to line number of job suite file. If you want you can include generating of screen log file too (with -L option IMO).
While you are on remote host, you can run this script (which was scp-ed) with -cls command: it will remove all log files there.
That's it.
#!/bin/sh
SCR=`basename "$0"`
SESSION=${USER}_screen
LOG=rmjobs.log
_wrapjob() {
_wrapjob_f=job-`date +%s`-$1.log
echo "Started $1 at `date`: $2" > $_wrapjob_f
echo '===============================================================================' >> $_wrapjob_f
echo >> $_wrapjob_f
$2|tee -a $_wrapjob_f
}
_runjob() {
[ $1 = "0" ] && {
# add -L to both if screelog.* logs are needed
screen -dmS $SESSION -t "job$1" sh -c "~/$SCR -run $1 \"$2\";sh"
} || {
screen -S $SESSION -p 0 -X screen -t "job$1" sh -c "~/$SCR -run $1 \"$2\";sh"
}
}
case "$1" in
-cls) rm -f screenlog.*; rm -f job*.log;;
-run) _wrapjob "$2" "$3";;
-runall)
i=0
rm -f $LOG 2>/dev/null
while read -r c; do
c=`echo $c|tr -d '[:cntrl:]'`
echo $c|egrep '^-' >/dev/null && continue
_runjob $i "$c"
echo "Ran $c;" >> $LOG
i=`expr $i + 1`
sleep 2
done;;
-submit)
rm -f .jobs-suite.tmp 2>/dev/null
while read -r c; do
echo $c|egrep '^-' >/dev/null && continue
eval "echo $c" >> .jobs-suite.tmp
done
scp "$0" "$USER@$RMHOST:$SCR"
scp ".jobs-suite.tmp" $USER@$RMHOST:".jobs-suite.tmp"
ssh -n -l $USER $RMHOST "sh $SCR -runall < .jobs-suite.tmp";;
*)
echo "Syntax:"
echo " -cls"
echo " -run JOB-NUMBER 'JOB-COMMAND'"
echo " -runall < FILE-WITH-COMMANDS"
echo " -submit < FILE-WITH-COMMANDS"
echo "Used env. vars: \$USER (login), \$RMHOST (hostname)"
exit 1;;
esac
:
