3

I am looking for a tool to specify pipelines of arbitrary multiple jobs with dependencies for the Sun Grid Engine. I can wire things together myself but I am hoping for something more declarative and reusable.

Can anyone recommend such a tool or has anyone adapted an existing build system/dependency manager for compute jobs on SGE?

Alex Stoddard
  • 131
  • 3

3 Answers3

2

Makeflow describes dependencies using the Make syntax. It natively supports SGE as well.

Derek
  • 181
  • 4
0

Have you considered the qmake command that comes with grid engine? A bit limited as the individual subjobs can't really be parallel but since you mentioned make...

William Hay
  • 101
0

Starcluster uses SGE fairly intelligently. It is part of a suite of tools that make numpy/scipy installs/deploy operations really easy via EC2.

We have moved away from using SGE, as it is in developmental flux at the moment.

meawoppl
  • 1,918
  • 11
  • 20