ESRF Workflow System (Ewoks)#

Motivation#

The main goal of Ewoks is to automate data processing and experiments at large-scale facilities as well as making data processing more scientific (reproducible) and FAIR (traceable).

Ewoks is not yet another workflow management system. To ensure the longevity of workflows and their implementation, Ewoks was designed to be a meta workflow system. This allows workflows to be isolated from the underlying software technologies used to execute and manage them.

Features#

The meta workflow approach allows for supporting a diverse set of use cases tailored to individual scientists on one end of the spectrum and large-scale facilities on the other end.

  • automated workflows distributed on a compute cluster

  • interactive worfkflows run as a desktop application

  • create, manage and execute worfklows as a web service

  • workflows with loops and conditional links

  • workflow tasks written in Python or defined by command line tools

  • parallel execution of tasks within the same workflow

  • job scheduling for integration in other systems (acquisition control, data portal)

How to cite Ewoks?#

What is Ewoks?#

Ewoks is a collection of Python projects to design and run data processing via workflows.

The Ewoks ecosystem is composed of:

  • ewoks to submit and execute workflows with any supported execution engine, be it locally or remotely.

  • ewokscore that defines the specification of Ewoks workflows.

  • ewoksweb: a web service to create, edit and execute workflows.

  • several bindings for workflow management systems, depending on the use case.

  • several ewoksapps exists that define workflows and tasks for specific domains (ex: tomwer for tomography, ewoksxrpd for X-Ray powder diffraction data processing).

Example

Ewoks is used at ESRF beamlines to azimuthally integrate diffraction images automatically during acquisition.

But it can do much more! See the task catalog for a list of possible operations (to be expanded soon with a workflow catalog).