Skip to content

Fireworks

FireWorks Workflow

FireWorks is a free, open-source code for defining, managing, and executing scientific workflows. It has been designed to coordinate scientific analysis on large systems like Livermore Computing (LC) HPC.

Strengths of FireWorks

  • Well-suited to LC-- we have many FireWorks users
  • Easy to install via conda
  • Server/worker model is inherently load-balancing
  • Extremely flexible and able to handle complex job structures

Configuration of FireWorks

  • Requires a MongoDB database which must be set up by LC WEG staff (contact lc-hotline@llnl.gov)
  • FireWorks requires some time to learn and understand (more powerful tools are more complex)

Overview of FireWorks

FireWorks uses a centralized server model where the server manages the workflows and workers run the jobs.

To use FireWorks you must request a mongoDB database. Please email lc-weg@llnl.gov to request your mongoDB database.

Below is an example of how to use FireWorks at LC. This is based heavily on the Fireworks tutorial which you will find here.

FireWorks is the primary workflow engine for the Materials Project, but this tool is general, well-documented, and suitable for a wide variety of scientific applications. For more information on Fireworks visit: https://materialsproject.github.io/fireworks/index.html

Terminology

FireWorks uses a number of terms to describe the different parts of the workflow manager.

FireWork Model

  • FireServer: MongoDB that controls the workflow. Also referred to as the LaunchPad. It contains all the tasks to be run, and whether they have run successfully etc.
  • FireTask: computing tasks to be performed.
  • FireWork: list of FireTasks.
  • Rocket: fetches a FireWork from the LaunchPad and runs it. Could be run on a separate machine (FireWorker) or through a batch system.