CatGo

English

简体中文

English

简体中文

Appearance

Workflow Engine ​

The workflow engine is CatGo's system for building and executing multi-step computational pipelines. It provides a visual node-based editor, REST/WebSocket monitoring surfaces, async backend execution, and persistent workflow/project state.

Two Paths to a Workflow ​

There are two ways to author a workflow in CatGo:

  • Visual editor — Drag nodes from the palette, connect edges, configure parameters, and run. Source: src/lib/workflow/.
  • CatBot (natural language) — Describe a calculation in plain English and let the assistant build the graph. Two layers handle this:
    • In-app chat tools (src/lib/chat/workflow-tools.ts) for the frontend conversation loop
    • MCP catgo_workflow tool (server/mcp_tools/server.py) for SDK-driven agent workflows

Both AI layers share the same node schema and CRUD endpoints but differ in a few behaviors worth knowing about:

  • The MCP create path auto-adds a structure_input node so agents always have a starting point.
  • The MCP run path starts execution immediately on call — no in-UI confirmation step. This is intentional for unattended agent runs; the in-app chat path uses CatBot's PermissionCard instead.

Connection handles matter. When you wire nodes, name the source and destination handles (e.g., out-0, in-1). Omitting handles falls back to a generic structure connection, which only works for simple single-structure chains. For nodes with multiple outputs (a relaxed structure + WAVECAR + DOS, for example), explicit handles are required.

Authoritative Sources ​

This page describes the workflow engine's structure and node catalog at the level useful for understanding how to compose pipelines. For the most current node names, parameter schemas, and handle aliases, the source code is authoritative:

  • Node definitions — src/lib/workflow/node-definitions.ts
  • Frontend chat workflow tools — src/lib/chat/workflow-tool-executor.ts
  • MCP workflow tool surface — server/mcp_tools/server.py
  • Known issues and authoring pitfalls — WORKFLOW_BUGS.md

Architecture ​

┌──────────────────────────────────┐
│  Workflow Editor (Svelte 5)      │  Visual graph builder
│  Node palette, edge connections, │  Templates, undo/redo
│  parameter forms, run config     │
├──────────────────────────────────┤
│  Workflow API (FastAPI)          │  REST + WebSocket
│  CRUD, execution control,       │  Real-time monitoring
│  results, templates             │
├──────────────────────────────────┤
│  Workflow Engine (async Python)  │  Topological execution
│  Job submission, polling,        │  Custodian error handling
│  result extraction, ASE DB       │
├──────────────────────────────────┤
│  HPC Cluster (SSH)               │  VASP, MLP, Bader
│  SLURM / PBS scheduler           │  Remote file I/O
└──────────────────────────────────┘

Execution Model ​

  1. Topological sort — Nodes are sorted into layers based on dependencies
  2. Layer-by-layer execution — All nodes in a layer run in parallel; the next layer starts only after the current one completes
  3. Routing — Each node is routed to the appropriate executor:
    • HPC: VASP calculations, MLP calculations, Bader analysis
    • Local: Structure transformations, analysis, logic nodes
  4. Result passing — Output structures and energies flow downstream via edges

State Machine ​

Workflow:  draft → running → (paused) → completed / failed
Step:      pending → queued → running → completed / failed / skipped

Node Reference ​

Input Nodes ​

Structure Input ​

Provides the starting structure for a workflow.

ParameterTypeDescription
sourceselectFile upload, Materials Project, OPTIMADE, ASE DB, or editor
structuretextPOSCAR-format structure data

DFT Calculation Nodes ​

All DFT nodes run on an HPC cluster via SSH. They generate VASP input files (INCAR, POSCAR, KPOINTS), submit a job, poll for completion, and extract results.

VASP Relax ​

Geometry optimization using conjugate gradient, quasi-Newton, or FIRE.

ParameterDefaultRangeDescription
ENCUT520200–900 eVPlane-wave cutoff energy
EDIFF1e-51e-4 to 1e-7Electronic convergence
EDIFFG-0.02—Ionic convergence (negative = force in eV/A)
ISIF32/3/4/72 = fix cell, 3 = full, 4 = fix volume, 7 = volume only
NSW2001–1000Maximum ionic steps
IBRION21/2/31 = Quasi-Newton, 2 = CG, 3 = FIRE (requires VTST)
KPOINTS4x4x4—k-point mesh
double_relaxfalse—Run twice (atomate2 DoubleRelaxMaker pattern)
NCORE41–64Cores per orbital band
LWAVEfalse—Write WAVECAR
LCHARGtrue—Write CHGCAR
custom_incar——Additional INCAR tags (free text)

Outputs: Relaxed structure (CONTCAR), total energy, forces, stress tensor.

VASP Static ​

Single-point energy calculation on a fixed geometry.

ParameterDefaultDescription
ENCUT520Plane-wave cutoff
EDIFF1e-6Tighter electronic convergence
ISMEAR-5Tetrahedron method with Blochl corrections
LORBIT11Project DOS onto atoms
NEDOS3001DOS grid points
KPOINTS6x6x6Denser k-grid for accurate DOS

Outputs: Energy, DOS data, charge density.

VASP MD ​

Ab initio molecular dynamics.

ParameterDefaultDescription
TEBEG300Starting temperature (K)
NSW5000Number of MD steps
POTIM1.0Timestep (fs)
SMASS-1Thermostat: -1 = NVE, 0 = NVT scaled, 3 = Nose-Hoover
ENCUT400Lower cutoff acceptable for MD

Outputs: Trajectory, energy vs. time, temperature profile.

Electronic Analysis ​

Post-processing for electronic structure properties.

ParameterDefaultDescription
analysisdosType: dos, bader, cohp
NEDOS3001DOS energy grid points
LORBIT11Orbital projection level
ISMEAR-5Tetrahedron for accurate DOS

Outputs: DOS data, Bader charges, or COHP bonding analysis (depends on analysis type).

Frequency ​

Vibrational frequency calculation via finite differences.

ParameterDefaultDescription
IBRION55 = finite differences, 6 = all directions
NFREE22 or 4 displacements per direction
POTIM0.015Displacement step size (A)
EDIFF1e-7Tight electronic convergence for forces

Outputs: Frequencies (cm-1), zero-point energy (ZPE), IR intensities.

ML Potential Nodes ​

ML potential nodes run on HPC but are much faster than DFT. Use them for pre-screening, pre-relaxation, or long-timescale dynamics.

MLP Relax ​

ParameterDefaultDescription
modelMACE-MPML potential: MACE-MP, CHGNet, M3GNet
fmax0.01Force convergence criterion (eV/A)

Outputs: Relaxed structure, energy.

MLP MD ​

ParameterDefaultDescription
modelMACE-MPML potential model
temperature300Temperature (K)
steps10000Number of MD steps
timestep1.0Timestep (fs)

Outputs: Trajectory, energy time series.

Surface Catalysis Nodes (NRR) ​

Specialized nodes for nitrogen reduction reaction studies.

Bulk Optimization ​

Dense k-grid relaxation for bulk crystals.

ParameterDefaultDescription
KPOINTS9x9x9Dense k-grid
ISIF3Full cell + ionic relaxation
ENCUT520Standard cutoff

Slab Generation ​

Cut a surface slab from a bulk crystal (runs locally).

ParameterDefaultDescription
miller_h, miller_k, miller_l1, 1, 1Miller indices
layers4Number of atomic layers
vacuum15.0Vacuum thickness (A)
supercell_a, supercell_b2, 2In-plane supercell

Slab Relaxation ​

Surface relaxation with frozen bottom layers and dipole correction.

ParameterDefaultDescription
ISIF2Fix cell shape, relax ions
freeze_layers2Number of bottom layers to freeze
LDIPOLtrueDipole correction for slab asymmetry

Adsorbate Placement ​

Place molecules on surface adsorption sites (runs locally).

ParameterDefaultDescription
adsorbateN2Molecule to place (*N2, *NH3, *H, etc.)
siteontopSite type: ontop, bridge, fcc, hcp
orientationend-onMolecule orientation: end-on, side-on

Reference Molecule ​

Gas-phase molecule calculation for thermodynamic reference.

ParameterDefaultDescription
moleculeN2Reference molecule (N2, H2, NH3)
box_size20Cubic box size (A)
KPOINTS1x1x1Gamma-point only

Free Energy Diagram ​

Calculate reaction free energies along a pathway.

ParameterDefaultDescription
pathwaydistalReaction mechanism: distal, alternating
potential-0.1Applied potential (V vs. RHE)
temperature298.15Temperature (K)

Formula: DG = DE + DZPE - TDS + neU

HER Analysis ​

Assess hydrogen evolution selectivity.

ParameterDefaultDescription
threshold0.2DG(*H) threshold for NRR selectivity (eV)

Structure Transformation Nodes ​

All transformation nodes run locally on the server. They modify the input structure and pass the result downstream.

Supercell Generation ​

ParameterDefaultDescription
scaling2x2x2Supercell expansion factors

Defect Generation ​

ParameterDefaultDescription
defect_typevacancyType: vacancy, substitution, interstitial
site_index0Atom index for the defect site
substitute—Replacement element (for substitution)

Enumerates symmetry-unique sites when multiple defects are possible.

Strain / Deformation ​

ParameterDefaultDescription
strain_typeuniaxialType: uniaxial, biaxial, hydrostatic, shear
magnitude0.02Strain magnitude (fractional)
scanfalseScan mode: generate multiple strained structures
scan_range-0.05 to 0.05Range for scan mode

Doping ​

ParameterDefaultDescription
dopant—Dopant element
host_element—Element to replace

Enumerates symmetry-unique substitution sites.

Intercalation ​

ParameterDefaultDescription
intercalantLiIntercalated species (Li, Na, K)

Heterostructure ​

ParameterDefaultDescription
methodZSLLattice matching algorithm
max_area200Maximum interface area (A^2)
max_strain0.05Maximum allowed strain

Nanotube ​

ParameterDefaultDescription
n, m10, 0Chiral indices

Water Solvation ​

ParameterDefaultDescription
density1.0Water density (g/cm^3)
modelTIP4PWater model

Passivation ​

ParameterDefaultDescription
methodpseudo-Hpseudo-hydrogen (fractional Z) or H

Analysis Nodes ​

DOS Analysis ​

Extracts d-band center and projected DOS from a parent VASP static calculation.

COHP Analysis ​

Runs LOBSTER for crystal orbital Hamilton population analysis.

MD Analysis ​

ParameterDefaultDescription
analysisallMetrics: RMSD, RDF, MSD, density profile, H-bonds

Convergence Check ​

ParameterDefaultDescription
energy_threshold1e-4Energy convergence (eV/atom)
force_threshold0.05Maximum force (eV/A)

Returns pass or fail. Connect to a Condition node for branching.

Energy Compare ​

Compares energies across multiple parent calculations. Outputs a ranking table with adsorption energies, surface energies, or formation energies.

Charge Analysis ​

ParameterDefaultDescription
methodbaderBader or DDEC6 charge analysis

Runs on HPC (requires Bader executable).

Export ​

ParameterDefaultDescription
formatjsonOutput format: json, csv, cif, poscar

Logic Nodes ​

Condition ​

Branches the workflow based on a criterion.

ParameterDefaultDescription
criterionenergy_diffWhat to check: energy_diff, max_force, convergence, n_steps
threshold0.01Threshold value
operator<Comparison operator

Loop ​

Iterates over a collection, executing downstream nodes for each item.

ParameterDefaultDescription
iterate_overstructuresWhat to loop: structures, parameters

Merge ​

Synchronization barrier. Waits for all incoming edges to complete before continuing. No parameters.

Built-in Templates ​

TemplateNodesDescription
Band Structure3Relax → Static → DOS analysis
Adsorption Screening5Parallel DFT + MLP relaxation, energy comparison
MLP MD Pipeline4Structure → MLP MD → MD analysis → Export
Batch Surface5Loop surfaces → Relax → Merge → Analyze
Defect Screening6Supercell → Defect gen → Loop → Relax → Compare
Heterostructure Study5Build interface → Relax → DOS + COHP

Templates are starting points — you can add, remove, or reconfigure any node after loading.

HPC Execution ​

Job Script Presets ​

PresetSchedulerDescription
Generic SLURMSLURMStandard SLURM submission script
Generic PBSPBSFor PBS/Torque clusters
Shaheen-IIISLURMKAUST HPC with module loading

Resource Configuration ​

ParameterDefaultDescription
nodes1Compute nodes
ntasks16MPI tasks
cpus_per_task8OpenMP threads per task
walltime02:00:00Maximum job duration
partition—Cluster partition/queue
memory—Memory per node
base_work_dir~/calculationsRemote working directory
poll_interval30sJob status polling interval

Remote Directory Structure ​

Each step creates a directory on the HPC cluster:

~/calculations/
├── vasp_relax_abc12345/
│   ├── INCAR
│   ├── POSCAR
│   ├── KPOINTS
│   ├── POTCAR (user-provided)
│   ├── submit.sh
│   ├── run_custodian.py
│   ├── CONTCAR (output)
│   ├── OUTCAR (output)
│   └── OSZICAR (output)
└── mlp_md_def45678/
    ├── POSCAR
    ├── run_mlp.py
    └── trajectory.xyz (output)

Custodian Error Handling ​

Custodian is enabled by default and automatically handles common VASP errors:

ErrorFix AppliedMax Retries
ZBRENT failureSwitch IBRION5
EDDDAV/EDWAVSwitch to algo=Normal/Fast5
KPOINTS too denseReduce mesh5
Charge mixingAdjust AMIX/BMIX5
Walltime exceededRestart from CONTCAR5

Disable Custodian in the run configuration if you want raw VASP execution.

Workflow Recovery ​

The engine persists all state to an SQLite database. Recovery is automatic:

  1. On server startup, recover_workflows() checks for interrupted workflows
  2. Queries HPC clusters for job status
  3. Extracts results from jobs that completed while offline
  4. Resumes execution from the next pending layer

Persistence Details ​

DataStorage
Workflow graphSQLite (graph_json)
Step statusSQLite (status, started_at, completed_at)
HPC job IDsSQLite (hpc_job_id, hpc_session_id)
Run configurationSQLite (WorkflowRunConfig)
DFT resultsASE database (energy, forces, structure)

API Reference ​

Workflow CRUD ​

EndpointMethodDescription
/workflow/POSTCreate a new workflow
/workflow/GETList all workflows
/workflow/{id}GETGet workflow details
/workflow/{id}PUTUpdate graph, name, or status
/workflow/{id}DELETEDelete a workflow

Execution Control ​

EndpointMethodDescription
/workflow/{id}/runPOSTStart workflow with run config
/workflow/{id}/pausePOSTPause execution
/workflow/{id}/resumePOSTResume paused workflow
/workflow/{id}/run-statusGETGet current execution status
/workflow/{id}/monitorWebSocketReal-time status stream

Steps and Results ​

EndpointMethodDescription
/workflow/{id}/stepsGETList all steps
/workflow/{id}/steps/{step_id}PUTUpdate step config
/workflow/{id}/steps/{step_id}/filesGETList output files
/workflow/{id}/steps/{step_id}/output/{file}GETDownload output file
/workflow/{id}/convergence/{step_id}GETOSZICAR convergence data
/workflow/{id}/resultsGETGet workflow results
/workflow/{id}/results-enrichedGETResults with formulas and volumes

Templates ​

EndpointMethodDescription
/workflow/templatesGETList available templates
/workflow/from-template/{id}POSTCreate workflow from template
/workflow/job-script-presetsGETGet job script presets

CatGo is licensed under AGPL-3.0-or-later. If CatGo contributes to your work, please include “This work used CatGo (https://app.catgo-ucsd.org).” and the preferred citation in CITATION.cff. This request is not an additional condition of the AGPL license.

Copyright 2024-present CatGo Contributors