Snapshots, simulation sets and batch runs in PSCAD

An EMT study spends much of its time on things it does not need: the same start-up simulated again for every case, and one case at a time on a machine with many cores. A snapshot fixes the first: initialise once, save the state, start every case from it. A simulation set fixes the second: PSCAD runs its cases several at a time.

Where a PSCAD study's time goes

A PSCAD case starts from zero: sources ramp up, machines are released, controls find their operating point. For a large case with converters that can take seconds of simulated time, and minutes of real time, before anything the study cares about happens. A study of twenty fault cases pays that twenty times.

A snapshot is the whole state of the case at one instant, saved to a .snp file. A run started from it carries on from that instant, at that simulated time, so the start-up is paid once.

A simulation set is a list of cases, or of one case with different settings, that PSCAD runs as a batch. Its volley is how many run at the same time, each its own process. That is where a PSCAD study gets real parallelism: four cases in a volley of four finish in about the time of one, as long as the machine has the cores and the licence allows it.

The method

Initialise once, then branch

  1. Run to a settled state with a snapshot time set, and the run saves the state there.
  2. Switch the start to that snapshot file. Every run from then on starts from it, until it is switched back.
  3. Time each event after the snapshot: a run from a snapshot taken at 2 s starts at 2 s, so a fault at 0.1 s is already in the past.
snapshot = snapshot_save(app, at=2.0)            # the next run saves the state at 2 s
px_run(app, duration=2.2, step=50, plot_step=250)  # initialise, once

snapshot_use(app, snapshot)                        # every run from here starts at 2 s
for duration in num_range(0.08, 0.2, 0.04):
    mark = px_mark()
    set_fault(app, "master:tfaultn", "", at=2.1, duration=duration)
    run = px_run(app, duration=2, step=50, plot_step=250)
    table("Snapshot").add({"Lowest voltage": peak(run.series("Vbus"), kind="min").value})
    px_undo(mark)
snapshot_off(app)                                  # back to starting from zero

Several cases at once

sim_set = simset_create(app, "batch", ["case_a", "case_b", "case_c", "case_d"], volley=4)
simset_override(app, sim_set, None, duration=5)    # every task: 5 s, without touching the cases
result = simset_run(app, sim_set)                  # builds and runs its own tasks
print(result.ok, result.seconds)

The cases in a set must be loaded in the workspace. Load them with the case, or list them under Libraries in the Builder's Start block. The volley is best kept at or below the number of processor cores.

Open this in the Builder. The Initialise once, then branch Quick Start loads in the PSCAD Workflow Builder, ready for your own case. Change any block and download the Python file.

Open in the Builder

The full Python scripts from the Builder

The Initialise once, then branch Quick Start saves a snapshot at 2 s, initialises once, then runs three fault durations from the snapshot and tables the lowest voltage of each. This is the file with the case set to C:/Studies/hvdc/hvdc.pscx:

initialise_once_then_branch.py as the Builder writes it: the settings and the workflow⬇ Download the full file (.py, 2,435 lines)

#!/usr/bin/env python3
# Snapshot then branch
"""Made with the PSCAD Workflow Builder, https://tool.adjiebrotots.com/pscad-scripter/builder/

BEFORE YOU RUN
  1. Check the SETTINGS below (lines marked REQUIRED).
  2. Have PSCAD installed and licensed; the script opens one, or uses the one already open (see RUN_MODE).
  3. Run it with Python 3.7 or newer and mhi-pscad installed, e.g.  py -3.12 snapshot_then_branch.py
Results and run.log go to a new folder next to this file (or in OUTPUT_DIR).
"""
# ==== IMPORTS ====
import bisect
import csv
import fnmatch
import importlib
import math
import os
import re
import sys
import time
import traceback

# ==== SETTINGS (edit here) ====
# The name of this workflow: used for the run folder and in the log.
WORKFLOW_NAME = "Snapshot then branch"
# REQUIRED. The PSCAD case to work on: a .pscx path, or the name of a case already in the
# workspace.
PROJECT = "C:/Studies/hvdc/hvdc.pscx"
# OPTIONAL. A .pswx workspace to load before the case ("" = keep the one that is open).
WORKSPACE = ""
# OPTIONAL. Extra .pslx / .pscx libraries to load before the case.
LIBRARIES = []
# OPTIONAL. Where each run writes its folder of results and run.log ("" = the folder of this
# script).
OUTPUT_DIR = ""
# OPTIONAL. True: put every parameter, layer state and substitution the workflow changed back
# when it ends.
UNDO_CHANGES = True
# OPTIONAL. How the results tables are saved in the run folder when the run ends: "csv", "xlsx"
# (Excel, needs pandas and openpyxl) or "both".
TABLE_FORMAT = "csv"
# OPTIONAL. Nominal frequency in Hz for the frequency checks: 50, 60 or "auto" (read the case's
# own global substitution).
F_NOMINAL = "auto"
# OPTIONAL. "auto", "launch" (start a new PSCAD), "connect" (use the one that is open) or
# "inside" (run from inside PSCAD).
RUN_MODE = "auto"
# OPTIONAL. True: show the PSCAD window while running.
SHOW_PSCAD = False
# OPTIONAL. The Fortran compiler that builds the case, as PSCAD names it ("" = the one PSCAD has
# selected).
FORTRAN_VERSION = ""
# OPTIONAL. Which installed PSCAD to start ("" = the newest).
PSCAD_VERSION = ""
# OPTIONAL. The port PSCAD listens on for automation (0 = let PSCAD choose).
PSCAD_PORT = 0
# OPTIONAL. Seconds to wait for PSCAD to start.
CONNECT_TIMEOUT = 120

# True: connect, find every element named in this workflow, say what was found, then stop
# without running anything.
CHECK_ONLY = False


# ==== WORKFLOW ====
# Note: A snapshot is the whole state of the case at one instant. A long start-up is simulated once, and each case after it begins there, which is often most of the run time of a study saved. Set the snaps...
def main():
    preflight([])
    app = px_connect()
    try:
        if CHECK_ONLY:
            check_elements(app, [])
            return
        # -- Step 1: Start (Case: hvdc.pscx) --
        # -- Step 2: Save a snapshot (at 2 s) --
        snapshot = snapshot_save(app, 2, None, "")
        # -- Step 3: Initialise (2.2 s, 50 us step) --
        initialise = px_run(app, 2.2, 50, 250, None, events=(), build="auto", read="out", keep_files=False)
        # Did not build: ends the workflow
        if not initialise.ok and (initialise.status == "failed"):
            log("The case did not build. The workflow ends here.", "warn")
            return
        # Stopped early: ends the workflow
        if not (initialise.ok or (initialise.status == "failed")):
            log("The simulation stopped before the end. The workflow ends here.", "warn")
            return
        # -- Step 4: Start from a snapshot (no snapshot chosen) --
        snapshot_n4 = snapshot_use(app, snapshot)
        # -- Step 5: Fault duration (s) (0.08 to 0.2, step 0.04) --
        fault_duration_s_values = num_range(0.08, 0.2, 0.04)
        for fault_duration_s_pass_no, fault_duration_s in enumerate(fault_duration_s_values, 1):
            ctx_push("Fault duration (s)", fault_duration_s, fault_duration_s_pass_no, len(fault_duration_s_values))
            mark = px_mark()
            try:
                # -- Step 6: Fault (master:tfaultn: 2.1 to 2.22 s) --
                fault = set_fault(app, "master:tfaultn", "", 2.1, fault_duration_s, True, "one", None, None)
                # -- Step 7: Run simulation (2 s, 50 us step) --
                run = px_run(app, 2, 50, 250, None, events=[(2.1, "Fault")], build="auto", read="out", keep_files=False)
                # Did not build: skips to the next pass of Fault duration (s)
                if not run.ok and (run.status == "failed"):
                    case_skipped("The case did not build")
                    continue
                # Stopped early: skips to the next pass of Fault duration (s)
                if not (run.ok or (run.status == "failed")):
                    case_skipped("The simulation stopped before the end")
                    continue
                # -- Step 8: Add row to results table (Into "Snapshot", saved at the end automatically) --
                vbus = run.series("Vbus", "")  # Read a channel
                peak_result = peak(vbus, kind="min", after=None, before=None)  # Peak
                table("Snapshot").add({"Lowest voltage": peak_result.value}, status=True)
            except StopWorkflow:
                raise
            except Exception as err:
                case_failed(err)
            finally:
                try:
                    px_undo(mark)
                except Exception as fin:
                    log(f"Could not put things back after this pass: {fin}", "error")
                ctx_pop(["Snapshot"])
    except StopWorkflow as stop:
        log(f"Stopped: {stop}")
    finally:
        finish(app)

The file goes on with 2,309 lines of helper functions copied from the Builder's runtime (connecting, building, running, reading the channels, the checks, tables, graphs, putting values back), only those this workflow needs. The download has them all.

How the script works

  • snapshot_save(app, 2, None, "") sets the case to save one snapshot at 2 s, named after the case, and the Initialise run of 2.2 s writes it.
  • snapshot_use(app, snapshot) switches the start to the file. The runs in the loop pass no snapshot of their own and keep starting from it.
  • The fault is at 2.1 s, 0.1 s after the snapshot. Each pass writes its duration on the Timed Fault Logic and puts it back after.
  • At the end, finish(app) puts the case's start and snapshot settings back as they were found, with every other setting the workflow changed.

The Run several cases at once Quick Start makes a simulation set of the case in use with a volley of 4, overrides its duration to 5 s, runs it and tables how it went. List your own cases in the set:

The Run several cases at once Quick Start, as the Builder writes it⬇ Download the full file (.py, 1,691 lines)

#!/usr/bin/env python3
# Several cases at once
"""Made with the PSCAD Workflow Builder, https://tool.adjiebrotots.com/pscad-scripter/builder/

BEFORE YOU RUN
  1. Check the SETTINGS below (lines marked REQUIRED).
  2. Have PSCAD installed and licensed; the script opens one, or uses the one already open (see RUN_MODE).
  3. Run it with Python 3.7 or newer and mhi-pscad installed, e.g.  py -3.12 several_cases_at_once.py
Results and run.log go to a new folder next to this file (or in OUTPUT_DIR).
"""
# ==== IMPORTS ====
import csv
import fnmatch
import importlib
import math
import os
import re
import sys
import time

# ==== SETTINGS (edit here) ====
# The name of this workflow: used for the run folder and in the log.
WORKFLOW_NAME = "Several cases at once"
# REQUIRED. The PSCAD case to work on: a .pscx path, or the name of a case already in the
# workspace.
PROJECT = "C:/Studies/hvdc/hvdc.pscx"
# OPTIONAL. A .pswx workspace to load before the case ("" = keep the one that is open).
WORKSPACE = ""
# OPTIONAL. Extra .pslx / .pscx libraries to load before the case.
LIBRARIES = []
# OPTIONAL. Where each run writes its folder of results and run.log ("" = the folder of this
# script).
OUTPUT_DIR = ""
# OPTIONAL. True: put every parameter, layer state and substitution the workflow changed back
# when it ends.
UNDO_CHANGES = True
# OPTIONAL. How the results tables are saved in the run folder when the run ends: "csv", "xlsx"
# (Excel, needs pandas and openpyxl) or "both".
TABLE_FORMAT = "csv"
# OPTIONAL. Nominal frequency in Hz for the frequency checks: 50, 60 or "auto" (read the case's
# own global substitution).
F_NOMINAL = "auto"
# OPTIONAL. "auto", "launch" (start a new PSCAD), "connect" (use the one that is open) or
# "inside" (run from inside PSCAD).
RUN_MODE = "auto"
# OPTIONAL. True: show the PSCAD window while running.
SHOW_PSCAD = False
# OPTIONAL. The Fortran compiler that builds the case, as PSCAD names it ("" = the one PSCAD has
# selected).
FORTRAN_VERSION = ""
# OPTIONAL. Which installed PSCAD to start ("" = the newest).
PSCAD_VERSION = ""
# OPTIONAL. The port PSCAD listens on for automation (0 = let PSCAD choose).
PSCAD_PORT = 0
# OPTIONAL. Seconds to wait for PSCAD to start.
CONNECT_TIMEOUT = 120

# True: connect, find every element named in this workflow, say what was found, then stop
# without running anything.
CHECK_ONLY = False


# ==== WORKFLOW ====
# Note: List the cases to run in the simulation set and load each of them in Start under Libraries, so they are in the workspace. Keep the volley at or below the number of processor cores: more than that a...
def main():
    preflight([])
    app = px_connect()
    try:
        if CHECK_ONLY:
            check_elements(app, [])
            return
        # -- Step 1: Start (Case: hvdc.pscx) --
        # -- Step 2: Make a simulation set (PXB_batch: the case in use, 4 at a time) --
        sim_set = simset_create(app, "PXB_batch", (), 4, False)
        # -- Step 3: Override a set’s settings (5 s) --
        simset_override(app, sim_set, None, 5, None, None)
        # -- Step 4: Run a simulation set (every case in the set) --
        set_run = simset_run(app, sim_set)
        # Had errors: ends the workflow
        if not set_run.ok:
            log("The simulation set had errors. The workflow ends here.", "warn")
            return
        # -- Step 5: Add row to results table (Into "Batch", saved at the end automatically) --
        table("Batch").add({
            "Finished": set_run.ok,
            "Seconds": set_run.seconds,
        })
    except StopWorkflow as stop:
        log(f"Stopped: {stop}")
    finally:
        finish(app)

Pitfalls

  • An event timed before the snapshot. A run from a snapshot starts at the snapshot's time. Anything timed before it never happens. Add the snapshot time to every event time.
  • A snapshot from a different build. A snapshot holds the state of one build of the case. Change the case's structure (a component added, a definition changed) and take the snapshot again.
  • A snapshot before the case has settled. Every run then starts mid-transient. Look at a run with no event first and take the snapshot after it has reached steady state.
  • A volley larger than the machine. More processes than cores slow each other down, and each needs its own licence seat where the licence counts them.
  • A simulation set changes PSCAD, not only the case. The set is kept in the workspace. The Builder deletes the sets it made at the end unless told to keep them.

Questions

How do I start a PSCAD run from a snapshot in Python?

Run once with a snapshot time set so the run saves a .snp file, then set the case's start to that file (StartType FROM_SNAP_FILE and startup_filename in the project's settings) and run again. The run carries on from the snapshot's time.

Can PSCAD run simulations in parallel?

Yes, through a simulation set: its volley says how many tasks run at the same time, each as its own process. A Python loop on its own runs one case after another.

What is the difference between a simulation set and Multiple Run?

Multiple Run repeats one case inside one run of its executable, stepping variables. A simulation set runs several cases, or one case with different settings, as separate processes, several at a time.

Related

Every parameter name a script writes is in the component reference, read out of PSCAD itself. The Workflow Builder writes the Python from blocks.