Python Example - Automated Model Testing

Navigation: User Guide ➔ COM Automation ➔ Python Automation ➔ Example - Unit Test Framework

Python Setup	Python Examples	Python Script			Optimisation		Visulisation	Python GUI	Tags and Data
Installation & Troubleshooting	List of Examples	Simple Script (pywin32)	SysCAD COM Python Class	Automated Model Testing	Constrained FEM (numpy|matplotlib)	Optimisation (numpy|scipy)	Adding Plots (numpy|matplotlib)	Dynamic with GUI	Accessing Data (sqlite3|pandas)

---

Introduction

Unit testing is a critical part of software development and model validation. Running a model and comparing current results to previous outputs after changes helps ensure consistency and correctness. COM Automation can be leveraged to facilitate this process. Development and testing teams often use automated workflows extensively, as described below. A simple Python example is also provided, which can serve as a foundation for your own unit testing setup.

Automated Model Testing Framework

Many teams use internal utilities designed for automated unit testing to support development of model components, solvers, thermodynamic calculations, and more. These tools help ensure that results do not change unexpectedly as the code evolves. They play an indispensable role in quality assurance and testing, helping maintain consistency across software updates and versions. COM Automation is commonly used to drive these workflows.

Such a framework typically cycles through a large collection of project files, solving each model and applying a series of step changes. For each solution, results are compared to a previously saved reference set. Any discrepancies trigger an investigation, which may lead to either:

• Identifying and fixing issues in the current code changes, or
• Confirming that the changes are expected improvements or beneficial updates.

In addition to validating solver results, these frameworks are also used to monitor performance metrics such as iteration speed and time per iteration. They can verify that projects load correctly from a saved state without requiring re-solving, especially when previously saved in a converged state.

The library of test projects continues to grow, encompassing simple and complex models of various sizes. The workflow involves adding new projects, selecting key result tags, and identifying input tags for step changes. The first solution run establishes a baseline for future comparisons. Projects with intentionally unrealistic or error-prone configurations are also included to test behavior under edge cases and invalid conditions. When a code change leads to understood and accepted result changes, the baseline is updated accordingly. Existing projects are periodically revised to test new features, add more step changes, and expand coverage.

Model Validation with Python

Beyond structured automated testing, developers and modelers may want to validate specific outputs across multiple project models. This could be useful when reviewing client models after software updates or when assessing the impact of changes to thermodynamic data.

A simple Python script can be used for these purposes. The process begins by creating a text-based data file listing project folders, followed by CSV-formatted lines containing tags and expected values. CSV is particularly convenient, as it allows easy generation from spreadsheet reports.

The input data is saved in a file named utest.dat. In this file, blank lines, lines starting with whitespace, and lines without commas are ignored—except for project names, which are always processed.

C:\SysCAD139\Examples\65 Smelting\GFEM Blast Furnace Examples.spf 

P_102.Qo.QM.Total (t/h)       ,    10.2730644002              
P_102.Qo.QM.Solids (t/h)      ,    2.433901724                
P_102.Qo.QM.Vapours (t/h)     ,    7.8391626762               
                              ,    ============== Solids      
P_102.Qo.QM.CaO(s) (t/h)      ,    0.042618824                
P_102.Qo.QM.Fe(s) (t/h)       ,    2.3912829                  

C:\SysCAD139\Examples\03 UnitModels\GFEM Simple Examples.spf

P_112.Qo.QM.H2O(g) (t/h)      ,    47.6731680080583    
P_112.Qo.QM.CO(g) (t/h)       ,    275.7948098716263  
P_112.Qo.QM.CO2(g) (t/h)      ,    358.8418906771008  
P_112.Qo.QM.H2(g) (t/h)       ,    9.179787443214618  
P_502.Qo.T (C)                ,   1738.67733669912

The following Python script reads this file, opens each listed project, runs the model, and checks whether the specified tags match the expected values.

## Unit testing using python
## Run various projects and compare results for changes

import time
import win32com.client as wc    ## PyWin32 COM Client
 
ProgID = "SysCADSimulator93.Application"
SysCAD = wc.DispatchEx(ProgID)  ## Fire up SysCAD
ScdPrj = r'\Project.spj'
Prj = None
Tags = None
Solver = None
ProBal = None
  
def readData(udata="utest.dat"):
    res = {}
    current = ""
    fp = open(udata)
    for x in fp.readlines():
        if x.startswith(r"C:"):
            current = x.strip()
            res[current] = []
            continue
        if not "," in x or x.startswith(" "):
            continue
        t, v = x.split(",")
        res[current].append((t.strip(), float(v)))
    return res
 
def Run():
    '''Start Probal and wait until it is solved'''
    ProBal.Start()                        ## Start ProBal
    while True:                           ## Wait until solved 
       time.sleep(.1)
       if ProBal.IsStopped:
           break
    
def Done(cl = True):
    global SysCAD, Solver, Tags, Prj
    time.sleep(1)
    del(Solver)
    del(Tags)
    del(Prj)
    del(SysCAD) 
 
def check(val, scval, eps=1.0e-14):
    mm = max(val, scval)
    if mm==0:  return True
    est = (val-scval)/mm
    return abs(est)<eps
 
def testProjects():
    global Prj, Tags, Solver, ProBal
    status = True
    work = readData()
    for prj, taglis in work.items():
        print ("Working on", prj)
        Prj = SysCAD.OpenProject(prj+ScdPrj)  ## Open project
        Tags = Prj.Tags
        Solver = Prj.Solver
        ProBal = Solver.ProBal
        Run()
        for tg, val in taglis:
            scval = Tags.TagValue(tg)
            if not check(val, scval):
                print (tg, val, scval)
                status = False
        SysCAD.CloseProject(False)
        time.sleep(1)
    if status:
        print ("All Looks Good")
    else:
        print ("Check Results")

if __name__ == "__main__":   
  testProjects()
  Done()