Python Example - Unit Test Framework

From SysCAD Documentation
Jump to navigation Jump to search

Navigation: User Guide ➔ COM Automation ➔ Python Automation ➔ Example - Unit Test Framework

Python Setup Python Examples
Installation &
Basic Usage & Scenarios Constrained FEM
(COM | scipy tools)
Model Testing
with GUI
External DLL
Model Generation
Importing data
to SysCAD


Unit Testing is important. Running a model to compare previous results to results after a change is very useful. COM Automation can be used to achieve this. The software development and testing team at Kenwalt use this extensively as described below. A simple Python example is also shown that can be used as a basis for your own unit testing with SysCAD.

SysCAD Batcher

At Kenwalt we have a utility referred to as "SysCAD Batcher" whose purpose is for automated unit testing as we work on the SysCAD code base related to unit models, solvers, thermodynamic calculations, etc. "Batcher" is for automated testing to ensure SysCAD results do not change unexpectedly as the code is revised. It play as an indispensable role in QA and Testing for maintaining quality and consistency for SysCAD updates and versions. COM Automation is used to achieve this. Batcher will cycle through a large collection of SysCAD projects solving each flowsheet model, including a number of step changes. For each solution, results are compared to a previously saved result set. Where there is a difference this is investigated. The investigation will trigger a review of code changes with two likely outcomes (a) some issue in the current code changes that requires fixing before release; or (b) the change is an understood and improvement in results or expected and beneficial.

In addition to structured testing of SysCAD solver results, "Batcher" is also used for other testing and product quality purposes. It is used to review changes in other statistics such as speed in terms of number of iterations and time per iteration. It is used to test that projects load correctly from a saved state (i.e. do not need to solve again when they were saved converged).

The library of SysCAD projects in Batcher for testing continues to grow. Simple, complex, small and large projects are included. The workflow is to add a new project and include a representative list of key result tags for the project. Identify some input tags that can be used for step changes. The first time the model is solved, the results are saved as the base case for future reference. Projects or step changes that are intentionally "bad" or "unrealistic" are intentionally included as it is important to test behaviour of error states and edge cases. The SysCAD solver needs to behave consistently in valid operational regions as well as outside of these. When a code change causes a change to the results that are understood and accepted, then the workflow is to update the base case results for future reference. Existing projects in the Batcher system are also updated and extended from time to time, for example test additional features & options, add more step changes, etc.

Model Testing

As described above, we have "Batcher" for ensuring that SysCAD results do not change as we develop and modify underlying algorithms in unit models, solvers, etc. However the developer (or modeller) may want to check specific tags in a number of different SysCAD project models; or they may want to check client SysCAD models when receiving SysCAD updates; or they want to check the results in a project after changes to thermodynamic data. A simple python script can be used for these purposes, with an example described here.

First create a text data file with the project folders and CSV values on subsequent lines with the tag and expected value. (CSV is useful here since we can just create the data file from an Excel report and save as CSV)

C:\SysCAD138\Examples\65 Smelting\FEM Blast Furnace Examples.spf 

P_102.Qo.QM.Total (t/h)       ,    10.2730644002              
P_102.Qo.QM.Solids (t/h)      ,    2.433901724                
P_102.Qo.QM.Vapours (t/h)     ,    7.8391626762               
                              ,    ============== Solids      
P_102.Qo.QM.CaO(s) (t/h)      ,    0.042618824                
P_102.Qo.QM.Fe(s) (t/h)       ,    2.3912829                  

C:\SysCAD138\Examples\03 UnitModels\FEM Simple Examples.spf

P_112.Qo.QM.H2O(g) (t/h)      ,    47.6731680080583    
P_112.Qo.QM.CO(g) (t/h)       ,    275.7948098716263  
P_112.Qo.QM.CO2(g) (t/h)      ,    358.8418906771008  
P_112.Qo.QM.H2(g) (t/h)       ,    9.179787443214618  
P_502.Qo.T (C)                ,   1738.67733669912

Blank lines, lines starting with whitespace and lines with no commas are ignored except for the project names.

Now the following script just opens up each of the listed projects, runs, then checks that the supplied tags have the given values.

## Unit testing using python
## Run various projects and compare results for changes

import time
import win32com.client as wc    ## PyWin32 COM Client
ProgID = "SysCADSimulator93.Application"
SysCAD = wc.DispatchEx(ProgID)  ## Fire up SysCAD
ScdPrj = r'\Project.spj'
Prj = None
Tags = None
Solver = None
ProBal = None
def readData(udata="utest.dat"):
    res = {}
    current = ""
    fp = open(udata)
    for x in fp.readlines():
        if x.startswith(r"C:"):
            current = x.strip()
            res[current] = []
        if not "," in x or x.startswith(" "):
        t, v = x.split(",")
        res[current].append((t.strip(), float(v)))
    return res
def Run():
    '''Start Probal and wait until it is solved'''
    ProBal.Start()                        ## Start ProBal
    while True:                           ## Wait until solved 
       if ProBal.IsStopped:
def Done(cl = True):
    global SysCAD, Solver, Tags, Prj
def check(val, scval, eps=1.0e-14):
    mm = max(val, scval)
    if mm==0:  return True
    est = (val-scval)/mm
    return abs(est)<eps
def testProjects():
    global Prj, Tags, Solver, ProBal
    status = True
    work = ReadData()
    for prj, taglis in work.items():
        print ("Working on", prj)
        Prj = SysCAD.OpenProject(prj+ScdPrj)  ## Open project
        Tags = Prj.Tags
        Solver = Prj.Solver
        ProBal = Solver.ProBal
        for tg, val in taglis:
            scval = Tags.TagValue(tg)
            if not check(val, scval):
                print (tg, val, scval)
                status = False
    if status:
        print ("All Looks Good")
        print ("Check Results")

if __name__ == "__main__":