View source for Python Example - External DLL

[[Category:Automation]] [[Category:Examples]]
{{Navigation|[[User Guide]]|[[COM Automation]]|[[Python Automation - Getting Started|Python Automation]]|altname=Example - External DLL}}
{{Python Example Header Tabs|currentpage=8}}
----
{{TOC|x=20%}}
== Using python to call functions in an external DLL ==

A common request with SysCAD is to be able to incorporate calculations from an external program or existing code into the SysCAD model. While the internal PGM language is powerful enough to do many engineering calculations, it may not be suitable for some situations, for example, the modeller might want to do a complex CFD (Computational Fluid Dynamics) simulation of the inside of a furnace, and use the data from that calculation to set the basic SysCAD parameters for a furnace model which is part of a large plant simulation. While PGM could in theory be used for writing a CFD model, there would be little point in doing so.

Even if the calculations involved are straightforward and could be implemented in PGM, there may be other reasons for needing to use an external library. The modeller may not have access to the actual code, which is often the case for third party libraries. The library or application is a "black box" which can be called via excel or other applications to do some necessary calculations, but the actual internal details (beyond what is necessary to make calls to the library) are unknown. You might have ten thousand lines of ancient (but functional and tested) FORTRAN code which does the necessary calculations, but would be a major exercise to convert to something more usable. 

For the purposes of this example we assume we have an external library as a DLL. (If all we have is an app of some sort, we would have to look at using the app to perform the calculations, saving the results to a file, and then reading the results back and into SysCAD)

We also assume we want to be repetitively calling the external library - in fact at every step of the SysCAD solution: if we just do the calculations once, we can simply save the results to an Excel spreadsheet or text file, and read them in using any of the standard procedures.

As an illustration, we will look at doing a dynamic simulation of a reaction tank, where we calculate the reaction rate externally. The reaction rate depends on the tank temperature, pressure and composition of the contents and we have a "black box" external library that does this calculation. So at each step of the simulation we

# Read the values of T, P and composition from the SysCAD model.
# Call the external library function with these values
# Set the rate in the SysCAD model
# Step the SysCAD model and repeat.

The first and third items are already covered in the general discussion of COM automation, and the final item is just a COM call as well, so we will concentrate on the external calling mechanism.

== Calling a dll from python ==
In fact many python libraries are already DLLs - any python file with an extension .pyd is nothing more than a DLL which python can use directly. If you have access to the source code for the external library, you can use SWIG (Simplified Wrapper Interface Generator) to compile this to a .pyd library which can be used directly in python.  

In this case though, all we have is the library and in it are some functions we need to access. In order to use this (in excel or wherever) we need to know the ''signature'' of the functions we want to call (and these need to be accessible externally). The signature describes the parameters the function takes, as well as the result. A typical signature might for the rate equation might be

<syntaxhighlight lang="cpp">
 double rateFn(double T, double p, double* X)
</syntaxhighlight>

This is a function that takes three parameters 
* two floating point numbers (temperature and pressure)
* a pointer to an array of floating point numbers (the concentrations of the components which affect the rate) 
It does some calculation, and returns a single floating point number as a result. We assume we have a dll available '''RATECALC.dll''' that exports a function rateFn with the appropriate signature.
* Signatures are typically specified in c/c++ header files 

There are a number of modules that can be used for accessing external DLLs from python, for this example we will use the ctypes module described here: https://docs.python.org/3/library/ctypes.html (and it is worthwhile reviewing the examples in this link).  

Before we can call the function from python we need to do a bit of setup:

<syntaxhighlight lang="python">
import ctypes                    ## import ctypes module
pd = ctypes.CDLL("RATECALC")     ## load the dll
print (pd.rateFn)                ## quick check that the function is there
</syntaxhighlight>

* The python interpreter needs to be able to find the DLL - so this needs to be in the same directory as the python script, or on the system PATH
* There are different types of dll, depending on the calling convention, and different ctypes methods for these: WinDLL for stdcall, OleDLL for windows functions returning HRESULT etc. You may need to experiment to see what works if your library is not documented fully.
* If the DLL is compiled as x86 (32 bit) then you '''must''' use a 32bit version of python, and similarly for 64bit compilations, you must use 64bit python. There are some workarounds to this using interprocess communication: https://pypi.org/project/msl-loadlib/

=== Function parameters and return value ===
In order to call the function, we need to provide parameters, and this means translating from the python representation to something the DLL can understand, as well as translating the result back to a python data type. The c_types module has primitive c-compatible data types (classes) that can take care of this. A '''c_double''' is used when the parameter or result is a double precision floating point number. (Integers and unicode strings can be passed directly) 

<syntaxhighlight lang="python">
from ctypes import c_double      ## we use this a lot so import it directly
pd.rateFn.restype = c_double     ## return value is a double
pd.rateFn.argtypes = [c_double, c_double, ctypes.POINTER(c_double)]
</syntaxhighlight>

Here we specify that the return value is a double, and that the arguments are two doubles and a pointer (to a double). The ctypes '''POINTER''' method returns a new type representing a pointer to its argument. 

Before we call the function, we still need to create a python array type object that can be used as the pointer argument. The recommended way of doing this is by multiplying by an array type by a positive integer and creating an instance of the resulting class, to which we can then assign values:

<syntaxhighlight lang="python">
XIn = (c_double*3)()
XIn[0] = 0.01
XIn[1] = 0.2
XIn[2] = 0.0
</syntaxhighlight>

Finally we can actually call the function:

<syntaxhighlight lang="python">
rate = pd.rateFn(T, P,  XIn)
</syntaxhighlight>

If all goes well, then rate is now a python double type with the right value!

== Putting it all together ==
The following python code opens the distributed Dynamic Plant Example (assuming SysCAD is installed in the default location), then runs for 50 steps while at each step calculating the dynamic reaction rate used in the tank by calling a function in an external dll:

<syntaxhighlight lang="python">
#! python3 -i
import ctypes
from ctypes import c_double    ## we use this a lot so import it directly
import SysCAD.syscadif as scif

pd = ctypes.CDLL(r"C:\devel\code\ReactionRates\x64\Release\ReactionRates")  ## Modify this to where the DLL is located (or add the location to the PATH)
pd.rateFn.restype = c_double
pd.rateFn.argtypes = [c_double, c_double, ctypes.POINTER(c_double)]
XIn = (c_double*3)()

sc = scif.SysCADCom()
ScdDir = r'C:\SysCAD139'
ScdPrj = r'\ExamplesDynamic\Dynamic Plant Example(Python DLL Control)-01.spf\Project.spj'
sc.OpenProject(ScdDir+ScdPrj)  ## Open project 

ModelTags = [
    "REACTION_TANK.Content.T (K)",
    "REACTION_TANK.OperatingP.Result (kPa)",
    "REACTION_TANK.Content.MF.H2SO4(aq) (%)", 
    "REACTION_TANK.Content.MF.NiSO4(aq) (%)" , 
    "REACTION_TANK.Content.MF.NiO(s) (%)"
    ]

RateTag = "REACTION_TANK.Cont.RB.R1.Extent.Rate (/s)"

def pyRateFn(T, P,  X):
    ''' A function calculated from temperature, pressure and concentrations'''
    x0, x1, x2  = X   # concentrations
    a, b, c = 0.1, 0.3, 0.5   # concentration tuning parameters.
    return 0.001*pow (T-273.15, 0.2)*(a*x0 + b*x1 + c*x2)


def SetRate(useDll = False):
    tagvals = sc.getTags(ModelTags)
    T, P, X = tagvals[0], tagvals[1], tagvals[2:]
    if useDll:     # Calculate rate from external DLL
        for i in range(3):
            XIn[i] = X[i]
        rate = pd.rateFn(T, P, XIn)
    else:
        rate = pyRateFn(T, P, X)   ## Calculate rate from python function
    print ("T, P, Concentrations, rate = ", T, P, X, rate)
    sc[RateTag] = rate


def RunModel(nsteps = 100, useDll = False):
    sc.idle()
    for i in range(nsteps):
        SetRate(useDll)
        sc.step()
    sc.stop()
</syntaxhighlight>

* We use the Dynamic methods Idle(), Step() and Stop() to drive the actual simulation
* Before each Step(), we get the tag values, calculate the rate using the DLL call and set the rate tag.

== Example DLL ==

If you want to experiment with this, you should build a DLL using whatever programming language you have available. You can use Visual Studio Community Edition or any other compiler. In Visual Studio, create a new DLL project RATECALC and add a suitable test function to the code: the function here does the same calculation as the python function pyRateFn in the previous code.

<syntaxhighlight lang="cpp">
// dllmain.cpp : Defines the entry point for the DLL application.
#include "pch.h"
#include <math.h>
BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved)
{
   switch (ul_reason_for_call)    {
   case DLL_PROCESS_ATTACH:
   case DLL_THREAD_ATTACH:
   case DLL_THREAD_DETACH:
   case DLL_PROCESS_DETACH:
       break;
   }
   return TRUE;
}

extern "C" {
    __declspec(dllexport)  double rateFn(double T, double p, double* x)
    {
        const double a = 0.1; 
        const double b = 0.3; 
        const double c = 0.5;
        return 0.001 * pow(T - 273.15, 0.2) * (a * x[0] + b * x[1] + c * x[2]);
    }
}
</syntaxhighlight>

* The function should be declared as '''extern "C"'''. This relates to the calling convention and name mangling in C++
* You have to export the function for it to be accessible in the library, hence __declspec(dllexport)

=== Calling FORTAN Code ===

A common case is FORTRAN code, and there are a few trick to remember here. If you have a DLL created from compiled FORTRAN, these tips will get you started. 

* FORTRAN function calls are always by ''reference'', and so POINTER data types.
* FORTRAN compilers may append an _ to the function name, so if the original FORTRAN code has a function RATE, the actual name in the DLL will be RATE_
* The return value is generally equivalent to the corresponding c data type; however for things like COMPLEX variables and strings, there are additional pointer arguments required in the function call to handle all the return data. 
* Multidimensional arrays are stored in column-major order.

A call to a FORTRAN function in a compiled library FORTRANDLL.dll

<syntaxhighlight lang="fortran">
DIMENSION X(3)
FUNCTION RATE(T, P, X)
</syntaxhighlight>

would use python code like
     
<syntaxhighlight lang="python">
pd = ctypes.CDLL("FORTRANDLL")
fm = pd.RATE_
fn.argtypes = [ctypes.POINTER(c_double), ctypes.POINTER(c_double), ctypes.POINTER(c_double)]
...
rate = fn(T, P, X)  # as before
</syntaxhighlight>

== Summary ==

* It is worth getting the python interface to the DLL sorted before bringing SysCAD into the model
* There are any number of things that can go wrong, some patience is needed.
* Good luck!!!