Extracting SPL operators from Python

Toolkits > com.ibm.streamsx.topology 2.1.0 > com.ibm.streamsx.topology.python > Creating SPL Operators from Python code > Extracting SPL operators from Python

Overview

To create SPL operators in a toolkit, execute: 

spl-python-extract -i toolkit-directory

When not using the streamsx package then the extraction script is available in the bin directory of this toolkit. 

python3 spl-python-extract.py -i toolkit-directory

Any Python module in the toolkit's opt/python/streams directory will have its decorated classes and functions converted to SPL operators.

These directories in a toolkit are automatically added to the Python search path during execution of an operator.
  • opt/python/streams - Contains modules that define Python callables that will be created as SPL operators
  • opt/python/packages - Root directory for Python packages hierarchy.
  • opt/python/modules - Arbitrary modules, not defined as a packages.
  • impl/nl - The directory for the resource files.

A single Python embedded runtime is used by an SPL processing element (PE), thus when multiple operators implemented in python are fused into the same PE they share the same runtime. The library and include paths to the Python runtime are set from the version of Python used to execute spl-python-extract.

The toolkit has no dependency on this toolkit (com.ibm.streamsx.topology) once spl-python-extract has been executed.

The toolkit is extended with resources files impl/nl/<locale>/TopologySplpyResource.xlf. These resource files must be added to the the resource section in file info.xml. The spl-python-extract.py program prepares or checks the info.xml file in the project directory
  • if the info.xml does not exist in the project directory, it copies the template info.xml into the project directory. The project name is obtained from the project directory name
  • If there is a info.xml file, the resource section is inspected. If the resource section has no valid message set description for the TopologySplpy Resource a warning message is printed'''.

The sample SPL toolkit samples/python/com.ibm.streamsx.topology.pysamples contains opt/python/streams/spl_samples.py for examples of how data is passed into and out of Python from SPL, using positional arguments.

E.g. 

spl-python-extract -i samples/python/com.ibm.streamsx.topology.pysamples

Usage 

usage: spl-python-extract [-h] -i DIRECTORY [--make-toolkit]

Extract SPL operators from decorated Python classes and functions.

optional arguments:
  -h, --help            show this help message and exit
  -i DIRECTORY, --directory DIRECTORY
                        Toolkit directory
  --make-toolkit        Index toolkit using spl-make-toolkit