Restrictions and known bugs¶
streamsx versions 2.0 and higher are incompatible with Cloud Pak for Data less than version 3.5.0.0
For use of streamsx 2.0, the com.ibm.streamsx.iot toolkit must be updated to minimum version 1.3.0 on the build service or a local Streams installation. Otherwise dependency errors will occur when the application is compiled:
/opt/ibm/streams/bin/sc -M `cat main_composite.txt` ... CDISP0384E ERROR: The <application> toolkit requires version 2.0.0 of the com.ibm.streamsx.topology toolkit, but version 1.17.0 of the required toolkit is currently installed. CDISP0884W WARNING: Errors occurred while the toolkits were loading. CDISP0092E ERROR: Because of previous compilation errors, the compile process cannot continue. make: *** [all] Error 1 Exception in thread "main" java.lang.IllegalStateException: Error submitting archive for build: <build-ID>
When this error occurs, the IOT-Toolkit must be updated or added to the toolkit path (for example environment variable STREAMS_SPLPATH). To update the IoT toolkit on the CP4D buildservice to version 1.3.0, follow these steps:
download the toolkit version 1.3.0 tarball from the toolkits release page on GitHub.
cd
to your download locationunpack the downloaded tarball:
tar -zxf streamsx.iot.toolkits-1.3.0-20201208-1147.tgz
upload the toolkit:
streamsx-streamtool [--disable-ssl-verify] uploadtoolkit --path com.ibm.streamsx.iot
you can verify that the new version is present, with
streamsx-streamtool [--disable-ssl-verify] lstoolkit --name com.ibm.streamsx.iot
. The old and the new version should be listed.
A job that is submitted with streamsx-streamtool to a Cloud Pak for Data with version above 3.0 is only visible in the Streams Console.
Job submission from a Python notebook within a CP4D project on Cloud Pak for Data with version above 3.0 may fail with exception message
CDIST3419E: Submission failed. Please provide the CP4D URL in your submission configuration.
In this case, specify the CP4D URL, for example from your browser’s address bar, in the submission configuration, for example:
cfg = ... cfg[ConfigParams.CP4D_URL] = 'https://cp4d-cluster.com' submission_result = submit(ContextTypes.DISTRIBUTED, topology, cfg)
For Python development outside of a CP4D, for example with a Jupyter notebook outside of CP4D, you must use an Anaconda or Miniconda Python installation.
No support for nested parallel regions at sources, i.e. nested
streamsx.topology.topology.Stream.set_parallel()
, for example:topo = Topology() s = topo.source(S()) s.set_parallel(3).set_parallel(2)
In this example, set_parallel(3) is ignored.
When tuples are nested within other tuples in a stream schema, the call style of instances of StreamSchema for a callable is always dict, whatever value the style property has. When the return value of a callable represents also a structured schema with nested tuples, the return type must also be a dict. Otherwise the behaviour is not defined.
No schema support for container types (list, map, set, and the like) with non-primitive value or element types as value or element types for other containers, also when encapsulated in a named tuple:
class A_schema(typing.NamedTuple): x: int y: int class B_schema(typing.NamedTuple): a_list: typing.List[A_schema] # supported, A_schema does not contain a container type at all class C_schema(typing.NamedTuple): c1: str c2: B_schema # supported, a container type can be nested at any depth class D_schema(typing.NamedTuple): d1: str d2: typing.Mapping[int, typing.List[int] # supported d3: typing.Mapping[int, typing.List[A_schema] # not supported: a container with non-primitive element type is direct value type of a map class E_schema(typing.NamedTuple): e1: bool e2: typing.Mapping[str, C_schema] # not supported: C_schema.c2.a_list is a list with non-primitive element type
Schemas support only primitive types for the key type of a map:
class A_schema(typing.NamedTuple): a1: int a2: int class B_schema(typing.NamedTuple): b1: str b2: typing.Mapping[str, A_schema] # supported b3: typing.Mapping[A_schema, str] # not supported, A_schema not supported as key type
Schemas support only primitive types as element type of a set:
class A_schema(typing.NamedTuple): a1: int a2: int class B_schema(typing.NamedTuple): b1: int b2: typing.Set[int] # supported b3: typing.Set[A_schema] # not supported
Python Composites (derived from
streamsx.topology.composite.Composite
) can have only one input port.No support to process final marker (end of stream) in Python Callables like in SPL operators
No hook for drain processing in consistent region for Python Callables
Submission time parameters, which are defined in SPL composites of other toolkits, or created by using streamsx.spl.op.Expression in the topology, cannot be accessed at runtime with streamsx.ec.get_submission_time_value(name).
The time-interval window (
streamsx.topology.topology.Stream.time_interval()
) is not supported bystreamsx.topology.topology.Window.aggregate()
. Use the spl.relational::Aggregate operator in an event-time stream.