Python Application API functions

Fetches information from an external system and presents that information as a stream. Takes a zero-argument callable that returns an iterable of tuples. Each tuple that is not None from the iterator returned from iter(func()) is present on the returned stream.

param func: A zero-argument callable that returns an iterable of tuples. A tuple is represented as a Python object that must be picklable.

return: A Stream whose tuples are the result of the output obtained by invoking the provided callable.

subscribe(self, topic, schema=CommonSchema.Python) function on Topology

Subscribe to a topic published by IBM Streams applications. A Streams application may publish a stream to allow other applications to subscribe to it. A subscriber matches a publisher if the topic and schema match.

schema defaults to CommonSchema.Python which subscribes to streams published by Python applications where the stream contains Python objects.

Setting schema to CommonSchema.Json subscribes to streams of JSON objects published by IBM Streams applications. JSON is used as a common interchange format between all languages supported by IBM Streams, including SPL, Java, Scala and Python.

Setting schema to CommonSchema.String subscribes to streams of strings published by IBM Streams applications. String is used as a common interchange format between all languages supported by IBM Streams, including SPL, Java, Scala and Python.

Setting schema to any other SPL schema subscribes to streams with the matching SPL tuple type published by IBM Streams applications. In general this will be SPL applications though Java and Scala applications can publish streams with an SPL schema. In Python tuples will be transformed into a Python dictionary object (based on the schema). Each tuple attribute will be converted into an appropriate Python type and added to the Python dictionary object using the name of the attribute as the dictionary key value.

int8,int16,int32,int64 : int
uint8,uint16,uint32,uint64 : int
float32,float64 : float
complex32,complex64 : complex
rstring : str
ustring : str
boolean : boolean
list : list
map : dictionary

See namespace:com.ibm.streamsx.topology.topic for more details.

param topic: Topic to subscribe to. param schema: Schema to subscriber to. Defaults to CommonSchema.Python representing Python objects. return: A Stream whose tuples have been published to the topic by IBM Streams applications.

sink(self, func) function on Stream

Sends information as a stream to an external system. Takes a user provided callable that does not return a value.

param func: A callable that takes a single parameter for the tuple and returns None. The callable is invoked for each incoming tuple.

return: None

filter(self, func) function on Stream

Filters tuples from a stream using the supplied callable func. For each tuple on the stream the callable is called passing the tuple, if the callable return evalulates to true the tuple will be present on the returned stream, otherwise the tuple is filtered out.

param func: A callable that takes a single parameter for the tuple, and returns True or False. If True, the tuple is included on the returned stream. If False, the tuple is filtered out. The callable is invoked for each incoming tuple.

return: A Stream containing tuples that have not been filtered out.

transform(self, func) or map(self, func) function on Stream

Transforms each tuple from this stream into 0 or 1 tuples using the supplied callable func. For each tuple on this stream, the returned stream will contain a tuple that is the result of the callable when the return is not None. If the callable returns None then no tuple is submitted to the returned stream.

param func: A callable that takes a single parameter for the tuple, and returns a tuple or None. The callable is invoked for each incoming tuple.

return: A Stream containing transformed tuples.

multi_transform(self, func) or flat_map(self, func) function on Stream

Transforms each tuple from this stream into 0 or more tuples using the supplied callable func. For each tuple on this stream, the returned stream will contain all non-None tuples from the iterable. Tuples will be added to the returned stream in the order the iterable returns them. If the return is None or an empty iterable then no tuples are added to the returned stream.

param func: A callable that takes a single parameter for the tuple, and returns an iterable of tuples or None. The callable must return an iterable or None, otherwise a TypeError is raised. The callable is invoked for each incoming tuple.

return: A Stream containing transformed tuples.

isolate(self) function on Stream

Guarantees that the upstream operation will run in a separate process from the downstream operation when the application is executed in distributed mode.

return: Stream

low_latency(self) function on Stream

The function is guaranteed to run in the same process as the upstream Stream function. All streams that are created from the returned stream are also guaranteed to run in the same process until end_low_latency() is called.

return: Stream

end_low_latency(self) function on Stream

Returns a Stream that is no longer guaranteed to run in the same process as the calling stream.

return: Stream

union(self, streamSet) function on Stream

The Union operator merges the outputs of the streams in the set into a single stream.

param streamSet - Set of streams outputs to merge

returns: Stream

parallel(self, width, routing=None, func=None) function on Stream

Parallelizes the stream into width parallel channels. Tuples are routed to parallel channels such that an even distribution is maintained. Each parallel channel can be thought of as being assigned its own thread. As such, each parallelized stream function are separate instances and operate independently from one another.

parallel will only parallelize the stream operations performed after the call to parallel and before the call to end_parallel

Parallel regions aren't required to have an output stream, and thus may be used as sinks. In other words, a parallel sink is created by calling parallel and creating a sink operation. It is not necessary to invoke end_parallel on parallel sinks.

Nested parallelism is not currently supported. A call to parallel should never be made immediately after another call to parallel without having an end_parallel in between.

Every call to end_parallel must have a call to parallel preceding it.

param width: degree of parallelism

param: func - (Optional) Function called when HASH_PARTIONED routing is specified. The function provides an int32 value to be used as the hash that determines the tuple routing to downstream operators

return: Stream whose subsequent processing will occur on width channels.

end_parallel(self) function on Stream

Ends a parallel region by merging the channels into a single stream

return: A Stream for which subsequent transformations are no longer parallelized

publish(self, topic, schema=schema.CommonSchema.Python) function on Stream

Publish this stream on a topic for IBM Streams applications to subscribe to. A Streams application may publish a stream to allow other applications to subscribe to it. A subscriber matches a publisher if the topic and schema match.

schema defaults to CommonSchema.Python which publishes the stream as Python objects.

Setting schema to CommonSchema.Json publishes the stream as JSON objects. Each tuple (a Python object) is converted using json.dumps. Note that each tuple object ype on the stream must be able to be converted to JSON. JSON is used as a common interchange format between all languages supported by IBM Streams, including SPL, Java, Scala and Python.

Setting schema to CommonSchema.String publishes the stream as strings. Each tuple (a Python object) is converted using str(). String is used as a common interchange format between all languages supported by IBM Streams, including SPL, Java, Scala and Python.

See namespace:com.ibm.streamsx.topology.topic for more details.

param topic: Topic to publish this stream to. param: schema: Schema to publish. Defaults to CommonSchema.Python representing Python objects.

return: None

autonomous(self) function on Stream Starts an autonomous region for downstream processing. By default IBM Streams processing is executed in an autonomous region where any checkpointing of operator state is autonomous (independent) of other operators.

This function may be used to end a consistent region by starting an autonomous region. This may be called even if this stream is in an autonomous region.

Autonomous is not applicable when a topology is submitted to a STANDALONE contexts and will be ignored.

Supported since v1.5