Developing toolkits

A toolkit is a collection of artifacts that are organized into a package. Toolkits make SPL or native functions and primitive or composite operators reusable across different applications. A toolkit provides one or more namespaces that contain functions, operators, and types that are packaged as part of the toolkit, all of which can then be reused in other Streams applications.

The Streams platform includes many toolkits, each with useful operators and functions. For example, the operators used to connect to Apache Kafka are in the Kafka toolkit.

However, you can create your own custom toolkit if the included toolkits do not contain the functionality that you need.

For more information, see Developing toolkits.

Building a toolkit

Build a custom toolkit that you can use in your streaming applications.

Notes:

  • Toolkit builds are intended for custom toolkits. Toolkits that are included with the Streams product are pre-built.
  • Developing a toolkit in a folder that is included in the Toolkit Paths setting is not recommended. Only indexed toolkits should be specified in the setting (i.e., toolkits where the toolkit.xml file is up-to-date with the contents of the toolkit directory).

Before you begin

You must open or add the folder for the toolkit you wish to build in Visual Studio Code.

A Makefile at the root of the toolkit folder is required. It must run the spl-make-toolkit command, which indexes the toolkit and makes the functions and operators it contains ready for use by SPL programs. A simple Makefile would look like the following.

all:
	spl-make-toolkit -i .

For other Makefile examples, see the toolkits in the Streams GitHub organization.

Gradle builds

If the toolkit uses Gradle as the build tool, the Makefile must run the target Gradle task. For example, consider the streamsx.kafka toolkit. A simple Makefile would look like the following.

all:
	cd com.ibm.streamsx.kafka; ../gradlew build

Procedure

  1. Add your Streams 5.5 instance to the Streams Explorer in VS Code if you haven’t already.
  2. Right-click on a custom toolkit folder (must include a Makefile at the root of the folder), Makefile file, info.xml file, or toolkit.xml file and select Build Toolkit.
  3. Follow the prompts to submit the build.

Watch and learn: This video demonstrates how to build a toolkit.

Results

After a build completes successfully, the indexed toolkit is extracted to the toolkit folder (overwriting the existing files). You will be presented with two options:

  • Download Toolkit: Downloads the toolkit archive file (.tgz) to the toolkit folder.
  • Add Toolkit to Toolkit Path: Adds the toolkit to a toolkit path folder (defined in the Toolkit Paths setting). This makes the indexed toolkit available to use in your Streams applications.

What to do next

Add the toolkit and use the toolkit in your Streams applications.