Developing SPL applications

A stream processing application is a collection of operators, all of which process streams of data. Typically, Streams Processing Language (SPL) is used for creating Streams applications.

Creating a SPL application

Procedure

  1. Bring up the Command Palette and select Create SPL Application.
  2. In the panel that appears, specify the following:
    • Application folder path: The path of the folder where you want to create the SPL application. It is recommended to use an empty folder.
    • Namespace: The namespace of the SPL application. You can use namespaces to organize your SPL code, similar to Python modules or Java packages.
      Note: The namespace must start with an ASCII letter or underscore, followed by ASCII letters, digits, underscores, or period delimiters.
    • Main composite name: The main composite name of the SPL application. The entry point of a SPL application is called a main composite and it is defined in a SPL source file (.spl).
      Note: The name must start with an ASCII letter or underscore, followed by ASCII letters, digits, or underscores.
  3. Click on the Create button to create the SPL application.

Watch and learn: This video demonstrates how to create an SPL application.

Results

A minimal SPL application is created in the folder location you specified. The folder structure is:

/+ <application-folder>
   /+ <namespace>
      /+ <main-composite>.spl
   /+ info.xml

Example SPL source file (<main-composite>.spl):

namespace yournamespace;

composite YourMainComposite {
  graph
    stream<rstring str> Src = Beacon() {
      // Insert operator clauses here
    }
}

Note that a toolkit information model (info.xml) file is also created since a SPL application is considered a toolkit. This file contains the name, description, and version information about the toolkit. It also contains any dependencies that might exist on other toolkits, in the form of a list of toolkit names and versions or range of versions required.

Example info.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<info:toolkitInfoModel xmlns:common="http://www.ibm.com/xmlns/prod/streams/spl/common" xmlns:info="http://www.ibm.com/xmlns/prod/streams/spl/toolkitInfo">
  <info:identity>
    <info:name>YourMainComposite</info:name>
    <info:description/>
    <info:version>1.0.0</info:version>
    <info:requiredProductVersion>4.3.0.0</info:requiredProductVersion>
  </info:identity>
  <info:dependencies/>
</info:toolkitInfoModel>

Editing a SPL application

Once you have created a SPL application, the next step is to develop the application. For more information, see Writing stream processing applications.

Defining operators

Connect operators together to ingest data from external data sources, process and analyze data, and write data to external data systems. SPL operators are defined in toolkits, which can be divided into two categories:

  • Toolkits developed by IBM

    • Standard toolkit: The SPL standard toolkit (spl) provides a set of operators, types, and functions that are available by default to develop SPL applications. For example, here are some commonly used operators: Aggregate, Beacon, FileSink, FileSource, Filter, Join, Split, and Throttle. You can find samples for these operators in the Samples Catalog.

    • Specialized toolkits: Specialized toolkits are sets of artifacts that are developed for specific business environments and activities. Some toolkits are included with the IBM product and others are publicly available on GitHub. For example, here are some commonly used toolkits: DateTime, DPS, Text, and TimeSeries.

      Note: Specialized toolkits have certain requirements and restrictions if you use Streams in an OpenShift or Kubernetes environment. Review Specialized toolkits before you develop applications with specialized toolkits to be deployed in these environments.

  • Custom toolkits

    • You can create toolkits outside the context of creating applications by developing all the necessary artifacts, organizing them into namespace directories, and running the spl-make-toolkit command to index the toolkit. For more information, see Developing toolkits and Building a toolkit.
    • Toolkits can also be created by third-party developers and integrated into your application.

To use an operator from a toolkit that is not part of the Streams product (e.g., a public streamsx toolkit), you must add the toolkit and import the toolkit.

Editing features

This extension includes rich SPL code editing features (IntelliSense).

  • Code completion: Type control + space or a trigger character (such as the dot character (.)) to show a list of suggestions. You can use this to add operators using templates.
  • Find references: To find out where a stream or operator is used within a file or workspace, right-click on it and select Go to References or Find All References.
  • Folding: If a closing bracket isn’t easily visible, fold portions of code using the folding icons on the left side of the editor.
  • Bracket matching: Bracket matching shows you the scope of a declaration when moving the cursor near a bracket.
  • Find problems: Syntax errors in your code are reported in the editor and in the PROBLEMS panel at the bottom.
  • View documentation: You can hover over any artifact, such as a parameter, stream, or operator, and its documentation is displayed if it is available.

Watch and learn: This video demonstrates some of the editing features available for SPL files.

Selecting a Streams color theme

For an enhanced development experience, it is recommended that you set the VS Code color theme to one of the provided themes: Streams Light (based on Streams Studio) or Streams Dark.

Themes