Developing and running applications that use the HBase Toolkit

IBMStreams com.ibm.streamsx.hbase Toolkit > com.ibm.streamsx.hbase 3.9.3 > Developing and running applications that use the HBase Toolkit

To create applications that use the HBase Toolkit, you must configure either Streams Studio or the SPL compiler to be aware of the location of the toolkit.

Before you begin

  • Install the Apache Ambari on you Hadoop server. https://ambari.apache.org/ Configure the product environment via ambari and install Apache Hadoop (http://hadoop.apache.org/) Install Apache HBase (http://hbase.apache.org/), and Apache Zookeeper (http://zookeeper.apache.org/). All of these products can be configured via ambari.

  • The com.ibm.streamsx.hbase toolkit can also access to the: IBM Biginsighs https://www.ibm.com/support/knowledgecenter/en/SSPT3X_4.2.0/com.ibm.swg.im.infosphere.biginsights.welcome.doc/doc/welcome.html Hortonworks data platform https://hortonworks.com/products/data-platforms/hdp/

About this task

After the location of the toolkit is communicated to the compiler, the SPL artifacts that are specified in the toolkit can be used by an application. The application can include a use directive to bring the necessary namespaces into scope. Alternatively, you can fully qualify the operators that are provided by toolkit with their namespaces as prefixes.

Procedure

  1. For Apache HBase, you must configure your compile and runtime environment by setting the HBASE_HOME environment variable:
    • If your Streams application will be running on the same resource where HBase is installed, set HBASE_HOME to the directory that contains HBase.
    • Otherwise, copy the /hbase-install-dir/lib directory to the file system where Streams will be running, making sure to follow symbolic links: 
        cp -Lr path-to-hbase/lib /path-on-streams-host
    • Set HBASE_HOME to the directory you created, for example, to path-on-streams-host.
  2. Supply the operator with HBase configuration information. By default, HBASE_HOME/conf is searched for the hbase-site.xml. If Streams will not be running on the same resource as the HBase installation, you can do one of the following:
    • Create a conf directory in the same directory you created in step 1 and copy the hbase-site.xml from your HBase installation to that directory.
    • Copy the hbase-site.xml file from the conf directory in your HBase installation into a directory accessible by your application and use the hbaseSite parameter to specify the path to the copied file.
  3. Configure the SPL compiler to find the toolkit root directory. Use one of the following methods:
    • Set the STREAMS_SPLPATH environment variable to the root directory of a toolkit or multiple toolkits (with : as a separator). For example: 
      export STREAMS_SPLPATH=$STREAMS_INSTALL/toolkits/com.ibm.streamsx.hbase
    • Specify the -t or --spl-path command parameter when you run the sc command. For example: 
      sc -t $STREAMS_INSTALL/toolkits/com.ibm.streamsx.hbase -M MyMain
      where MyMain is the name of the SPL main composite. Note: These command parameters override the STREAMS_SPLPATH environment variable.
    • Add the toolkit location in IBM Streams Studio.
  4. Develop your application. To avoid the need to fully qualify the operators, add a use directive in your application.
    • For example, you can add the following clause in your SPL source file: 
      use com.ibm.streamsx.hbase::*;
      You can also specify a use clause for individual operators by replacing the asterisk (*) with the operator name. For example: 
      use com.ibm.streamsx.hbase::HBASEDelete;

  5. Build your application. You can use the sc command or Streams Studio.

  6. Start the IBM Streams instance. Remember to set HBASE_HOME and HADOOP_HOME This can be done using the Streams Console or the streamtool utility. The streamsx.hbase toolkit from version 3.1.0 delivers all needed apache clint jar libaries. If you use the hbaseSite parameter to specify the hbase-site.xml file, it is not necessary to set the HBASE_HOME and HADOOP_HOME environment variables.

  7. Run the application. You can submit the application as a job by using the streamtool submitjob command or by using Streams Studio.