Installing Streams Quick Start Edition with Docker
Edit meIBM® Streams Quick Start Edition can help you get started with Streams quickly, without having to install a Streams cluster environment.
Introduction
This document describes the installation, configuration, first steps, and common Docker management scenarios for Streams Quick Start Edition running in a Docker environment.
Overview
Here’s an overview of the steps to get up and running:
- Install and configure Docker.
- Set up a mapped directory on your local host file system for the Docker container.
- Download and install Streams Quick Start Edition.
- Configure your
hosts
file. - Access the Streams Quick Start Edition:
- Use a VNC client.
- Use a Secure Shell (SSH).
- Manage the Docker container.
- View the known issues.
Supported environments
The following operating system and Docker environments are supported for Streams Quick Start Edition 4.3.
Operating system | Docker |
---|---|
Windows 10 | Docker Community Edition 17.03.1-ce or later |
MacOS El Capitan 10.11 or later | Docker Community Edition 17.03.1-ce or later |
Linux systemd-based distribution | Docker Community Edition 17.03.1-ce or later. Note: If you use Standard Docker (Red Hat), you must adjust the Docker environment manually to allow for 50 GB of Docker storage. |
Docker configuration requirements
Configure your Docker environment as follows.
Setting | Minimum | Recommended |
---|---|---|
CPUs | 2 | 4 |
Memory | 4 GB | 8 GB |
Disk space | 20 GB | 50 GB or greater depending upon number and size of projects. |
Installing and configuring Docker
- Download and install Docker-CE 17.03.1 or later from:
https://store.docker.com/editions/community/docker-ce-desktop-windows
Note: Installing Docker might conflict with settings required for other VM technologies such as Oracle VM VirtualBox. The installation might also require that you set your VM technology settings in your BIOS settings. If this is necessary, Docker warns you when you try to start it and tells you which settings to change.
- After you install Docker, configure it as follows:
- Right-click the Docker icon in the system tray and select Settings.
- Under Setting, select Shared Drives. Share your hard disk drive (usually your C: drive).
- Under Settings, select Advanced and configure the CPUs (minimum 2, recommended 4) and memory (minimum 4 GB, recommended 8 GB).
- Open a PowerShell session from the Windows menu and test that Docker is set up correctly by running this command:
docker run hello-world
If Docker is set up correctly, you will receive a response that includes the following text:
Hello from Docker! This message shows that your installation appears to be working correctly.
-
Download and install Docker Community Edition for Mac from:
https://store.docker.com/editions/community/docker-ce-desktop-mac
-
After you install Docker, configure it as follows:
- From the Docker icon in the menu, select Preferences > Advanced.
- Configure the CPUs (minimum 2, recommended 4) and memory (minimum 4 GB, recommended 8 GB).
-
Open a Terminal session and test that Docker is set up correctly by
running this command:
docker run hello-world
If Docker is set up correctly, you will receive a response that includes this text:Hello from Docker! This message shows that your installation appears to be working correctly.
- Use the OS package manager to install Docker Community Edition. (For Red Hat, install Standard Docker).
- If necessary, increase the Docker installation storage space for
the Streams Quick Start Edition image.
Tip: The amount of storage space that is needed depends on whether you map directories to the local file system and the amount and the size of your applications and data.
- The Red Hat Standard Docker storage space is set to 10 GB. In this case, you must increase the storage space.
- Storage space for Docker-ce is usually set to 20 GB. In this case, you might need to increase the storage space.
- To set the Docker default image size (Base Device Size), log in as root or use
sudo
to create or edit the/etc/docker/daemon.json
file. - Add or modify the following code, where
XX
represents the default image size in GB.- If you keep the
/home/streamsadmin/workspace
directory in the Docker image, set the storage space to at least 50 GB. - If you map the
/home/streamsadmin/workspace
to a local host directory, you can set the Docker storage to 20 GB, because projects and data can be stored on your local drive instead of inside the Docker container.
{ "storage-opts": ["dm.basesize=XXG"] }
- If you keep the
- Restart the docker service:
sudo systemctl restart docker
- Verify the default image size by entering the following command:
docker info |grep "Base Device Size:"
Mapping Docker container directories to the local host file system
(Optional, but recommended)
During the Streams Quick Start Edition installation, you are
prompted to map two Docker directories,
/home/streamsadmin/workspace
and /home/streamsadmin/hostdir
, to
local host directories.
Because the container has limited space, in most cases the best option is to map to external directories. Doing so will help prevent the container from exceeding its internal disk limits and make it easier for you to back up your project data and upgrade your Streams Docker container in the future.
-
Create a directory on your host machine where you can keep both mapped directories isolated from other host machine files. For example, you might create a directory called
mappedDockerFiles
in the home directory of your host machine:mkdir <HOME DIRECTORY>/mappedDockerFiles
Important: Do not map the Docker files directly to the top-level user home directory.
-
Make note of the directory. You will provide it during the Quick Start Edition installation.
The installation creates the mapped directories in your local host
file system within mappedDockerFiles
for two directories, the
internal workspace and hostdir subdirectories. During
installation, you are prompted for the names that you want to use
under ~/mappedDockerFiles
.
The following explains the use of the two directories.
-
The workspace directory: Streams projects created using Streams Studio would be stored in
/home/streamsadmin/workspace
on the Docker container. By mapping the workspace directory to a folder on your local host, the projects stored in/home/streamsadmin/workspace
will be stored on the mapped workspace directory on your local host machine. Storing them here instead of in the internal Docker container means that you can reuse this directory for easy recovery of your projects. For example, if you later upgrade your Streams4Docker installation, you use the Import function of Streams Studio to import the projects from a previous Streams4Docker installation. -
The hostdir directory: If this directory is mapped to the local host file system, it corresponds to
/home/streamsadmin/hostdir
in the Docker container. Use it to store files that you want to share between your Docker container and your local host. Keep any large files that are used in your Streams4Docker container in this directory when possible, because files in this directory do not use up space inside the Docker container.
Download and install Streams Quick Start Edition
After you configure Docker and map your directories, you are ready to download and install Quick Start Edition.
You have two options:
Download and install from Docker Hub
The following sections contain sample commands for downloading and installing Quick Start Edition from Docker Hub.
The commands that you use to download and install the container on Windows depend on whether you choose to map the container directories.
Container with mapped directories
This command is based on using the following mapped directories, replacing user with your user name:
C:\user\Documents\DockerMapped\workspace C:\user\Documents\DockerMapped\hostdir
Enter the following command, replacing user with your user name:
docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v //c/user/Documents/DockerMapped/workspace:/home/streamsadmin/workspace -v //c/user/Documents/DockerMapped/hostdir:/home/streamsadmin/hostdir -p 8443:8443 -p 9975:9975 -p 8006-8016:8006-8016 -p 8444:8444 -p 8080:80 -p 5905:5901 -p 4022:22 --name streamsdocker4300 -h 'streamsqse.localdomain' ibmcom/streams-qse:4.3.0.0 INSTALL
Container without mapped directories
Enter the following command:
docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 8443:8443 -p 9975:9975 -p 8006-8016:8006-8016 -p 8444:8444 -p 8080:80 -p 5905:5901 -p 4022:22 --name streamsdocker4300 -h 'streamsqse.localdomain' ibmcom/streams-qse:4.3.0.0 INSTALL
The commands that you use to download and install the container on Mac or Linux depend on whether you choose to map the container directories.
Container with mapped directories
This command is based on setting the MAPPED_WORKSPACE
and MAPPED_HOSTDIR
variables to the mapped directories you created, for example:
export MAPPED_WORKSPACE=$HOME/Documents/DockerMapped/workspace export MAPPED_HOSTDIR=$HOME/Documents/DockerMapped/hostdir
After you export the mapped drives, enter the following command:
docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v $MAPPED_WORKSPACE:/home/streamsadmin/workspace -v $MAPPED_HOSTDIR:/home/streamsadmin/hostdir -p 8443:8443 -p 9975:9975 -p 8006-8016:8006-8016 -p 8444:8444 -p 8080:80 -p 5905:5901 -p 4022:22 --name streamsdocker4300 -h 'streamsqse.localdomain' ibmcom/streams-qse:4.3.0.0 INSTALL
Container without mapped directories
Enter the following command:
docker run --privileged -d -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 8443:8443 -p 9975:9975 -p 8006-8016:8006-8016 -p 8444:8444 -p 8080:80 -p 5905:5901 -p 4022:22 --name streamsdocker4300 -h 'streamsqse.localdomain' ibmcom/streams-qse:4.3.0.0 INSTALL
Manual download and installation
Prerequisite: Make sure you are connected to the Internet.
- Click Download Streams Quick Start Edition to go to the IBM Streams Quick Start Edition page. Download IBM Streams Quick Start Edition V4.3.0 for Docker on Windows 10.
-
Extract the
Streams4WindowsDocker4300EL7.zip
file into a directory. -
Open PowerShell and go to the
Streams4WindowsDocker4300EL7
directory. -
Ensure that you have permissions to run a PowerShell script:
-
In the PowerShell window, determine your PowerShell execution policy:
get-executionpolicy
-
If your permissions are restricted, change them to unrestricted:
set-executionpolicy unrestricted
Tip: After you complete the installation, optionally reset your execution policy back to restricted by entering the following command:
set-executionpolicy restricted
-
In the PowerShell window, determine your PowerShell execution policy:
-
Run the install script by entering the following command:
./streamsdockerInstall.ps1
-
Follow the installation instructions.
- Read and accept the license agreement page and the Notices page.
- Choose whether to map host local directories into the Docker container. If you have not created a local directory for this purpose, see Mapping Docker container directories to the local host file system.
- Confirm the directories and start the installation.
- Click Download Streams Quick Start Edition to go to the IBM Streams Quick Start Edition page and use the following instructions for your operating system.
-
Extract the
Streams4Docker4300EL7
file into a directory. -
From your Terminal session, change to the
Streams4Docker4300EL7
directory. -
Run the install script by entering the following command:
./streamsdockerInstall.sh
-
Follow the installation instructions.
- Read and accept the license agreement page and the Notices page.
- Choose whether to map host local directories into the Docker container. If you have not created a local directory for this purpose, see Mapping Docker container directories to the local host file system.
- Confirm the directories and start the installation.
The installation takes from 25 minutes
to an hour or more depending on your host system configuration. When
the installation is complete, you are returned to a Streams4Docker
command prompt.
Configuring the hosts file
Streams operates by using a host name and
expects DNS to provide the conversion from host name to IP
address. Because Streams Quick Start Edition does not have a DNS server,
you can simulate one by using the hosts
file.
Before you can access Streams Quick Start Edition, you must set the
hosts
file to point the host name streamsqse.localdomain to the
127.0.0.1 loopback address.
- Open a text editor as an administrator. For example, for Notepad, find Notepad in your Windows menu, right-click it, and select **Run as Administrator**.
- Open
C:\Windows\System32\drivers\etc\hosts
.Tip: Set the Notepad filename filter to "All Files" to see the `hosts` file.
- Append
streamsqse streamsqse.localdomain
to the line that has the loopback address. For example:127.0.0.1 localhost streamsqse streamsqse.localdomain
- Save and close the file.
- Open a text editor with root authority.
- Open
/etc/hosts
. - Append
streamsqse streamsqse.localdomain
to the line that has the loopback address. For example:127.0.0.1 localhost streamsqse streamsqse.localdomain
- Save and close the file.
Accessing Streams Quick Start Edition with a VNC client
Use port 5905 to access Streams Quick Start Edition in the Docker container with a VNC client.
-
Open your VNC Client and connect to the VNC server:
streamsqse.localdomain:5905
-
When you are prompted for a password, enter:
passw0rd
The default user name for the Streams Console is streamsadmin.
On the Streams desktop, access the Streams applications from the Applications menu.
Accessing Streams Quick Start Edition with Secure Shell (SSH)
You can use ssh to access the container by specifying the streamsqse.localdomain and using port 4022. For example:
ssh -p 4022 streamsadmin@streamsqse.localdomain
By default, all passwords are passw0rd within an ssh session. If you use the user name streamsadmin and you want to run a command as root, you can use one of two methods:
-
sudo [command] Provide your streamsadmin password when you are prompted. You run commands by using sudo and return to streamsadmin ID after the command is completed.
-
sudo su - Provide your streamsadmin password when you are prompted. You will be logged in as root. To return to the streamsadmin user ID, enter: exit.
Managing the Docker container
The following commands are useful for managing your Docker container with Streams Quick Start Edition.
Opening a session on Docker
You can use the attach command or the exec command to open a session on Docker.
The difference between the attach and exec commands is that the attach command takes you into the main root session. The main root session which might be busy doing other functions (for example, it might be in the process of starting streams), which will prevent you from doing useful work, but will allow for better troubleshooting in case of a problem. The exec command opens a new separate root session.
Attaching to a running Docker container
-
Open PowerShell (Windows) or Terminal (MacOS or Linux).
-
Determine the name of the session by entering docker ps. The container streamsdocker4300 is displayed with a status of
running
. -
Attach to the container by entering this command:
docker attach streamsdocker4300
This command takes you to the Docker container Linux prompt.
Running a command in a running container with the exec command
Enter the Docker exec command:
docker exec -ti streamsdocker4300 /bin/bash
Detaching from the container
From within your attached Docker session, you can detach from the session and still leave session active.
- Use the appropriate key combination for your environment:
- Windows or Linux: Ctrl-q-p
- MacOS: command-q-p
Important: If you are attached to a container and enter exit, it might cause the container to stop.
- (Optional) Close the PowerShell or Terminal session.
Pausing and restarting the container
You can pause a container when it is not in use. The container pauses and all activity inside the container stops. You can unpause the container to use it again.
-
If you are running external programs that are connected to the container, close them.
-
From a PowerShell (Windows) or Terminal (MacOS or Linux) session, enter the appropriate command:
- To pause the container:
docker pause streamsdocker4300
- To unpause the container:
docker unpause streamsdocker4300
- To pause the container:
Stopping and restarting the container
Typically, you manage a container by pausing and unpausing it. But sometimes you might need to stop and restart the container, for example, for a reboot.
From a PowerShell (Windows) or Terminal (MacOS or Linux) session, enter the appropriate command:
-
To stop a container:
docker stop streamsdocker4300
-
To restart a container:
docker restart streamsdocker4300
Tip: When a container is restarted, it takes a few minutes for the system to come back up and start the domains. To see the status, you can use VNC to access the desktop, where you can see the yellow desktop icon (Streams Domain Starting) while the domain is starting. The icon turns green (Streams Domain Started) when the domain is ready. The icon eventually disappears.
Adjusting the desktop screen size for VNC
When you first access Streams Quick Start Edition with VNC, you will see the Streams desktop.
If you are accustomed to earlier versions of Streams Quick Start Edition, you will notice that there are no longer any desktop icons. All Streams Quick Start Edition resources have been moved to the Applications > Favorites menu, which you can access from the top left of the screen. There you will find the Streams applications and links to resource web pages.
To adjust the display size to closer match your physical display, follow these steps:
-
Go to Applications > System Tools > Settings.
-
Click Devices > Displays.
-
In the Displays app, click Resolution.
-
Select the resolution that best matches your physical display. Close the Resolution list by clicking ‘x’ in the corner. Click Apply.
-
Click Keep Changes.
Known issues
Description: Firefox crashes often
Workaround:
- Open Firefox and type
about:config
in the address bar. Accept the warning message. - Search for
browser.tabs.remote.autostart
and set it tofalse
. - Repeat the above for the
browser.tabs.remote.autostart.2
, set it tofalse
. - Restart Firefox.
Workaround #2:
If that still does not help, try installing and using Google Chrome.
wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm
sudo yum localinstall google-chrome-stable_current_x86_64.rpm
Description: Running streamsdockerInstall.sh
fails with the following message:
TASK [install Streams prereq packages with yum] ********************************
changed: [streamsqse.localdomain] => (item=readline)
failed: [streamsqse.localdomain] (item=requests) => {“changed”: false, “cmd”: “/usr/bin/pip2 install -U requests”, “item”: “requests”, “msg”: “stdout: Collecting requests\n
...<snip>...
Installing collected packages: certifi, chardet, idna, urllib3, requests\n Found existing installation: chardet 2.2.1\n Uninstalling chardet-2.2.1:\n Successfully uninstalled chardet-2.2.1\n Found existing installation: idna 2.4\n Uninstalling idna-2.4:\n Successfully uninstalled idna-2.4\n Found existing installation: urllib3 1.10.2\n Uninstalling urllib3-1.10.2:\n Successfully uninstalled urllib3-1.10.2\n
Found existing installation: requests 2.6.0\n\n:stderr: ipapython 4.5.0 requires pyldap>=2.4.15, which is not installed.\nrtslib-fb 2.1.63 has requirement pyudev>=0.16.1, but you’ll have pyudev 0.15 which is incompatible.\nipapython 4.5.0 has requirement dnspython>=1.15, but you’ll have dnspython 1.12.0 which is incompatible.\nCannot uninstall ‘requests’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.\n”}
changed: [streamsqse.localdomain] => (item=future)
changed: [streamsqse.localdomain] => (item=dill)
to retry, use: --limit @/root/ansible/streamsdockerCentos7.retry
PLAY RECAP *********************************************************************
streamsqse.localdomain : ok=41 changed=32 unreachable=0 failed=1
Cause: This might be caused by a change in the Python repositories.
Workaround:
- In the Quick Start Edition installation directory, locate the
Ansible/streamsdockerCentos7.yaml
file and open it in a text editor. - Locate the following section:
######### Python Module installation #################################### name: install Streams prereq packages with yum pip: name= state=latest with_items: readline requests future dill
- Change
requests
torequest
. - Save the file.
- Remove the old container:
docker stop streamsdocker4300 docker rm streamsdocker4300
- Reinstall Quick Start Edition.
Description: Streams Studio Project Explorer: Right-clicking a file or folder, and then selecting Show in > System Explorer throws a dbus error.
Cause: CentOS 6 Nautilus is not compatible with dbus API.
Workaround:
-
Open Streams Studio.
-
Go to Window > Preferences > General > Workspace.
-
Set Command for launching system explorer as follows:
nautilus "${selected_resource_parent_loc}"
Getting help
If your problem is not discussed in the known issues section, you can ask a question on the Streamsdev forum.