Streams is now part of IBM Cloud Pak for Data, for better integration with data from across an organization. Follow the link to learn more and get started.
You no longer need to download and install the Streams runtime to start developing Streams applications. Develop Streams applications and deploy them in IBM Cloud Pak for Data or IBM Cloud using the extension for Microsoft VS Code.
Notebooks showing Streams applications written in Python.
Scale Stream Applications at Runtime
Monitor Applications and Finding Errors Using Streams Console
Compute Averages Using Time Based Windows
Streaming Analytics in IBM Cloud Pak for Data
This sample notebook is included in IBM Cloud Pak for Data. The video walks you through creating the Streams application in Python, executing it on the Streams instance, and visualizing the results.
IBM Streams Introduction
IBM Streams Software Engineer Alex Cook walks you through a technical overview of IBM Streams. Explore how to capture data streams and analyze them faster with IBM Streams.
Streaming Analytics in Python Demo
In this video, you can see a Streams application analyzing data from IoT devices. The application is created in a Python notebook on IBM Watson Studio and executed in the Streaming Analytics service on the IBM Cloud. The results of the application are displayed on a map embedded in the notebook. Try the notebook for yourself at https://ibm.biz/WeatherNotebook.
IBM Streaming Analytics on IBM Cloud
If you are new to Streams and/or to the Streaming Analytics service on IBM Cloud, this video is a great way to get started using the stock trades starter application. This is a simple application that analyzes stock quotes and computes the rolling average price for each company.
IBM Streams + Excel
IBM Streams Community Architect Samantha Chan walks you through how IBM Streams and Microsoft Excel integrate to enable rapid analysis of data streams. Bring the full power of Excel to data streams.
Resiliency
How resilient is Streams?
We have customers that have been running for over 2 years without downtime, and that was before we made high availability even easier. If systems go down, Streams self-heals while Guaranteed Tuple Processing makes sure that no data is lost. Streams provides enterprise resiliency with simple set-up and configuration.
What happens when a Streams host goes down?
The controller service will automatically restart processing elements and services on the other Streams hosts. If a host configured for High Availability comes back online, services will be automatically restarted.
Will I lose data if a Streams host goes down?
In Streams 4.0, we introduced consistent regions, which not only guarantees that the application does not lose data, but, in most cases, ensures that each incoming record is processed exactly once. Consistent regions are enabled using a simple annotation (@consistent) and HA compliant operators. Read more about consistent regions.
What happens if an Application Processing Element fails?
Failures are detected and Processing Elements (PEs) are automatically restarted by the controller service. If needed, the PE will be relocated to a new host.
Streams has made developing Streams applications easier than ever with the introduction of our Java Application API. Millions of Java developers worldwide can now write Streams code without learning a new language. Get started with our Java API development guide.
Graphical Editing with an Eclipse IDE
Writing streaming applications can be hard. With the Streams Studio graphical editor, it’s not. Build your applications using your ideal mix of drag-and-drop editing and rich text editing. Learn more about Streams Studio here.
Admin Console for Configuration and Monitoring
Setting up, configuring, and monitoring your Streams environment is easier and more efficient than ever. Check application status in a single glance, configure high-availability with just a few clicks, and dig deep into metrics within seconds. Read an overview here.
Streaming Data to Microsoft Excel
Visualizing streaming data is easier than ever with the Streams Microsoft Excel Plug-in. Drag and drop live data-feeds onto your worksheet, insert a chart, and watch as your chart changes in real-time to reflect the reality of NOW. Start Streaming to Excel here.
Cloud
Streaming Analytics Service in IBM Cloud
Running and scaling real-time analytics just got easier with the introduction of IBM's Streaming Analytics IBM Cloud service. Simply develop your applications locally using IBM Streams and then deploy those applications in the cloud using IBM's Streaming Analytics service. Sign up to get 120 free node hours per month and use our Streaming Analytics roadmap to get in the cloud fast.
Geospatial Analytics service in IBM Cloud
The Internet of Things already connects billions of devices, with forecasts predicting steep growth rates in the coming years. Many of these devices, such as smartphones and connected vehicles, are mobile. Awareness of the location of on-the-move devices opens up exciting new application opportunities. Support for these new applications requires highly scalable services that can analyze high volumes of data in real time. Get started on the cloud fast and for free here.
Performance
What does NOW mean for Streams?
NOW means that Streams outperforms Apache Storm by 2.6 to 12.3 times in terms of throughput while simultaneously consuming 5.5 to 14.2 times less CPU time. Furthermore, the throughput and CPU time gaps widen as data volume, degree of parallelism, and/or number of processing nodes grows. Read this article on the performance seen in these Streams benchmarks. NOW means not waiting for a batch to complete or a timer to fire. NOW means that while your competitors are making decisions based on yesterday, you will be making them based on the present - gaining insight not hindsight.
How do I make sure I’m getting the best performance?
Getting the most out of your hardware is a great way to save money. IBM Streams handles an incredible amount of data per core. Read these best practices to configure Streams for the best performance and this article on how to optimize your applications.
Analytics
Cybersecurity Toolkit
If you’re not using real-time processing to protect your network, then by the time you know you’ve been attacked, it will be too late. With IBM Streams’ new Cybersecurity toolkit, you can monitor and detect active, real-time attacks within your network and allow you to respond to these serious threats before damage can be done. Bring your network security into the Cognitive Era with the IBM Streams Cybersecurity toolkit.
Spark MLlib
True stream processing is here for your Spark analytics. With our new Spark MLlib toolkit, you can leverage analytics being used extensively in the open source community. As you develop and improve your Spark analytics models, plug them into your Streams application to get answers continuously and in-stream. Get started here.
Geospatial Analytics
Real-time geospatial analytics opens the door to a whole new set of business use-cases. Retailers can interact with customers in real-time based on their location, agencies that monitor environmental conditions can provide targeted warnings to residents, and the connected cars of the near-future can become a reality. Try out our connected car demo here, and read the geospatial toolkit overview here.
Time Series Analytics
Our time series analytics were designed from the bottom up for processing streaming data. Any type of data that changes over time can be considered time-series data—sales figures, sensor readings, energy-usage, and network traffic. Streams provides up-to-the-millisecond analysis of how your data is changing now, and predictions on how it will change in the future. Read about how our time series analytics are being used for anomaly detection here, and get a more in-depth overview of the time series toolkit here.
Text Analytics
Streaming text analytics allows businesses to tap into the wealth of knowledge that text has to offer and react in real-time. Businesses can catch outgoing communication of sensitive or confidential information before being compromised, financial companies can act on news reports within milliseconds, and call-centers can respond to customer complaints immediately. Streams enables you to perform a wide range of analytics from simple extraction of keywords and phrases from text to complex semantic analysis on speech. Learn more about IBM Streams text analytics here.
Business Analytics
The longer it takes to analyze data and update business logic, the harder it is for businesses to keep pace. In order to take full advantage of real-time analytics, businesses must be able to respond to the results of that analysis. Streams provides the ability to integrate business rules directly into a streaming application. Harness the power of IBM Operational Decision Manager (ODM) as Streams enables you to modify business rules on the fly and have those rules applied to your real-time streaming analytic applications. Check out are Rules Toolkit here.
More Prepackaged Toolkits for Rapid Analytics Development