We propose a framework for complex event processing (CEP) coupled with predictive analytics to predict simple events and complex events on Internet of Things (IoT) data streams. The data is consumed through a REST service containing the traffic data of around 2,000 locations in the city of Madrid, Spain. This prediction of complex events will help users in understanding the future state of road traffic and hence take meaningful decisions. For predicting events, we propose a framework that uses WSO2 Siddhi as CEP, along with InfluxDB as persistent storage. The data is consumed in the CEP with the help of a high-speed Apache Kafka messaging pipeline. This data is used to build predictive models inside the CEP that helps users to derive meaningful insights. However, in these event analytics engines, the events are created via rules that are triggered when the streaming data exceeds a certain threshold. The calculation of the “threshold” is utmost necessary as it acts as the means for the generation of simple events and complex events in an event analytics scenario. We have proposed a novel 2-fold approach for finding out the thresholds in such large datasets. We have taken the help of unsupervised learning to get the idea of thresholds. The first phase uses Node-RED and serverless computing to create the thresholds and then supply them back to the CEP for prediction. The machine learning models run on a cloud service, and the predictions or thresholds are returned back through REST services into the CEP. In the second phase, it not only creates the thresholds but also uses novel hypothesis testing techniques along with windowing mechanism on data streams to implement clustering and supply the result back into the CEP. This approach leverages on the usage of statistical techniques to understand the change in distribution of data. The changes in the data distributions trigger the retraining of the machine learning models, and the results are given back into the CEP for being used in an event generation scenario. We have also included a section in which we have incorporated a statistical analysis on the dataset used.

Chapter Contents:

• 7.1 Introduction
• 7.2 Related works
• 7.3 Research preliminaries
• 7.3.1 Dataset description
• 7.3.2 Data ingestion
• 7.3.3 Complex event processing
• 7.3.4 Clustering approaches
• 7.3.5 OpenWhisk
• 7.3.6 Evaluation metrics
• 7.3.6.1 Silhouette coefficient
• 7.3.6.2 Calinski–Harabasz index
• 7.4 Proposed methodology
• 7.4.1 Statistical approach to optimize the number of retrainings
• 7.5 Experimental results and discussion
• 7.6 Conclusion
• Acknowledgment
• References

Inspec keywords: smart cities; traffic engineering computing; cloud computing; road traffic; unsupervised learning; Internet of Things; data analysis

Other keywords: persistent storage; Internet of Things data streams; Spain; serverless computing; unsupervised learning; event analytics engines; InfluxDB; traffic prediction; Node-RED; WSO2 Siddhi; cloud service; high-speed Apache Kafka messaging pipeline; complex event processing; predictive analytics; IoT data streams; Madrid; REST service; clustering; hypothesis testing techniques; smart city; data distributions; road traffic; windowing mechanism; statistical techniques

Subjects: Mobile, ubiquitous and pervasive computing; Internet software; Unsupervised learning; Traffic engineering computing; Information networks; Data handling techniques