Energy Consumption Prediction using Big Data and Process Mining

Big Data is a term used to describe a collection of data that is huge in size and yet growing exponentially with time (Catarina G. & Mariam A. M.). These data can be structured, unstructured as well as real-time. Big Data is the name used to describe the theory and practice of applying advanced computer analysis to the ever-growing amount of digital information that we can collect and store from the world around us. Being able to predict future performance based on historical results, or to identify sub-par production zones, can be used to shift assets to more productive areas (Catarina G. & Mariam A. M.). With Big Data analysis, revenue recovery rates can be improved and real time energy consumption can be optimized. To further optimize the energy consumption rate, the idea of process mining can be employed. Process mining can be used to discover, monitor and improve real processes (i.e., not assumed processes) by extracting knowledge from event logs readily available in today’s information systems in order to automatically construct the models of business processes, compare existing business process models with the new automatically constructed models to identify deviations and bottlenecks and to enhance the business processes Badr O. & Ahmed Z. E..

In recent years, advances in sensor technologies and expansion of smart meters have resulted in massive growth of energy data sets. These Big Data have created new opportunities for energy prediction, but at the same time, they impose new challenges for traditional technologies (Catarina G. & Mariam A. M.). On the other hand, new approaches for handling and processing these Big Data have emerged, such as MapReduce, Spark, Storm, and Oxdata H2O. This article explores how findings from machine learning with Big Data can benefit energy consumption prediction. Although local learning itself is not a novel concept, it has great potential in the Big Data domain because it reduces computational complexity. Modeling and forecasting electrical energy consumption has been an active research area for more than a decade. In the United States, retail sales of electricity exceed $3,760 billion and the electricity sector generates the largest share of greenhouse gas emissions (31%) (Catarina G. & Mariam A. M.). Today, with climate change and the focus on environment, it is even more important to model and forecast electricity consumption accurately in pursuit of conservation opportunities. The importance of measuring and collecting electricity data, together with recent advances in sensor technology, have led to the proliferation of smart meters that measure and communicate electricity consumption. These smart meters measure electricity at intervals of an hour or less, whereas some sensor devices can measure consumption in real time (Wang Z. et al). These Big Data have created opportunities to develop new ways of analyzing energy consumption, identifying potential savings, and measuring energy efficiency. Sensor-based approaches to energy forecasting rely on readings from sensors or smart meters and contextual information such as meteorological information or work schedules to infer future energy behavior (Jose C. R. et al). Typically, historical data such as temperature, day of the week, time of day, and energy consumption are fed into a machine learning model that learns from them and consequently can forecast future energy consumption. A typical assumption of Big Data is that more data can lead to deeper insights and higher business value. This is especially true in machine learning, where algorithms can learn better from bigger data sets. However, massive data sets can be challenging to process (Mel K. et al). Many machine learning algorithms were designed with the assumption that the whole dataset fits into the memory. Often these algorithms are of high algorithmic complexity and require large amounts of memory (Catarina G. & Mariam A. M.). This gave rise to distributed processing approaches, such as MapReduce, which are suitable for algorithms that can be parallelized to a degree sufficient to take advantage of available nodes (Catarina G. & Mariam A. M.).

Part two of the article will be available in this blog soon….