Screen Shot 2016-10-21 at 11.26.47 AM

Project Status:


Performance Evaluation of Event-Based Non-Intrusive Load Monitoring Approaches

Faculty & Collaborators

Mario Berg├ęs , Nuno Nunes

Current Students

Lucas Pereira

Project Description

The ability of correctly evaluating the learned models is one of the most crucial aspects of machine learning. On the one hand, it is necessary to assess that one model is better than another, whereas on the other hand it is of vital importance to understand which metric should be used to evaluate a particular learned model.

In this project, we wish to extend the existing body of knowledge in evaluating machine learning algorithms (e.g., medical diagnosis) to the domain of Non-Intrusive Load Monitoring in order to: 1) provide insights on how to correctly evaluate the different algorithms that constitute a NILM system; and 2) assess which performance metrics make more sense when evaluating each of those algorithms.

To this end, we will follow a twofold approach: first we will evaluate the learned models to understand if there are statistically significant different between them, and if so, which models rank better than the others; second we will investigate if there are any significant differences between the metrics being considered and if these (eventual) differences affect the final decision regarding which algorithms are better than the others.

Work packages

As the title suggests, in this work we are interest in evaluating the performance of event-based approaches. To this end we have defined three short-term (WP-1, WP-2 and WP-3) and one long-term work package (WP4):

  • WP-1: Dataset labeling
  • WP-2: Evaluate event detection algorithms
  • WP-3: Evaluate event classification algorithms
  • WP-4: Evaluate energy estimation algorithms

Selected Publications

  1. L. Pereira and N. Nunes, "Semi-Automatic Labeling for Non-Intrusive Load Monitoring Datasets," in IFIP Conference on Sustainable Internet and ICT for Sustainability (SustainIT textquoteright15) [poster abstract], Madrid, Spain, 2015.
    bibtex Go to Publication
    @conference {2760, title = {Semi-Automatic Labeling for Non-Intrusive Load Monitoring Datasets},
      booktitle = {IFIP Conference on Sustainable Internet and ICT for Sustainability (SustainIT {textquoteright}15) [poster abstract]},
      year = {2015},
      month = {04/2015},
      publisher = {IEEE Explore},
      organization = {IEEE Explore},
      address = {Madrid, Spain},
      abstract = {<p class="p1">In this paper we present and evaluate a semi- automatic labeling prototype to enable the creation of fully labeled energy disaggregation datasets from sub-metered data. Our results advocate in favor of our approach and show that it is possible to extract individual appliance transitions with considerable precision, as long as the individual appliance information is present in the sub-metered data, and its resolution is high enough.</p> },
      author = {Pereira, Lucas and Nuno Nunes} }
  2. L. Pereira, N. Nunes, and M. Berg'es, "SURF and SURF-PI: A File Format and API for Non-Intrusive Load Monitoring Public Datasets," in International Conference on Future Energy Systems (e-Energy textquoteright14) [short paper], Cambridge, UK, 2014.
    bibtex Go to Publication
    @conference {2315, title = {SURF and SURF-PI: A File Format and API for Non-Intrusive Load Monitoring Public Datasets},
      booktitle = {International Conference on Future Energy Systems (e-Energy {textquoteright}14) [short paper]},
      series = {e-Energy {textquoteright}14: Proceedings of the 5th International Conference on Future Energy Systems},
      year = {2014},
      month = {06/2014},
      publisher = {ACM},
      organization = {ACM},
      address = {Cambridge, UK},
      abstract = {<p>In this paper we propose a common file format and API for public Non-Intrusive Load Monitoring (NILM) datasets such that researchers can easily evaluate their approaches across the different datasets and benchmark their results against prior work. The proposed file format enables storing the power demand of the whole house along with individual appliance consumption, and other relevant metadata in a single compact file, whereas the API supports the creation and manipulation of individual files and datasets in the proposed format.</p> },
      keywords = {API, Datasets, Energy Disaggregation, File Format},
      doi = {10.1145/2602044.2602078},
      author = {Pereira, Lucas and Nuno Nunes and Berg{'e}s, M.} }