eprintid: 1492674 rev_number: 25 eprint_status: archive userid: 608 dir: disk0/01/49/26/74 datestamp: 2016-05-09 16:17:09 lastmod: 2021-10-18 00:23:26 status_changed: 2016-05-09 16:17:09 type: article metadata_visibility: show creators_name: Adepeju, M creators_name: Rosser, G creators_name: Cheng, T title: Novel evaluation metrics for sparse spatio-temporal point process hotspot predictions - a crime case study ispublished: pub divisions: UCL divisions: B04 divisions: C05 divisions: F44 keywords: Point process, hotspot, prediction, space-time, crime note: Copyright © 2016 The Author(s). Published by Taylor & Francis. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. abstract: Many physical and sociological processes are represented as discrete events in time and space. These spatio-temporal point processes are often sparse, meaning that they cannot be aggregated and treated with conventional regression models. Models based on the point process framework may be employed instead for prediction purposes. Evaluating the predictive performance of these models poses a unique challenge, as the same sparseness prevents the use of popular measures such as the root mean squared error. Statistical likelihood is a valid alternative, but this does not measure absolute performance and is therefore difficult for practitioners and researchers to interpret. Motivated by this limitation, we develop a practical toolkit of evaluation metrics for spatio-temporal point process predictions. The metrics are based around the concept of hotspots, which represent areas of high point density. In addition to measuring predictive accuracy, our evaluation toolkit considers broader aspects of predictive performance, including a characterisation of the spatial and temporal distributions of predicted hotspots and a comparison of the complementarity of different prediction methods. We demonstrate the application of our evaluation metrics using a case study of crime prediction, comparing four varied prediction methods using crime data from two different locations and multiple crime types. The results highlight a previously unseen interplay between predictive accuracy and spatio-temporal dispersion of predicted hotspots. The new evaluation framework may be applied to compare multiple prediction methods in a variety of scenarios, yielding valuable new insight into the predictive performance of point process-based prediction. date: 2016-04-07 date_type: published official_url: http://dx.doi.org/10.1080/13658816.2016.1159684 oa_status: green full_text_type: pub language: eng primo: open primo_central: open_green verified: verified_manual elements_id: 1128066 doi: 10.1080/13658816.2016.1159684 lyricists_name: Cheng, Tao lyricists_id: TCHEN23 actors_name: Cheng, Tao actors_id: TCHEN23 actors_role: owner full_text_status: public publication: International Journal of Geographical Information Science volume: 30 number: 11 pagerange: 2133-2154 issn: 1362-3087 citation: Adepeju, M; Rosser, G; Cheng, T; (2016) Novel evaluation metrics for sparse spatio-temporal point process hotspot predictions - a crime case study. International Journal of Geographical Information Science , 30 (11) pp. 2133-2154. 10.1080/13658816.2016.1159684 <https://doi.org/10.1080/13658816.2016.1159684>. Green open access document_url: https://discovery.ucl.ac.uk/id/eprint/1492674/1/IJGIS%20Adepejue%20Rosser%20Cheng%202016.pdf