Volume 77, Issue 2
Original Article

Multiple‐change‐point detection for high dimensional time series via sparsified binary segmentation

Haeran Cho

Corresponding Author

University of Bristol, UK

Address for correspondence: Haeran Cho, School of Mathematics, University of Bristol, University Walk, Bristol, BS8 1TW, UK. E‐mail: haeran.cho@bristol.ac.ukSearch for more papers by this author
Piotr Fryzlewicz

London School of Economics and Political Science, UK

Search for more papers by this author
First published: 24 July 2014
Citations: 72

Summary

Time series segmentation, which is also known as multiple‐change‐point detection, is a well‐established problem. However, few solutions have been designed specifically for high dimensional situations. Our interest is in segmenting the second‐order structure of a high dimensional time series. In a generic step of a binary segmentation algorithm for multivariate time series, one natural solution is to combine cumulative sum statistics obtained from local periodograms and cross‐periodograms of the components of the input time series. However, the standard ‘maximum’ and ‘average’ methods for doing so often fail in high dimensions when, for example, the change points are sparse across the panel or the cumulative sum statistics are spuriously large. We propose the sparsified binary segmentation algorithm which aggregates the cumulative sum statistics by adding only those that pass a certain threshold. This ‘sparsifying’ step reduces the influence of irrelevant noisy contributions, which is particularly beneficial in high dimensions. To show the consistency of sparsified binary segmentation, we introduce the multivariate locally stationary wavelet model for time series, which is a separate contribution of this work.

Number of times cited according to CrossRef: 72

  • More effective prognostics with elbow point detection and deep learning, Mechanical Systems and Signal Processing, 10.1016/j.ymssp.2020.106987, 146, (106987), (2021).
  • Change-point detection based on adjusted shape context cost method, Information Sciences, 10.1016/j.ins.2020.08.112, 545, (363-380), (2021).
  • Estimating multiple breaks in nonstationary autoregressive models, Journal of Econometrics, 10.1016/j.jeconom.2020.06.005, (2020).
  • Nonparametric Spectral Analysis of Multivariate Time Series, Annual Review of Statistics and Its Application, 10.1146/annurev-statistics-031219-041138, 7, 1, (361-386), (2020).
  • Time series analysis of COVID-19 infection curve: A change-point perspective, Journal of Econometrics, 10.1016/j.jeconom.2020.07.039, (2020).
  • Evidence for intensification of meteorological droughts in Oman over the past four decades, Atmospheric Research, 10.1016/j.atmosres.2020.105126, 246, (105126), (2020).
  • Conversion of encroached juniper woodland back to native prairie and to switchgrass increases root zone soil moisture and watershed runoff, Journal of Hydrology, 10.1016/j.jhydrol.2020.124640, (124640), (2020).
  • Testing and estimating change-points in the covariance matrix of a high-dimensional time series, Journal of Multivariate Analysis, 10.1016/j.jmva.2019.104582, (104582), (2020).
  • Multiple Change Points Detection in Low Rank and Sparse High Dimensional Vector Autoregressive Models, IEEE Transactions on Signal Processing, 10.1109/TSP.2020.2993145, 68, (3074-3089), (2020).
  • A unified data‐adaptive framework for high dimensional change point detection, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12375, 82, 4, (933-963), (2020).
  • Long term belt conveyor gearbox temperature data analysis – Statistical tests for anomaly detection, Measurement, 10.1016/j.measurement.2020.108124, 165, (108124), (2020).
  • Determining the number of change‐point via high‐dimensional cross‐validation, Stat, 10.1002/sta4.284, 9, 1, (2020).
  • A new hybrid approach to panel data change point detection, Communications in Statistics - Theory and Methods, 10.1080/03610926.2020.1760298, (1-12), (2020).
  • Change point estimation in panel data with time‐varying individual effects, Journal of Applied Econometrics, 10.1002/jae.2769, 0, 0, (2020).
  • Multiscale change point detection for dependent data, Scandinavian Journal of Statistics, 10.1111/sjos.12465, 0, 0, (2020).
  • Change point detection for nonparametric regression under strongly mixing process, Statistical Papers, 10.1007/s00362-020-01196-y, (2020).
  • Multivariate analysis of variance and change points estimation for high‐dimensional longitudinal data, Scandinavian Journal of Statistics, 10.1111/sjos.12460, 0, 0, (2020).
  • Segmentation Method for Ship-Radiated Noise Using the Generalized Likelihood Ratio Test on an Ordinal Pattern Distribution, Entropy, 10.3390/e22040374, 22, 4, (374), (2020).
  • Adaptive G–G clustering for fuzzy segmentation of multivariate time series, Stochastic Environmental Research and Risk Assessment, 10.1007/s00477-020-01817-w, (2020).
  • Most recent changepoint detection in censored panel data, Computational Statistics, 10.1007/s00180-020-01028-5, (2020).
  • Joint Structural Break Detection and Parameter Estimation in High-Dimensional Nonstationary VAR Models, Journal of the American Statistical Association, 10.1080/01621459.2020.1770097, (1-14), (2020).
  • Estimating a Change Point in a Sequence of Very High-Dimensional Covariance Matrices, Journal of the American Statistical Association, 10.1080/01621459.2020.1785477, (1-11), (2020).
  • Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection, Journal of the Korean Statistical Society, 10.1007/s42952-020-00060-x, (2020).
  • BayesProject: Fast computation of a projection direction for multivariate changepoint detection, Statistics and Computing, 10.1007/s11222-020-09966-2, (2020).
  • undefined, 2019 15th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), 10.1109/SITIS.2019.00021, (56-60), (2019).
  • Selective review of offline change point detection methods, Signal Processing, 10.1016/j.sigpro.2019.107299, (107299), (2019).
  • Greedy Kernel Change-Point Detection, IEEE Transactions on Signal Processing, 10.1109/TSP.2019.2953670, 67, 24, (6204-6214), (2019).
  • Show Me Your Friends and I’ll Tell You Who You Are. Finding Anomalous Time Series by Conspicuous Cluster Transitions, Data Mining, 10.1007/978-981-15-1699-3_8, (91-103), (2019).
  • Inference of Break-Points in High-Dimensional Time Series, SSRN Electronic Journal, 10.2139/ssrn.3378221, (2019).
  • A Setwise EWMA Scheme for Monitoring High-Dimensional Datastreams, Random Matrices: Theory and Applications, 10.1142/S2010326320500045, (2019).
  • Estimation of Dynamic Networks for High-Dimensional Nonstationary Time Series, Entropy, 10.3390/e22010055, 22, 1, (55), (2019).
  • Oracally efficient estimation for dense functional data with holiday effects, TEST, 10.1007/s11749-019-00655-5, (2019).
  • Comparative analysis of nonparametric change-point detectors commonly used in hydrology, Hydrological Sciences Journal, 10.1080/02626667.2019.1669792, (1-21), (2019).
  • Structural breaks in panel data: Large number of panels and short length time series, Econometric Reviews, 10.1080/07474938.2018.1454378, 38, 7, (828-855), (2018).
  • Most Recent Changepoint Detection in Panel Data, Technometrics, 10.1080/00401706.2018.1438926, 61, 1, (88-98), (2018).
  • Simultaneous multiple change-point and factor analysis for high-dimensional time series, Journal of Econometrics, 10.1016/j.jeconom.2018.05.003, 206, 1, (187-225), (2018).
  • A practical adaptive nonlinear tracking algorithm with range rate measurement, International Journal of Distributed Sensor Networks, 10.1177/1550147718776863, 14, 5, (155014771877686), (2018).
  • Nonparametric multiple change-point estimation for analyzing large Hi-C data matrices, Journal of Multivariate Analysis, 10.1016/j.jmva.2017.12.005, 165, (143-165), (2018).
  • Graph-based structural change detection for rotating machinery monitoring, Mechanical Systems and Signal Processing, 10.1016/j.ymssp.2017.06.003, 99, (73-82), (2018).
  • Abrupt transitions in time series with uncertainties, Nature Communications, 10.1038/s41467-017-02456-6, 9, 1, (2018).
  • undefined, Proceedings of the 27th ACM International Conference on Information and Knowledge Management - CIKM '18, 10.1145/3269206.3271669, (507-516), (2018).
  • Change-Point Estimation in the Multivariate Model Taking into Account the Dependence: Application to the Vegetative Development of Oilseed Rape, Journal of Agricultural, Biological and Environmental Statistics, 10.1007/s13253-018-0324-y, 23, 3, (374-389), (2018).
  • Consistent and powerful non-Euclidean graph-based change-point test with applications to segmenting random interfered video data, Proceedings of the National Academy of Sciences, 10.1073/pnas.1804649115, 115, 23, (5914-5919), (2018).
  • Multi-sensors based condition monitoring of rotary machines: an approach of multidimensional time-series analysis, Measurement, 10.1016/j.measurement.2018.10.089, (2018).
  • Asymptotic properties and information criteria for misspecified generalized linear mixed models, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12270, 80, 4, (817-836), (2018).
  • Modified path algorithm of fused Lasso signal approximator for consistent recovery of change points, Journal of Statistical Planning and Inference, 10.1016/j.jspi.2018.10.003, (2018).
  • A Fast Screen and Shape Recognition Algorithm for Multiple Change-Point Detection, Mathematical Problems in Engineering, 10.1155/2018/8371085, 2018, (1-10), (2018).
  • undefined, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 10.1109/GlobalSIP.2018.8646607, (773-777), (2018).
  • Nonparametric change point detection in multivariate piecewise stationary time series, Journal of Nonparametric Statistics, 10.1080/10485252.2018.1504943, 30, 4, (926-956), (2018).
  • Oracle Estimation of a Change Point in High-Dimensional Quantile Regression, Journal of the American Statistical Association, 10.1080/01621459.2017.1319840, 113, 523, (1184-1194), (2018).
  • Multiple change-points detection in high dimension, Random Matrices: Theory and Applications, 10.1142/S201032631950014X, (1950014), (2018).
  • Assessing Change-Points in Surface Air Temperature Over Alaska, Frontiers in Environmental Science, 10.3389/fenvs.2018.00121, 6, (2018).
  • Detecting Multiple Change Points Using Adaptive Regression Splines With Application to Neural Recordings, Frontiers in Neuroinformatics, 10.3389/fninf.2018.00067, 12, (2018).
  • FreSpeD: Frequency-Specific Change-Point Detection in Epileptic Seizure Multi-Channel EEG Data, Journal of the American Statistical Association, 10.1080/01621459.2018.1476238, (1-14), (2018).
  • High dimensional change point estimation via sparse projection, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12243, 80, 1, (57-83), (2017).
  • The locally stationary dual-tree complex wavelet model, Statistics and Computing, 10.1007/s11222-017-9784-0, 28, 6, (1139-1154), (2017).
  • Key-frame selection for video summarization: an approach of multidimensional time series analysis, Multidimensional Systems and Signal Processing, 10.1007/s11045-017-0513-9, 29, 4, (1485-1505), (2017).
  • Semi‐supervised approaches to efficient evaluation of model prediction performance, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12264, 80, 3, (579-594), (2017).
  • Consistent and powerful graph-based change-point test for high-dimensional data, Proceedings of the National Academy of Sciences, 10.1073/pnas.1702654114, 114, 15, (3873-3878), (2017).
  • undefined, 2017 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), 10.1109/SDPC.2017.47, (210-214), (2017).
  • undefined, Proceedings of the International Conference on Compute and Data Analysis - ICCDA '17, 10.1145/3093241.3093282, (143-149), (2017).
  • Estimation of structural mean breaks for long-memory data sets, Statistics, 10.1080/02331888.2017.1335314, 51, 4, (904-920), (2017).
  • High-dimensional change-point estimation: Combining filtering with convex optimization, Applied and Computational Harmonic Analysis, 10.1016/j.acha.2015.11.003, 43, 1, (122-147), (2017).
  • A survey of methods for time series change point detection, Knowledge and Information Systems, 10.1007/s10115-016-0987-z, 51, 2, (339-367), (2016).
  • Detection of change in the spatiotemporal mean function, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12156, 79, 1, (29-50), (2016).
  • Estimating whole‐brain dynamics by using spectral clustering, Journal of the Royal Statistical Society: Series C (Applied Statistics), 10.1111/rssc.12169, 66, 3, (607-627), (2016).
  • On high-dimensional change point problem, Science China Mathematics, 10.1007/s11425-016-0058-5, 59, 12, (2355-2378), (2016).
  • Classification of multiple time signals using localized frequency characteristics applied to industrial process monitoring, Computational Statistics & Data Analysis, 10.1016/j.csda.2015.07.009, 94, (351-362), (2016).
  • Splitting Algorithm for Detecting Structural Changes in Predictive Relationships, Advances in Data Mining. Applications and Theoretical Aspects, 10.1007/978-3-319-41561-1_30, (405-419), (2016).
  • Detecting relevant changes in time series models, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 10.1111/rssb.12121, 78, 2, (371-394), (2015).
  • Detection of Changes in Multivariate Time Series With Application to EEG Data, Journal of the American Statistical Association, 10.1080/01621459.2014.957545, 110, 511, (1197-1216), (2015).
  • Large-Scale Multi-Stream Quickest Change Detection via Shrinkage Post-Change Estimation, IEEE Transactions on Information Theory, 10.1109/TIT.2015.2495361, 61, 12, (6926-6938), (2015).