Analyzing large scale recurrent event data using parametric frailty model: A divide-and-conquer approach
Jerry Q Cheng
Rutgers University, USA
: J Comput Eng Inf Technol
Abstract
Currently when analyzing large scale recurrent event data, we are facing challenges such as memory limitations, un-scalable computing time, etc. In this research, we propose a divide-and-conquer approach using parametric frailty models. Specifically, we randomly divide the data into many subsets and obtain the maximum likelihood estimator from each individual data set. Then, we propose a weighted method to combine these individual estimators as the final estimator. We show that this divide-and-conquer estimator is asymptotically equivalent to the estimator based on the full data. We conduct simulation studies to demonstrate the performance of this proposed method and apply it to a large real data set of repeated heart failure hospitalizations.