A Markov Chain-Based Framework for Predicting Dynamic Customer Behavior Using Transition Probability Modeling
Article Sidebar
Main Article Content
Abstract
Markov chains provide a transparent probabilistic approach for modeling dynamic systems whose behavior evolves through discrete states over time. This study develops and empirically validates a Markov chain-based framework for predicting dynamic customer behavior from observed transaction histories. The proposed framework transforms time-stamped retail transactions into a monthly customer-state panel, defines interpretable behavioral states, estimates transition probability matrices, evaluates short-term and multi-step state movement, and derives long-run steady-state behavior. Using a two-year online retail transaction dataset as the empirical case, customer activity is represented through five states: inactive, at-risk, occasional, active, and loyal. After cleaning the data, the analysis uses 805,549 valid purchase records from 5,878 customers and constructs 92,679 consecutive customer-month transitions. The estimated transition matrix shows strong persistence in inactive behavior, substantial instability among occasional and active customers, and meaningful but limited loyalty persistence. The long-run steady-state distribution suggests that, without intervention, the system tends toward a high inactive-state share, indicating structural retention risk. Predictive validation using a holdout period shows that the Markov model achieves 75.59% one-step state prediction accuracy, outperforming a simple state-persistence baseline of 66.32%. The findings demonstrate that Markov chains can convert historical state movements into interpretable forecasts, identify stable and unstable behavioral states, and support retention-oriented decision-making. The framework is adaptable to other stochastic dynamic systems, including machine reliability, disease progression, educational performance, weather-state evolution, and financial behavior.
Downloads
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
1. Ascarza, E. (2018). Retention futility: Targeting high-risk customers might be ineffective. Journal of Marketing Research, 55(1), 80-98. https://doi.org/10.1509/jmr.16.0163 DOI: https://doi.org/10.1509/jmr.16.0163
2. Ascarza, E., Neslin, S. A., Netzer, O., Anderson, Z., Fader, P. S., Gupta, S., Hardie, B. G. S., Lemmens, A., Libai, B., Neal, D., Provost, F., & Schrift, R. (2018). In pursuit of enhanced customer retention management: Review, key issues, and future directions. Customer Needs and Solutions, 5(1-2), 65-81. https://doi.org/10.1007/s40547-017-0080-0 DOI: https://doi.org/10.1007/s40547-017-0080-0
3. Baghla, S., & Gupta, G. (2022). Performance evaluation of various classification techniques for customer churn prediction in e-commerce. Microprocessors and Microsystems, 94, 104680. https://doi.org/10.1016/j.micpro.2022.104680 DOI: https://doi.org/10.1016/j.micpro.2022.104680
4. Burez, J., & Van den Poel, D. (2009). Handling class imbalance in customer churn prediction. Expert Systems with Applications, 36(3), 4626-4636. https://doi.org/10.1016/j.eswa.2008.05.027 DOI: https://doi.org/10.1016/j.eswa.2008.05.027
5. Chen, D. (2012). Online Retail II [Dataset]. UCI Machine Learning Repository. https://doi.org/10.24432/C5CG6D
6. Chen, D., Sain, S. L., & Guo, K. (2012). Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining. Journal of Database Marketing & Customer Strategy Management, 19, 197-208. https://doi.org/10.1057/dbm.2012.17 DOI: https://doi.org/10.1057/dbm.2012.17
7. Coussement, K., & Van den Poel, D. (2008). Churn prediction in subscription services: An application of support vector machines while comparing two parameter-selection techniques. Expert Systems with Applications, 34(1), 313-327. https://doi.org/10.1016/j.eswa.2006.09.027 DOI: https://doi.org/10.1016/j.eswa.2006.09.038
8. Dinh, D. H., Nguyen, Q. M., & Bui, H. C. (2022). A Markov chain model for predicting brand switching behavior toward online food delivery services. In Financial Econometrics: Bayesian Analysis, Quantum Uncertainty, and Related Topics (pp. 781-797). Springer. https://doi.org/10.1007/978-3-030-98689-6_51 DOI: https://doi.org/10.1007/978-3-030-98689-6_51
9. Fader, P. S., & Hardie, B. G. S. (2009). Probability models for customer-base analysis. Journal of Interactive Marketing, 23(1), 61-69. https://doi.org/10.1016/j.intmar.2008.11.003 DOI: https://doi.org/10.1016/j.intmar.2008.11.003
10. Fader, P. S., Hardie, B. G. S., & Lee, K. L. (2005). RFM and CLV: Using iso-value curves for customer base analysis. Journal of Marketing Research, 42(4), 415-430. https://doi.org/10.1509/jmkr.2005.42.4.415 DOI: https://doi.org/10.1509/jmkr.2005.42.4.415
11. Grunt, O., Plucar, J., Stakova, M., Janecko, T., & Zelinka, I. (2017). An approach to customer behavior modeling using Markov Decision Process. MENDEL, 23(1), 141-148. https://doi.org/10.13164/mendel.2017.1.141 DOI: https://doi.org/10.13164/mendel.2017.1.141
12. Gupta, S., Hanssens, D., Hardie, B. G. S., Kahn, W., Kumar, V., Lin, N., Ravishanker, N., & Sriram, S. (2006). Modeling customer lifetime value. Journal of Service Research, 9(2), 139-155. https://doi.org/10.1177/1094670506293810 DOI: https://doi.org/10.1177/1094670506293810
13. Hadden, J., Tiwari, A., Roy, R., & Ruta, D. (2007). Computer assisted customer churn management: State-of-the-art and future trends. Computers & Operations Research, 34(10), 2902-2917. https://doi.org/10.1016/j.cor.2005.11.007 DOI: https://doi.org/10.1016/j.cor.2005.11.007
14. Harish, A. S., & Malathy, C. (2023). Customer segment prediction on retail transactional data using K-means and Markov model. Intelligent Automation & Soft Computing, 36(1), 589-600. https://doi.org/10.32604/iasc.2023.032030 DOI: https://doi.org/10.32604/iasc.2023.032030
15. Huang, B., Kechadi, M. T., & Buckley, B. (2012). Customer churn prediction in telecommunications. Expert Systems with Applications, 39(1), 1414-1425. https://doi.org/10.1016/j.eswa.2011.08.024 DOI: https://doi.org/10.1016/j.eswa.2011.08.024
16. Jandera, A., & Skovranek, T. (2022). Customer behaviour hidden Markov model. Mathematics, 10(8), 1230. https://doi.org/10.3390/math10081230 DOI: https://doi.org/10.3390/math10081230
17. Jerath, K., Fader, P. S., & Hardie, B. G. S. (2016). Customer-base analysis using repeated cross-sectional summary data. European Journal of Operational Research, 249(1), 340-350. https://doi.org/10.1016/j.ejor.2015.09.002 DOI: https://doi.org/10.1016/j.ejor.2015.09.002
18. Keramati, A., Jafari-Marandi, R., Aliannejadi, M., Ahmadian, I., Mozaffari, M., & Abbasi, U. (2014). Improved churn prediction in telecommunication industry using data mining techniques. Applied Soft Computing, 24, 994-1012. https://doi.org/10.1016/j.asoc.2014.08.041 DOI: https://doi.org/10.1016/j.asoc.2014.08.041
19. Kim, K., Jun, C.-H., & Lee, J. (2014). Improved churn prediction in the telecommunications industry by analyzing a large network. Expert Systems with Applications, 41(15), 6575-6584. https://doi.org/10.1016/j.eswa.2014.05.014 DOI: https://doi.org/10.1016/j.eswa.2014.05.014
20. Liberali, G., Urban, P., Hauser, J., & Urban, G. L. (2022). Morphing for consumer dynamics: Bandits meet hidden Markov models. Marketing Science, 41(3), 593-614. https://doi.org/10.1287/mksc.2021.1346 DOI: https://doi.org/10.1287/mksc.2021.1346
21. Mitrović, S., & De Weerdt, J. (2020). Churn modeling with probabilistic meta-paths-based representation learning. Information Processing & Management, 57(2), 102052. https://doi.org/10.1016/j.ipm.2019.06.001 DOI: https://doi.org/10.1016/j.ipm.2019.06.001
22. Okwuashi, O., & Ndehedehe, C. E. (2021). Integrating machine learning with Markov chain and cellular automata models for modelling urban land use change. Remote Sensing Applications: Society and Environment, 21, 100461. https://doi.org/10.1016/j.rsase.2020.100461 DOI: https://doi.org/10.1016/j.rsase.2020.100461
23. Pondel, M., Wuczyński, M., Gryncewicz, W., Łysik, Ł., Hernes, M., Rot, A., & Kozina, A. (2021). Deep learning for customer churn prediction in e-commerce decision support. Business Information Systems, 1, 3-12. https://doi.org/10.52825/bis.v1i.42 DOI: https://doi.org/10.52825/bis.v1i.42
24. Vermeer, S., & Trilling, D. (2020). Toward a better understanding of news user journeys: A Markov chain approach. Journalism Studies, 21(7), 879-894. https://doi.org/10.1080/1461670X.2020.1722958 DOI: https://doi.org/10.1080/1461670X.2020.1722958
25. Verbeke, W., Dejaeger, K., Martens, D., Hur, J., & Baesens, B. (2012). New insights into churn prediction in the telecommunication sector: A profit driven data mining approach. European Journal of Operational Research, 218(1), 211-229. https://doi.org/10.1016/j.ejor.2011.09.031 DOI: https://doi.org/10.1016/j.ejor.2011.09.031
26. Xiahou, X., & Harada, Y. (2022). B2C e-commerce customer churn prediction based on K-means and SVM. Journal of Theoretical and Applied Electronic Commerce Research, 17(2), 458-475. https://doi.org/10.3390/jtaer17020024 DOI: https://doi.org/10.3390/jtaer17020024
27. Zhu, B., Baesens, B., & Van den Broucke, S. K. L. M. (2017). An empirical comparison of techniques for the class imbalance problem in churn prediction. Information Sciences, 408, 84-99. https://doi.org/10.1016/j.ins.2017.04.015 DOI: https://doi.org/10.1016/j.ins.2017.04.015