Is your machine learning algorithm worth the pharmaceutical industry?

---

Ewa J Kleczyk

Symphony Health, USA

: J Pharm Drug Deliv Res

Abstract

---

The holy grail in health care is not fancier technology and tools, it is physician and patient behavior change. Machine learning will truly come of age when it can systematically and reliably do one of two things – improve the decisionmaking of clinicians and patients or improve their efficiency in carrying out the actions that follow from those decisions (Jean Drouin, M.D., Founder and CEO - Clarify Health Solutions, 2018). The quote above presents well the current state of machine learning in the healthcare industry. Every aspect of the area seems to be influenced by some set of models and their results; however, with now almost every analytics organization leveraging machine learning algorithms to provide insights into healthcare decision-making processes, there is an everincreasing need for establishing a set of guidelines for machine learning research to aid data scientists with the ability to validate and replicate the applied algorithms and models. The discussion has been often focused on how to accurately identify at-risk patients to aid their disease education, diagnoses, and treatment, but also how to accurately attribute the patient population to physicians to ensure proper care for these patients. The application of such algorithms span from personal promotion triggering to addressable TV targeting, and patient journey / treatment identification. Often healthcare data along with sociodemographic variables are leveraged to predict at- risk patients or their specific treatment pathways, noting the variables of significance that predict those currently in the at-risk group or their next treatment steps. However, more and more data scientists question the relevance, validity, and directionality of the machine learning algorithm insights. Given the fact that someone’s diagnosis or treatment pathway might be impacted by the insights from the algorithms, there is an increased need for these models to be scrutinized and validated, when leveraged in the decision-making process. For example, can being on a diabetes medication be a predictor of breast cancer diagnosis or is it merely descriptors of the selected patient cohort that can help inform, but not predict the outcome? Further, questions are being posed on the ability to replicate results from the machine learning algorithms. Can an independent third party using the same data and assumptions arrive at the same results / final set of algorithms? Validation of the research has come to question in the recent couple of years, with a few cases of un- successful applications of the machine learning algorithms (i.e. IBM Watson (Mary Chris Jaklevic, 2017)). As a result, there is a growing concern in the scientific community about applying these techniques in certain areas of the healthcare industry due to the pitfalls listed above. Trying to think through on how to improve the process at-risk patient prediction along with the ability to validate and replicate the modeling outcomes, this presentation will review case studies in which we propose techniques and business reasonings to inform objective evaluation of the algorithms and their application in healthcare. We will outline selected benchmark rules that we think can help in validation of the research, along with examples of applications for case studies with the validated outcomes. As a result, the audience will be able to learn and then apply same rules and techniques when working on machine learning projects to ensure the results are not only informed in science and valid but can also be replicated if needed by others.

Recent Publications

1. Jonathan Bernick (May 2015) The Role of Machine Learning in Drug Design and Delivery. Journal of Developing Drugs.

2. Nitin Choudhary, Ewa Kleczyk, PhD, Rajkumar Rajabathar (Spring 2018) Clinical In-Market Leading Indicators for Brand Performance. Journal of the Pharmaceutical Management Science Association, ISSN 2473-9685.

3. Solutions. (Feb 2018) ‘Breaking through the hype in health care: What can machine learning really do for your patients?’ Academy of Health.

4. Mary Chris Jaklevic (Feb 2017) ‘MD Anderson Cancer Center’s IBM Watson project fails, and so did the journalism related to it’ Health News Review.

5. Tim Hare, Pratibha Sharan, Ewa Kleczyk, PhD, Derek Evans. (Spring 2018) Improving Accuracy in Rare Disease Patient Identification Using Pattern Recognition Ensembles. Journal of the Pharmaceutical Management Science Association, ISSN 2473-9685.

Biography

---

Ewa Kleczyk is a Vice President leading the Client Analytics group at Symphony Health. Ewa has more than 15 years of experience in pharmaceutical market research, having worked on both primary & secondary side of the market research area. With her broad experience in commercial effectiveness, digital & media analytics, as well as forecasting & promotional impact measurement, she is one of the industry recognized leaders in the pharmaceutical marketing research area. Ewa is also a highly sought-after conference speaker with her experience speaking at the leading industry conferences, including PMSA, PMRG & DTC Perspectives. She also has published in multiple academic & industry journals and is a board member of several peer-reviewed publications, including the PMSA Journal. Ewa has been an active advocate of mentoring future women leaders of the pharma industry for which she has been recognized with multiple leadership awards, including HBA’s ‘Rising Star’ & ‘Luminary’ recognitions. Ewa has earned her PhD in Economics from Virginia Tech and has been a part-time graduate faculty in the School of Economics at the University of Maine.