Aaron Richter

BACKGROUND:
Building cancer risk models from real-world data requires overcoming challenges in data preprocessing, efficient representation, and computational performance. We present a case study of a cloud-based approach to learning from de-identified electronic health record data and demonstrate its effectiveness for melanoma risk prediction.

METHODS:
We used a hybrid distributed and non-distributed approach to computing in the cloud: distributed processing with Apache Spark for… Show more

BACKGROUND:
Building cancer risk models from real-world data requires overcoming challenges in data preprocessing, efficient representation, and computational performance. We present a case study of a cloud-based approach to learning from de-identified electronic health record data and demonstrate its effectiveness for melanoma risk prediction.

METHODS:
We used a hybrid distributed and non-distributed approach to computing in the cloud: distributed processing with Apache Spark for data preprocessing and labeling, and non-distributed processing for machine learning model training with scikit-learn. Moreover, we explored the effects of sampling the training dataset to improve computational performance. Risk factors were evaluated using regression weights as well as tree SHAP values.

RESULTS:
Among 4,061,172 patients who did not have melanoma through the 2016 calendar year, 10,129 were diagnosed with melanoma within one year. A gradient-boosted classifier achieved the best predictive performance with cross-validation (AUC = 0.799, Sensitivity = 0.753, Specificity = 0.688). Compared to a model built on the original data, a dataset two orders of magnitude smaller could achieve statistically similar or better performance with less than 1% of the training time and cost.

CONCLUSIONS:
We produced a model that can effectively predict melanoma risk for a diverse dermatology population in the U.S. by using hybrid computing infrastructure and data sampling. For this de-identified clinical dataset, sampling approaches significantly shortened the time for model building while retaining predictive accuracy, allowing for more rapid machine learning model experimentation on familiar computing machinery. A large number of risk factors (>300) were required to produce the best model. Show less

Melanoma is one of the fastest growing cancers in the world, and can affect patients earlier in life than most other cancers. Therefore, it is imperative to be able to identify patients at high risk for melanoma and enroll them in screening programs to detect the cancer early. In this study, we explore data from dermatology outpatients to build a risk model for the disease. Using millions of patient records with thousands of data points in each record, we show that we can build a melanoma risk… Show more

Melanoma is one of the fastest growing cancers in the world, and can affect patients earlier in life than most other cancers. Therefore, it is imperative to be able to identify patients at high risk for melanoma and enroll them in screening programs to detect the cancer early. In this study, we explore data from dermatology outpatients to build a risk model for the disease. Using millions of patient records with thousands of data points in each record, we show that we can build a melanoma risk model from real-world Electronic Health Record (EHR) data without any expert knowledge or manually engineered features. While other risk models for melanoma have been developed, this is the first to use routinely collected EHR data rather than expert features targeted specifically for melanoma. The random forest model achieves similar or better performance than these previous models (AUC 0.79, sensitivity 0.71, specificity 0.72), which allows larger populations of patients to get screened for melanoma risk without having to perform specialized and time-consuming data collection. Important features from the model can be extracted and studied, and features influencing a specific prediction can be explained to providers and patients. The process for building this model can be further refined to improve performance, as well as used for risk prediction of other diseases. Show less

Advancements are constantly being made in oncology, improving prevention and treatment of cancers. To help reduce the impact and deadliness of cancers, they must be detected early. Additionally, there is a risk of cancers recurring after potentially curative treatments are performed. Predictive models can be built using historical patient data to model the characteristics of patients that developed cancer or relapsed. These models can then be deployed into clinical settings to determine if new… Show more

Advancements are constantly being made in oncology, improving prevention and treatment of cancers. To help reduce the impact and deadliness of cancers, they must be detected early. Additionally, there is a risk of cancers recurring after potentially curative treatments are performed. Predictive models can be built using historical patient data to model the characteristics of patients that developed cancer or relapsed. These models can then be deployed into clinical settings to determine if new patients are at high risk for cancer development or recurrence. For large-scale predictive models to be built, structured data must be captured for a wide range of diverse patients. This paper explores current methods for building cancer risk models using structured clinical patient data. Trends in statistical and machine learning techniques are explored, and gaps are identified for future research. The field of cancer risk prediction is a high-impact one, and research must continue for these models to be embraced for clinical decision support of both practitioners and patients. Show less

Melanoma is the fastest growing cancer worldwide, and 1 in 50 Americans will develop it in their lifetime. Sentinel lymph node (SLN) metastasis is one of the most important prognostic indicators for melanoma survival. We present several machine learning models for predicting SLN metastasis using data from a real-world dermatology electronic health record (EHR) system. The class label is the result of a sentinel lymph node biopsy, an elective procedure that can be performed for newly-diagnosed… Show more

Melanoma is the fastest growing cancer worldwide, and 1 in 50 Americans will develop it in their lifetime. Sentinel lymph node (SLN) metastasis is one of the most important prognostic indicators for melanoma survival. We present several machine learning models for predicting SLN metastasis using data from a real-world dermatology electronic health record (EHR) system. The class label is the result of a sentinel lymph node biopsy, an elective procedure that can be performed for newly-diagnosed melanoma patients to determine if there is metastasis in the nearest lymph node. We show that a simple model, using solely Breslow thickness, can achieve predictive performance (AUC=0.769) comparable to a logistic regression model using 5 features (AUC=0.772, p=0.518). Current clinical recommendations are to perform a biopsy for patients with melanomas thicker than 1mm, however, when applying this 1mm threshold to the simple thickness model, it achieves 0% sensitivity for melanomas <1mm. Using a random forest model, we achieve 78.9% sensitivity (p<0.001) for melanomas <1mm. Our study shows that the probability of sentinel lymph node positivity is indeed linearly correlated to the tumor thickness (R2=0.934), and that machine learning models can effectively detect thin melanomas that warrant an SLN biopsy. Show less

We present the Modernizing Analytics for MELanoma (MAMEL) dataset: a real-world,
dermatology-specific research dataset specifically crafted to advance data mining and
machine learning research in the field of melanoma diagnosis, analysis, and treatment. This
dataset was collected and curated from Modernizing Medicine's EMA Dermatology™
application, a cloud-based Electronic Health Record (EHR) platform. A big data processing
architecture, built on Apache Hadoop and Apache… Show more

We present the Modernizing Analytics for MELanoma (MAMEL) dataset: a real-world,
dermatology-specific research dataset specifically crafted to advance data mining and
machine learning research in the field of melanoma diagnosis, analysis, and treatment. This
dataset was collected and curated from Modernizing Medicine's EMA Dermatology™
application, a cloud-based Electronic Health Record (EHR) platform. A big data processing
architecture, built on Apache Hadoop and Apache Spark, was used to collect all patient
data, identify patients for the MAMEL dataset, and create and document all data elements.
This paper outlines the application and data processing architectures, provides an
exploratory analysis of data elements available in MAMEL, and discusses avenues for using
this dataset in clinical decision support applications for melanoma. Show less

While cancer treatments are constantly advancing, there is still a real risk of relapse after potentially curative treatments. At the risk of adverse side effects, certain adjuvant treatments can be given to patients that are at high risk of recurrence. The challenge, however, is in finding the best tradeoff between these two extremes. Patients that are given more potent treatments, such as chemotherapy, radiation, or systemic treatment, can suffer unnecessary consequences, especially if the… Show more

While cancer treatments are constantly advancing, there is still a real risk of relapse after potentially curative treatments. At the risk of adverse side effects, certain adjuvant treatments can be given to patients that are at high risk of recurrence. The challenge, however, is in finding the best tradeoff between these two extremes. Patients that are given more potent treatments, such as chemotherapy, radiation, or systemic treatment, can suffer unnecessary consequences, especially if the cancer does not return. Predictive modeling of recurrence can help inform patients and practitioners on a case-by-case basis, personalized for each patient. For large-scale predictive models to be built, structured data must be captured for a wide range of diverse patients. This paper explores current methods for building cancer recurrence risk models using structured clinical patient data. Show less

With an ever-increasing amount of options, the task of selecting machine learning tools for big data can be difficult. The available tools have advantages and drawbacks, and many have overlapping uses. The world’s data is growing rapidly, and traditional tools for machine learning are becoming insufficient as we move towards distributed and real-time processing. This paper is intended to aid the researcher or professional who understands machine learning but is inexperienced with big data. In… Show more

With an ever-increasing amount of options, the task of selecting machine learning tools for big data can be difficult. The available tools have advantages and drawbacks, and many have overlapping uses. The world’s data is growing rapidly, and traditional tools for machine learning are becoming insufficient as we move towards distributed and real-time processing. This paper is intended to aid the researcher or professional who understands machine learning but is inexperienced with big data. In order to evaluate tools, one should have a thorough understanding of what to look for. To that end, this paper provides a list of criteria for making selections along with an analysis of the advantages and drawbacks of each. We do this by starting from the beginning, and looking at what exactly the term “big data” means. From there, we go on to the Hadoop ecosystem for a look at many of the projects that are part of a typical machine learning architecture and an understanding of how everything might fit together. We discuss the advantages and disadvantages of three different processing paradigms along with a comparison of engines that implement them, including MapReduce, Spark, Flink, Storm, and H2O. We then look at machine learning libraries and frameworks including Mahout, MLlib, SAMOA, and evaluate them based on criteria such as scalability, ease of use, and extensibility. There is no single toolkit that truly embodies a one-size-fits-all solution, so this paper aims to help make decisions smoother by providing as much information as possible and quantifying what the tradeoffs will be. Additionally, throughout this paper, we review recent research in the field using these tools and talk about possible future directions for toolkit-based learning. Show less

Data generated by social media are frequently leveraged to build machine learning models that can accurately profile human behavior and sentiment. Twitter is a readily available source of population data that can be collected and used by any organization. Therefore, accurate machine learning models must be created to learn from this user-generated content. In this paper, we explore the task of classifying a user's preference towards a specific entity. Particularly, we study the accuracy of… Show more

Data generated by social media are frequently leveraged to build machine learning models that can accurately profile human behavior and sentiment. Twitter is a readily available source of population data that can be collected and used by any organization. Therefore, accurate machine learning models must be created to learn from this user-generated content. In this paper, we explore the task of classifying a user's preference towards a specific entity. Particularly, we study the accuracy of classification models as an increasing number of tweets (status posts) per user is provided to the models. New users and tweets are constantly being created, warranting the use of techniques to reduce the size of data needed for machine learning algorithms. We find that there is a diminishing return on model performance as the number of tweets per user is increased, and identify a threshold where adding more tweets per user does not result in statistically better performance. Utilizing this threshold, as opposed to the maximum amount of tweets per user, data collection time is reduced by 80% while dataset size is reduced by 75%. Show less

Online reviews are often the primary factor in a customer’s decision to purchase a product or service, and are a valuable source of information that can be used to determine public opinion on these products or services. Because of their impact, manufacturers and retailers are highly concerned with customer feedback and reviews. Reliance on online reviews gives rise to the potential concern that wrongdoers may create false reviews to artificially promote or devalue products and services. This… Show more

Online reviews are often the primary factor in a customer’s decision to purchase a product or service, and are a valuable source of information that can be used to determine public opinion on these products or services. Because of their impact, manufacturers and retailers are highly concerned with customer feedback and reviews. Reliance on online reviews gives rise to the potential concern that wrongdoers may create false reviews to artificially promote or devalue products and services. This practice is known as Opinion (Review) Spam, where spammers manipulate and poison reviews (i.e., making fake, untruthful, or deceptive reviews) for profit or gain. Since not all online reviews are truthful and trustworthy, it is important to develop techniques for detecting review spam. By extracting meaningful features from the text using Natural Language Processing (NLP), it is possible to conduct review spam detection using various machine learning techniques. Additionally, reviewer information, apart from the text itself, can be used to aid in this process. In this paper, we survey the prominent machine learning techniques that have been proposed to solve the problem of review spam detection and the performance of different approaches for classification and detection of review spam. The majority of current research has focused on supervised learning methods, which require labeled data, a scarcity when it comes to online review spam. Research on methods for Big Data are of interest, since there are millions of online reviews, with many more being generated daily. To date, we have not found any papers that study the effects of Big Data analytics for review spam detection. The primary goal of this paper is to provide a strong and comprehensive comparative study of current research on detecting review spam using various machine learning techniques and to devise methodology for conducting further investigation. Show less

Big data is a big business, and effective modeling of this data is key. This paper provides a comprehensive multi- dimensional analysis of various open source tools for machine learning with big data. An evaluation standard is proposed along with detailed comparisons of the frameworks discussed, with regard to algorithm availability, scalability, speed, and more. The major tools profiled are Mahout, MLlib, H2O, and SAMOA, along with the big data processing engines they utilize, including Hadoop… Show more

Big data is a big business, and effective modeling of this data is key. This paper provides a comprehensive multi- dimensional analysis of various open source tools for machine learning with big data. An evaluation standard is proposed along with detailed comparisons of the frameworks discussed, with regard to algorithm availability, scalability, speed, and more. The major tools profiled are Mahout, MLlib, H2O, and SAMOA, along with the big data processing engines they utilize, including Hadoop MapReduce, Apache Spark, and Apache Storm. There is not yet one framework that “does it all”, but this paper provides insight into each tool’s strengths and weaknesses along with guidance on tool choice for specific needs. Show less