Performance Evaluation of Distributed Machine Learning Algorithms in Big Data Platforms
M.V. Anjana Devi,
Dr. Koneti Krishnaiah,
Bhaskar K
The exponential growth of data generated from social media, Internet of Things (IoT) devices, healthcare
systems, financial transactions, and enterprise applications has significantly increased the demand for
scalable machine learning techniques capable of processing massive datasets efficiently. Traditional
machine learning algorithms are often limited by computational resources, memory constraints, and
prolonged training times when handling large-scale data. Distributed machine learning has emerged
as an effective solution by leveraging parallel computing frameworks and distributed data processing
platforms to improve scalability, computational efficiency, and model performance. Modern big data
platforms such as Apache Hadoop, Apache Spark, and distributed cloud infrastructures enable machine
learning algorithms to process large datasets across multiple computing nodes while reducing execution
time and improving resource utilization. This paper presents a comprehensive performance evaluation
of distributed machine learning algorithms implemented on big data platforms. The proposed study
compares widely used algorithms including Distributed Linear Regression, Random Forest, Gradient
Boosting, Support Vector Machine, and Distributed Deep Neural Networks using Apache Spark as the
distributed processing framework. Performance evaluation is conducted using large-scale benchmark
datasets containing structured and semi-structured data. Comparative analysis is performed using
classification accuracy, execution time, scalability, resource utilization, throughput, and speedup
metrics. Experimental results demonstrate that distributed machine learning algorithms significantly
outperform conventional standalone implementations in terms of computational efficiency, scalability,
and processing speed while maintaining high predictive accuracy. The proposed evaluation framework
provides valuable insights for researchers and practitioners in selecting suitable distributed learning
algorithms for big data analytics applications.
