Xinxin Shao, Limei Fu · 2020
In order to improve the accuracy and efficiency of customer clustering, an improved Canopy algorithm is proposed. The initial cluster centers are generated based on the principle of maximum and minimum by using Canopy algorithm. Then the overlapping data is classified by shared nearest neighbor algorithm. For different sizes of order data sets, the comparison between the proposed algorithm and the SNN algorithm in terms of the actual effect is conducted. The experimental results show that the proposed method effectively solves the problem of processing overlapping data of traditional Canopy algorithm and the redundant distance calculation in iterative is avoided. The improved algorithm has better performance than the traditional algorithm. The algorithm effectively improves analysis time and has great application value in the analysis of bank data and forecasting customer behavior. The clustering results can provide support for the bank's sales behavior.
Cited by 4
Jun-Wu Zhai Jun-Wu Zhai, Yu-Chen Tian Jun-Wu Zhai, Wen-Tao Li Yu-Chen Tian et al. · 電腦學刊 · 2023
<p>Aiming at the problems that traditional K-means text clustering cannot automatically determine the number of clusters and is sensitive to initial cluster centers, this paper proposes a Canopy-MMD text clustering algorithm based on simulated annealing and silhouette coefficient optimization. The algorithm uses the simulated annealing algorithm combined with the silhouette coefficient to optimize the Canopy algorithm to find the optimal number of clusters, and uses the optimal number of clusters to determine the scale coefficient in the MMD algorithm, and finally achieves a better text clustering effect. The Sohu News dataset of Sogou Lab is experimentally analyzed and compared with the clustering results obtained by traditional K-means and algorithms in the literature. The experimental results show that the clustering performance of the algorithm is better than the traditional K-means algorithm and the algorithm in the literature, and the accuracy, precision, recall and F value are improved by 8.02%, 8.91%, 8.02%, 9.51% compared with the traditional K-means algorithm, which can be widely used in fields such as text mining, knowledge graph and natural language processing.</p> <p>&nbsp;</p>
Cited by 0
Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2009
Elsevier’s Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.
Cited by 0
Huiling Zhao · 2021 International Conference on Electronic Information Engineering and Computer Science (EIECS) · 2021
For the traditional K-Means algorithm in the face of large-scale data, due to the randomness of the initial cluster centers selected lead to local optima, slow speed clustering and other issues, this paper presents an improved algorithm Canopy-KMeans. First use the “minimum maximum principle” to optimize the selection of the center point of the Canopy algorithm, obtain a more accurate set of cluster center points and the number of clusters k; then use the “triangular inequality principle” to optimize the K-Means algorithm, reduce unnecessary distance calculation in the iterative process, finally combined with the Hadoop platform Mapreduce calculation framework for the parallel design and implementation of the algorithm. By experimental tests show, K-Means optimized analysis and comparison before optimization algorithm, fast convergence, you can get better clustering quality, suitable for large-scale data clustering.
Cited by 0
Di Fan, Leihua Fan, Fuyan Zhao et al. · 2024
As online courses are more and more widely used in college courses, the analysis of online learning data has become an important means to improve teaching quality and learning effect. In this paper, an online learner clustering algorithm that integrates Autoencoder and Canopy-Kmeans is proposed. The Autoencoder is used to perform feature degradation and feature extraction for high-dimensional data which effectively solves the problems of data sparsity and dimensionality disaster. The Canopy algorithm is introduced to precluster the data which reduces the randomness of the K-means algorithm during initialization and improves the stability of clustering. And the improved K-means algorithm is used to perform the clustering analysis of text data. The experimental results show that the algorithm proposed behaved better in both clustering accuracy and noise resistance compared with traditional clustering algorithm, which is helpful to a better understanding of the behavioral characteristics of online learners and a better support for personalized teaching is provided based on this.
Cited by 0
Hua Zhang, Xiangbing Zhou · 2018
Given a dataset, it is difficult to find the number of natural clusters and moreover the clustering result is sensitive to the selection of the initial seeds. This sensitivity can make many clustering algorithms converge to the local optima. This paper proposes a novel niching genetic algorithm (NNGA) with K-means (NClust) that is capable of automatically finding the better number of clusters and identifying the right genes/chromosomes through a novel initial population approach based on an improved canopy and K-means++ which, may effectively accelerate the convergence speed and enhance the global searching ability for the purpose of a more efficient result. Furthermore, adaptive probabilities of crossover and mutation are also employed to prevent the convergence of NClust to a local optimum. With the help of the performance of our algorithm, the existing cluster indices, including SSE, DBI, PBM, and COSEC are employed as fitness functions. Using the real-world data sets, this paper compares the performance of NClust with other GA-based clustering methods (GAK and GenClust). Experiment results indicate that the NClust has high performance.
Cited by 13
Jianfei Tong · Journal of Interdisciplinary Mathematics · 2017
Poor understanding and low clustering efficiency of massive data is a problem under the context of big data. To solve this problem, MapReduce programming model is adopted to combine Canopy and K-means clustering algorithms within cloud computing environment, so as to fully make use of the computing and storing capacity of Hadoop clustering. Large quantities of buyers on taobao are taken as application context to do case study through Hadoop platform’s data mining set Mahout. General procedure for miming with Mahout is also given.
Cited by 8
Lan Luan, Shaobo Du · 2022 IEEE 10th International Conference on Information, Communication and Networks (ICICN) · 2022
Fuzzy C-Means Clustering (FCM) algorithm is a flexible clustering algorithm, which can be used for large-scale data mining. However, FCM algorithm is sensitive to the selection of cluster centers and runs slowly when clustering large-scale text data. Therefore, a parallel fuzzy C-means clustering algorithm based on improved Canopy is proposed. Canopy algorithm needs to specify a reasonable threshold to calculate, and the concept of weight is introduced to improve the Canopy algorithm to improve the FCM clustering accuracy. Firstly, the density value of each sample point in the data set is calculated, and the maximum value is selected as the first clustering center. In the calculation of the remaining sample points in the weight, and then select the center point of other clusters. Finally, the parallel computing is carried out through the Spark framework to improve the efficiency of the algorithm. The experimental results show that the fuzzy C-means parallel text clustering algorithm based on improved Canopy improves the clustering accuracy by 10% -20% compared with FCM algorithm, and the overall performance of the algorithm has been improved.
Cited by 1
Gongjian Zhou · 2018
How to apply clustering algorithm to effectively cluster large-scale data is an important research topic in data mining. Based on an in-depth analysis of the Hadoop platform architecture and Canopy-kmeans clustering algorithm, the Canopy-kmeans algorithm was optimized and parallelized. The data packets are clustered after grouping and sampling by statistical thinking to facilitate parallelization and reduce time complexity. The Canopy initial center point selection was optimized using the minimum-maximum principle, and data outlier average sampling method was used to ensure the uniform extraction of data samples from the original data, and the k-means iterative calculation process was optimized. Combined with the MapReduce framework under the Hadoop platform, the improved algorithm is designed and implemented in parallel. Experiments show that the improved Canopy-Kmeans parallel algorithm is effective and convergent when clustering massive amounts of numerical data, and it has a certain degree of improvement in the clustering accuracy and timeliness.
Cited by 3
D. Akila, Sivakami Raja, S. Jeyalaksshmi et al. · 2024
Text mining programs often offer numerous options for processing textual data. The inclusion of extensive meta-data-such as links to other websites, titles, authors, and publication dates-can enrich text analysis and aid in text grouping. However, this side information, or “metadata,” can also introduce noise if not handled correctly. When used indiscriminately for clustering, this noise can degrade the quality of the resulting clusters. To address this challenge, we propose a methodology that leverages a Feature Selection procedure to identify and utilize metadata that enhances clustering effectiveness. The methodology, known as Co-Clustering with Side Information (C-CSI), is designed to maximize the benefits of relevant metadata while mitigating the effects of irrelevant or noisy data. C-CSI is implemented using datamining technologies, specifically focusing on Co-Clustering, which involves clustering both rows and columns of a matrix. In our study, we employed the canopy and k-means algorithms to perform the clustering tasks. This approach ensures that the metadata used in clustering is both relevant and beneficial, improving the overall quality of the clustering results
Cited by 2
QIN Xinyu SHEN Xueli · SHILAP Revista de lepidopterología · 2021
As the most widely used supervised classification algorithm, K nearest neighbor (KNN) algorithm is often inefficient in the processing of large-scale and multidimensional data. Therefore, an improved KNN algorithm for high dimensional large data processing is proposed. Firstly, deep neural networks (DNN) is used as feature extractor and dimension reduction is carried out to learn the most appropriate representation form of depth feature. Then, the appropriate number of clusters and the initial clustering center, obtained through the density Canopy algorithm become the input parameters of the subsequent K-means clustering. Finally, the learned data are clustered, and Hashing strategy in approximate similarity search (ASS) is used to cluster partitioning according to its approximate similarity, and the result is token as a new training sample of KNN classifier. In addition, considering that the nearest neighbor samples to be searched may fall in different clusters and the performance of KNN search is reduced, an additional clustering enhancement strategy is adopted in clustering, which effectively alleviates this situation. Five different data sets are used for comparison test. The results show that, compared with the experimental algorithms, this algorithm can not only greatly improve the accuracy of KNN classification, but also effectively improves the classification efficiency of the algorithm, reduces the distance required for searching, and has good robustness for noise data.
Cited by 3Open access
Zhihe Wang, Gang Zhang, Hui Du et al. · Lecture notes in electrical engineering · 2022
Cited by 0
Wei Dai, Changjun Yu, Zilong Jiang · Journal of Computing Science and Engineering · 2016
The fuzzy c-means (FCM) is a frequently utilized algorithm at present. Yet, the clustering quality and convergence rate of FCM are determined by the initial cluster centers, and so an improved FCM algorithm based on canopy cluster concept to quickly analyze the dataset has been proposed. Taking advantage of the canopy algorithm for its rapid acquisition of cluster centers, this algorithm regards the cluster results of canopy as the input. In this way, the convergence rate of the FCM algorithm is accelerated. Meanwhile, the MapReduce scheme of the proposed FCM algorithm is designed in a cloud environment. Experimental results demonstrate the hybrid canopy-FCM clustering algorithm processed by MapReduce be endowed with better clustering quality and higher operation speed.
Cited by 11Open access
Yue Zhang, Ruan Peng, Jingfeng Zhao · Frontiers in Psychology · 2022
With the continuous improvement of the level of science and technology, the popularization of the Internet and the development of applications, online consumption has become a major force in personal consumption. As a result, digital consumption is born, and digital consumption is not only reflected in transaction consumption at the monetary level. Like some intangible services similar to the use of dating software, it can also become digital consumption. In this environment, a new economic concept, the digital economy, has emerged as the times require. The digital economy helps to achieve the rapid optimal allocation and regeneration of resources and achieve high-quality economic development. Therefore, as a new economic form, the digital economy has penetrated into all fields of human society. The Canopy algorithm is a fast clustering technique that requires only one pass through the data technology to get the results. But it is inaccurate for large-scale data clustering. Therefore, when analyzing the data, it is necessary to use the Canopy algorithm for preliminary clustering, and then combine with other algorithms or model software for refinement. This article introduces the development of the digital economy in the Internet era. It conducts a theoretical discussion on consumer psychology in the context of the digital economy. It introduces the basic calculation formula of the clustering algorithm and the algorithm flow of the Canopy clustering algorithm. It does model optimization for the Canopy clustering algorithm. On this basis, it designs questionnaires for experimental design. The indicators are divided into commodity attributes and consumer psychology. It builds a consumer psychology prediction model and tests the prediction results. The results show that the maximum difference between the prediction results of digital economy consumption psychology based on the Canopy clustering algorithm and the actual results is 0.047. It can be shown that the psychological prediction model of digital economy consumption based on the Canopy clustering algorithm has certain practicability.
Cited by 5Open access
Jingting Liang, Xiangguo Yin, Mingxing Lin · Biomedical Physics & Engineering Express · 2025
<i>Background</i>. Electroencephalogram (EEG) microstate analysis can capture transient patterns of brain activity and provide valuable insights into brain motor and cognitive functions. However, the performance of traditional microstate analysis algorithms limits a deeper understanding of the neural mechanisms behind complex conditions.<i>Methods</i>. This study proposed a Canopy-KM-GSA algorithm, which combines Canopy clustering algorithm, K-means algorithm and genetic simulated annealing framework to automatically determine the optimal number of microstates and refine the clustering sequence. Utilizing the proposed algorithm, the study performed microstate analysis of pedaling motor datasets, Passive Auditory Oddball Paradigm task datasets, and epileptic patients datasets. The performance of the proposed algorithm is compared with seven baseline algorithms (including traditional K-means algorithm, K-medoids algorithm, ICA algorithm, PCA algorithm, GMD driven density canopy K-means algorithm, modified K-means algorithm and Agglomerative Hierarchical Clustering(AAHC) algorithm).<i>Results</i>. The results demonstrated the superior performance of Canopy-KM-GSA, achieving a significantly higher total evaluation compared to baseline microstate analysis algorithms. With an average Global Explained Variance (GEV) of 94.43%, an average Calinski-Harabasz Index (CHI) of 537.99, and an average Davies-Bouldin Index (DBI) of 1.57 in pedaling motor datasets; an average GEV of 94.46%, an average CHI of 389.29, and an average DBI of 1.44 in Passive Auditory Oddball Paradigm task datasets; an average GEV of 58.40%, an average CHI of 254.11, and an average DBI of 1.53 in epileptic patients datasets.<i>Conclusions</i>. The novel microstate analysis algorithms offers a more accurate tool for EEG microstate analysis.
Cited by 0
Jun Ma, Shen Luo, Liuyang Wu et al. · 2017
In order to satisfy the needs of highway passenger precise segmentation with massive historical data, a novel clustering algorithm under parallel computing framework is proposed. The average number of tickets purchased in a period is considered to build an evaluation model of highway passenger segmentation CFMY. To accurately determine the initial center point and K value, Canopy algorithm is introduced to improve the K-means clustering algorithm. The improved k-means algorithm is conducted using the parallel computing framework in Spark platform. Finally, the proposed method using the parallel computing framework is applied to the highway passenger segmentation cluster analysis, where the CFMY model is used as the evaluation index. The effectiveness of the proposed method is verified by experiments.
Cited by 3
Qìng Yu, Zhimin Ding · 2015
In order to solve the problem that the Fuzzy C-Means algorithm is sensitive to the initial clustering center, we use the Canopy algorithm to carry out the quick and rough clustering. At the same time, to avoid the blindness of the Canopy algorithm, we put forward an improved Canopy-FCM algorithm based on a max-min principle. In allusion to the problem that the FCM algorithm has high time complexity, this article use the parallel computing frame of MapReduce to design and realize the improved Canopy-FCM algorithm. Experimental results show: the improved Canopy-FCM algorithm based on MapReduce has better clustering quality and running speed than the Canopy-FCM and the FCM algorithm based on MapReduce, and the improved Canopy-FCM algorithm based on Hadoop has better speed-up ratio than the Canopy-FCM based on the Standalone mode.
Cited by 11
Yufeng Zhuang, Jingwen Han, Yanzhu Hu · 2020
With the sharp increase in the number of orders and the amount of dismantling and sorting in the pharmaceutical logistics center, how to save labor in a limited time and improve the efficiency of sorting orders for dismantling is a problem that needs to be solved urgently in the field of medicine circulation. This article focuses on the issue of order batching strategy based on the "delivery to person" picking system in the pharmaceutical industry to improve the efficiency of picking. Based on the characteristics of pharmaceutical orders and related policies, the number of identical items between the two orders is used as the order coupling factor to establish the order batching model. In order to solve the model, this paper proposes the Canopy-k-means two-stage clustering algorithm in which the Canopy algorithm is used to coarsely cluster the orders to reduce the dimensionality firstly, and then the K-means clustering algorithm is run to subdivide the batch. In addition, the MATLAB platform is used to simulate and verify the algorithm, the implementation effects of the algorithm and the first come first service (FCFS) algorithm and the K-means clustering algorithm are compared, and the effectiveness of the algorithm in terms of clustering effect and operating efficiency is verified.
Cited by 3
Yuxue LI, Jun JI, Yang DONG · SHILAP Revista de lepidopterología · 2024
In order to solve the problem that random selection of clustering number k can easily lead to instability and low computational efficiency of k-means clustering algorithm in the zoning calculation of wind load on the surface of long-span roof structures, an improved Canopy-k-means clustering algorithm was proposed. Firstly, the Canopy algorithm was introduced, and the selection of its initial threshold and clustering center was improved to reduce the blindness of the initial value selection so as to improve the reliability of the results of wind load zoning. Secondly, the improved Canopy algorithm was used to preprocess the wind load dataset to determine the cluster number k quickly and accurately. Thirdly, the improved Canopy algorithm was combined with k-means to achieve the accurate identification of the optimal classification number k, so that the improved Canopy-k-means clustering algorithm can get the zoning results quickly and accurately when the wind load on the surface of long-span roof structure was divided. Finally, taking a long-span cylindrical roof dry coal shed structure as an example, based on the test results of the surface wind load data obtained from the wind tunnel test, the improved Canopy-k-means clustering algorithm was used to calculate the surface wind load. The results show that by using the improved Canopy-k-means clustering algorithm, the wind loads on the surface of long-span roofs at 0 °, 50 °and 90 °wind angles are divided into three different zones, and the corresponding SD values are 2.36,3.51 and 2.52,respectively, which are significantly lower than those obtained by the traditional k-means clustering algorithm, and the intra-class compactness and inter-class dispersion are obviously improved. Therefore, the improved Canopy-k-means clustering algorithm can obtain the optimal zoning results quickly and accurately, and has good engineering application value for wind load zoning on the surface of long-span roofs.
Cited by 0Open access
Peizhang Xie, Minming Mao, Xuguang Jin et al. · 2021
With the rapid development of “Internet +”, mobile application and communication technology, e-commerce has increasingly become the main shopping way for consumers in China. Aiming at quality keywords, this paper designs a distributed data clustering system based on Hadoop and mahout. In order to overcome the randomness, low accuracy and many iterations of K-Means algorithm, Canopy and K-Means Clustering Algorithm based on Mahout is designed by using the advantages of Canopy algorithm, that is, no need to specify the number of clusters, high efficiency and conciseness Clustering keywords. Based on the algorithm, good results have been obtained.
Cited by 3