What is the LSH Forest Algorithm

LSH-Based Graph Partitioning Algorithm

tip

Access other chapters in this book by swiping

2018 | OriginalPaper | Book chapter

Abstract

The well-partitioned graph is capable of accelerating the parallel graph algorithms significantly, but few of them adopt the well partitioning algorithms in large scale graph computing. The high time complexity, which even exceed that of the final algorithms occasionally, is the main factor to prevent their applicabilities. Existing graph partitioning algorithms are mostly based on multilevel k-way scheme or iterative label propagation. Most of these algorithms can yield a high-quality result, but the high time / space complexities limit their applications in big data. In this paper, we propose the locality-sensitive hashing (LSH) based graph partitioning algorithm whose time / space complexity is O (n), n is the number of vertices in graph. For all kinds of hyperscale graphs, it works at the speed of random partitioning method approximately. Compared with the latest mainstream graph partitioning algorithms, the new algorithm owns a simple processing pipeline and avoids irregular memory access generated by graph traversals. The experimental result show that the new algorithm achieves 10x faster than Metis and 2x faster than label propagation algorithm at the cost of reasonable precision loss.

Would you like to get access to this content? Then find out more about our products now:

Springer Professional "Business + Technology"

With Springer Professional "Business + Technology" you get access to:

  • above 69,000 books
  • above 500 magazines

from the following fields:

  • Automobile + engines
  • Construction + real estate
  • Business IT + informatics
  • Electrical engineering + electronics
  • Energy + environment
  • Finance + Banking
  • Management + leadership
  • Marketing + sales
  • Mechanical engineering + materials
  • Insurance + risk

Try now for 30 days free of charge.

Springer Professional "Technology"

With Springer Professional "Technology" you get access to:

  • above 50,000 books
  • above 380 magazines

from the following fields:

  • Automobile + engines
  • Construction + real estate
  • Business IT + informatics
  • Electrical engineering + electronics
  • Energy + environment
  • Mechanical engineering + materials



Try now for 30 days free of charge.

Springer Professional "Economy"

With Springer Professional "Economy" you get access to:

  • above 58,000 books
  • above 300 magazines

from the following fields:

  • Construction + real estate
  • Business IT + informatics
  • Finance + Banking
  • Management + leadership
  • Marketing + sales
  • Insurance + risk



Try now for 30 days free of charge.

literature
Go back to reference Zhang, X.K., Ren, J., Song, C., Jia, J., Zhang, Q .: Label propagation algorithm for community detection based on node importance and label influence. Phys. Lett. A 381 (2017) Zhang, X.K., Ren, J., Song, C., Jia, J., Zhang, Q .: Label propagation algorithm for community detection based on node importance and label influence. Phys. Lett. A 381 (2017)
Go back to reference Karypis, G., Schloegel, K., Kumar, V .: Parmetis: parallel graph partitioning and sparse matrix ordering library, Version, Department of Computer Science, University of Minnesota (2003) Karypis, G., Schloegel, K ., Kumar, V .: Parmetis: parallel graph partitioning and sparse matrix ordering library, Version, Department of Computer Science, University of Minnesota (2003)
Back to reference Bulu \ (\ dot {\ rm c} \), A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C .: Recent advances in graph partitioning, vol. 77, no. 1, pp. 207–220 (2013) Bulu \ (\ dot {\ rm c} \), A., Meyerhenke, H., Safro, I., Sanders, P., Schulz, C .: Recent advances in graph partitioning, vol. 77, no. 1, pp. 207–220 (2013)
Go back to reference Jin, K., Hwang, I., Kim, Y.H., Moon, B.R .: Genetic approaches for graph partitioning: a survey. In: Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2011, Dublin, Ireland, July 2011, pp. 473-480 (2011) Jin, K., Hwang, I., Kim, YH, Moon, BR: Genetic approaches for graph partitioning: a survey. In: Proceedings of Genetic and Evolutionary Computation Conference, GECCO 2011, Dublin, Ireland, July 2011, pp. 473-480 (2011)
Go back to reference Wang, L., Xiao, Y., Shao, B., Wang, H .: How to partition a billion-node graph (2014) Wang, L., Xiao, Y., Shao, B., Wang , H .: How to partition a billion-node graph (2014)
Go back to reference Zhang, W., He, B., Chen, Y., Zhang, Q .: GMR: graph-compatible mapreduce programming model. Multimedia Tools Appl. 1, 1–19 (2017) Zhang, W., He, B., Chen, Y., Zhang, Q .: GMR: graph-compatible mapreduce programming model. Multimedia Tools Appl. 1, 1–19 (2017)
Go back to reference Bawa, M., Condie, T., Ganesan, P .: LSH forest: self-tuning indexes for similarity search. In: International Conference on World Wide Web, pp. 651-660 (2005) Bawa, M., Condie, T., Ganesan, P .: LSH forest: self-tuning indexes for similarity search. In: International Conference on World Wide Web, pp. 651–660 (2005)
Go back to reference Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K .: Multi-probe LSH: efficient indexing for high-dimensional similarity search. In: International Conference on Very Large Data Bases, pp. 950-961 (2007) Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K .: Multi-probe LSH: efficient indexing for high-dimensional similarity search. In: International Conference on Very Large Data Bases, pp. 950–961 (2007)
Go back to reference Berchtold, S., Keim, D.A., Kriegel, H.P .: The X-tree: an index structure for high-dimensional data. In: Proceedings of VLDB, September 1996, Mumbai, India, pp. 28-39 (1996) Berchtold, S., Keim, D.A., Kriegel, H.P .: The X-tree: an index structure for high-dimensional data. In: Proceedings of VLDB, September 1996, Mumbai, India, pp. 28-39 (1996)
Raghavan, U.N., Albert, R., Kumara, S .: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 76 (2), 036106 (2007) CrossRef Raghavan, U.N., Albert, R., Kumara, S .: Near linear time algorithm to detect community structures in large-scale networks. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 76 (2), 036106 (2007) CrossRef
About this chapter
title
LSH-Based Graph Partitioning Algorithm
book
Artificial Intelligence

Print ISBN: 978-981-13-2121-4

Electronic ISBN: 978-981-13-2122-1

Copyright Year: 2018

https://doi.org/10.1007/978-981-13-2122-1

DOI
https://doi.org/10.1007/978-981-13-2122-1_5
Authors:
Weidong Zhang
Mingyue Zhang
publishing company
Springer Singapore

premium partner