Advanced search
1 file | 2.50 MB Add to list

A framework for big Earth observation data using horizontal scaling strategy

Author
Organization
Abstract
The rapid growth of Earth observation (EO) data poses a challenge to the way of data management. An efficient framework based on big data technology can bring new solutions. Some excellent frameworks have been proposed, which provide efficient organization and management of EO data. However, they are not optimized for data distribution in the storage environment. In this letter, an optimized EO data management strategy is proposed. Different horizontal scaling strategies are designed to explore the optimal scheme of EO data distribution. The MapReduce parallel computing model was used to test the performance of data retrieval in the experiment. The results show that the proposed strategy contributes to the efficient organization and arrangement of data. Remote sensing (RS) data blocks can be evenly distributed to different shards according to the time characteristics and hash characteristics of the strategy, and the logical index of the data reduces the time consumed by the routing process. This distributed management mode that achieves load balancing provides a framework foundation for parallel computing. Therefore, the framework with an efficient strategy can improve the performance of data management.
Keywords
Distributed databases, Big Data, Earth, Metadata, Indexing, Load management, Routing, Data retrieval, Earth observation (EO) data, horizontal scaling strategy, parallel computing, Electrical and Electronic Engineering, Geotechnical Engineering and Engineering Geology

Downloads

  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 2.50 MB

Citation

Please use this url to cite or link to this publication:

MLA
Cheng, Yinyi, et al. “A Framework for Big Earth Observation Data Using Horizontal Scaling Strategy.” IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, vol. 19, 2022, doi:10.1109/LGRS.2022.3192640.
APA
Cheng, Y., Zhou, K., Wang, J., Cui, S., Yan, J., De Maeyer, P., & Van de Voorde, T. (2022). A framework for big Earth observation data using horizontal scaling strategy. IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 19. https://doi.org/10.1109/LGRS.2022.3192640
Chicago author-date
Cheng, Yinyi, Kefa Zhou, Jinlin Wang, Shichao Cui, Jining Yan, Philippe De Maeyer, and Tim Van de Voorde. 2022. “A Framework for Big Earth Observation Data Using Horizontal Scaling Strategy.” IEEE GEOSCIENCE AND REMOTE SENSING LETTERS 19. https://doi.org/10.1109/LGRS.2022.3192640.
Chicago author-date (all authors)
Cheng, Yinyi, Kefa Zhou, Jinlin Wang, Shichao Cui, Jining Yan, Philippe De Maeyer, and Tim Van de Voorde. 2022. “A Framework for Big Earth Observation Data Using Horizontal Scaling Strategy.” IEEE GEOSCIENCE AND REMOTE SENSING LETTERS 19. doi:10.1109/LGRS.2022.3192640.
Vancouver
1.
Cheng Y, Zhou K, Wang J, Cui S, Yan J, De Maeyer P, et al. A framework for big Earth observation data using horizontal scaling strategy. IEEE GEOSCIENCE AND REMOTE SENSING LETTERS. 2022;19.
IEEE
[1]
Y. Cheng et al., “A framework for big Earth observation data using horizontal scaling strategy,” IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, vol. 19, 2022.
@article{01GKGV7V8S0KR03GCN8NEBBGD4,
  abstract     = {{The rapid growth of Earth observation (EO) data poses a challenge to the way of data management. An efficient framework based on big data technology can bring new solutions. Some excellent frameworks have been proposed, which provide efficient organization and management of EO data. However, they are not optimized for data distribution in the storage environment. In this letter, an optimized EO data management strategy is proposed. Different horizontal scaling strategies are designed to explore the optimal scheme of EO data distribution. The MapReduce parallel computing model was used to test the performance of data retrieval in the experiment. The results show that the proposed strategy contributes to the efficient organization and arrangement of data. Remote sensing (RS) data blocks can be evenly distributed to different shards according to the time characteristics and hash characteristics of the strategy, and the logical index of the data reduces the time consumed by the routing process. This distributed management mode that achieves load balancing provides a framework foundation for parallel computing. Therefore, the framework with an efficient strategy can improve the performance of data management.}},
  articleno    = {{3513005}},
  author       = {{Cheng, Yinyi and Zhou, Kefa and Wang, Jinlin and Cui, Shichao and Yan, Jining and De Maeyer, Philippe and Van de Voorde, Tim}},
  issn         = {{1545-598X}},
  journal      = {{IEEE GEOSCIENCE AND REMOTE SENSING LETTERS}},
  keywords     = {{Distributed databases,Big Data,Earth,Metadata,Indexing,Load management,Routing,Data retrieval,Earth observation (EO) data,horizontal scaling strategy,parallel computing,Electrical and Electronic Engineering,Geotechnical Engineering and Engineering Geology}},
  language     = {{eng}},
  pages        = {{5}},
  title        = {{A framework for big Earth observation data using horizontal scaling strategy}},
  url          = {{http://doi.org/10.1109/LGRS.2022.3192640}},
  volume       = {{19}},
  year         = {{2022}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: