Developing Data Engineering Solutions for Edge Computing in IoT Deployments
Keywords:
Edge Computing, Internet of Things (IoT), Data Engineering, Real-Time Data Processing, Fog Computing, Microservices Architecture, Data Ingestion, Distributed Systems, Data Storage OptimizationAbstract
The growing adoption of Internet of Things (IoT) devices has led to an explosion of data generated at the network's edge, necessitating efficient data engineering solutions to handle, process, and analyze this data closer to the source. Traditional centralized cloud architectures face limitations in latency, bandwidth, and scalability, making edge computing a compelling approach to overcome these challenges. This paper presents a comprehensive framework for implementing data engineering solutions tailored to edge computing environments in IoT deployments. Key considerations include data ingestion, storage optimization, real-time processing, and data security at the edge. Through the integration of distributed computing models, such as Fog Computing and microservices architectures, the proposed solutions aim to reduce latency and enhance responsiveness while maintaining data integrity and security. The paper discusses various data engineering techniques that optimize resource usage, enable dynamic scaling, and support diverse IoT applications. Experimental evaluations demonstrate how edge-based data processing can lead to significant improvements in performance, especially in latency-sensitive applications like autonomous vehicles, healthcare monitoring, and smart manufacturing systems.
References
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637-646. doi:10.1109/JIOT.2016.2579198.
Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30-39. doi:10.1109/MC.2017.9.
Bonomi, F., Milito, R., Zhu, J., & Addepalli, S. (2012). Fog computing and its role in the internet of things. In Proceedings of the first edition of the MCC workshop on Mobile cloud computing (pp. 13-16). doi:10.1145/2342509.2342513.
Shi, W., & Dustdar, S. (2016). The promise of edge computing. Computer, 49(5), 78-81. doi:10.1109/MC.2016.145.
Hu, Y. C., Patel, M., Sabella, D., Sprecher, N., & Young, V. (2015). Mobile edge computing—A key technology towards 5G. ETSI white paper, 11(11), 1-16.
Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems, 29(7), 1645-1660. doi:10.1016/j.future.2013.01.010.
Premsankar, G., Di Francesco, M., & Taleb, T. (2018). Edge computing for the Internet of Things: A case study. IEEE Internet of Things Journal, 5(2), 1275-1284. doi:10.1109/JIOT.2018.2818722.
Dastjerdi, A. V., Gupta, H., Calheiros, R. N., Ghosh, S. K., & Buyya, R. (2016). Fog computing: Principles, architectures, and applications. In Internet of Things (pp. 61-75). Springer, Cham.
Chiang, M., & Zhang, T. (2016). Fog and IoT: An overview of research opportunities. IEEE Internet of Things Journal, 3(6), 854-864. doi:10.1109/JIOT.2016.2584538.
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017). A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys & Tutorials, 19(4), 2322-2358. doi:10.1109/COMST.2017.2745201.
Published
Issue
Section
License
Copyright (c) 2020 MANDA NAVYA (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
