AI-Driven Data Engineering in the Internet of Things: Scaling Data Pipelines for Smart Device Ecosystems
Keywords:
Data Engineering, Ecosystem, Analytics, PipelineAbstract
AI driven data engineering for driving a data pipeline at scale in IoT ecosystems is what this study covers. It looks into the impact of AI on e.g. data processing, latency reduction, scalability of the system. We show improvements in the efficiency as well as in the adaptability, using quantitative analysis. The findings illustrate how AI can help in intelligent data workflows sitting with IoT so data becomes easily processable and available for real time decisions of smart devices.
References
Zeydan, E., & Mangues-Bafalluy, J. (2022). Recent advances in data engineering for networking. IEEE Access, 10, 34449-34496. 10.1109/ACCESS.2022.3162863
Multamäki, M. (2024). Near real-time IoT data pipeline architectures (Master's thesis, M. Multamäki). https://urn.fi/URN:NBN:fi:oulu-202409135845
Malikireddy, S. K. R., Algubelli, B., & Tadanki, S. (2021). Knowledge graph-driven real-time data engineering for context-aware machine learning pipelines. European Journal of Advances in Engineering and Technology, 8(5), 65-76. https://www.researchgate.net/profile/Sai-Kiran-Reddy-Malikireddy-3/publication/387675951_Knowledge_Graph-Driven_Real-Time_Data_Engineering_for_Context-_Aware_Machine_Learning_Pipelines/links/6777928000aa3770e0d32efb/Knowledge-Graph-Driven-Real-Time-Data-Engineering-for-Context-Aware-Machine-Learning-Pipelines.pdf
Singu, S. K. (2021). Designing scalable data engineering pipelines using Azure and Databricks. ESP Journal of Engineering & Technology Advancements, 1(2), 176-187. 10.56472/25832646/JETA-V1I2P119
Okafor, K. C., Ndinechi, M. C., & Misra, S. (2022). Cyber‐physical network architecture for data stream provisioning in complex ecosystems. Transactions on Emerging Telecommunications Technologies, 33(4), e4407. https://doi.org/10.1002/ett.4407
Bhaskaran, S. V. (2020). Integrating data quality services (dqs) in big data ecosystems: Challenges, best practices, and opportunities for decision-making. Journal of Applied Big Data Analytics, Decision-Making, and Predictive Modelling Systems, 4(11), 1-12. http://polarpublications.com/index.php/JABADP/article/view/4
Yadav, H. (2024). Scalable ETL pipelines for aggregating and manipulating IoT data for customer analytics and machine learning. International Journal of Creative Research In Computer Technology and Design, 6(6), 1-30. https://jrctd.in/index.php/IJRCTD/article/view/45
Muñoz Arcentales, J. A. (2021). Contribution to the advancement of data engineering for smart spaces through data usage control and context-aware systems (Doctoral dissertation, Telecomunicacion). https://doi.org/10.20868/UPM.thesis.69244.
Madaan, N., Ahad, M. A., & Sastry, S. M. (2018). Data integration in IoT ecosystem: Information linkage as a privacy threat. Computer law & security review, 34(1), 125-133. http://library.oapen.org/handle/20.500.12657/22846
Elsaleh, T., Enshaeifar, S., Rezvani, R., Acton, S. T., Janeiko, V., & Bermudez-Edo, M. (2020). IoT-Stream: A lightweight ontology for internet of things data streams and its use with data analytics and event detection services. Sensors, 20(4), 953. https://doi.org/10.3390/s20040953
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Sunil Kumar Mudusu (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
