中圖分類號：TP39文獻標識碼：ADOI:10.19358/j.issn.2097-1788.2025.09.006
引用格式：許文靜，安寧，于重,等. 大規(guī)模異構數(shù)據(jù)遷移的自適應清洗與智能轉換框架［J］.網絡安全與數(shù)據(jù)治理，2025，44（9）：35-45.

Adaptive cleaning and intelligent transformation framework for large-scale heterogeneous data migration

Abstract： In the context of digital transformation, migrating from traditional centralized databases to distributed architectures presents core challenges including semantic conflicts in heterogeneous data models, business continuity requirements, and inefficient manual conversion processes. This paper proposes an intelligent transformation framework named AUTO-MIG, whose core innovations lie in an intelligent decision-making mechanism that deeply mines intrinsic data relationships and a high-performance execution engine adapted to large-scale heterogeneous environments. AUTO-MIG innovatively employs graph neural networks(GNN) to automatically uncover complex inter-table relationships embedded within database schemas and combines a multi-objective optimization model to intelligently determine the optimal storage strategy, thereby enhancing the automation of cross-model data transformation. Furthermore, the framework incorporates a uniquely designed dual-mode log capture mechanism and a stream-batch hybrid cleaning pipeline to achieve low-latency, highly reliable synchronization and cleaning of massive historical data and high-frequency real-time changes. The framework has been successfully deployed on containerized platforms and validated through a large-scale educational training system data migration case. The results demonstrate that the GNN-driven relationship discovery significantly improves complex query performance, while the dual-mode collaborative execution engine considerably reduces total migration time and optimizes resource utilization efficiency. This provides reliable technical support and a practical pathway for enterprise digital transformation.

Key words : heterogeneous data；data migration；intelligent transformation framework；metadata awareness；graph neural network