E-commerce Data Migration & Validation System
A Python-based ETL and validation system built to migrate interconnected operational data from a legacy platform into a new e-commerce website.
Contribution
What I built
I designed and developed a migration pipeline that transforms interconnected JSON datasets into structured, import-ready CSV files for products, users, orders, transactions, designers, and product views. I implemented configurable column mapping, data normalization, duplicate handling, product and variant SKU generation, status and payment-method mapping, ID reconciliation, currency conversion, and preservation of translated product content.
Approach
Process
The pipeline resolves relationships across products, customers, orders, transactions, designers, shipments, payments, and product-view data before applying entity-specific business rules. Timestamped and versioned outputs create a traceable history of each run. Multi-stage validation then compares the original JSON files with both the generated CSVs and post-import exports, reporting missing records, unmatched IDs, incorrect values, invalid relationships, SKU issues, seller sales totals, and migrated product specifications.
Current result
Outcome
The automated workflow reduced manual data preparation and exposed inconsistencies before and after import. Configurable templates made the output adaptable without rewriting migration logic, while detailed reports provided confidence that complex business data was transferred accurately into the new e-commerce website.