Akshansh .

Lead Data Engineer | Scalable ETL Pipelines | Real-Time Streaming | ML-Driven Systems | LLM Ops AWS, Kafka, Python, SQL Results-driven Lead Data and AI Platform with expertise in scalable data architectures, real-time ML pipelines, and processing structured and unstructured data (geospatial, image, video). Proven ability to collaborate cross-functionally, leveraging expertise in Kafka, Spark, and cloud technologies to enhance data-driven decision-making. Eager to engage in cutting-edge projects that foster innovation and excellence in technology. Key Expertise & Achievements Scalable ETL & Data Transformation Built high-throughput ETL pipelines using AWS Glue, PySpark, and Airflow, reducing batch processing times by 40%. Designed metadata-driven ingestion frameworks to handle structured, semi-structured, and unstructured data at scale. Automated data validation, deduplication, and schema evolution, ensuring data quality and accuracy. Real-Time Streaming & Analytics Developed low-latency streaming pipelines using Apache Flink, Kafka Streams, and Kinesis, reducing event processing lag by 50%. Implemented real-time aggregations and transformations using KSQL and Flink SQL, enabling sub-second insights. Optimized Kafka partitioning, compression, and retention strategies, reducing storage costs by 25%. Data Lake Architecture & Governance Built multi-layered data lakes using Delta Lake, Iceberg, and Hudi, ensuring efficient storage and ACID compliance. Implemented data lineage tracking with AWS Glue Catalog and Apache Atlas for better discoverability and governance. Developed fine-grained access control policies using AWS Lake Formation and IAM, ensuring data security and compliance. Machine Learning Data Pipelines Designed feature engineering pipelines for predictive analytics, improving forecasting model accuracy by 20%. Integrated ML workflows with Spark ML and SageMaker, optimizing model training and inference at scale. Developed recommendation systems using graph-based ML, increasing user engagement. Performance Optimization & Cost Reduction Reduced ETL processing time by 30% with distributed computing and optimized SQL transformations. Implemented data partitioning, clustering, and indexing, cutting query execution times by 50%. Reduced compute and storage costs by 20% through efficient resource allocation and query tuning. I am passionate about building future-ready data ecosystems, enabling real-time insights, machine learning, and high-scale analytics to drive business impact.