Data Engineer - Chennai / Bengaluru
About this role
Must Needed Skills: Apache Airflow, Python, PySpark ,Hive(Trino), SQL and Snowflake
- Develop and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python, PySpark ,Hive(Trino) and Snowflake to ensure the efficient and reliable delivery of data.
- Design and implement custom connectors to facilitate the ingestion of diverse data sources into our platform, including structured and unstructured data from various document formats .
- Collaborate closely with cross-functional teams to gather requirements, understand data needs, and translate them into technical solutions.
- Design and implement data CI/CD pipelines to enable automated and efficient data integration, transformation, and deployment processes.
- Monitor and troubleshoot data pipelines, proactively identifying and resolving issues related to data ingestion, transformation, and loading.
- Conduct data validation and testing to ensure the accuracy, consistency, and compliance of data.
- Stay up-to-date with emerging technologies and best practices in data engineering.
- Document data workflows, processes, and technical specifications to facilitate knowledge sharing and ensure data governance.
Company at a glance
Photon, a global leader in AI and digital solutions, helps clients accelerate AI adoption and embrace DigitalHyper-expansion® to ‘make tomorrow happen today’. We work with 40% of the Fortune 100, enabling them to stay agile and future-ready in an era of converging digital and AI boundaries. Powering billions of touchpoints a day, Photon combines AI management, digital innovation, product design thinking, and engineering excellence to drive lasting transformation for F500 clients. We employ several thousand people across dozens of countries. Learn more at www.photon.com
Tired of cold applications?
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
Know someone who'd be great for this?