Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things

Job Title: Senior Data Engineer

Location: Bangalore-Manyata Tech Park (Hybrid)

 Business & Team: Retail Technology
 

Impact & contribution

• Contribute to internal online discussion around engineering, delivery and technology e.g. blog posts, knowledge base articles.
• Deliver high quality, sustainable solutions to meet business requirements, leveraging approved delivery frameworks and by applying industry best practice.
• Develop reusable frameworks and tooling to accelerate data engineering maturity.
• Develop strategic data programs and data products.
• Identify and drive opportunities for continuous improvement within the team and in delivery of products.
• Lead the design and development of highly reliable and scalable data pipelines and data platforms with comprehensive test coverage.
• Participate in domain planning, providing guidance to ensure that technical deliveries are aligned to engineering direction and strategy.
• Partner with teams to engineer improvements to the scaling and reliability of our services.
• Provide data engineering practice consultancy across platforms, products and delivery teams.
• Provide thought leadership, mentorship and up-skilling to other engineering teams.
• Take responsibility for nominated technical assets related to areas of expertise including roadmaps and technical direction.
• Actively participate in On-Call rosters from time to time to own resolution of any critical issues, ensuring the seamless operation of our systems, products and platforms.
• Architect and implement scalable, secure, and high-performance data pipelines using technologies such as Spark, Python, SQL, and AWS 
• Lead the development and optimisation of ETL/ELT processes for structured and unstructured data sources 
• Collaborate with data scientists, analysts, and product teams to understand data requirements and translate them into technical solutions
• Ensure data quality, lineage, and governance across all stages of the data lifecycle.
• Drive adoption of engineering best practices including CI/CD, DevSecOps, and observability tooling 
• Provide technical leadership and mentorship to junior engineers, fostering a culture of continuous learning and improvement
• Participate in hiring, onboarding, and bar-raiser interviews for senior and staff-level engineering roles 
• Contribute to the development of semantic data models and data product lifecycle frameworks

Roles & Responsibilities:

• Participate in on-call rotations to provide 24/7 support for critical systems
• Maintain and enhance the productivity of data delivery and its processes.
• Build strategy to migrate ETL Load from legacy systems to cloud.
• Manage the hyper care period
• Support platform engineer to improve platform resilience, reduce incident rates, and optimize solution performance as well as reduce operations cost

Essential Skills:

• 10+ years of experience in data engineering, with at least 2 years in a staff or lead role
• Proficiency in distributed computing frameworks (e.g., Spark, Hadoop), cloud platforms (AWS preferred), and data orchestration tools.
• Strong programming skills in Python, Scala, or Java.
• Experience with data modelling, data warehousing, and advanced storage technologies (e.g., NoSQL, HDFS, Graph Stores)
• Experience in AWS cloud (Certification Preferred)
• Familiarity with observability tools like CloudWatch, PagerDuty, and Tableau
• Familiarity with implementation of Amazon RDS services
• Excellent communication and stakeholder management skills.
• Sound experience in building data pipelines & ETL tools - AWS Data Stack (Ab Initio is good to have)
• Experience in designing and developing ETL test scripts for structured and unstructured data
• Sound skills testing solutions with large data volumes and complex data transformation rules
• Sound Leadership and behavioural skills including Active Listening, Constructive Challenge, Problem Solving and Risk management
• Sound in AI and Data Literacy with an experience and ability to use AI Tools into development lifecycle

Education Qualifications

• Bachelor's or master’s degree in computer science, Engineering, or related field