Lead AI Data Engineer

We’re determined to make a difference and are proud to be an insurance company that goes well beyond coverages and policies. Working here means having every opportunity to achieve your goals – and to help others accomplish theirs, too. Join our team as we help shape the future.   

The Enterprise Data Services Department’s Customer Data Ecosystem Operations Team is looking for a skilled Lead AI Data Engineer to join our team. This is an exciting opportunity to join us on our multi-year Cloud Modernization journey. The role focuses on integrating data from multiple systems, curating and transforming it into high-quality data products for actionable insights, using a mix of solutions spanning AI and Cloud tools and technologies. You uphold data integrity, accessibility, and compliance while creating curated Data Products to support diverse analytics needs—including descriptive, diagnostic, predictive, and prescriptive use cases for visualization and advanced data science.

Ideal candidates bring deep expertise in data engineering frameworks and tools (e.g., Spark, Snowflake), proficiency in programming languages (Python, SQL, PL/SQL), and experience with DevOps / DataOps pipelines, cloud platforms (AWS services), and agile methodologies (Scrum, Kanban). Familiarity with data warehousing, streaming technologies, and modern architecture such as Lakehouse and Data Mesh is highly desirable. Strong problem-solving, communication, and collaboration skills are essential, along with a proactive mindset and the ability to thrive in complex, fast-paced environments.

To succeed in this role, you should be a strong critical thinker, technical acumen and be able to derive the root causes of business problems.

This role will have a Hybrid work schedule, with the expectation of working in an office (Columbus, OH, Chicago, IL, Hartford, CT or Charlotte, NC) 3 days a week (Tuesday through Thursday).

Key Responsibilities

• Design, develop, and optimize ETL/ELT pipelines for both structured and unstructured data.

• Mentor junior team members and engage in communities of practice to deliver high-quality data and AI solutions while promoting best practices, standards, and adoption of reusable patterns.

• Partner with architects and stakeholders to influence and implement the vision of the AI and data pipelines while safeguarding the integrity and scalability of the environment.

• Ingest and process large-scale datasets into the Enterprise Data Lake and downstream systems.

• Curate and publish Data Products to support analytics, visualization, and machine learning use cases.

• Collaborate with data analysts, data scientists, and BI teams to build data models and pipelines for research, reporting, and advanced analytics.

• Apply best practices for data modeling, governance, and security across all solutions.

• Partner with cross-functional teams to ensure alignment and delivery of high-value outcomes.

• Monitor and fine-tune data pipelines for performance, scalability, and reliability.

• Automate auditing, balance, reconciliation, and data quality checks to maintain high data integrity.

• Develop self-healing pipelines with robust re-startability mechanisms for resilience.

• Schedule and orchestrate complex, dependent workflows using tools like MWAA, Autosys, or Control-M.

• Leverage CI/CD pipelines to enable automated integration, testing, and deployment processes.

• Lead Proof of Concepts (POCs) and technology evaluations to drive innovation.

• Develop AI-driven systems to improve data capabilities, ensuring compliance with industry best practices.

• Implement efficient Retrieval-Augmented Generation (RAG) architectures and integrate with enterprise data infrastructure.

• Implement data observability practices to proactively monitor data health, lineage, and quality across pipelines, ensuring transparency and trust in data assets.

Required Skills & Experience:

• Bachelor’s or master’s degree in computer science or a related discipline.

• 5+ years of experience in data analysis, transformation, and development, with ideally 2+ years in the insurance or a related industry.

• 3+ years of experience developing and deploying large-scale data and analytics applications on cloud platforms such as AWS and Snowflake.

• Strong proficiency in SQL, Python, and ETL tools such as Informatica IDMC for data integration and transformation  (3+ years).

• Experience designing and optimizing data models for Data Warehouses, Data Marts, and Data Fabric, including dimensional modeling, semantic layers, metadata management, and integration for scalable, governed, and high-performance analytics (3+ years).

• 3+ years of hands-on experience in processing large-scale structured and unstructured data in both batch and near-real-time environments, leveraging distributed computing frameworks and streaming technologies for high-performance data pipelines.

• Strong technical knowledge (AI solution leveraging Cloud and modern solutions)

• 3+ years of experience in Agile methodologies, including Scrum and Kanban frameworks.

• 2+ years of experience in leveraging DevOps pipelines for automated testing and deployment, ensuring continuous integration and delivery of data solutions.

• Proficient in data visualization tools such as Tableau and Power BI, with expertise in creating interactive dashboards, reports, and visual analytics to support data-driven decision-making.

• Ability to analyze source systems, provide business solutions, and translate these solutions into actionable steps.

Nice to Have

• Certifications in AWS Data & Analytics Services, Snowflake, and Informatica IDMC.

• Experience in building and integrating AI agents into data workflows.

• Strong knowledge of data governance, including metadata management and lineage tracking, with hands-on experience collaborating with the Data Governance Office.

Candidate must be authorized to work in the US without company sponsorship. The company will not support the STEM OPT I-983 Training Plan endorsement for this position.

Compensation

The listed annualized base pay range is primarily based on analysis of similar positions in the external market. Actual base pay could vary and may be above or below the listed range based on factors including but not limited to performance, proficiency and demonstration of competencies required for the role. The base pay is just one component of The Hartford’s total compensation package for employees. Other rewards may include short-term or annual bonuses, long-term incentives, and on-the-spot recognition. The annualized base pay range for this role is:

Equal Opportunity Employer/Sex/Race/Color/Veterans/Disability/Sexual Orientation/Gender Identity or Expression/Religion/Age

About Us | Our Culture | What It’s Like to Work Here | Perks & Benefits