About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
Senior Software Development Engineer \n Primary Skills DB2 Job requirements AS400 Developer Experience: 3–5 years Department: Digital Engineering Employment Type: Full-time Role Overview We are looking for an experienced…
About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
Skills: BI Reporting, AI Analytics, Pharma/Life Sciences Domain, SQL, Data Warehousing
AI Architect, Principal Frontline AI Engineer - Healthcare
Jersey City, New Jersey, United States · Remote OK
$200k–$300k/yr
Senior+
About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
Skills: Claude API, Full-stack AI Engineering, RAG, Agentic Workflows, Healthcare Operations
About Brillio: Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative…
Skills: Snowflake, Dbt, Microsoft Azure, Power BI, SQL
Senior SRE Lead Role Overview We are seeking a highly experienced Senior SRE Lead to lead reliability engineering and observability initiatives for critical platforms supporting GM Financials’ ecosystem, with a primary f…
Skills: Site Reliability Engineering, Microsoft Azure, Salesforce, Observability, Incident Management
Senior Observability / Monitoring Engineer Role Overview We are looking for a Senior Observability / Monitoring Engineer to design, implement, and optimize observability solutions for large-scale enterprise platforms. Th…
BI Architect \n Primary Skills Power BI SQL Fabric AI/Copilot Azure Stack Specialization Other Vizulization tool experience Job requirements Sr Power BI Architect Role Overview We are looking for experienced BI Architect…
Skills: Power BI, SQL, Microsoft Fabric, AI/Copilot, Azure Stack
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
$125k–$130k/yr
Full-time
Posted 50d ago
~40 hrs/week
Responsibilities
Design and maintain scalable ETL/ELT pipelines using PySpark and Databricks within a medallion architecture. Manage cloud infrastructure via Terraform and implement CI/CD pipelines for automated deployment and testing.
Requirements
Requires strong proficiency in Python, PySpark, and advanced SQL, with hands-on experience in Databricks and Terraform. Candidates must have experience with Azure or AWS cloud environments and a solid understanding of big data and lakehouse principles.
Full job description
About Brillio:
Brillio is one of the fastest growing digital technology service providers and a partner of choice for many Fortune 1000 companies seeking to turn disruption into a competitive advantage through innovative digital adoption. Brillio, renowned for its world-class professionals, referred to as "Brillians", distinguishes itself through their capacity to seamlessly integrate cutting-edge digital and design thinking skills with an unwavering dedication to client satisfaction.
Brillio takes pride in its status as an employer of choice, consistently attracting the most exceptional and talented individuals due to its unwavering emphasis on contemporary, groundbreaking technologies, and exclusive digital projects. Brillio's relentless commitment to providing an exceptional experience to its Brillians and nurturing their full potential consistently garners them the Great Place to Work® certification year after year.
Senior Lead Data Engineer
\nJob requirements
Key Responsibilities
Design, develop, and maintain ETL/ELT pipelines using PySpark and Python in Databricks (including notebooks, jobs, Delta Lake tables, and Unity Catalog for governance).
Implement medallion architecture (bronze/silver/gold layers) and optimize Spark jobs for performance, cost, scalability, and reliability (handling partitioning, skew, caching, adaptive query execution, etc.).
Write efficient SQL queries for data transformation, validation, and analytics within Databricks.
Provision and manage cloud infrastructure using Terraform (IaC) for Databricks workspaces, clusters, jobs, storage (ADLS/S3), networking, IAM roles/permissions, and related resources on Azure and/or AWS.
Implement and maintain CI/CD pipelines using Jenkins, GitHub (Actions/Repositories), and branching strategies for automated testing, deployment of notebooks, jobs, Delta Live Tables, and Terraform configurations.
Integrate data from diverse sources (databases, APIs, streaming, files) into cloud storage and processing layers.
Ensure data quality, lineage, security, and compliance (including Delta Lake ACID transactions, schema evolution, time travel, and access controls).
Monitor pipeline performance, troubleshoot failures, and implement alerting/observability (using Databricks tools, cloud monitoring services, or third-party solutions).
Optimize cloud costs through auto-scaling clusters, spot instances, job scheduling, and efficient resource usage.
Collaborate in agile teams, participate in code reviews, and contribute to best practices for data engineering.
Required Skills & Experience
Strong proficiency in Python and PySpark for distributed data processing and ETL.
Advanced SQL skills with experience in complex querying, window functions, and optimization.
Hands-on experience with Databricks (clusters, notebooks, Delta Lake, Unity Catalog, Delta Live Tables, workflows/jobs).
Proficiency in Terraform for infrastructure provisioning and management (Databricks resources, cloud storage, IAM, networking).
Experience with GitHub for version control and collaboration (branching, pull requests, code reviews).
Solid knowledge of CI/CD practices and tools, particularly Jenkins (pipelines, plugins for Databricks/GitHub/Terraform).
Working experience on Azure (Data Lake, Data Factory, Synapse, Key Vault, etc.) and/or AWS (S3, Glue, EMR, IAM, Lambda, etc.).
Understanding of big data concepts, data modeling (star/snowflake, dimensional), and lakehouse principles.
Familiarity with performance tuning in Spark/Databricks environments.
Brillio is an equal opportunity employer to all, regardless of age, ancestry, colour, disability (mental and physical), exercising the right to family care and medical leave, gender, gender expression, gender identity, genetic information, marital status, medical condition, military or veteran status, national origin, political affiliation, race, religious creed, sex (includes pregnancy, childbirth, breastfeeding, and related medical conditions), and sexual orientation.
#LI-AY1
\n
$125,000 - $130,000 a year
\n
Know what it’s like to work and grow at Brillio: Click here
Brillio is The Enterprise AI Accelerator helping Fortune 1000 companies move from AI ambition to scaled impact, faster. Powered by our AI accelerator platform - Agentic Data and Application Management (ADAM), Brillio is one of the fastest-growing digital technology service providers, delivering transformation across five core workstreams: business-led transformation, customer experience transformation, AI and data engineering, digital engineering, and infrastructure engineering.
With 14 delivery locations across North America, Europe, and Asia and a team of over 6,000 customer-obsessed professionals, Brillio combines deep industry expertise, modern engineering, and accelerators to deliver measurable outcomes. Brillio accelerates AI by industrializing it across every layer of the business so companies don't just use AI, they operate by it, at reliable enterprise scale.
Offices: 2999 Olympus Blvd, Suite 275 Coppell, Dallas, Texas 75019, US · 399 Thornall Street, 1st Floor, Edison, New Jersey 08837, US · 6111 Bollinger Canyon Rd, Suite #100, San Ramon, California 94583, US · No 8/2, Dr.M.H Marigowda Road, 4th Floor of Bren Optimus, Bangalore, Karnataka 560029, IN · Baner Pashan Link Road, 9th Floor, The Kode, Pune, Maharashtra 411027, IN
Digital TransformationCustomer ExperienceProduct and Platform EngineeringDigital InfrastructureArtificial IntelligenceCloud ComputingAgentic AIDigital EngineeringHealthcareLife Sciences
Brillio is The Enterprise AI Accelerator helping Fortune 1000 companies move from AI ambition to scaled impact, faster. Powered by our AI accelerator platform - Agentic Data and Application Management (ADAM), Brillio is one of the fastest-growing digital technology service providers, delivering transformation across five core workstreams: business-led transformation, customer experience transformation, AI and data engineering, digital engineering, and infrastructure engineering.
With 14 delivery locations across North America, Europe, and Asia and a team of over 6,000 customer-obsessed professionals, Brillio combines deep industry expertise, modern engineering, and accelerators to deliver measurable outcomes. Brillio accelerates AI by industrializing it across every layer of the business so companies don't just use AI, they operate by it, at reliable enterprise scale.
Offices: 2999 Olympus Blvd, Suite 275 Coppell, Dallas, Texas 75019, US · 399 Thornall Street, 1st Floor, Edison, New Jersey 08837, US · 6111 Bollinger Canyon Rd, Suite #100, San Ramon, California 94583, US · No 8/2, Dr.M.H Marigowda Road, 4th Floor of Bren Optimus, Bangalore, Karnataka 560029, IN · Baner Pashan Link Road, 9th Floor, The Kode, Pune, Maharashtra 411027, IN
Digital TransformationCustomer ExperienceProduct and Platform EngineeringDigital InfrastructureArtificial IntelligenceCloud ComputingAgentic AIDigital EngineeringHealthcareLife Sciences