Area(s) of responsibility JD, for a Databricks developer` We are looking for an experienced Databricks Developer to design, develop, and maintain scalable data pipelines and analytics solutions using Databricks. The idea…
Area(s) of responsibility • Perform functional, SIT, regression, and UAT testing across D365 and integrated web applications • Analyze business requirements, workflows, and system integrations to design effective test sc…
Skills: D365 Testing, Web Applications Testing, Test Design, Defect Management, Integration Testing
Area(s) of responsibility Data Engineer (Migration Consultant) Role Summary We are seeking a Senior Data Engineer with strong enterprise data migration experience to own the end-to-end data pipeline — from source extract…
Skills: Data Migration, ERP Data Structures, SQL, Python, PySpark
Area(s) of responsibility Cloud Consultant Role Summary We are seeking an experienced Cloud Consultant to provision, configure, and operationalize the cloud infrastructure required for a data migration engagement. The ro…
Area(s) of responsibility We are hiring hands-on Developers and Engineers to design, build, and deploy production-grade Agentic AI solutions on Aisera and similar platforms (ServiceNow AI and open agent frameworks) acros…
Skills: Agentic AI, LLM Frameworks, Python, RAG Pipelines, Multi-agent Systems
Area(s) of responsibility Role Summary A DevOps Consultant with 3+ years of Azure DevOps experience, strong exposure to SAP program delivery, and hands-on expertise in Tricentis (Tosca) integration with Azure pipelines. …
Skills: Azure DevOps, SAP Program Delivery, Tricentis Tosca, CI/CD Pipelines, S/4HANA
Area(s) of responsibility Required Qualifications Ability to perform customer demonstration , lead use-case workshops 9+ years of hands-on experience in software engineering, with at least 1+ years in AI/GenAI engineerin…
Area(s) of responsibility Windchill Integration SME The expectation is this person should have 9-15 years of experience with Windchill development and Integration .As Subject Matter Experts (SMEs) in Windchill Integratio…
Area(s) of responsibility MDM OCUST Basic Skill Set : Core Java Selenium Basic knowledge of bBot Framework We are seeking a skilled Automation Engineer with expertise in Java and Selenium to join our team. The ideal cand…
Area(s) of responsibility About Birlasoft: Birlasoft, a global leader at the forefront of Cloud, AI, and Digital technologies, seamlessly blends domain expertise with enterprise solutions. The company’s consultative and …
Area(s) of responsibility Job Title: Senior Business Analyst Location: Offshore – India Experience: 8–12 Years Employment Type: Full-time Job Summary We are seeking a Senior Business Analyst with deep Life Sciences Regul…
Skills: Business Analysis, Life Sciences, Regulatory Affairs, Veeva Vault RIM, Snowflake
Area(s) of responsibility Full stack (JAVA+Reactjs) developerJD S.No Reactjs JD 1.Strong proficiency in JavaScript, including DOM manipulation and the JavaScript object model Strong proficiency in React JS-Component arch…
Area(s) of responsibility . Offshore Project Team Lead Experience: 12–15+ years managing Oracle CX/Cloud projects Lead offshore delivery activities for Oracle CX implementation programs. • Manage project plans, resources…
Area(s) of responsibility Role & responsibilities: Strong experience in leading multiple Alliances and Partnerships for key SIs, particularly Microsoft, AWS. Microsoft operations lead – Ability to manage Partner Center p…
Skills: Alliance Management, Microsoft Partner Center, AWS, Cloud Computing, AI
Area(s) of responsibility OIC Consultant JD Total 7-10 Yrs of IT Experience in Oracle Integration Cloud (OIC) with Oracle SOA Suite as prior Integration Experience. Experience in design and development of Interfaces with…
Area(s) of responsibility API Automation Tester JD • The key purpose of this role is to gather requirement, design test automation scripts, execute, debug, fix, share results and collaborate with existing resources on th…
Skills: API Automation, Selenium, Java, Cucumber, BDD
Area(s) of responsibility About Birlasoft: Birlasoft, a global leader at the forefront of Cloud, AI, and Digital technologies, seamlessly blends domain expertise with enterprise solutions. The company’s consultative and …
Area(s) of responsibility Vendor Resource - Job Description Technology: JDE Technical Lead Essential Duties & Responsibilities - 1. Lead the technical team and responsible for the deliverables,review, meetings etc 2. Han…
Area(s) of responsibility About Birlasoft: Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and des…
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
Contract
Posted 6d ago
Apply by Oct 30
~40 hrs/week
Responsibilities
Design, develop, and maintain scalable ETL/ELT data pipelines and analytics solutions using Databricks and Apache Spark. Integrate data from various sources and optimize storage using Delta Lake and Z-Ordering.
Requirements
Requires strong expertise in Python, SQL, and PySpark with a focus on cloud-based data engineering. Experience with Delta Lake and version control is mandatory, while knowledge of Azure, AWS, or GCP is preferred.
Full job description
Area(s) of responsibility
JD, for a Databricks developer` We are looking for an experienced Databricks Developer to design, develop, and maintain scalable data pipelines and analytics solutions using Databricks. The ideal candidate should have strong expertise in Apache Spark, Python, SQL, and cloud-based data engineering technologies to support enterprise data integration and analytics initiatives. Key Responsibilities • Design, develop, and maintain ETL/ELT pipelines using Databricks. • Develop scalable data processing frameworks using Apache Spark (PySpark). • Build and optimize batch and streaming data pipelines. • Integrate data from multiple sources such as databases, APIs, cloud storage, and enterprise applications. • Implement data transformation, cleansing, and validation processes. • Develop Delta Lake tables and optimize data storage using partitioning and Z-Ordering. • Create reusable notebooks, workflows, and jobs in Databricks. • Monitor and troubleshoot pipeline failures and performance issues. • Collaborate with business analysts, architects, and stakeholders to understand data requirements. • Follow coding standards, version control, and CI/CD best practices. • Participate in code reviews and technical design discussions. Technical Skills Mandatory Skills • Strong experience with Databricks • Expertise in Apache Spark / PySpark • Proficiency in Python • Strong SQL skills • Experience with ETL/ELT development • Experience with Delta Lake • Knowledge of data warehousing concepts • Experience with Git and version control Preferred Skills • Experience with cloud platforms: o Microsoft Azure – Azure Databricks, ADLS Gen2, Azure Data Factory o Amazon Web Services – S3, Glue, EMR o Google Cloud – BigQuery, Cloud Storage • Experience with streaming technologies such as Spark Structured Streaming. • Knowledge of orchestration tools such as Airflow. • Understanding of data governance and security concepts. Nice to Have • Experience with Medallion Architecture (Bronze, Silver, Gold layers). • Knowledge of Unity Catalog. • Experience in performance tuning and cost optimization in Databricks. • Exposure to DevOps and CI/CD pipelines. • Certification in Databricks or cloud platforms. Key Competencies • Data Modeling • ETL/ELT Development • Performance Optimization • Problem Solving • Team Collaboration • Agile/Scrum Methodology • Documentation and Technical Design
Related keywords
DatabricksApache SparkPySparkPythonSQLETLELTDelta LakeData WarehousingGitAzureADLS Gen2Azure Data FactoryAWSS3Glue
Navigating Change. Powering Progress. | Reimagining the Future with Birlasoft
Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses with unmatched efficiency and innovation. As part of the CKA Birla Group, a multibillion-dollar enterprise, we boast a 12,000+ professional team committed to upholding the Group's 170-year legacy. Our core values prioritize Diversity, Equity, and Inclusion (DEI) initiatives, along with Corporate Social Responsibility (CSR) activities, demonstrating our dedication to building inclusive and sustainable communities. Join us in shaping a future where technology seamlessly aligns with purpose.
For further information, visit www.birlasoft.com
Offices: 35 & 36, Rajiv Gandhi Infotech Park, Phase - I, Hinjawadi MIDC, Pune, Maharashtra 411057, IN · The Assotech Business Cresterra Tower 3, Plot No. 22, Noida-Greater Noida Expressway, Sector 135, Noida, UP 201306, IN · 4th Floor, 53-54 Grosvenor Street, London, London W1K 3HU, GB · Building A, Plot No. EL-207/1, MIDC,, TTC Industrial Area, Mahape Road, Mahape,, Navi Mumbai, Maharashtra 400710, IN · Tower 2.1, Level 15 & 16, TSI Business Park, Waverock Project, APIIC IT/ITES SEZ,, Nanakramguda Village, Serillingampally Mandal, Ranga Reddy District, Hyderabad, Telangana 500008, IN
Enterprise Application ServicesApplication Development and MigrationQuality Assurance and Testing ServicesERP and CRM ServicesMobility ServicesAdvanced Analytics ServicesCloud ServicesContent Management SolutionsIntegrated Management ServicesTesting as a Service
Navigating Change. Powering Progress. | Reimagining the Future with Birlasoft
Birlasoft, a powerhouse where domain expertise, enterprise solutions, and digital technologies converge to redefine business processes. We take pride in our consultative and design thinking approach, driving societal progress by enabling our customers to run businesses with unmatched efficiency and innovation. As part of the CKA Birla Group, a multibillion-dollar enterprise, we boast a 12,000+ professional team committed to upholding the Group's 170-year legacy. Our core values prioritize Diversity, Equity, and Inclusion (DEI) initiatives, along with Corporate Social Responsibility (CSR) activities, demonstrating our dedication to building inclusive and sustainable communities. Join us in shaping a future where technology seamlessly aligns with purpose.
For further information, visit www.birlasoft.com
Offices: 35 & 36, Rajiv Gandhi Infotech Park, Phase - I, Hinjawadi MIDC, Pune, Maharashtra 411057, IN · The Assotech Business Cresterra Tower 3, Plot No. 22, Noida-Greater Noida Expressway, Sector 135, Noida, UP 201306, IN · 4th Floor, 53-54 Grosvenor Street, London, London W1K 3HU, GB · Building A, Plot No. EL-207/1, MIDC,, TTC Industrial Area, Mahape Road, Mahape,, Navi Mumbai, Maharashtra 400710, IN · Tower 2.1, Level 15 & 16, TSI Business Park, Waverock Project, APIIC IT/ITES SEZ,, Nanakramguda Village, Serillingampally Mandal, Ranga Reddy District, Hyderabad, Telangana 500008, IN
Enterprise Application ServicesApplication Development and MigrationQuality Assurance and Testing ServicesERP and CRM ServicesMobility ServicesAdvanced Analytics ServicesCloud ServicesContent Management SolutionsIntegrated Management ServicesTesting as a Service