Newcastle upon Tyne, England, United Kingdom · Hybrid
$400/hr–$450/hr
Apache Spark · Python · AWS · Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark,…
Skills: Apache Spark, PySpark, Python, AWS, Apache Airflow
Newcastle upon Tyne, England, United Kingdom · On-site
$50k–$60k/yr
About Scrumconnect Consulting Scrumconnect Consulting is a multi-award-winning digital consultancy, recognised for delivering impactful and innovative technology solutions across UK government departments. Our work has p…
Skills: Aws, Terraform, Python, Sql, Pyspark
Pybus Recruitment LtdNewcastle upon Tyne, England, United Kingdom · On-site
Job Title: Workshop Production Manager Location: Manufacturing Facility Reports to: Managing Director Direct Reports: Workshop Supervisors, Team Leaders, and up to 35 Industrial Engineers Contract: Full‑time, Permanent R…
Skills: Workshop management, Production management, Industrial engineering, Sheet metal fabrication, Vehicle assembly
London, England, United Kingdom · On-site
As our Regional Operations Manager, you’ll be responsible for leading, coordinating, and delivering commissioning and technical operations across multiple projects within their assigned region. You will ensure that all t…
Skills: Data centre commissioning, Technical leadership, Project management, Electrical engineering, Mechanical engineering
Newcastle upon Tyne, England, United Kingdom • Hybrid
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
Design, build, and maintain scalable cloud data pipelines using Apache Spark and PySpark within an AWS environment. Manage workflow orchestration via Apache Airflow and provision infrastructure using Terraform.
Must hold active SC clearance and have 5+ years of experience in data engineering. Proficiency in Python, SQL, and AWS services is required, along with experience in dimensional data modelling.
Apache Spark · Python · AWS · Cloud Data Pipelines
A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets in a modern, cloud-native environment.
About Scrumconnect
Scrumconnect is a leading UK technology consultancy delivering digital transformation across public and private sectors, contributing to over 20% of the UK’s major citizen-facing public services. We specialise in cloud engineering, data platforms, and agile delivery, helping clients build scalable, secure, and user-centred digital solutions that create real impact.
Active SC clearance is a mandatory, non-negotiable requirement. Candidates must hold current, in-date Security Check (SC) clearance at the time of application. Sponsorship is not available. Applications without active SC clearance will not be considered.
Working arrangement:
This role is hybrid. Candidates must be willing and able to travel to the London office three days per week. Remaining days may be worked remotely from anywhere in the UK.
About the role
You will work as a Data Engineer on a complex, cloud-based data programme — designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable, well-governed data assets to downstream users.
You will apply strong data analysis skills to identify root causes of data issues, work with dimensional data models and slowly changing dimensions, and implement infrastructure as code using Terraform. Familiarity with DWP engineering best practices and the ability to translate customer expectations into applied technical functionality are key to success in this role.
Key responsibilities
Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure.
Workflow orchestration
Configure and manage Apache Airflow DAGs for task orchestration, ensuring reliable scheduling, monitoring, and execution of data processing workflows.
Root cause analysis
Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues — including reviewing EMR output logs and CloudWatch metrics.
Data modelling
Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured, analytically trusted data assets.
Infrastructure as code
Provision and manage cloud infrastructure using Terraform. Containerise solutions using Docker and manage deployments through GitLab CI/CD pipelines and release tagging.
Security & encryption
Apply understanding of both server-side and client-side encryption patterns within AWS. Work within IAM policies and data governance standards appropriate to a regulated government environment.
Technical skills required
Data processing & orchestration
AWS services
Infrastructure, DevOps & delivery
Technology stack at a glance
PythonPySparkSQLApache SparkApache AirflowJupyter NotebooksDimensional modelling / SCDAWS EMRAmazon AthenaAWS S3AWS IAMAWS CloudWatchAWS EC2 / ECRAmazon TextractAmazon ComprehendTerraformDockerGitLab CI/CDGitLab Tags
Tech consultancy developing and delivering outcomes at pace.
Scrumconnect, an acclaimed consultancy firm, sets the gold standard in crafting sophisticated, user-centric software solutions underpinned by data-driven insights. Our robust software solutions have cast a global footprint, influencing over 50M individuals via 64 robust digital offerings. As trusted allies in public sector transformation, we've partnered with 20% of the top 75 Govt Services, yielding savings exceeding £24 million for UK taxpayers. Our mission is simple: transform public services into exceptional digital experiences. Our vision is far-sighted. Through digital channels, we aim to democratize public service access, ensuring convenience, wider accessibility, and cost efficiency. As we move forward, we envision ourselves to be an esteemed SME in UK public sector transformation, collaborating with at least 8 central government departments. In the next 5 years, we aspire to grow by expanding our consulting business. We're dedicated to achieving a 99% Customer Net Promoter Score and plan to create 750 new jobs across the UK. Our core values of openness, integrity, excellence, learning, and unwavering commitment guide us in every endeavor. As a testament to our work, we have bagged prestigious accolades- Special Award - Organisational Exit from Pandemic, 'Digital Transformation Project of the Year' at the UKIT Awards and the 'Digital, Data and Technology Award' at the Civil Service Awards. Scrumconnect was born out of an identified gap in the digital transformation landscape. Traditional consultancies often overlooked key areas like design thinking, user-centric designing, customer experience, and product innovation. To disrupt this paradigm, we developed our unique model. We handpick high-performing teams for each project, streamlining talent acquisition and retention to deliver value swiftly and efficiently. Our success lies in this novel approach - bringing together the finest minds Join us at Scrumconnect, and be part of the revolution
Offices: 56 Kingston Road, Staines-upon-Thames, London, England TW18 4NL, GB · Newcastle, GB · Manchester, GB · Bengaluru, Karnataka 560078, IN
There are currently 264 open engineering positions in Newcastle upon Tyne, United Kingdom listed on Clera. New openings are added daily as companies post roles.
Companies currently hiring include WSP, Turner & Townsend, Scrumconnect Consulting, AECOM, Assystem, among others. Browse the listings above to see every active employer.
Yes — 167 of the 264 open engineering positions offer remote or hybrid work (20 remote, 147 hybrid).
Each listing links directly to the employer's application page. Apply early — fresh listings get the most recruiter attention in the first two weeks.
