Company Description Bringing Ingenuity to Life We’re an innovation and transformation consultancy that believes in the power of ingenuity to build a positive-human future in a technology-driven world. Our diverse teams o…
Skills: Google Cloud Platform, BigQuery, Dataflow, Dataproc, Cloud Composer
City of Westminster, England, United Kingdom · Hybrid
Senior+
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Data Protection Law, Privacy Consulting, CCPA, EU GDPR, OneTrust Configuration
Belfast, Northern Ireland, United Kingdom · Hybrid
Senior
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Data Protection Law, Privacy Consulting, CCPA, EU GDPR, OneTrust Configuration
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Data Protection Law, Privacy Consulting, CCPA, EU GDPR, OneTrust Configuration
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Data Protection Law, Privacy Consulting, CCPA, EU GDPR, OneTrust Configuration
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Security Architecture, Information Security, Risk Management, Agile Methodology, Problem Solving
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Security Architecture, Information Security, Risk Management, Agile Methodology, Problem Solving
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Security Architecture, Information Security, Risk Management, Agile Methodology, Problem Solving
Belfast, Northern Ireland, United Kingdom · Hybrid
Senior
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Information Security, Security Architecture, Infrastructure Security, Integration Projects, Problem Solving
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
City of Westminster, England, United Kingdom · Hybrid
Mid level
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Agile Delivery, Project Management, Stakeholder Engagement, Problem Solving, Scoping
Senior Management Consultants with Organisational Agility Expertise
Copenhagen, Capital Region of Denmark, Denmark · On-site
Senior
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Senior Management Consultants with Organisational Agility Expertise
Copenhagen, Capital Region of Denmark, Denmark · On-site
Senior
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Senior Management Consultants with Organisational Agility Expertise
Copenhagen, Capital Region of Denmark, Denmark · On-site
Senior
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Senior Management Consultants with Organisational Agility Expertise
Copenhagen, Capital Region of Denmark, Denmark · On-site
Senior
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Managing Consultant - C&M - Google Cloud (Technical Lead)
London, England, United Kingdom · Hybrid
Senior
Company Description We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our teams of interdisciplinary expert…
Skills: Google Cloud Platform, Cloud Architecture, DevSecOps, Kubernetes Engine, BigQuery
Senior Marketing Manager - Financial Services, Life Sciences, and Public Services
Copenhagen, Capital Region of Denmark, Denmark · On-site
Senior
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Skills: B2B Marketing, Integrated Campaign Delivery, Stakeholder Management, Digital Marketing, Public Relations
Copenhagen, Capital Region of Denmark, Denmark · On-site
Mid level
Company Description About us We believe in the power of ingenuity to build a positive human future. As strategies, technologies, and innovation collide, we create opportunity from complexity. Our diverse teams of experts…
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
Full-time
Private Medical Insurance, Interest Free Season Ticket Loan, 25 Days Annual Leave, Option To Buy 5 Additional Days, Company Pension Scheme, Annual Performance-based Bonus
Posted 27d ago
~40 hrs/week
Responsibilities
Lead the design and delivery of robust data processing solutions and automated pipelines using Google Cloud Platform. Act as a subject matter expert to mentor teams and define end-to-end software development lifecycles and engineering standards.
Requirements
Requires expert-level experience in GCP data tools, batch and streaming frameworks, and the ability to translate business requirements into technical designs. Proficiency in CI/CD pipelines and Agile methodologies is essential.
Full job description
Company Description
Bringing Ingenuity to Life
We’re an innovation and transformation consultancy that believes in the power of ingenuity to build a positive-human future in a technology-driven world. Our diverse teams of experts combine innovative thinking with breakthrough-technologies to progress further, faster.
With a global network of FTSE 100 and Fortune 500 clients, we’ll offer you unrivalled opportunities for growth and the freedom to excel. Combining strategies, technologies and innovation, we turn complexity to opportunity and deliver enduring results, enabling you to build a lasting career.
Isn’t it time you joined us?
Job Description
As a Principal GCP Data Engineer, you'll be a true subject matter expert in using the data processing and management capabilities of Google Cloud to develop data-driven solutions for our clients.
You will typically lead a team or the solution delivery effort, demonstrating technical excellence through leading by example. You could be providing technical support, leading an engineering team or working across multiple teams as a subject matter expert who is critical to the success of a large programme of work.
Your team members will look to you as a trusted expert and will expect you to define the end-to-end software development lifecycle in line with modern best practices. As part of your responsibilities, you will be expected to:
Develop robust data processing jobs using tools such as Google Cloud Dataflow, Dataproc and BigQuery
Design and deliver automated data pipelines that use orchestration tools such as Cloud Composer
Design end-to-end solutions and contribute to architecture discussions beyond data processing
Own the development process for your team, building strong principles and putting robust methods and patterns in place across architecture, scope, code quality and deployments.
Shape team behaviour for writing specifications and acceptance criteria, estimating stories, sprint planning and documentation.
Actively define and evolve PA’s data engineering standards and practices, ensuring we maintain a shared, modern and robust approach.
Lead and influence technical discussions with client stakeholders to achieve the collective buy-in required to be successful
Coach and mentor team members, regardless of seniority, and work with them to build their expertise and understanding.
Qualifications
To be successful in this role, you will need to have:
Experience delivering and deploying production-ready data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc
Experience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam.
Expert understanding of when to use a range of data storage technologies including relational/non-relational, document, row-based/columnar data stores, data warehousing and data lakes.
Expert understanding of data pipeline patterns and approaches such as event-driven architectures, ETL/ELT, stream processing and data visualisation.
Experience working with business owners to translate business requirements into technical specifications and solution designs that satisfies the data requirements of the business.
Experience working with metadata management products such as Cloud Data Catalog and Collibra and Data Governance tools like Dataplex
Experience in developing solutions on GCP using cloud-native principles and patterns.
Experience building data quality alerting and data quarantine solutions to ensure downstream datasets can be trusted.
Experience implementing CI/CD pipelines using techniques including as git code control/branching, automated tests and automated deployments.
Comfortable working in an Agile team using Scrum or Kanban methodologies.
In addition to the above, we would be thrilled if you also had:
Experience of working on migrations of enterprise scale data platforms including Hadoop and traditional data warehouses
An understanding of machine learning model development lifecycle, feature engineering, training and testing
Good understanding or hands-on experience of Kafka
Experience as a DBA or developer on RDBMS such as PostgreSQL, MySQL, Oracle or SQL Server
Experience designing data applications to meet non-functional requirements such as performance and availability
Personal qualities
You are pragmatic and already understand that writing code is only part of what a data engineer does.
You can clearly communicate with both clients and peers, describing technical issues and solutions in both written and meeting/workshop contexts.
You are able to clearly explain technical concepts to non technical audiences at all levels of an organisation.
You are able to influence and persuade senior and specialist client stakeholders, potentially across multiple organisational boundaries without direct authority.
You are a confident problem solver and troubleshooter.
You are confident and generous in sharing your specialist knowledge, ideas and solutions.
You are constantly learning and able to make others better by consciously teaching and unconsciously inspiring.
Additional Information
Benefits package at PA:
Private medical insurance
Interest free season ticket loan
25 days annual leave with the opportunity to buy 5 additional days
Company pension scheme
Annual performance-based bonus
Life and Income protection insurance
Tax efficient benefits (cycle to work, give as you earn, childcare benefits)
Voluntary benefits (Dental, critical illness, spouse/partner life assurance)
PA is committed to building an inclusive and supportive culture where diversity thrives, and all of our people can excel. We believe that greater diversity stimulates innovation, enabling us to fulfil our purpose of ‘Bringing Ingenuity to Life’, supporting the growth of our people, and delivering more enduring results for our clients.
#LI-NF1
Related keywords
Google Cloud PlatformGCPBigQueryDataflowDataprocCloud ComposerPub/SubApache SparkApache BeamETLELTEvent-driven ArchitectureData LakeData WarehousingCloud Data CatalogCollibra
We believe in the power of ingenuity to build a positive human future.
As strategies, technologies, and innovation collide, we create opportunity from complexity.
Our diverse teams of experts combine innovative thinking and breakthrough technologies to progress further, faster. Our clients adapt and transform, and together we achieve enduring results.
We are about 4,000 strategists, innovators, designers, consultants, digital experts, scientists, engineers, and technologists. And we have deep expertise in consumer and manufacturing, defence and security, energy and utilities, financial services, government and public services, health and life sciences, and transport.
Our teams operate globally from offices across the UK, Ireland, US, Nordics, and Netherlands.
PA. Bringing Ingenuity to Life.
Offices: PA Consulting Group, 10 Bressenden Place, London, London SW1E 5DN , GB · Jakobsbergsgatan 17, Stockholm, Stockholm County SE 111 44, SE · Portland Towers, Goteborg Plads 1, Copenhagen, Nordhavn DK-2150, DK · Papendorpseweg 97, Utrecht, Utrecht 3528 BJ, NL · Back Lane, Melbourn, England SG8 6DD, GB
We believe in the power of ingenuity to build a positive human future.
As strategies, technologies, and innovation collide, we create opportunity from complexity.
Our diverse teams of experts combine innovative thinking and breakthrough technologies to progress further, faster. Our clients adapt and transform, and together we achieve enduring results.
We are about 4,000 strategists, innovators, designers, consultants, digital experts, scientists, engineers, and technologists. And we have deep expertise in consumer and manufacturing, defence and security, energy and utilities, financial services, government and public services, health and life sciences, and transport.
Our teams operate globally from offices across the UK, Ireland, US, Nordics, and Netherlands.
PA. Bringing Ingenuity to Life.
Offices: PA Consulting Group, 10 Bressenden Place, London, London SW1E 5DN , GB · Jakobsbergsgatan 17, Stockholm, Stockholm County SE 111 44, SE · Portland Towers, Goteborg Plads 1, Copenhagen, Nordhavn DK-2150, DK · Papendorpseweg 97, Utrecht, Utrecht 3528 BJ, NL · Back Lane, Melbourn, England SG8 6DD, GB