Lead Data Engineer - Technology Data Product

Location:

Job Summary
We are seeking a Lead Data Engineer to support the design, implementation, maintenance, and lifecycle of enterprise data products starting with our DevOps Data Product transforming technology, operational, and business data into trusted, governed, and reusable insights.

In this hands-on technical leadership role, you will partner with Data Product Owners, data modelers, data architects, platform engineers, technology teams, and business stakeholders to develop scalable data pipelines, reusable data models, cloud-native services, and deployment automation on Google Cloud Platform.

This is a strong fit for someone who enjoys solving complex data and platform problems, improving data quality and reliability, reducing friction in the data delivery and pipeline lifecycle, and building standardized solutions that encourage broad adoption. The role offers the opportunity to influence data engineering standards, strengthen platform capabilities, and help shape a delivery ecosystem built for speed, quality, governance, and resilience.

Essential Functions

• Data Pipeline Engineering: Design, build, and support production batch and incremental pipelines using Python, SQL, and Cloud Composer to collect and process data and telemetry from enterprise tools and source systems.
• BigQuery Engineering & Optimization: Implement and maintain BigQuery datasets, tables, views, and transformation workloads. Optimize partitioning, clustering, query performance, access, storage, and compute costs.
• Data Transformation & Testing: Write and deploy automated transformation workflows and data quality checks using Dataform.
• CI/CD & Infrastructure as Code: Create and maintain repeatable Terraform configurations and GitLab CI/CD pipelines to automate testing, credential handling, and environment deployments across development, test, and production.
• System Integration & Security: Build secure integrations with enterprise APIs and source systems using appropriate authentication and credential management, incorporating retry, error handling, and rate-limiting patterns.
• Data Quality & Reliability: Configure pipeline monitoring, alerting, schema validation, and reconciliation jobs.
• Data Product Delivery: Partner with the Data Product Owner, data modelers, architects, and platform teams to translate product requirements, metric definitions, data-quality expectations, and governance requirements into reliable technical solutions.
• Documentation & Governance: Maintain technical documentation, lineage, data contracts, implementation guides, and evidence needed to support governance and audit requirements.
• Technical Mentorship & Standards: Conduct code reviews, establish reusable coding templates, and mentor team engineers on best practices for data engineering and version control.

Required Technical Skills

• SQL & BigQuery: Advanced SQL proficiency with hands-on BigQuery experience managing datasets, partitioning, clustering, views, schema updates, access controls, and query optimization.
• Data Pipeline & Orchestration: Proven experience designing and running production ELT/ETL pipelines, state management, incremental loading, and workflow scheduling with Cloud Composer or Apache Airflow.
• Programming & Scripting: Strong hands-on programming skills in Python and Bash shell scripting for container and pipeline workflows.
• API Ingestion: Practical experience extracting data from enterprise REST or GraphQL APIs, handling pagination, rate limits, OAuth flows, and integration with secret management systems.
• Dataform / SQL Transformation: Hands-on experience developing modular data transformations, dependencies, and automated test assertions using Dataform or comparable SQL-based frameworks.
• Terraform & GCP Infrastructure: Experience writing reusable Terraform modules to provision GCP data and IAM resources, including BigQuery, Cloud Storage, service accounts, Cloud Run, and Cloud Composer.
• GitLab CI/CD: Practical experience maintaining GitLab CI/CD pipelines, including environment stages, automated testing, container builds, and branch management.
• Data Quality & Production Reliability: Experience implementing automated data-quality controls, reconciliation, observability, alerting, failure recovery, and production support.
• Security & Governance: Experience applying access controls, secure credential handling, lineage, auditability, and data-governance requirements within a regulated or highly governed environment.

Preferred Technical Skills

• Containerization: Experience packaging and running lightweight workloads using Docker, Cloud Build, and Cloud Run.
• Developer Telemetry: Prior experience building reporting data models around software engineering tools, such as Jira, ServiceNow, GitLab.
• Streaming & Event Ingestion: Familiarity with event-driven data ingestion using Pub/Sub or Apache Beam / Google Cloud Dataflow.
• Analytics & AI Readiness: Familiarity with engineering, operational, productivity, quality, cost, risk, or AI-related data.

Work Experience & Education

• Education: Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related technical discipline, or equivalent practical experience.
• Relevant Experience: 5+ years of hands-on data engineering experience building production cloud data pipelines and data warehouse solutions.
• DevOps / CI/CD Experience: 2+ years of practical experience integrating automated testing, infrastructure as code, and CI/CD deployment workflows.

 Core Competencies

• All KeyBank employees are expected to demonstrate Key’s Values and abide by Key’s Code of Conduct.

Physical Demands

• General Office - Prolonged sitting, ability to communicate face to face in person or on the phone with teammates and clients, frequent use of PC/laptop, occasional lifting/pushing/pulling of backpacks, computer bags up to 10 lbs.

Work Location Category

• Hybrid (2+ days)

COMPENSATION AND BENEFITS

Please click here for a list of benefits for which this position is eligible.

Key has implemented an approach to employee workspaces which prioritizes in-office presence, while providing flexible options in circumstances where roles can be performed effectively in a mobile environment.

Qualified individuals with disabilities or disabled veterans who are unable or limited in their ability to apply on this site may request reasonable accommodations by emailing HR_Compliance@keybank.com.