Data engineer - Seattle WA

Full Job Description

Data engineer who has experience with Databricks and Unity Catalog.

Seattle WA

10 months

Core Skills -

Experience – 10+ years

Azure Data bricks

Azure data piplelines

Azure data factory

Azure data lakes

Real time experience with migration to Unity Catalog- 1-2 years of hands on experience is good to have.

Experience in implementing Unity catalog

Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices.

Good to have Privacera, and Collibra experience

Responsibilities

Design and implement data architecture leveraging technologies such as Databricks, Unity Catalog, Privacera, and Collibra.

Develop, optimize, and manage data pipelines for ETL processes using Databricks, with a focus on data integrity and quality.

Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices.

Ensure data security and compliance with regulations using Databricks and Privacera's features.

Establish a robust data governance strategy, defining standards, metadata management, lineage, and quality practices.

Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups.

Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions.