2,228 Etl Processes jobs in the United States

Manager, Data Engineering

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transformation within Capital One.

What You’ll Do:

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies

• Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems

• Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake

• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

• Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment

• Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance

Basic Qualifications:

• Bachelor’s Degree

• At least 4 years of experience in application development (Internship experience does not apply)

• At least 2 years of experience in big data technologies

• At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)

• At least 2 years of people management experience

Preferred Qualifications:

• 7+ years of experience in application development including Python, SQL, Scala, or Java

• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)

• 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)

• 4+ year experience working on real-time data and streaming applications

• 4+ years of experience with NoSQL implementation (Mongo, Cassandra)

• 4+ years of data warehousing experience (Redshift or Snowflake)

• 4+ years of experience with UNIX/Linux including basic commands and shell scripting

• 2+ years of experience with Agile engineering practices

At this time, Capital One will not sponsor a new applicant for employment authorization, or offer any immigration related support for this position (i.e. H1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN, or another type of work authorization).

The minimum and maximum full-time annual salaries for this role are listed below, by location. Please note that this salary information is solely for candidates hired to perform work within one of these locations, and refers to the amount Capital One is willing to pay at the time of this posting. Salaries for part-time roles will be prorated based upon the agreed upon number of hours to be regularly worked.

Candidates hired to work in other locations will be subject to the pay range associated with that location, and the actual annualized salary amount offered to any candidate at the time of hire will be reflected solely in the candidate’s offer letter.

Capital One offers a comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being. Learn more at theCapital One Careers website. Eligibility varies based on full or part-time status, exempt or non-exempt status, and management level.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at or via email at . All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

For technical support or questions about Capital One's recruiting process, please send an email to

Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.

Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).

Manager, Data Engineering

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative,inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you’ll have the opportunity to be on the forefront of driving a major transformation within Capital One.

What You’ll Do:

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies

• Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems

• Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake

• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

• Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment

• Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance

Basic Qualifications:

• Bachelor’s Degree

• At least 4 years of experience in application development (Internship experience does not apply)

• At least 2 years of experience in big data technologies

• At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)

• At least 2 years of people management experience

Preferred Qualifications:

• 7+ years of experience in application development including Python, SQL, Scala, or Java

• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)

• 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)

• 4+ year experience working on real-time data and streaming applications

• 4+ years of experience with NoSQL implementation (Mongo, Cassandra)

• 4+ years of data warehousing experience (Redshift or Snowflake)

• 4+ years of experience with UNIX/Linux including basic commands and shell scripting

• 2+ years of experience with Agile engineering practices

At this time, Capital One will not sponsor a new applicant for employment authorization, or offer any immigration related support for this position (i.e. H1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN, or another type of work authorization).

The minimum and maximum full-time annual salaries for this role are listed below, by location. Please note that this salary information is solely for candidates hired to perform work within one of these locations, and refers to the amount Capital One is willing to pay at the time of this posting. Salaries for part-time roles will be prorated based upon the agreed upon number of hours to be regularly worked.

Candidates hired to work in other locations will be subject to the pay range associated with that location, and the actual annualized salary amount offered to any candidate at the time of hire will be reflected solely in the candidate’s offer letter.

Capital One offers a comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being. Learn more at theCapital One Careers website. Eligibility varies based on full or part-time status, exempt or non-exempt status, and management level.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at or via email at . All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

For technical support or questions about Capital One's recruiting process, please send an email to

Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.

Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).

Manager, Data Engineering

Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you'll have the opportunity to be on the forefront of driving a major transformation within Capital One.

What You'll Do:

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies

• Work with a team of developers with deep experience in machine learning, distributed microservices, and full stack systems

• Utilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and Snowflake

• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

• Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment

• Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance

Basic Qualifications:

• Bachelor's Degree

• At least 4 years of experience in application development (Internship experience does not apply)

• At least 2 years of experience in big data technologies

• At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)

• At least 2 years of people management experience

Preferred Qualifications:

• 7+ years of experience in application development including Python, SQL, Scala, or Java

• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)

• 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)

• 4+ year experience working on Real Time data and streaming applications

• 4+ years of experience with NoSQL implementation (Mongo, Cassandra)

• 4+ years of data warehousing experience (Redshift or Snowflake)

• 4+ years of experience with UNIX/Linux including basic commands and Shell Scripting

• 2+ years of experience with Agile engineering practices

At this time, Capital One will not sponsor a new applicant for employment authorization, or offer any immigration related support for this position (ie H1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN, or another type of work authorization).

The minimum and maximum Full time annual salaries for this role are listed below, by location. Please note that this salary information is solely for candidates hired to perform work within one of these locations, and refers to the amount Capital One is willing to pay at the time of this posting. Salaries for part-time roles will be prorated based upon the agreed upon number of hours to be regularly worked.

Candidates hired to work in other locations will be subject to the pay range associated with that location, and the actual annualized salary amount offered to any candidate at the time of hire will be reflected solely in the candidate's offer letter.

Capital One offers a comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being. Learn more at the Capital One Careers website . Eligibility varies based on full or part-time status, exempt or non-exempt status, and management level.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at or via email at (see below) . All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.

For technical support or questions about Capital One's recruiting process, please send an email to (see below)

Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.

Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).

Job Title:

Manager, Data Engineering

Overview:

OVERVIEWThe Data and Analytics group within Services is seeking a highly experienced and strategic professional to manage the Mastercard Performance (MyMPA) product. MyMPA is a system designed to provide comprehensive data on Mastercard, Maestro, and Cirrus brands. It includes various data dimensions such as QMR and Processed Data, Country and Regional Totals, and Issuing, Acquiring, and Cross Border data. You will work closely with internal stakeholders, external partners, and subject matter experts to drive the growth of the product aligned with Mastercard's business goals and core values.

Services within Mastercard is responsible for acquiring, engaging, and retaining customers by managing fraud and risk, enhancing cybersecurity, and improving the digital payments experience. We provide value-added services and leverage expertise, data-driven insights, and execution.

Role
• Responsible for MyMPA application and build the team to achieve objectives defined around MyMPA delivery
• Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object storage.
• Leverage new technologies and approaches to innovate with increasingly large data sets.
• Work closely with business stakeholders and embedded engineering teams within business teams in a collaborative manner to help them build scalable products in quick time.
• Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency.

All About You
• 10+ years of experience in Data Warehouse related projects or data oriented projects
• Experience of building data pipelines through Spark with Scala/Python/Java on Hadoop or Object storage
• Experience of building Nifi pipelines
• Understanding of Mastercard CORE datasets
• Experience of working with Databases like Oracle, Netezza would be added advantage
• Strong analytical skills required for debugging production issues, providing root cause and implementing mitigation plan
• Strong communication skills - both verbal and written - and strong relationship, collaboration skills and organisational skills
• Experience managing teams located across multiple locations
• Ability to be high-energy, detail-oriented, proactive and able to function under pressure in an independent environment along with a high degree of initiative and self-motivation to drive results

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title and Summary

Director, Data Engineering

Overview:
The Mastercard Services Technology team is looking for Director of Data Engineering, to lead our mission of unlocking the full potential of Mastercard's data assets. In this high-impact role, you will drive innovation, reduce friction in the management of big data assets, and ensure seamless data accessibility across both public cloud (AWS, Azure) and on-premise environments. You will champion standards, best practices, and governance principles to enable modern, scalable, and secure data platforms.
As the Director, you will collaborate with cross-geography teams of data engineers and platform engineers, aligning efforts across multiple initiatives into a unified execution framework. Your expertise in technical program management will be critical in harmonizing delivery approaches, ensuring accountability, and enabling our teams to consistently deliver high-value data products and platforms.

Role:
• Lead the design and delivery of end-to-end cloud-native solutions with a focus on Lakehouse architectures (S3, Azure Data Lake, Databricks, EMR, Athena, etc.).
• Provide both technology and people leadership, mentoring engineers and fostering a culture of excellence, ownership, and continuous improvement.
• Drive CI/CD pipeline development, infrastructure-as-code, and automated environment provisioning across AWS and Azure.
• Set and enforce security best practices (IAM, encryption, network security) and ensure compliance with frameworks such as GDPR and HIPAA.
• Establish and enforce enterprise-grade data governance frameworks, including data cataloging, lineage, classification, and access control, leveraging tools like Lake Formation or Azure Purview.
• Design and lead implementation of robust data engineering pipelines, including ETL/ELT, data lakes, and data warehouses.
• Implement Lakehouse architectures using the medallion model (Bronze, Silver, Gold) with deep integration into AWS services (Glue, Lake Formation) and analytics engines (Databricks, EMR, Athena).
• Continuously monitor, troubleshoot, and optimize systems for performance, reliability, scalability, and cost efficiency.
• Work closely with product, engineering, and data stakeholders to translate business requirements into scalable technical solutions.
• Participate in architectural discussions, sprint planning, feature sizing, and Agile ceremonies to ensure seamless execution.
• Stay current with emerging technologies in cloud, data, and DevOps to keep our platforms future-ready.
• Effectively communicate technical vision, trade-offs, and roadmaps to diverse stakeholders.

All About You:
• Extensive experience in cloud and data platform architecture, DevOps, or related roles, with proven leadership of cross-functional teams.
• Hands-on expertise with AWS and Azure services: S3, EC2, Lambda, Glue, Redshift, Lake Formation, Azure Data Factory, Fabric, Synapse, and Databricks.
• Strong track record of building DevOps pipelines and infrastructure-as-code using Terraform, CloudFormation, ARM templates, Azure DevOps, or GitLab CI/CD.
• Solid understanding of cloud security concepts, IAM design, data encryption, and governance frameworks.
• Experience with data engineering frameworks (Spark, PySpark) and large-scale ETL systems.
• Proficiency in scripting and automation using Python, Bash, or PowerShell.
• Demonstrated success delivering production-ready medallion architecture solutions.
• Experience with containerization & orchestration (Docker, Kubernetes, ECS, AKS).
• Familiarity with data governance tools (Lake Formation, Purview) and observability stacks (CloudWatch, Azure Monitor, ELK).
• Knowledge of cloud cost optimization and financial management.
• Exposure to MLOps workflows and ML pipeline integrations.
• Comfortable working in Agile/Scrum environments with strong collaboration skills.
• Strong problem-solving skills, with the ability to identify discordant views and engage in constructive dialogue to resolve them.
• Excellent verbal and written communication skills, with the ability to explain technical problems clearly and succinctly and effectively sharing vision and roadmaps with diverse stakeholders.
• Bachelor's degree in Computer Science, Information Systems, Engineering, Data Analytics, or a related field. Equivalent practical experience will also be considered.

#LI-NF1

Mastercard is a merit-based, inclusive, equal opportunity employer that considers applicants without regard to gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law. We hire the most qualified candidate for the role. In the US or Canada, if you require accommodations or assistance to complete the online application process or during the recruitment process, please contact and identify the type of accommodation or assistance you are requesting. Do not include any medical or health information in this email. The Reasonable Accommodations team will respond to your email promptly.

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard's security policies and practices;

• Ensure the confidentiality and integrity of the information being accessed;

• Report any suspected information security violation or breach, and

• Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.

In line with Mastercard's total compensation philosophy and assuming that the job will be performed in the US, the successful candidate will be offered a competitive base salary and may be eligible for an annual bonus or commissions depending on the role. The base salary offered may vary depending on multiple factors, including but not limited to location, job-related knowledge, skills, and experience. Mastercard benefits for full time (and certain part time) employees generally include: insurance (including medical, prescription drug, dental, vision, disability, life insurance); flexible spending

• Lead and mentor a team of data engineers (onshore and offshore) in building scalable, secure, and high-performing data solutions.
• Oversee the development and maintenance of our enterprise data warehouse and data lake, aligning them with healthcare data standards and regulatory requirements (e.g., HIPAA).
• Design and optimize data pipelines using Azure Databricks, Spark, and other Azure data services to ingest, transform, and deliver healthcare data from multiple sources, including EMRs, claims, and patient portals.
• Collaborate with clinical, operational, and analytics teams to deliver reliable, self-service data and insights.
• Ensure robust data governance, privacy, and quality practices across the organization.
• Drive adoption of Agile and iterative development practices within the team.
• Play a key role in workforce planning, hiring, coaching, and performance management.
• Communicate data strategy, roadmap, and results clearly to leadership and cross-functional teams.

• Bachelor’s degree in computer science, Information Systems, or a related field (or equivalent experience).
• 7+ years of experience in data engineering or related roles, with 2+ years in a leadership or people management capacity.
• Strong background in SQL, data modeling, and ETL/ELT development.
• Hands-on expertise with Azure Databricks, including:
• Building Spark-based data pipelines
• Managing cluster performance and cost optimization
• Integrating with Azure Data Factory and Data Lake Storage.
• Deep understanding of Azure cloud architecture and healthcare-specific data challenges.
• Familiarity with healthcare data formats and standards, such as HL7, FHIR, or claims data.
• Proficiency in BI tools like Power BI, Tableau, or Sigma.
• Strong communication skills and the ability to collaborate with technical and non-technical stakeholders.

Seeking an experienced and highly skilled Data Engineering Technical Lead to spearhead the design, development, and optimization of robust and scalable data pipelines and data infrastructure. This role requires a deep understanding of data engineering principles, advanced proficiency in distributed data processing technologies, and a proven track record of leading technical teams in an agile environment. The Technical Lead will play a critical role in shaping our data architecture, ensuring data quality and governance, and enabling data-driven initiatives across the organization, particularly within the healthcare and supply chain logistics domains. This role is 100% remote.

Key Responsibilities (Bullets):

• Lead, mentor, and guide a team of data engineers in best practices for data pipeline development, data modeling, and infrastructure management. Provide technical oversight and ensure adherence to architectural standards.

• Drive the design and implementation of highly scalable, reliable, and efficient data architectures, including data lakes, data warehouses, and streaming platforms. Define technical roadmaps and strategies for data engineering initiatives.

• Design, develop, and maintain complex ETL/ELT processes and data pipelines using modern data engineering tools and frameworks (e.g., Apache Spark, Kafka, Airflow). Ensure data ingestion, transformation, and loading processes are optimized for performance and reliability.

• Establish and enforce data governance policies, data quality standards, and data security protocols. Implement robust monitoring and alerting for data pipelines and data quality issues.

• Leverage expertise in cloud-based data platforms (e.g., AWS, Azure, GCP) to design and implement cloud-native data solutions, optimizing for cost, performance, and scalability.

• Collaborate closely with product owners, data scientists, data analysts, and other engineering teams to translate business requirements into technical specifications and deliver data solutions that align with strategic objectives. Act as a primary interface with customers, business stakeholders, and cross-functional teams to gather requirements, provide updates, and ensure alignment on data initiatives.

• Identify and resolve complex data-related performance bottlenecks and architectural challenges. Implement strategies for query optimization and data storage efficiency.

• Actively participate in agile ceremonies, including sprint planning, backlog refinement, daily stand-ups, and sprint reviews, contributing to a culture of continuous delivery and improvement. Lead and own the full Software Development Life Cycle (SDLC) for data solutions, from conceptualization and design through deployment, testing, and operational support.

• Create and maintain comprehensive technical documentation, including data flow diagrams, architectural designs, data dictionaries, and operational runbooks.

Minimum Qualifications

• Bachelor's Degree in a related field; Master's Degree preferred

• Industry leading certification for technical expertise preferred

• 12+ years of experience in a technical field with at least 2 years in a technical leadership or lead engineer role.

Other Job Specific Skills

• Expert-level proficiency in SQL and experience with relational and NoSQL databases.

• Extensive experience with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and stream processing technologies (e.g., Apache Kafka, Flink).

• Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.

• Proven experience with cloud-based data platforms (e.g., AWS Redshift, S3, Glue, EMR; Azure Data Lake, Synapse, Databricks; GCP BigQuery, Dataflow).

• Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java).

• Demonstrated experience with CI/CD practices for data pipelines and infrastructure as code (IaC).

• Solid understanding of data governance, data security, and compliance best practices (e.g., HIPAA, GDPR).

Compensation Ranges

Compensation ranges for ASM Research positions vary depending on multiple factors; including but not limited to, location, skill set, level of education, certifications, client requirements, contract-specific affordability, government clearance and investigation level, and years of experience. The compensation displayed for this role is a general guideline based on these factors and is unique to each role. Monetary compensation is one component of ASM's overall compensation and benefits package for employees.

EEO Requirements

It is the policy of ASM that an individual's race, color, religion, sex, disability, age, sexual orientation or national origin are not and will not be considered in any personnel or management decisions. We affirm our commitment to these fundamental policies.

All recruiting, hiring, training, and promoting for all job classifications is done without regard to race, color, religion, sex, disability, or age. All decisions on employment are made to abide by the principle of equal employment.

Physical Requirements

The physical requirements described in "Knowledge, Skills and Abilities" above are representative of those which must be met by an employee to successfully perform the primary functions of this job. (For example, "light office duties' or "lifting up to 50 pounds" or "some travel" required.) Reasonable accommodations may be made to enable individuals with qualifying disabilities, who are otherwise qualified, to perform the primary functions.

Disclaimer

The preceding job description has been designed to indicate the general nature and level of work performed by employees within this classification. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications required of employees assigned to this job.

$175-180,000

EEO Requirements

It is the policy of ASM that an individual's race, color, religion, sex, disability, age, gender identity, veteran status, sexual orientation or national origin are not and will not be considered in any personnel or management decisions. We affirm our commitment to these fundamental policies.

All recruiting, hiring, training, and promoting for all job classifications is done without regard to race, color, religion, sex, veteran status, disability, gender identity, or age. All decisions on employment are made to abide by the principle of equal employment.

• Builds software to extract, transform and load data - SSIS, Azure Data Factory, Synapse
• Models data for efficient consumption by reporting and analytics tools - Azure Analysis Services, Power BI, SQL Databases
• Maintains previously deployed software and reports - Power BI, SSIS, AAS, SQL Server
• Designs dashboards and reports to meet business needs
• Presents technical problems with solutions in mind, in a constructive and understood fashion
• Demonstrates expert proficiency by sharing learnings with team and in technical showcases
• Independently discovers solutions and collaborates on insights and best practices
• Takes ownership of team's work deliverables and actively participates in mentoring of team members
• Actively seeks out opportunities to help improve the team's practices and processes to achieve fast flow
• Works with other developers to facilitate knowledge

Seeking an experienced and highly skilled Data Engineering Technical Lead to spearhead the design, development, and optimization of robust and scalable data pipelines and data infrastructure. This role requires a deep understanding of data engineering principles, advanced proficiency in distributed data processing technologies, and a proven track record of leading technical teams in an agile environment. The Technical Lead will play a critical role in shaping our data architecture, ensuring data quality and governance, and enabling data-driven initiatives across the organization, particularly within the healthcare and supply chain logistics domains. This role is 100% remote.

Key Responsibilities (Bullets):

• Lead, mentor, and guide a team of data engineers in best practices for data pipeline development, data modeling, and infrastructure management. Provide technical oversight and ensure adherence to architectural standards.

• Drive the design and implementation of highly scalable, reliable, and efficient data architectures, including data lakes, data warehouses, and streaming platforms. Define technical roadmaps and strategies for data engineering initiatives.

• Design, develop, and maintain complex ETL/ELT processes and data pipelines using modern data engineering tools and frameworks (e.g., Apache Spark, Kafka, Airflow). Ensure data ingestion, transformation, and loading processes are optimized for performance and reliability.

• Establish and enforce data governance policies, data quality standards, and data security protocols. Implement robust monitoring and alerting for data pipelines and data quality issues.

• Leverage expertise in cloud-based data platforms (e.g., AWS, Azure, GCP) to design and implement cloud-native data solutions, optimizing for cost, performance, and scalability.

• Collaborate closely with product owners, data scientists, data analysts, and other engineering teams to translate business requirements into technical specifications and deliver data solutions that align with strategic objectives. Act as a primary interface with customers, business stakeholders, and cross-functional teams to gather requirements, provide updates, and ensure alignment on data initiatives.

• Identify and resolve complex data-related performance bottlenecks and architectural challenges. Implement strategies for query optimization and data storage efficiency.

• Actively participate in agile ceremonies, including sprint planning, backlog refinement, daily stand-ups, and sprint reviews, contributing to a culture of continuous delivery and improvement. Lead and own the full Software Development Life Cycle (SDLC) for data solutions, from conceptualization and design through deployment, testing, and operational support.

• Create and maintain comprehensive technical documentation, including data flow diagrams, architectural designs, data dictionaries, and operational runbooks.

Minimum Qualifications

• Bachelor's Degree in a related field; Master's Degree preferred

• Industry leading certification for technical expertise preferred

• 12+ years of experience in a technical field with at least 2 years in a technical leadership or lead engineer role.

Other Job Specific Skills

• Expert-level proficiency in SQL and experience with relational and NoSQL databases.

• Extensive experience with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and stream processing technologies (e.g., Apache Kafka, Flink).

• Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.

• Proven experience with cloud-based data platforms (e.g., AWS Redshift, S3, Glue, EMR; Azure Data Lake, Synapse, Databricks; GCP BigQuery, Dataflow).

• Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java).

• Demonstrated experience with CI/CD practices for data pipelines and infrastructure as code (IaC).

• Solid understanding of data governance, data security, and compliance best practices (e.g., HIPAA, GDPR).

Compensation Ranges

Compensation ranges for ASM Research positions vary depending on multiple factors; including but not limited to, location, skill set, level of education, certifications, client requirements, contract-specific affordability, government clearance and investigation level, and years of experience. The compensation displayed for this role is a general guideline based on these factors and is unique to each role. Monetary compensation is one component of ASM's overall compensation and benefits package for employees.

EEO Requirements

It is the policy of ASM that an individual's race, color, religion, sex, disability, age, sexual orientation or national origin are not and will not be considered in any personnel or management decisions. We affirm our commitment to these fundamental policies.

All recruiting, hiring, training, and promoting for all job classifications is done without regard to race, color, religion, sex, disability, or age. All decisions on employment are made to abide by the principle of equal employment.

Physical Requirements

The physical requirements described in "Knowledge, Skills and Abilities" above are representative of those which must be met by an employee to successfully perform the primary functions of this job. (For example, "light office duties' or "lifting up to 50 pounds" or "some travel" required.) Reasonable accommodations may be made to enable individuals with qualifying disabilities, who are otherwise qualified, to perform the primary functions.

Disclaimer

The preceding job description has been designed to indicate the general nature and level of work performed by employees within this classification. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications required of employees assigned to this job.

$175-180,000

EEO Requirements

It is the policy of ASM that an individual's race, color, religion, sex, disability, age, gender identity, veteran status, sexual orientation or national origin are not and will not be considered in any personnel or management decisions. We affirm our commitment to these fundamental policies.

All recruiting, hiring, training, and promoting for all job classifications is done without regard to race, color, religion, sex, veteran status, disability, gender identity, or age. All decisions on employment are made to abide by the principle of equal employment.