1,884 Data Engineering jobs in the United States
Data Engineering & Big Data Architect
Posted today
Job Viewed
Job Description
Data Engineering & Big Data Architect Technology Architect - US Company ITL USA Requisition ID 135930BR Infosys is seeking a highly experienced Data Engineering & Big Data Architect to join our team and perform Technical Product management/Data analyst tasks and own the development of a Tiered data architecture for our data platform . In this role, they would be responsible for crafting and building a comprehensive data architecture that will enable seamless data integration and enable the delivery of high-quality insights to our leadership and business stakeholders. You will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions, lead, and participate in sales and pursuits focused on our clients' business needs. You will be part of a learning culture, where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued. Candidate must be located within commuting distance of Austin, TX/ Sunnyvale, CA or be willing to relocate to the area. This position may require travel to project locations Required Qualifications: Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education. At least 7 years of experience in Information Technology. Experience with Hadoop distributed frameworks while handling large amount of data using Spark or PYSpark and Hadoop Ecosystems. Proven experience in data engineering, data architecture, or a related field Experience with Spark or PySpark is required. Strong understanding of data modeling, data warehousing, and ETL concepts Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop or Spark Experience with Scala or Python is required. Preferred Qualifications: Experience in understanding of Design Patterns, ability to discuss tradeoffs between RDBMS vs Distributed Storage At least 5 years of Experience with Spark or PySpark, Python, Scala and Data Engineering Experience in design and implementing a tiered data architecture that integrates analytics data from multiple sources in an efficient and effective manner. Experience with data orchestration tools like Airflow is a nice to have. Excellent problem-solving and analytical skills, and the ability to work well under tight deadlines. Excellent interpersonal skills and the ability to collaborate effectively with cross-functional teams. Experience in developing data models and mapping rules to transform raw data into actionable insights and reports. Experience in collaborating with the analytics and business teams to understand their requirements, with cross-functional teams to define and implement data governance policies and standards. Experience in developing data validation and reconciliation processes to ensure data quality and accuracy is met. Experience with development and maintenance of user documentation, including data models, mapping rules, and data dictionaries. Estimated annual compensation range for candidate based in the below locations will be: Sunnyvale, CA- $104079 -$75310 Along with competitive pay, as a full-time Infosys employee you are also eligible for the following benefits :- Long-term/Short-term Disability Health and Dependent Care Reimbursement Accounts Insurance (Accident, Critical Illness , Hospital Indemnity, Legal) 401(k) plan and contributions dependent on salary level The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements. Job description Infosys is seeking a highly experienced Data Engineering & Big Data Architect to join our team and perform Technical Product management/Data analyst tasks and own the development of a Tiered data architecture for our data platform . In this role, they would be responsible for crafting and building a comprehensive data architecture that will enable seamless data integration and enable the delivery of high-quality insights to our leadership and business stakeholders. You will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions, lead, and participate in sales and pursuits focused on our clients' business needs. You will be part of a learning culture, where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued. Candidate must be located within commuting distance of Austin, TX/ Sunnyvale, CA or be willing to relocate to the area. This position may require travel to project locations Required Qualifications: Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education. At least 7 years of experience in Information Technology. Experience with Hadoop distributed frameworks while handling large amount of data using Spark or PYSpark and Hadoop Ecosystems. Proven experience in data engineering, data architecture, or a related field Experience with Spark or PySpark is required. Strong understanding of data modeling, data warehousing, and ETL concepts Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop or Spark Experience with Scala or Python is required. Preferred Qualifications: Experience in understanding of Design Patterns, ability to discuss tradeoffs between RDBMS vs Distributed Storage At least 5 years of Experience with Spark or PySpark, Python, Scala and Data Engineering Experience in design and implementing a tiered data architecture that integrates analytics data from multiple sources in an efficient and effective manner. Experience with data orchestration tools like Airflow is a nice to have. Excellent problem-solving and analytical skills, and the ability to work well under tight deadlines. Excellent interpersonal skills and the ability to collaborate effectively with cross-functional teams. Experience in developing data models and mapping rules to transform raw data into actionable insights and reports. Experience in collaborating with the analytics and business teams to understand their requirements, with cross-functional teams to define and implement data governance policies and standards. Experience in developing data validation and reconciliation processes to ensure data quality and accuracy is met. Experience with development and maintenance of user documentation, including data models, mapping rules, and data dictionaries. Estimated annual compensation range for candidate based in the below locations will be: Sunnyvale, CA- $1 4079 - 175310 Along with competitive pay, as a full-time Infosys employee you are also eligible for the following benefits :- Medical/Dental/Vision/Life Insurance Long-term/Short-term Disability Health and Dependent Care Reimbursement Accounts Insurance (Accident, Critical Illness , Hospital Indemnity, Legal) 401(k) plan and contributions dependent on salary level Paid holidays plus Paid Time Off The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements. About Us Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem. Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. #J-18808-Ljbffr
Senior Data Engineering Leader (Big Data, AWS)
Posted 2 days ago
Job Viewed
Job Description
Are you passionate about leveraging technology to solve complex business challenges? At Capital One, you'll join a dynamic team of innovators dedicated to addressing real problems and fulfilling customer needs. We are looking for a Senior Data Engineering Leader who thrives on merging data with cutting-edge technologies. Take the next step in driving significant transformation at Capital One.
Key Responsibilities:
Collaborate within Agile teams to design, develop, test, implement, and support comprehensive technical solutions using full-stack development tools.
Work alongside experienced developers in machine learning, distributed microservices, and full-stack systems.
Utilize programming languages such as Java, Scala, Python, along with Open Source RDBMS and NoSQL databases, and cloud-based data warehousing services like Redshift and Snowflake.
Stay at the forefront of technology trends by experimenting with new technologies, participating in technology communities, and mentoring fellow engineers.
Partner with digital product managers to create robust cloud solutions that deliver exceptional experiences for millions striving for financial empowerment.
Conduct unit tests and collaborate with team members to ensure your code is well-designed, optimized, and performs effectively.
Basic Qualifications:
Bachelor's Degree.
Minimum 4 years of experience in application development (Internship experience does not apply).
At least 2 years of experience in big data technologies.
At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud).
Preferred Qualifications:
7+ years of experience in application development including proficiency in Python, SQL, Scala, or Java.
4+ years of experience in a public cloud environment (AWS, Microsoft Azure, Google Cloud).
4+ years of experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL).
4+ years of experience working with Real-Time data and streaming applications.
4+ years of experience with NoSQL implementation (Mongo, Cassandra).
4+ years of experience in data warehousing (Redshift or Snowflake).
4+ years of experience with UNIX/Linux, including basic commands and Shell Scripting.
2+ years of experience with Agile engineering practices.
This position is not eligible for employment authorization sponsorship. The salary range for this role is as follows:
- McLean, VA: $193,400 - $20,700
- Richmond, VA: 175,800 - 200,700
Please note that salaries for positions outside of these locations will vary accordingly. This role is also eligible for performance-based incentives, which may include cash bonuses and/or long-term incentives. We offer a comprehensive benefits package that supports your overall well-being.
Capital One is an equal opportunity employer (EOE) committed to a diverse and inclusive workplace. We will consider qualified applicants with criminal histories in line with applicable laws.
If you need accommodation in the application process, please contact Capital One Recruiting. All information will remain confidential.
Apple Services Engineering - Big Data Analyst, Analytics & Data Engineering
Posted today
Job Viewed
Job Description
**Cupertino, California, United States**
**Software and Services**
**Summary**
Posted: **Jul 08, 2025**
Weekly Hours: **40**
Role Number: **200610106**
The Apple Service Engineering team is one of the most exciting examples of Apple's long-held passion for combining art and technology. These are the people who power the App Store, Apple TV, Apple Music, Apple Podcasts, Apple Books, Apple Pay. And they do it at a massive scale, meeting Apple's high expectations with dedication to deliver a variety of entertainment in over 35 languages to more than 150 countries.
Do you have a passion for data and how it can be used to improve customer experience, protect user privacy, make products better, and increase revenue for a business? Do you thrive working with data engineers, data scientists and project managers in an agile, distributed and project focused work environment? If so, join us!
Apple is seeking an experienced data analyst to work on an exciting set of problems outstanding to ASE. You will help find interesting data trends, identify root causes, and help transform up-and-coming ideas to products and systems. You will do so in an environment that highly values collaboration, mentoring and continuous learning.
**Description**
Dive deep into large-scale data to uncover trends and identify key insights.
Build and automate dashboards used to present key insights to partners.
Partner with data engineers and project managers to conduct data investigation analysis.
Collaborate with researchers and machine learning engineers for building data science and machine learning solutions such as anomaly detection.
Facilitate cross-functional discussion to find out what is the right technical resolution to the business problems.
**Minimum Qualifications**
+ Bachelors or higher degree in a quantitative field (e.g. Computer Science, Engineering, Mathematics, Operations Research or a related field)
+ At least 5+ years in an analytics or data engineering role
+ Background in data analytics, proficient in SQL or Python Pandas
+ Experience with data visualization tools, such as Tableau, GGplot, Keynote etc
+ Fundamental understanding of statistical analysis, forecasting and modeling
+ Experience in coding for automation (e.g. Scala, Python, Java etc.)
+ Experience writing data pipelines using one or more big data technologies (e.g. Spark, Hive, Trino, Flink, Airflow etc.)
+ Familiarity with modern cloud technologies such as Kubernetes
+ Excellent written and verbal communication skills
**Preferred Qualifications**
+ Familiarity with software development lifecycle for data science and engineering
+ Familiarity with commerce, payments, and user experience datasets at online services
**Pay & Benefits**
At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $147,400 and $272,100, and your base pay will depend on your skills, qualifications, experience, and location.
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation.Learn more about Apple Benefits. ( Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.
Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.Learn more about your EEO rights as an applicant ( .
Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.Learn more about your EEO rights as an applicant ( .
Apple will not discriminate or retaliate against applicants who inquire about, disclose, or discuss their compensation.
Apple participates in the E-Verify program in certain locations as required by law.Learn more about the E-Verify program ( .
Apple is committed to working with and providing reasonable accommodation to applicants with physical and mental disabilities. Reasonable Accommodation and Drug Free Workplace policy Learn more .
Apple is a drug-free workplace. Reasonable Accommodation and Drug Free Workplace policy Learn more .
Apple will consider for employment all qualified applicants with criminal histories in a manner consistent with applicable law. If you're applying for a position in San Francisco, review the San Francisco Fair Chance Ordinance guidelines applicable in your area.
It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Data Engineering
Posted today
Job Viewed
Job Description
Join to apply for the Data Engineering role at targetjobs UK 3 days ago Be among the first 25 applicants Join to apply for the Data Engineering role at targetjobs UK Get AI-powered advice on this job and more exclusive features. Summary Digital Futures is looking for aspiring data engineering consultants to join our Future Talent Programme. The Future Talent Programme offers a unique entry point for individuals looking to start their career in technology. The programme begins with immersive training at the Digital Futures Academy, where trainees gain the essential technical expertise, consulting skills, and practical experience required for success. Following the successful completion of our Academy and attainment of an industry certification, trainees will then be deployed as consultants within high-performing technology teams at one of Digital Futures clients. Summary Digital Futures is looking for aspiring data engineering consultants to join our Future Talent Programme. The Future Talent Programme offers a unique entry point for individuals looking to start their career in technology. The programme begins with immersive training at the Digital Futures Academy, where trainees gain the essential technical expertise, consulting skills, and practical experience required for success. Following the successful completion of our Academy and attainment of an industry certification, trainees will then be deployed as consultants within high-performing technology teams at one of Digital Futures clients. Your journey with us Training – Your journey with Digital Futures starts with 12-weeks of free immersive training, during which our team of expert instructors will help you learn essential technical and consulting skills to prepare you for your career in data. Our training is delivered remotely, with one-to-one mentoring and group project work. Certification – As part of your graduation from the Digital Futures Academy, you will achieve the AWS Data Engineer Associate certification. The cost of your certification is covered by Digital Futures. Employment – Once your deployment has been secured, we will offer you a full-time employment contract with Digital Futures. Throughout this time, you will work as a consultant in a technology team at one of the world’s leading companies, applying your skills to exciting, innovative and impactful projects. Your immersive training Technical Skills The Data Engineering pathway at the Digital Futures Academy covers the key technical skills and concepts needed to be successful in your first role in data. The technical skills training is underpinned by a comprehensive consulting skills curriculum to ensure that our consultants are workplace ready ahead of starting roles with our clients. Modules taught at the Digital Futures Academy include: Coding fundamentals (Python, SQL) Data modelling and data warehousing Extract-Transform-Load (ETL) Creating data pipelines Ethics in data and AI Cloud and big data technologies (PySpark, AWS) Working with unstructured data Consulting Skills Effective teamwork Business acumen Time management Stakeholder management Presentations Agile methodology Being a brand ambassador Being a Digital Futures consultant Role Consulting roles with our clients are varied and give you the hands-on opportunities to gain the experience you need for success. As a data engineering consultant, you will be: Working with fellow data engineers to design and implement data pipelines. Building and maintaining ETL processes to ingest, transform, and store data from various sources. Collaborating with data scientists and analysts to ensure that data is properly formatted and accessible. Troubleshooting and resolving data pipeline and infrastructure issues. Helping to design and maintain data warehouses, data lakes, and other data storage solutions. Developing Generative AI powered data solutions. Ensuring that data is properly secured and follows industry best practices Staying up to date with new data engineering technologies and techniques What are we looking for? Education You don’t need any previous formal education or commercial experience in tech or data to join our Future Talent Programme, as long as you can demonstrate the following: An understanding of the basics of Python A genuine interest in tech and how it can help the world A growth mindset with a commitment to lifelong learning Self-starting and motivated by personal development Intellectually curious with great problem-solving skills A team player and collaborative in nature A delivery mindset with the ability to get things done Truly motivated to take the first step to starting a career in tech Key programme information Please note the following important information before submitting an application to the Future Talent Programme: Start Date - 15 September 2025 Locations: Following the training, roles with our clients will be located across the UK, including in: London, Stevenage, Bristol, Norwich, Leeds, Sheffield and Halifax. You must be available to commute or relocate to one of these locations to apply. Salary: Upon starting your deployment with one of our clients and becoming a permanent employee of Digital Futures, the salary is: Year 1 - £27,500 per annum plus benefits Year 2 - £2,000 per annum plus benefits, subject to performance review Benefits: Once you become a permanent employee of Digital Futures you will have access to: a comprehensive benefits package and pension contributions an Employee Referral Scheme – earn up to £1 00 per referral an Incentives Scheme which will allow you to access funds for additional training and certifications. You must have the right to work in the UK from the start date of the programme. Why join Digital Futures? Digital Futures is a mission-led technology services company specialising in strategic workforce solutions. Our mission is to accelerate the transition to a skills-based economy and create sustainable employment opportunities across the UK by challenging conventional thinking and bridging the gap between education and industry — equipping exceptional talent from all backgrounds with the skills, knowledge, and confidence to succeed. Our Future Talent programme focuses on uncovering potential, rather than on background or experience. In doing so, we unlock exceptional talent and help people to thrive in their first role in tech. We focus not only on preparing you for your first consulting role with us, but also on setting you up for success for your future career. If you have the right mindset and drive, we will help you reach your career goals, preparing you to become a leader of tomorrow. Seniority level Seniority level Internship Employment type Employment type Full-time Job function Job function Information Technology Industries Technology, Information and Internet Referrals increase your chances of interviewing at targetjobs UK by 2x Get notified about new Data Specialist jobs in Norwich, CT . Norwich, CT $27,500.00-$2,000.00 1 month ago Child Development Mentor & Data Specialist-Little Learners Early Education Program Intervention Specialist - Regional Multicultural Magnet School West Greenwich, RI 59,795.00- 122,400.00 5 days ago New London County, CT 55.00- 60.00 1 day ago Waterford, CT 85,000.00- 110,000.00 1 month ago West Greenwich, RI 57,795.00- 122,400.00 20 hours ago Senior Cloud Engineer- Governance (Azure) West Greenwich, RI 74,961.00- 164,800.00 20 hours ago Automated Test Equipment (ATE) Software Engineer Waterford, CT 85,000.00- 125,000.00 3 weeks ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago West Greenwich, RI 55,631.00- 98,467.00 5 days ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago Insurance Software Engineer – Pplus & DSS Mansfield City, CT $1 5,100.00- 195,200.00 5 days ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago West Greenwich, RI 74,961.00- 164,800.00 5 days ago West Greenwich, RI 59,795.00- 122,400.00 5 days ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr
Director, Data Engineering
Posted today
Job Viewed
Job Description
**Relocation Assistance Available** Business Context Marriott Vacations Worldwide (MVW) is a leading global vacation company that offers vacation ownership, exchange, rental and resort and property management, along with related businesses, products, and services. The Company has over 1.
Data Engineering Manager
Posted 1 day ago
Job Viewed
Job Description
As a Data Engineering Manager here at Advanced Materials, you will be responsible for leading technical development and delivery across our comprehensive data platform including Snowflake, ETL/ELT, DevSecOps, Cloud Infrastructure, Data Pipelines, and Data Lakes. You will drive innovation and technology strategy while providing technical mentorship to the team.
You will report to our Director of Gen AI, Data & Analytics Leader who oversees our enterprise data strategy, delivery and platform operations.
In this role, you will have significant impact on our technical architecture, roadmap innovation, and excellence in engineering. You will provide hands-on technical leadership while establishing best practices and driving continuous improvement.
KEY RESPONSIBILITIES
-
Data Warehousing & Engineering, Development and Support: Lead development, delivery and support for Enterprise Data Warehouse, Analytics, DataOps, Data/AI Applications, and pipelines.
-
Technical Leadership & Development: Manage and mentor technical contract service workers while establishing standards, innovative approaches and best practices, fostering excellence
-
Infrastructure Management: Implement cloud infrastructure and code management using Infrastructure as Code practices with comprehensive governance
-
Data Architecture, Ops & Governance: Oversee Metadata catalogs and report enhancement initiatives while establishing data quality standards. Lead efforts in observability, and lineage tracking to maintain high integrity in AI/ML datasets.
-
Technology Modernization: Lead evaluation of emerging technologies and execute incremental modernization with minimal business disruption
-
Innovation & Strategy: Drive technology roadmap with a first-principles mindset while leading standardization efforts using automation tools
-
Vendor & Service Management: Establish frameworks for vendor management and technical escalation procedures with clear accountability
YOU MUST HAVE
-
Bachelor's or Master's degree in computer science, Engineering, Applied Mathematics or related field.
-
Minimum 10 years of progressive experience in Data and Analytics, with a strong focus on engineering and enterprise-scale analytics platforms.
-
Expert-level proficiency in languages such as SQL, Python and Spark with advanced knowledge of modern data engineering tools and practices.
-
Strong knowledge of data modeling concepts like Change Data Capture, ETL/ ELT processes and tools and data integration techniques and data governance.
-
Prior experience with enterprise-scale platforms:
-
Snowflake or similar Data Warehouses
-
Databricks or similar AI/ML Platforms
-
Informatica Intelligent Cloud Services (IICS), HVR, Fivetran or similar Data Ingestion and Transformation Tools
WE VALUE
-
Experience managing high-performing engineering teams and mentoring junior engineers in complex technical environments.
-
AI & Advanced Analytics: Hands-on experience with AI/ML platforms (Azure OpenAI, Databricks, MLOps) and implementing enterprise-scale machine learning solutions.
-
Experience implementing Infrastructure as Code (IaC) practices and end-to-end data observability solutions.
-
Strong ability to translate complex technical concepts to business stakeholders and drive consensus across diverse teams.
-
Experience and domain knowledge of enterprise systems of records such as SAP for ERP, Salesforce for CRM and associated E2E processes etc.
-
Background in enterprise analytics with measurable business outcomes for domains such as Commercial, Finance, Supply Chain, Customer Experience and others.
-
Prior experience with enterprise-scale visualization platforms:
-
PowerBI, Tablaeu or similar visualization tools
-
Cloud & Infrastructure Skills: Advanced experience with Terraform/Bicep, Docker/Kubernetes, and cloud-native architecture in enterprise environments.
-
Innovation Leadership: Proven track record of driving technical innovation and leading transformational initiatives with measurable business impact
-
Excellent ability to translate complex problems into smaller, iteratively solvable problems while navigating uncertainty.
-
Familiarity with IT Operations, ITIL practices and other service management frameworks.
-
Ownership and Pride in your work and willingness to work in a diverse, cross-functional environment.
Honeywell helps organizations solve the world's most complex challenges in automation, the future of aviation and energy transition. As a trusted partner, we provide actionable solutions and innovation through our Aerospace Technologies, Building Automation, Energy and Sustainability Solutions, and Industrial Automation business segments - powered by our Honeywell Forge software - that help make the world smarter, safer and more sustainable.
Honeywell is an equal opportunity employer. Qualified applicants will be considered without regard to age, race, creed, color, national origin, ancestry, marital status, affectional or sexual orientation, gender identity or expression, disability, nationality, sex, religion, or veteran status.
Data Engineering Advisor
Posted 1 day ago
Job Viewed
Job Description
Job Description
An Oil & Gas customer of Insight Global is looking for a Data Engineering Advisor to join their team full-time to sit in The Woodlands, TX, on a hybrid schedule 3 days a week and as needed with travel up to 5% within the Texas area. The IT division's primary goal over the next few years is focused on their Data Transformation efforts that will support all of the company's domains for midstream assets. This person will be designing, developing, and deploying innovative data and analytical solutions to enhance current operations and shape the future of the organization. The salary range for this role is dependent on level of experience with a flexible range of $120,000-$200,000 annually.
Responsibilities:
Design and create data architecture and strategies to facilitate efficient storage and retrieval of structured and unstructured data.
Design, build, and manage data pipelines to ingest, transform, and distribute data between on-premises and Azure-based destinations.
Implement measures to ensure the quality and integrity of data, including data cleaning, validation, and error handling within data pipelines.
Utilize technologies from Qlik, OpenText, SAP Datasphere, and Azure cloud services, including Copilot, Dataverse, Fabric, Data Factory, Logic Apps, Power Automate, Azure Synapse Analytics, Fabric to enable custom applications and BI efforts.
Employ programming languages like Java or Python for automation and API integration tasks.
Work closely with data scientists and developers to provide data necessary for custom applications, visualizations, and analytic models.
Assess, recommend, and integrate new data management technologies, data modeling, relational data structures, and change data capture.
Identify bottlenecks, optimize queries, and fine-tune database or data processing configurations.
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form ( . The EEOC "Know Your Rights" Poster is available here ( .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: .
Skills and Requirements
12+ years of IT experience as a Data Engineer, Data Architect, or Software Developer
In-depth knowledge of Azure cloud services, relational database design principles, data warehousing, performance tuning, and optimization
Strong analytic skills related to working with both structured and unstructured datasets
High level proficiency in T-SQL, ADF Pipelines, SSIS packages, Lift and Shift of existing On-premises processes, database design principles, data modeling and data warehousing
Experience using/setting up Azure DevOps integration with cloud services and setting up CI/CD processes
Understanding of performance analysis and tuning activities to ensure optimal database performance
Excellent communication skills with the ability to interact with developers supporting requirements gathering Oil and Gas/Midstream industry experience
Heavy Data Transformation experience null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to
Be The First To Know
About the latest Data engineering Jobs in United States !
Data Engineering Advisor
Posted 1 day ago
Job Viewed
Job Description
Position: Data Engineering Advisor As a Data Engineering Advisor, you will be responsible for creating and supporting analytical data marts by developing data integration pipelines (ETL) sourcing from various databases and systems. You will develop programs using coding languages (e.g., Python, SQL, PROC SQL, R) and/or low-code environments (e.g., Palantir Foundry) to create analyses and applications for stakeholders across the organization. You will work on data architecture, data engineering solutions, and best practices for analyzing very large datasets. Additionally, you will perform data mining activities to derive insights, work with cloud OLAP systems (e.g., Snowflake), and stay current on new and emerging data science, data engineering, and big data techniques. Responsibilities: Manage and scale data pipelines from internal and external sources to support new product launches and improve data quality. Facilitate data engineering activities including acquisition, extraction, normalization, transformation, and management of large and complex data sets. Stay informed about emerging data engineering, big data, and data science techniques; contribute to best practices, standards, and guidelines. Collaborate with subject matter experts to design and develop front-end applications and supporting data models/pipelines. Provide governance and oversight of data assets, environments, and procedures; enforce data asset naming conventions. Build and maintain data pipelines using big data technologies and ETL processes. Develop and maintain data warehouse schemas and data models to ensure consistency and accuracy. Align technical and business strategies with changing business and technology drivers. Create advanced visualizations and tools to provide insights and drive decision-making. Identify and develop Data-as-a-Service opportunities to improve data availability and consumption. Minimum Qualifications: Bachelors Degree in Computer Science, Information Systems, Engineering, Statistics, Mathematics, or equivalent STEM major. Seven or more years of experience in processing large data sets and hands-on experience in data transformation, aggregation, and filtering. Preferred Qualifications: Certifications such as Palantir Foundry Data Engineer, Azure Data Engineer, Certified Data Management Professional, SnowPro Advanced, etc. Experience building and maintaining APIs and ETL pipelines using tools like Python, SQL, Apache Airflow, Azure Data Factory, etc. Familiarity with platforms such as Palantir Foundry, Snowflake, SAP, Oracle. Understanding of cloud-based data storage and architecture. Proven ability to implement data validation, metadata management, and maintain data consistency across systems. Experience in the electric utility industry or infrastructure investment planning. Ability to collaborate with cross-functional teams including IT, engineering, planning, and business units to deliver tailored data solutions. #J-18808-Ljbffr
Data Engineering Manager
Posted 2 days ago
Job Viewed
Job Description
Specialty/Competency: Data, Analytics & AI
Industry/Sector: EUR X-Sector
Time Type: Full time
Travel Requirements: Up to 80%
At PwC, we are on the cutting edge of data analytics and engineering, committed to crafting innovative data solutions that empower our clients. As a Data Engineering Manager, you will take the lead in designing and constructing advanced data systems and infrastructure, transforming raw data into impactful insights that drive strategic decision-making.
Your primary responsibilities will include:
- Developing and implementing efficient data pipelines, integration, and transformation solutions.
- Coaching and inspiring your team to achieve quality outcomes while fostering a collaborative environment.
- Taking ownership of projects from planning to execution, ensuring they stay on budget and deliver value.
- Engaging with business stakeholders to accurately define and translate their data needs into technical solutions.
- Evaluating and recommending innovative data technologies to enhance our systems.
- Monitoring and optimizing data processing across batch and real-time environments.
Your success in this role will stem from strong analytical and problem-solving skills, as well as the ability to communicate effectively with multiple stakeholders. Additionally, embracing technology and innovation will be crucial as you lead your team towards improved data delivery.
Qualifications:
- Minimum Degree Required: Bachelor's Degree in relevant fields such as Management Information Systems, Computer Science, Engineering, Mathematics, or Statistics.
- Experience: At least 5 years in data engineering or a related field.
- Proficiency in SQL and familiarity with programming languages such as Python, Java, or Scala.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of data governance and security best practices.
At PwC, we prioritize diversity and equal opportunity in our hiring process. We welcome applicants from all backgrounds and strive to create an inclusive environment for our team. We look forward to your application!
Data Engineering Manager
Posted 2 days ago
Job Viewed
Job Description
Specialty/Competency: Data, Analytics & AI
Industry/Sector: EUR X-Sector
Time Type: Full time
Travel Requirements: Up to 80%
At PwC, we are on the cutting edge of data analytics and engineering, committed to crafting innovative data solutions that empower our clients. As a Data Engineering Manager, you will take the lead in designing and constructing advanced data systems and infrastructure, transforming raw data into impactful insights that drive strategic decision-making.
Your primary responsibilities will include:
- Developing and implementing efficient data pipelines, integration, and transformation solutions.
- Coaching and inspiring your team to achieve quality outcomes while fostering a collaborative environment.
- Taking ownership of projects from planning to execution, ensuring they stay on budget and deliver value.
- Engaging with business stakeholders to accurately define and translate their data needs into technical solutions.
- Evaluating and recommending innovative data technologies to enhance our systems.
- Monitoring and optimizing data processing across batch and real-time environments.
Your success in this role will stem from strong analytical and problem-solving skills, as well as the ability to communicate effectively with multiple stakeholders. Additionally, embracing technology and innovation will be crucial as you lead your team towards improved data delivery.
Qualifications:
- Minimum Degree Required: Bachelor's Degree in relevant fields such as Management Information Systems, Computer Science, Engineering, Mathematics, or Statistics.
- Experience: At least 5 years in data engineering or a related field.
- Proficiency in SQL and familiarity with programming languages such as Python, Java, or Scala.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of data governance and security best practices.
At PwC, we prioritize diversity and equal opportunity in our hiring process. We welcome applicants from all backgrounds and strive to create an inclusive environment for our team. We look forward to your application!