15,794 Senior Data Engineer jobs in the United States

Akkodis is seeking a Big Data Engineer for a Contract job with a client in Plano, TX(Hybrid) . Responsibilities include developing processes to interact with HDFS and Oracle, creating workflows using Autosys, and ensuring ML/AI models are production-ready while working in an agile environment.

Rate Range: $ 60 /hour to $0/hour; The rate may be negotiable based on experience, education, geographic location, and other factors.

Big Data Engineer job responsibilities include:

• Develop and enhance application components for supporting ML/AI models and data ingestion processes with a focus on code resiliency and stability.
• Build processes to interact with HDFS and Oracle using Python/PySpark and Oracle PL/SQL.
• Create, schedule, and manage workflows and jobs using Autosys.
• Collaborate with development teams to contribute to story refinement and deliver data requirements throughout the delivery life cycle.
• Leverage architecture components to integrate, clean, transform, and control data as per acceptance criteria.
• Develop and execute test plans to identify issues, errors, and underlying causes, ensuring data accuracy and reliability.
• Drive complex information technology projects, ensuring on-time delivery and adherence to team delivery and release processes.
• Identify, define, and document data engineering requirements for deployment, maintenance, and business functionality.
• Work independently with strong analytical skills, while also being an excellent team player in a fast-paced agile environment.
• Contribute to data resiliency capabilities and ensure that ML/AI models are ready for production.

• BS/master's in computer science, Engineering, or a related field.
• 10+ years of relevant experience in Big Data engineering and development.
• Expertise in Oracle PL/SQL, Hadoop ecosystem, Hive Tables, and Python/PySpark.
• Experience in creating workflows and scheduling jobs using Autosys, with a strong background in machine learning and data resiliency.

• The California Fair Chance Act
• Los Angeles City Fair Chance Ordinance
• Los Angeles County Fair Chance Ordinance for Employers
• San Francisco Fair Chance Ordinance

Job Description

Insight Global is seeking a Big Data Engineer to assist with a application modernization and taking reports currently in the mainframe and recreate in a new modernized application. The output of those reports would be in Tableau or Excel and while this role is focused on Big Data Technologies, will primarily focus on the reporting for the initial duration of the contract.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

At least 6+ Years of experience with PySpark and MapR

Programming background with Python

Experience with SQL (Relational Databases) doing data modeling, Oracle experience is a plus.

Experience with PowerBI and/or Tableau null

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to

Outstanding long-term contract opportunity! A well-known Financial Services Company is looking for a Big Data Engineer in Chandler, AZ (Hybrid).

Work with the brightest minds at one of the largest financial institutions in the world. This is a long-term contract opportunity that includes a competitive benefit package! Our client has been around for over 150 years and is continuously innovating in today's digital age. If you want to work for a company that is not only a household name, but also truly cares about satisfying customers' financial needs and helping people succeed financially, apply today.

Contract Duration: 24 Months
Required Skills & Experience

• 4+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education.
• Minimum 4 years of hand on experience with
  • Building data pipeline using big-data stack (Hadoop, Hive, pySpark, python)
  • Amazon AWS S3 - Object storage, security, data service integration with S3
  • Data modelling and database design.
  • Job Scheduler - Autosys
  • PowerBI, Dremio
  • Unix/shell scripting, CICD pipeline

• Exposure in GCP cloud data engineering

• Consult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables.
• Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors.
• Contribute to the resolution of moderately complex issues and consult with others to meet Software Engineering deliverables while leveraging solid understanding of the function, policies, procedures, and compliance requirements.
• Collaborate with client personnel in Software Engineering.

• Medical Insurance - Four medical plans to choose from for you and your family
• Dental & Orthodontia Benefits
• Vision Benefits
• Health Savings Account (HSA)
• Health and Dependent Care Flexible Spending Accounts
• Voluntary Life Insurance, Long-Term & Short-Term Disability Insurance
• Hospital Indemnity Insurance
• 401(k) including match with pre and post-tax options
• Paid Sick Time Leave
• Legal and Identity Protection Plans
• Pre-tax Commuter Benefit
• 529 College Saver Plan

• Software Engineering

Big Data Engineer

CHANDLER, Arizona

Hybrid

Contract

$53.56/hr - $60.35/hr

Outstanding long-term contract opportunity! A well-known Financial Services Company is looking for a Big Data Engineer in Chandler, AZ (Hybrid).

Work with the brightest minds at one of the largest financial institutions in the world. This is a long-term contract opportunity that includes a competitive benefit package! Our client has been around for over 150 years and is continuously innovating in today's digital age. If you want to work for a company that is not only a household name, but also truly cares about satisfying customers' financial needs and helping people succeed financially, apply today.

Contract Duration: 24 Months

Required Skills & Experience

• 4+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work or consulting experience, training, military experience, education.

• Minimum 4 years of hand on experience with

• Building data pipeline using big-data stack (Hadoop, Hive, pySpark, python)

• Amazon AWS S3 - Object storage, security, data service integration with S3

• Data modelling and database design.

• Job Scheduler - Autosys

• PowerBI, Dremio

• Unix/shell scripting, CICD pipeline

Desired Skills & Experience

• Exposure in GCP cloud data engineering

What You Will Be Doing

• Consult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables.

• Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors.

• Contribute to the resolution of moderately complex issues and consult with others to meet Software Engineering deliverables while leveraging solid understanding of the function, policies, procedures, and compliance requirements.

• Collaborate with client personnel in Software Engineering.

You will receive the following benefits:

• Medical Insurance - Four medical plans to choose from for you and your family

• Dental & Orthodontia Benefits

• Vision Benefits

• Health Savings Account (HSA)

• Health and Dependent Care Flexible Spending Accounts

• Voluntary Life Insurance, Long-Term & Short-Term Disability Insurance

• Hospital Indemnity Insurance

• 401(k) including match with pre and post-tax options

• Paid Sick Time Leave

• Legal and Identity Protection Plans

• Pre-tax Commuter Benefit

• 529 College Saver Plan

Motion Recruitment Partners (MRP) is an Equal Opportunity Employer. All applicants must be currently authorized to work on a full-time basis in the country for which they are applying, and no sponsorship is currently available. Employment is subject to the successful completion of a pre-employment screening. Accommodation will be provided in all parts of the hiring process as required under MRP's Employment Accommodation policy. Applicants need to make their needs known in advance.

Posted by: Blair Richardson

Specialization:

• Software Engineering

Innova Solutions has a client that is immediately hiring for a Big Data Engineer.

Job title: Big Data Engineer
Position type: Full-time/Contract
Duration: 24 Months
Location: Chandler, AZ (Hybrid)

As a Big Data Engineer, you will:

• Design, develop, and maintain scalable data pipelines using Hadoop, Hive, PySpark, and Python.
• Integrate and manage data services with Amazon AWS S3, ensuring security and compliance.
• Perform data modeling and database design using MySQL or similar relational databases.
• Implement and manage job scheduling usingAutosys.
• Automate data workflows and transformations using Spark, Python, and shell scripting.
• Collaborate with stakeholders to understand data requirements and deliver actionable insights.
• Utilize tools like Power BI and Dremio for data visualization and exploration.

• Experiencein software/data engineering.
• Strong experience with Hadoop, Hive, PySpark, Python
• AWS S3 and data service integration
• Data modeling and relational database design
• Autosys job scheduler
• Unix/Shell scripting
• CI/CD pipelines
• Power BI and Dremio
• Solid understanding of software engineering policies, procedures, and compliance requirements.

• Named One of America's Best Employers for New Grads by Forbes (2024)
• Named One of the Best Companies to Work for by U.S. News & World Report (Private Companies List, 2024-2025)
• One of the Largest IT Staffing Firms in the US - Ranked #3 by Staffing Industry Analysts (SIA, 2024)
• One of the Largest Staffing Firms in the US - Ranked #13 by Staffing Industry Analysts (SIA, 2024; includes Innova Solutions, Volt, & HireGenics)
• Named One of the Top Ten Private Companies in Atlanta by the Atlanta Business Chronicle (2023)
• One of the Largest Certified Minority Business Enterprises (MBEs) in the United States (NMSDC, 2024)
• AWS Advanced Tier Services Partner with 100+ certifications

Headquartered in the United States, TP-Link Systems Inc. is a global provider of reliable networking devices and smart home products, consistently ranked as the world’s top provider of Wi-Fi devices. The company is committed to delivering innovative products that enhance people’s lives through faster, more reliable connectivity. With a commitment to excellence, TP-Link serves customers in over 170 countries and continues to grow its global footprint.

We believe technology changes the world for the better! At TP-Link Systems Inc, we are committed to crafting dependable, high-performance products to connect users worldwide with the wonders of technology.

Embracing professionalism, innovation, excellence, and simplicity, we aim to assist our clients in achieving remarkable global performance and enable consumers to enjoy a seamless, effortless lifestyle.

KEY RESPONSIBILITIES:

• Design, develop, and maintain scalable big data infrastructure and pipelines, including data ingestion, cleansing, transformation, and data warehouse modeling for large-scale datasets.

• Design and maintain vector databases and embedding pipelines to support LLM applications, RAG (Retrieval Augmented Generation) systems, semantic search and agentic capabilities.

• Collaborate with cross-functional teams to deliver reliable, actionable data solutions that support business and product decisions.

• Implement and manage batch and streaming ETL/ELT workflows using distributed data processing frameworks Spark and orchestration tools.

• Participate in data integration and ETL pipeline development, ensuring secure and efficient data processing.

• Investigate system issues, perform troubleshooting, and assist in optimizing data processing workflows.

Requirements

REQUIRED QUALIFICATIONS

• Bachelor’s degree in Computer Science, Information Systems, or related field.

• 3-5 years of hands-on experience in data engineering or big data infrastructure, working with large scale datasets in a production environment.

• Proficiency in Python with experience developing scalable ETL/ELT pipelines or made significant contributions to open source python library

• Ability to work effectively in a team-oriented environment with good communication and problem-solving skills.

PREFERRED QUALIFICATIONS

• Experience with LLM frameworks and libraries (e.g. LangChain, LlamaIndex) is strongly preferred

Benefits

Salary Range: $100,000 - 150,000

• Free snacks and drinks, and provided lunch on Fridays

• Fully paid medical, dental, and vision insurance (partial coverage for dependents)

• Contributions to 401k funds

• Bi-annual reviews, and annual pay increases

• Health and wellness benefits, including free gym membership

• Quarterly team-building events

At TP-Link Systems Inc., we are continually searching for ambitious individuals who are passionate about their work. We believe that diversity fuels innovation, collaboration, and drives our entrepreneurial spirit. As a global company, we highly value diverse perspectives and are committed to cultivating an environment where all voices are heard, respected, and valued. We are dedicated to providing equal employment opportunities to all employees and applicants, and we prohibit discrimination and harassment of any kind based on race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws. Beyond compliance, we strive to create a supportive and growth-oriented workplace for everyone. If you share our passion and connection to this mission, we welcome you to apply and join us in building a vibrant and inclusive team at TP-Link Systems Inc.

Please, no third-party agency inquiries, and we are unable to offer visa sponsorships at this time.

A client of Innova Solutions is looking for a
Big Data/ Java developer

Title: Big Data developer
Position type: Fulltime Contract (w2)
Duration: 12 Months
Location: Charlotte, NC/ Irving, TX/ Phoenix, AZ

As a Big Data Engineer, you would:

• Design and implement large-scale data processing systems using Apache Spark.
• Develop robust, scalable, and high-performance data pipelines
• Integrate and manage data from various sources including MongoDB
• Optimize data workflows for performance, scalability, and reliability.
• Ensure data quality, governance, and security best practices are followed.
• Monitor and troubleshoot production data pipelines and systems.

• Experience in Big data engineering
• Proficiency in Apache Spark (batch and/or streaming).
• Strong programming skills in Java
• Experience with MongoDB

• Named One of America's Best Employers for New Grads by Forbes (2024)
• Named One of the Best Companies to Work For by U.S. News & World Report (Private Companies List, 2024-2025)
• One of the Largest IT Staffing Firms in the US - Ranked #3 by Staffing Industry Analysts (SIA, 2024)
• One of the Largest Staffing Firms in the US - Ranked #13 by Staffing Industry Analysts (SIA, 2024; includes Innova Solutions, Volt, & HireGenics)
• Named One of the Top Ten Private Companies in Atlanta by the Atlanta Business Chronicle (2023)
• One of the Largest Certified Minority Business Enterprises (MBEs) in the United States (NMSDC, 2024)
• AWS Advanced Tier Services Partner with 100+ certifications

Responsibilities:

Vanguard's Trading Analytics and Strategy (TAS) team collaborates with global trading desks to optimize trading strategies, saving millions of dollars for clients annually. The team partners with traders and portfolio managers across various asset classes and mandates, both passive and active, to conduct data-driven analyses and develop tools that shape trading strategies. As a Full Stack Engineer, you will design, implement, and maintain a modern, robust, and scalable platform to meet the increasing demands of the trading desks.

Qualifications:

Programming Skills: Proficiency in Python programming.
Database Expertise: Strong expertise in SQL, Presto, HIVE, and Spark.
Domain Knowledge: Knowledge of trading and investment data.
Big Data Technologies: Experience with Spark and developing distributed computing applications using PySpark.
Data Manipulation: Proficiency with libraries such as Pandas, Polars, and NumPy.
Data Engineering: Understanding of data pipelines, ETL processes, and data warehousing concepts.
API Development: Experience in building APIs.
Testing: Write, maintain, and execute automated unit tests using Python. Follow Test-Driven Development (TDD) practices.
AWS Services: Extensive experience with key AWS services/components including EMR, Lambda, Glue ETL, Step Functions, S3, ECS, Kinesis, IAM, RDS PostgreSQL, DynamoDB, Timeseries database, CloudWatch Events/Event Bridge, Athena, SNS, SQS, and VPC.
Serverless Architectures: Proficiency in developing serverless architectures using AWS services.
Database Skills: Experience with both relational and NoSQL databases.
Data Modeling: Skills in designing and implementing data models, including normalization, denormalization, and schema design.
Data Warehousing: Knowledge of data warehousing solutions like Amazon Redshift.
Analytical Skills: Strong analytical skills with the ability to troubleshoot data issues.
Development Practices: Good understanding of source control, unit testing, test-driven development, and CI/CD.
Code Quality: Ability to write clean, maintainable code and comprehend code written by others.
Communication: Strong communication skills.
Data Visualization: Proficiency in data visualization tools and ability to create visual representations of data, particularly using Tableau.

• Big Data Engineer with MapReduce , spark , hive , sql skillset "Expert in SQL and Data warehousing concepts. Hands -on experience with public cloud data warehouse (GCP, Azure, AWS). GCP certification will be very good to have. MapR experience is must
• Strong Hands on experience with one or more programming languages ( Python or Java).
• Hands-on expertise with application design and software development in Big Data (Spark(Pyspark), HIVE). Experience with CICD pipelines, Automated test frameworks, DevOps and source code management tools (XLR, Jenkins, Git, Maven).
• Strong communication and analytical skills including effective presentation skills.
• Familiarity with Agile & scrum ceremonies. "
• Digital : BigData and Hadoop Ecosystem - MapR
• 6-8 Years of experience