9,175 Spark jobs in the United States
Spark Developer
Posted 28 days ago
Job Viewed
Job Description
About the job Spark Developer
Spark Developer needs Java/Spark developer with over 5 Years of hands on experience
Spark Developer requires:
- Locations; Charlotte, NC; Iselin, NJ
- Java Spark
- AWS, CDP
- Big Data Platform 5 years
- Experience with sqoop
- Experience working with large data set and understands how to tune code for performance.
- Experience working in retirement industry/financial institutions.
- Experience working in cloud (AWS/CDP Cloud)
- Assess the technical viability of emerging products and technologies.
- Work with developers and infrastructure specialists to test and evaluate new technologies.
- Participate in the development of business cases and obtaining approvals for capital expenditures.
- Monitor and analyze new technology product performance and resolving issues regarding potential improvements or modifications to complex situations, as needed.
Apache Spark Developer
Posted 28 days ago
Job Viewed
Job Description
The Candiadte will be responsible for leading technical teams and projects related to apache spark, scala, and python. The role involves overseeing the design, development, and implementation of scalable and efficient solutions using these technologies.
Key Responsibilities
1. Design and implementation of solutions using apache spark, scala, and python
2. Provide technical expertise and guidance to team members in resolving complex technical issues
3. Collaborate with stakeholders to gather requirements and define project scope
4. Ensure adherence to best practices in coding, testing, and deployment processes
5. Conduct code reviews and performance optimization activities
6. Troubleshoot and debug technical issues to ensure seamless project delivery
7. Stay updated with the latest trends and advancements in apache spark, scala, and python technologies
8. Mentor team members and facilitate knowledge sharing within the team
Skill Requirements
1. Strong proficiency in apache spark, scala, and python
2. Experience in leading technical teams and projects
3. Excellent problem-solving skills and ability to think critically
4. Solid understanding of software development life cycle
5. Good communication and interpersonal skills
6. Ability to work effectively in a collaborative team environment
7. Strong organizational and leadership abilities
8. Experience in performance tuning and optimization techniques
9. Knowledge of big data concepts and tools is a plus
Certifications : Relevant certifications in apache spark, scala, or python are a plus
Spark & Scala Developer
Posted 28 days ago
Job Viewed
Job Description
Job Title: Scala and Spark Developer
Job Description:
We are seeking a talented and experienced Scala and Spark Developer to join our dynamic team. As a key member of our development team, you will play a crucial role in designing, implementing, and maintaining scalable and high-performance data processing applications using Scala and Apache Spark.
Responsibilities:
Design, develop, and implement distributed data processing applications using Scala and Apache Spark.
Collaborate with data engineers, data scientists, and other stakeholders to understand and define requirements for data processing tasks.
Optimize and tune Spark applications for performance and efficiency.
Develop and maintain ETL processes for large-scale data pipelines.
Work with large datasets and implement efficient data processing algorithms.
Troubleshoot and resolve issues related to data processing and performance.
Stay abreast of industry trends and advancements in Spark and Scala technologies.
Collaborate with cross-functional teams to integrate Spark applications into existing systems.
Requirements:
Proven experience as a Scala Developer with a focus on Spark development.
Strong programming skills in Scala and a solid understanding of functional programming concepts.
Experience with Apache Spark, including Spark SQL and Spark Streaming.
Proficiency in building and optimizing Spark jobs for large-scale data processing.
Knowledge of data modeling, ETL processes, and data warehousing concepts.
Familiarity with distributed computing and cluster management.
Experience with big data technologies such as Hadoop and Hive is a plus.
Excellent problem-solving and analytical skills.
Strong communication and collaboration skills.
Spark developer (San Jose/CA)
Posted 5 days ago
Job Viewed
Job Description
A career in IBM Software means you'll be part of a team that transforms our customer's challenges into solutions.
Seeking new possibilities and always staying curious, we are a team dedicated to creating the world's leading AI-powered, cloud-native software solutions for our customers. Our renowned legacy creates endless global opportunities for our IBMers, so the door is always open for those who want to grow their career.
We are seeking a skilled Software Developer to join our IBM Software team. As part of our team, you will be responsible for developing and maintaining high-quality software products, working with a variety of technologies and programming languages.
IBM's product and technology landscape includes Research, Software, and Infrastructure. Entering this domain positions you at the heart of IBM, where growth and innovation thrive.
**Your role and responsibilities**
Job Summary:
We are seeking an experienced and highly skilled Spark Scala Developer The candidate will have a deep understanding of distributed computing, data pipelines, and real-time and batch data processing architectures.
Key Responsibilities:
*
Design, develop, and optimize big data applications using Apache Spark and Scala.
*
Architect and implement scalable data pipelines for both batch and real-time processing.
*
Collaborate with data engineers, analysts, and architects to define data strategies.
*
Optimize Spark jobs for performance and cost-effectiveness on distributed clusters.
*
Build and maintain reusable code and libraries for future use.
*
Work with various data storage systems like HDFS, Hive, HBase, Cassandra, Kafka, and Parquet.
*
Implement data quality checks, logging, monitoring, and alerting for ETL jobs.
*
Mentor junior developers and lead code reviews to ensure best practices.
*
Ensure security, governance, and compliance standards are adhered to in all data processes.
*
Troubleshoot and resolve performance issues and bugs in big data solutions.
**Required technical and professional expertise**
*
12+ years of total software development experience.
*
Minimum 5+ years of hands-on experience with Apache Spark and Scala.
*
Strong experience with distributed computing, parallel data processing, and cluster computing frameworks.
*
Proficiency in Scala with deep knowledge of functional programming.
*
Solid understanding of Spark tuning, partitions, joins, broadcast variables, and performance optimization techniques.
*
Experience with cloud platforms such as AWS, Azure, or GCP (especially EMR, Databricks, or HDInsight).
*
Hands-on experience with Kafka, Hive, HBase, NoSQL databases, and data lake architectures.
*
Familiarity with CI/CD pipelines, Git, Jenkins, and automated testing.
*
Strong problem-solving skills and the ability to work independently or as part of a team.
**Preferred technical and professional experience**
*
Exposure to machine learning pipelines using Spark MLlib or integration with ML frameworks.
*
Experience with data governance tools (e.g., Apache Atlas, Collibra).
*
Contributions to open-source big data projects are a plus.
*
Excellent communication and leadership skills.
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Spark developer (San Jose/CA)
Posted 5 days ago
Job Viewed
Job Description
A career in IBM Software means you'll be part of a team that transforms our customer's challenges into solutions.
Seeking new possibilities and always staying curious, we are a team dedicated to creating the world's leading AI-powered, cloud-native software solutions for our customers. Our renowned legacy creates endless global opportunities for our IBMers, so the door is always open for those who want to grow their career.
We are seeking a skilled Software Developer to join our IBM Software team. As part of our team, you will be responsible for developing and maintaining high-quality software products, working with a variety of technologies and programming languages.
IBM's product and technology landscape includes Research, Software, and Infrastructure. Entering this domain positions you at the heart of IBM, where growth and innovation thrive.
**Your role and responsibilities**
We are seeking an experienced and highly skilled Spark Scala Developer.The candidate will have a deep understanding of distributed computing, data pipelines, and real-time and batch data processing architecture.Key Responsibilities:
*
Design, develop, and optimize big data applications using Apache Spark and Scala.
*
Architect and implement scalable data pipelines for both batch and real-time processing.
*
Collaborate with data engineers, analysts, and architects to define data strategies.
*
Optimize Spark jobs for performance and cost-effectiveness on distributed clusters.
*
Build and maintain reusable code and libraries for future use.
*
Work with various data storage systems like HDFS, Hive, HBase, Cassandra, Kafka, and Parquet.
*
Implement data quality checks, logging, monitoring, and alerting for ETL jobs.
*
Mentor junior developers and lead code reviews to ensure best practices.
*
Ensure security, governance, and compliance standards are adhered to in all data processes.
*
Troubleshoot and resolve performance issues and bugs in big data solutions.
**Required technical and professional expertise**
*
12+ years of total software development experience.
*
Minimum 5+ years of hands-on experience with Apache Spark and Scala.
*
Strong experience with distributed computing, parallel data processing, and cluster computing frameworks.
*
Proficiency in Scala with deep knowledge of functional programming.
*
Solid understanding of Spark tuning, partitions, joins, broadcast variables, and performance optimization techniques.
*
Experience with cloud platforms such as AWS, Azure, or GCP (especially EMR, Databricks, or HDInsight).
*
Hands-on experience with Kafka, Hive, HBase, NoSQL databases, and data lake architectures.
*
Familiarity with CI/CD pipelines, Git, Jenkins, and automated testing.
*
Strong problem-solving skills and the ability to work independently or as part of a team.
**Preferred technical and professional experience**
*
Experience with Databricks, Delta Lake, or Apache Iceberg.
*
Exposure to machine learning pipelines using Spark MLlib or integration with ML frameworks.
*
Experience with data governance tools (e.g., Apache Atlas, Collibra).
*
Contributions to open-source big data projects are a plus.
*
Excellent communication and leadership skills.
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
PL/SQL PY Spark Developer
Posted 3 days ago
Job Viewed
Job Description
# Key Responsibilities
**Primary Project (Enterprise Customer Table Pilot)**
- Design and implement data consolidation solutions moving from SQL Server to Databricks data lake
- Work with business stakeholders and cross-functional teams to define enterprise customer table specifications
- Determine optimal approach for data processing - either within existing Oracle systems or in the data lake environment
- Collaborate with enterprise data lake team to leverage existing PySpark resources and infrastructure
- Produce modified data inputs for the new enterprise customer table consolidation process
- Ensure data quality and consistency across three different billing platform feeds
**Secondary Pilot (Code Conversion)**
- Convert existing Oracle/PL/SQL code to PySpark for data lake processing
- Evaluate feasibility of migrating current data warehouse operations to PySpark
- Provide proof-of-concept for future large-scale migration initiatives
- Test and validate converted code performance in the data lake environment
**Team Development & Knowledge
Senior Spark Engineer
Posted 6 days ago
Job Viewed
Job Description
At Superlinked we’re building the backbone of how people and data connect. Our Python framework turns messy, complex data into vector embeddings - the magic that makes Search, RecSys, RAG & Analytics actually work (check our product intro !).
- $9.5M Seed raised from Index Ventures, Theory Ventures + top San Francisco investors
- Founded by ex-Google & McKinsey leaders (our CEO Daniel scaled YouTube’s ads to $10B+)
- Partnered with MongoDB, Redis, Starburst, Dataiku
As our Senior Spark Engineer, you’ll own the batch processing engine of the framework - the part that makes Superlinked’s intelligence possible. You’ll work closely with Balazs Kemenes (Head of Engineering) and the team to build, optimize and scale Spark-based systems for vector computations.
What You’ll Do- Own Spark, PySpark, Dataproc & EMR batch pipelines end-to-end.
- Tune and optimize Spark jobs for speed, stability and cost.
- Build tools for monitoring, debugging and performance.
- Collaborate with engineers to improve abstractions for computation.
- Provide technical leadership on Spark best practices.
- 6-8+ years in software engineering with focus on data-intensive apps.
- Deep Spark/PySpark & Python expertise.
- Hands-on experience with Dataproc, EMR, Databricks.
- Strong grasp of batch processing, DAG execution & big data systems.
- Pragmatic, performance-driven, DevOps mindset.
Nice to have : compiler architectures, GPU acceleration, LLM workflows, or deep IR/vector DB knowledge.
Why Join Us
We’re a small team (22 people across Budapest, UK, Tel Aviv, San Francisco) with no politics and lots of ownership. If you want your work to actually matter, you’ll feel at home here.
Competitive salary + equity
Team offsites (Mallorca, Istanbul, the Alps, Portugal next)
Hybrid setup: 1 office day/week @ Create26, Király utca
And here is the crew that would love to row with you. Join us! :)
Be The First To Know
About the latest Spark Jobs in United States !
Senior java developer with Spark - VP - TAMPA
Posted 12 days ago
Job Viewed
Job Description
The Java Applications Developer with Spark is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.
Responsibilities:
- Lead integration of functions to meet goals, deploy new products, and enhance processes
- Analyze complex business processes, system processes, and industry standards to define and develop solutions to high level problems
- Provide expertise in area of advanced knowledge of applications programming and plan assignments involving large budgets, cross functional project, or multiple projects
- Develop application methodologies and standards for program analysis, design, coding, testing, debugging, and implementation
- Utilize advanced knowledge of supported main system flows and comprehensive knowledge of multiple areas to achieve technology goals
- Consult with end users to identify system function specifications and incorporate into overall system design
- Allocate work, and act as an advisor/coach developers, analysts, and new team members
- Influence and negotiate with senior leaders and communicate with external parties
- Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Qualifications:
- 8 + years of application/software development
- 8 + years of experience with JAVA (Core Java, J2EE, Spring Boot RESTful Services), Python, Web services (REST, SOAP), XML, Java Script, Micro services , SOA etc
- 5 + Years of experience on BigData Technologies like Apache Spark , Hive, Hadoop & Storm.
- Extremely strong with data structure, algorithms and design patterns.
- Knowledge on technologies like ELK , Docker, Kubernetes , Azure Cloud, AWS S3 etc
- Knowledge on NOSQL Databases like MongoDB , Hbase, Cassandra etc.
- Knowledge in version control systems (e.g Git) and CI/CD pipelines.
- Working experience with financial application / Finance processes is a plus
- Extensive years of working in a multi-cultural environment delivering results with virtual teams.
- A well-diversified background with a successful track record of leadership, experience, and performance
- Strong problem-solving skills with the ability to work independently, multi-task, and take ownership of various analyses or reviews
- Traits of "Taking Ownership", "Delivering with Pride" and "Succeeding Together"
- Demonstrated leadership, project management, and development skills
- Relationship and consensus building skills
Education:
- Bachelor’s degree/University degree or equivalent experience
- Master’s degree preferred
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
Job Family Group:
Technology---
Job Family:
Applications Development---
Time Type:
Full time---
Primary Location:
Tampa Florida United States---
Primary Location Full Time Salary Range:
$113,840.00 - $170,760.00
In addition to salary, Citi’s offerings may also include, for eligible employees, discretionary and formulaic incentive and retention awards. Citi offers competitive employee benefits, including: medical, dental & vision coverage; 401(k); life, accident, and disability insurance; and wellness programs. Citi also offers paid time off packages, including planned time off (vacation), unplanned time off (sick leave), and paid holidays. For additional information regarding Citi employee benefits, please visit citibenefits.com. Available offerings may vary by jurisdiction, job level, and date of hire.
---
Most Relevant Skills
Please see the requirements listed above.---
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.---
Anticipated Posting Close Date:
Jul 18, 2025---
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Senior Spark Data Engineer
Posted 1 day ago
Job Viewed
Job Description
At Ascyndent we believe Digital transformation is all about leveraging information and technology to enhance the human experience. It is also about transformative execution. Our teams bring business visions and strategies into reality leveraging technology capabilities.
Our consultants leverage innovative capabilities and deep domain expertise to develop short and long-term strategies aligned with your enterprise goals, skill requirements and technology platforms. As an experienced member of our team, we look first and foremost for people who are passionate around solving business problems through innovation and engineering practices. We embrace a culture of experimentation and constantly strive for improvement and learning. You'll work in a collaborative, trusting, thought-provoking environment-one that encourages diversity of thought and creative solutions that are in the best interests of our customers globally.
This job is for you if…
- You're interested in a meaningful job at a growing technology company where you'll work with a variety of technology solutions and be appreciated for doing your job well.
- You want to help shape the future of emerging technology.
- You enjoy collaborating with highly skilled professionals with a passion for selling.
- You want to join a culture that embraces dynamic individuals with a focus on innovation.
A day in the life…
- As a part of our team, you will be developing innovative, high quality, and robust operational engineering capabilities
- Develop software in our technology stack which is constantly evolving but currently includes Java, Hive, MapReduce, Spark, MAPR-DB, Cloud, RESTful APIs, Unix Shell Script, etc.
- Work with Business partners and with stakeholders to understand functional requirements, architecture dependencies, and business capability roadmaps
- Contribute to technical solution design to meet business requirements
- Taking your place as a core member of an Agile team driving the latest development practices
- Identify and drive reengineering opportunities, and opportunities for adopting new technologies
What we are looking for…
Required
- Experience in EMR and Java is a must.
- Experience in API's
- Experience in writing complex SQL
- Good to have Python
What you'll Love Working Here…
- A career with a company where you will be challenged, respected, and valued.
- The people here do their jobs well, care about one another, care about our customers and exercise good ethics and judgement.
- Our team members health and wellness are our priority as well as rewarding them for their hard work.
- Be a part of a team that values creative solutioning with peers and one that honors experimentation and sharing.
- Be trusted to make decisions that are in the best interests of our clients and our business. We think and act like owners. We care – and that makes all the difference.
- Results fuel our excitement, and we know how our personal accomplishments tie to the success of the company.
- We are courageous and take risks that scare us. Our enthusiasm for experimenting is how we will find the next breakthrough.
Senior Spark Data Engineer
Posted today
Job Viewed
Job Description
At Ascyndent we believe Digital transformation is all about leveraging information and technology to enhance the human experience. It is also about transformative execution. Our teams bring business visions and strategies into reality leveraging technology capabilities.
Our consultants leverage innovative capabilities and deep domain expertise to develop short and long-term strategies aligned with your enterprise goals, skill requirements and technology platforms. As an experienced member of our team, we look first and foremost for people who are passionate around solving business problems through innovation and engineering practices. We embrace a culture of experimentation and constantly strive for improvement and learning. You'll work in a collaborative, trusting, thought-provoking environment-one that encourages diversity of thought and creative solutions that are in the best interests of our customers globally.
This job is for you if…
- You’re interested in a meaningful job at a growing technology company where you’ll work with a variety of technology solutions and be appreciated for doing your job well.
- You want to help shape the future of emerging technology.
- You enjoy collaborating with highly skilled professionals with a passion for selling.
- You want to join a culture that embraces dynamic individuals with a focus on innovation.
A day in the life…
- As a part of our team, you will be developing innovative, high quality, and robust operational engineering capabilities
- Develop software in our technology stack which is constantly evolving but currently includes Java, Hive, MapReduce, Spark, MAPR-DB, Cloud, RESTful APIs, Unix Shell Script, etc.
- Work with Business partners and with stakeholders to understand functional requirements, architecture dependencies, and business capability roadmaps
- Contribute to technical solution design to meet business requirements
- Taking your place as a core member of an Agile team driving the latest development practices
- Identify and drive reengineering opportunities, and opportunities for adopting new technologies
What we are looking for…
Required
- Experience in EMR and Java is a must.
- Experience in API’s
- Experience in writing complex SQL
- Good to have Python
What you’ll Love Working Here…
- A career with a company where you will be challenged, respected, and valued.
- The people here do their jobs well, care about one another, care about our customers and exercise good ethics and judgement.
- Our team members health and wellness are our priority as well as rewarding them for their hard work.
- Be a part of a team that values creative solutioning with peers and one that honors experimentation and sharing.
- Be trusted to make decisions that are in the best interests of our clients and our business. We think and act like owners. We care – and that makes all the difference.
- Results fuel our excitement, and we know how our personal accomplishments tie to the success of the company.
- We are courageous and take risks that scare us. Our enthusiasm for experimenting is how we will find the next breakthrough.