515 Big Data jobs in Sunnyvale

As a Principal Big Data Engineer, you will be an integral member of our data ingestion and processing platform team responsible for architecture, design and development. Having the dynamic ability to adapt to conventional big-data frameworks and tools with the use-cases required by the project Ability to communicate with research and development teams and data scientists, finding bottlenecks and resolving them Design and implement different architectural models for our scalable data processing, as well as scalable data storage Build tools for proper data ingestion from multiple heterogeneous sources
Skills
1) 6+ years of experience in design and implementation in an environment with hundreds of terabytes of data
2) 6+ years of experience with large data processing tools such as: Dataflow, GKE, BQ, Beam
3) 6+ years of experience with Java
4) Can-do attitude on problem solving, quality and ability to execute
5) Excellent inter-personal and teamwork skills

Headquartered in New Jersey (U.S), Cygnus Professionals Inc. is a next-generation global information technology solutions and consulting company, powered by a strong management and leadership team with over 30 person-years of experience. Cygnus has a presence in more than 4 countries with over 25 satisfied customers. We aim to expand across industries and geographies with our industry-focused business excellence. Cygnus Professionals Inc. has been recognized by the US Pan Asian American Chamber of Commerce Education Foundation (USPAACC) as one of the “Fast 100 Asian American Businesses” for its rapid revenue growth over the past two years. Job Description Greetings from Cygnus Professionals, We are currently seeking candidates for the following position. If you are interested, please submit your updated profile, contact details, and rate. Duration: Potential Contract to Hire Mode of Interview: Skype after phone screening Eligibility: Must be a Green Card holder, US Citizen, or hold an EAD. Candidates with other statuses cannot be considered at this time. Position Responsibilities: Experience with Java, Spark, and Hadoop Minimum of 2 years experience with distributed systems Knowledge in distributed system design, data pipelining, and implementation Knowledge of machine learning algorithms Experience building large-scale applications using software design patterns and OO principles Experience with distributed computing (Hadoop/Spark/Cloud) or parallel processing (CUDA/threads/MPI) Expertise in design patterns (UML diagrams) and data modeling for large-scale analytic systems Experience converting raw data into structured data suitable for productization Familiarity with data warehousing, distributed/parallel processing, Hadoop, Cloud technologies, HDFS, and Linux clusters Experience with Agile, Scrum, and SDLC methodologies Ability to work in a research-oriented, fast-paced, technical environment Strong communication and interpersonal skills, quick learner, collaborative attitude Additional Information **U.S. Citizens and authorized work candidates are encouraged to apply. We are unable to sponsor at this time. **All information will be kept confidential according to EEO guidelines. #J-18808-Ljbffr

Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the future of work. To support the active project demands and skills gaps, our staffing experts can help you find the best job for you.

Role:
Location:
Duration:
Required Skills:
Job Description:

For more information and other jobs available please contact our recruitment team at To view all the jobs available in the USA and Asia please visit our website at

Company: Qualcomm India Private Limited Job Area: Engineering Group, Engineering Group > Software Engineering General Summary: As a leading technology innovator, Qualcomm pushes the boundaries of what's possible to enable next-generation experiences and drives digital transformation to help create a smarter, connected future for all. As a Qualcomm Software Engineer, you will design, develop, create, modify, and validate embedded and cloud edge software, applications, and/or specialized utility programs that launch cutting-edge, world class products that meet and exceed customer needs. Qualcomm Software Engineers collaborate with systems, hardware, architecture, test engineers, and other teams to design system-level software solutions and obtain information on performance requirements and interfaces. Minimum Qualifications: Bachelor's degree in Engineering, Information Systems, Computer Science, or related field and 2+ years of Software Engineering or related work experience. OR Master's degree in Engineering, Information Systems, Computer Science, or related field and 1+ year of Software Engineering or related work experience. OR PhD in Engineering, Information Systems, Computer Science, or related field. 2+ years of academic or work experience with Programming Languages such as C, C++, Java, Python, etc. Preferred Qualifications: 3+ years of experience as a Data Engineer or in a similar role Experience with data modeling, data warehousing, and building ETL pipelines Solid working experience with Python, AWS analytical technologies and related resources (Glue, Athena, QuickSight, SageMaker, etc.) Experience with Big Data tools , platforms and architecture with solid working experience with SQL Experience working in a very large data warehousing environment, Distributed System. Solid understanding of various data exchange formats and complexities Industry experience in software development, data engineering, business intelligence, data science, or related field with a track record of manipulating, processing, and extracting value from large datasets Strong data visualization skills Basic understanding of Machine Learning; Prior experience in ML Engineering a plus Ability to manage on-premises data and make it inter-operate with AWS based pipelines Ability to interface with Wireless Systems/SW engineers and understand the Wireless ML domain; Prior experience in Wireless (5G) domain a plus Education: Bachelor's degree in computer science, engineering, mathematics, or a related technical discipline Preferred Qualifications: Masters in CS/ECE with a Data Science / ML Specialization Principal Duties and Responsibilities: Completes assigned coding tasks to specifications on time without significant errors or bugs. Adapts to changes and setbacks in order to manage pressure and meet deadlines. Collaborates with others inside project team to accomplish project objectives. Communicates with project lead to provide status and information about impending obstacles. Quickly resolves complex software issues and bugs. Gathers, integrates, and interprets information specific to a module or sub-block of code from a variety of sources in order to troubleshoot issues and find solutions. Seeks others' opinions and shares own opinions with others about ways in which a problem can be addressed differently. Participates in technical conversations with tech leads/managers. Anticipates and communicates issues with project team to maintain open communication. Makes decisions based on incomplete or changing specifications and obtains adequate resources needed to complete assigned tasks. Prioritizes project deadlines and deliverables with minimal supervision. Resolves straightforward technical issues and escalates more complex technical issues to an appropriate party (e.g., project lead, colleagues). Writes readable code for large features or significant bug fixes to support collaboration with other engineers. Determines which work tasks are most important for self and junior engineers, stays focused, and deals with setbacks in a timely manner. Unit tests own code to verify the stability and functionality of a feature. #J-18808-Ljbffr

Objective & Deliverables The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high level analysis, professional reports and presentations, and mentoring, support and training. Technical Knowledge and Skills: 4+ years of hands-on Development, Deployment and production Support experience in Big Data environment. 4-5 years of programming experience in Java, Scala, Python. Proficient in SQL and relational database design and methods for data retrieval. Knowledge of NoSQL systems like HBase or Cassandra Hands-on experience in Cloudera Distribution 6.x Hands-on experience in creating, indexing Solr collections in Solr Cloud environment. Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL. Must have experience with developing Hive QL, UDF's for analyzing semi structured/structured datasets. Must have experience with Spring framework, Web Services, REST API's and MicroServices. Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc. Must have working experience in the data warehousing and Business Intelligence systems. Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs. Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process. Experience in building Client models using MLLib or any Client tools. Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm Experience with Graph Databases like Neo4J, Tiger Graph, Orient DB Agile development methodologies. Top Must Haves: 5+ years of strong hands-on Design and Development experience in Hadoop/Big Data Environment. Strong Hands-on coding experience in Java, Scala and Python languages Strong Hands-on experience with Spring framework, Web Services, REST API's and MicroServices. 3+ years of Hands-on experience building data pipelines using Hadoop components Sqoop, Kafka, Hive, SOLR, Map Reduce, Spark, Spark SQL, HBase etc. #J-18808-Ljbffr

We offer a hybrid work environment. Most US-based positions can alsobeperformed remotely (any exceptions will be noted in the Minimum Qualifications below.)

Our Mission:

To actively connect people to their next great opportunity.

Who We Are:

ZipRecruiter is a leading online employment marketplace. Powered by AI-driven smart matching technology, the company actively connects millions of all-sized businesses and job seekers through innovative mobile, web, and email services, as well as through partnerships with the best job boards on the web. ZipRecruiter has the #1 rated job search app on iOS & Android.

Summary :

Our team has a unique opportunity to work on applications and data at scale, serving millions of jobseekers and tens of thousands of customers. We're working on building an efficient marketplace of jobseekers and employers and need generalist software engineers to build fast, scalable, and effective applications, stream and batch data processing, ML infrastructure and a variety of other systems all to help connect people to their next job. We provide an essential service and have a thriving business as a result.

Our stack is complex and we're looking for engineers who know how to write evolvable, properly instrumented, and efficient code as part of a growing distributed system. We're working on data driven systems and applications and need people who can build fast services and processes to power all of the intelligence we use to help change people's lives.

Responsibilities:

• Build data processing and exploration pipelines along with ML infrastructure to power our intelligence
• Deploy a range of cloud-based technologies for critical projects
• Write, test, instrument, and deploy code to our Kubernetes environment
• Help drive the innovation and evolution of ZipRecruiter

• 5+ years of professional software development experience with a focus on big data technologies
• Experience with Hadoop, Spark, Hive, and/or other big data technologies
• Comprehensive computer science fundamentals in coding, object-oriented programming, data structures, and algorithms
• Experience with containerization technologies like Docker and/or Kubernetes

• 8+ years of professional software development experience, with a focus on big data technologies
• BS/MS/PhD in Mathematics, Computer Science, Physics, related technical field or equivalent practical experience
• Experience with data integration tools like Apache Kafka, Flume, and NiFi
• Familiarity with data storage technologies such as Delta Lake, HBase, Cassandra, and MongoDB
• Familiarity with Apache Hudi, Apache Beam, Apache Flink, Google Cloud Dataflow, Amazon Kinesis Data Analytics, and/or Azure Databricks

• Competitive compensation
• Exceptional benefits package
• Flexible Vacation & Paid Time Off
• Employer-matched 401(k) plan

Palo Alto Networks is the next-generation security company maintaining trust in the digital age by helping tens of thousands of organizations worldwide prevent cyber breaches. With our deep cybersecurity expertise, commitment to innovation, and game-changing Next-Generation Security Platform, customers can confidently pursue a digital-first strategy and embark on new technology initiatives, such as cloud and mobility. We have achieved this by having devices on the network, agents on endpoint devices, and services in our cloud to detect and prevent malicious activities. The Threat Intelligence Cloud Team is looking for a SENIOR-LEVEL BIG DATA SOFTWARE ENGINEER responsible for developing and operating this big data distributed system platform. We are looking for an experienced hands-on Developer to be responsible for developing Palo Alto Network's AutoFocus. This is our contextual threat intelligence service that accelerates analysis, correlation, and prevention workflows. Unique, targeted attacks are automatically prioritized with full context, allowing security teams to respond to critical attacks faster, without additional IT security resources. If your passion is Big Data and you want to join us in building our next-generation security platform, then we want to hear from you! KEYS TO SUCCESS FOR THIS POSITION: Experience working on Big Data computing systems like Hadoop MapReduce, Spark, etc. Real-time streaming like Kafka, Spark Streaming. No-SQL databases like HBase, Cassandra, etc., and working with ElasticSearch is a big plus. A successful candidate must have: 8+ years overall Software Development experience and 2+ years in developing large scale big data projects and/or distributed systems Hands-on experience with object-oriented languages like Java or C++ Ability to contribute technically to design and development Experience partnering with dev-ops for all operational aspects of the system Great communication skills and ability to work smoothly with cross-functional teams Hands-on experience working with large scale data ingestion, processing, and storage Master’s degree in computer science or related fields Preferred skills: Hands-on experience with the Hadoop ecosystem, Spark, non-relational databases (NoSQL, MongoDB, Cassandra), messaging systems (Kafka, RabbitMQ). Comfortable building automation systems for quality control, builds, monitoring, and alerting Learn more about Palo Alto Networks HERE and check out our FAST FACTS #LI-PALSAAS #J-18808-Ljbffr

Principal Staff Software Engineer, Big Data Pipelines & Platform Infrastructure Principal Staff Software Engineer, Big Data Pipelines & Platform Infrastructure Get AI-powered advice on this job and more exclusive features. Direct message the job poster from LinkedIn LinkedIn was built to help professionals achieve more in their careers, and every day, millions of people use our products to connect, discover opportunities, and gain insights. Our global reach allows us to make a direct impact on the world’s workforce in ways no other company can. We’re much more than a digital resume—we transform lives through innovative products and technology. Searching for your next big opportunity? At LinkedIn, we strive to help our employees find passion and purpose. Join us in changing the way the world works. We believe that leadership is about fostering growth, maintaining balance, and achieving results. Our leaders value their craft, inspire their teams, and set the standard for high-quality work. They drive the strategy and implementation of critical initiatives, motivating and developing their teams to make a lasting impact on our business and community. Job Description At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The work location of this role is hybrid, meaning it will be performed both from home and from a LinkedIn office on select days, as determined by the business needs of the team. We are looking for a Principal-level Technical Lead for the Big Data Platform infrastructure organization in LinkedIn. The candidate would be the organizational leader’s technical partner in building end-to-end solutions for the platform. The organization includes orchestrators, data processing query engines, streams compute, data warehouse, and catalog. The candidate should have proven experience in most of these technologies so that the person can help design and lead end-to-end solutions that span the entire platform. This would include partnering with the technical leads to help build solutions in the following areas: Building a seamless managed data experience for the platform users Strategize and help build a single multipurpose query engine that spans both batch and streams compute. Strategizing on the SQL-first strategy for infrastructure. Building an observable platform that has true end-to-end error handling. The candidate is expected to work with partners and customer stakeholders to strategize and lead solutions along with a deep hands-on approach to problem-solving. The candidate should have demonstrated strong leadership skills with a focus on customer experience and quality. Basic Qualifications: BS/BA in Computer Science or related technical field or equivalent technical experience 10+ years of experience in software design, building solutions Preferred Qualifications: 12+ years' experience in software design 5+ of those years in a technical leadership position Experience with big data platform infrastructure Experience leading and influencing within large, matrixed organization Experience leading high-impact, cross-organization initiatives. "Suggested skills" - big data platform infrastructure - Developing technology strategies at a company-wide scale. - Technical leadership LinkedIn is committed to fair and equitable compensation practices. The pay range for this role is $207,000 to $40,000. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to skill set, depth of experience, certifications, and specific work location. This may be different in other locations due to differences in the cost of labor. The total compensation package for this position may also include annual performance bonus, stock, benefits and/or other applicable incentive compensation plans. For more information, visit Equal Opportunity Statement We seek candidates with a wide range of perspectives and backgrounds and we are proud to be an equal opportunity employer. LinkedIn considers qualified applicants without regard to race, color, religion, creed, gender, national origin, age, disability, veteran status, marital status, pregnancy, sex, gender expression or identity, sexual orientation, citizenship, or any other legally protected class. LinkedIn is committed to offering an inclusive and accessible experience for all job seekers, including individuals with disabilities. Our goal is to foster an inclusive and accessible workplace where everyone has the opportunity to be successful. If you need a reasonable accommodation to search for a job opening, apply for a position, or participate in the interview process, connect with us at and describe the specific accommodation requested for a disability-related limitation. Reasonable accommodations are modifications or adjustments to the application or hiring process that would enable you to fully participate in that process. Examples of reasonable accommodations include but are not limited to: -Documents in alternate formats or read aloud to you -Having interviews in an accessible location -Being accompanied by a service dog -Having a sign language interpreter present for the interview A request for an accommodation will be responded to within three business days. However, non-disability related requests, such as following up on an application, will not receive a response. LinkedIn will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by LinkedIn, or (c) consistent with LinkedIn's legal duty to furnish information. Pay Transparency Policy Statement As a federal contractor, LinkedIn follows the Pay Transparency and non-discrimination provisions described at this link: Global Data Privacy Notice for Job Candidates This document provides transparency around the way in which LinkedIn handles personal data of employees and job applicants: Seniority level Seniority level Not Applicable Employment type Employment type Full-time Job function Industries Technology, Information and Internet and IT Services and IT Consulting Referrals increase your chances of interviewing at LinkedIn by 2x Sign in to set job alerts for “Principal Staff Software Engineer” roles. Software Engineer, AI Intern (Fall 2025) San Francisco Bay Area $57. 0- 61.00 20 hours ago Software Engineer, AI Intern (Summer 2025) Mountain View, CA 125,400.00- 188,100.00 2 weeks ago Software Engineer, AI Platform - New Grad Software Engineer (L4), Content & Business Products New Grads 2025 - Software Engineer, Algorithm San Jose, CA 120,000.00- 165,000.00 9 months ago New Grads 2025 - General Software Engineer San Jose, CA 120,000.00- 165,000.00 4 months ago San Jose, CA 112,000.00- 126,000.00 36 minutes ago Software Engineer(s) - New Grad (Fall 2025 Graduation) Sunnyvale, CA 167,000.00- 185,500.00 1 day ago Software Engineer 4 - TV & Web Player Platform Software Engineer I (Intern) United States San Jose, CA 44,000.00- 130,000.00 2 days ago San Jose, CA 100,500.00- 173,250.00 22 hours ago Full Stack Software Engineer - Post-training Palo Alto, CA 180,000.00- 440,000.00 2 weeks ago San Jose, CA 133,900.00- 242,000.00 1 week ago Menlo Park, CA 56.25- 173,000.00 4 hours ago San Jose, CA 113,400.00- 206,300.00 2 weeks ago Full Stack Software Engineer (L4), Product Localization Engineering Software Engineer - Intern (Summer 2025) San Jose, CA 3,000.00- 4,000.00 8 months ago Santa Clara, CA 150,000.00- 175,000.00 7 months ago Software Engineer(s) - New Grad (Fall 2025 Graduation) New College Grad Software Engineer, Software Engineering Development (Apps) San Jose, CA 92,735.00- 131,300.00 1 week ago Frontend Software Engineer - University Graduate 2025 San Mateo, CA 120,000.00- 280,000.00 2 weeks ago We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr