438 Big Data jobs in Sunnyvale

As a Principal Big Data Engineer, you will be an integral member of our data ingestion and processing platform team responsible for architecture, design and development. Having the dynamic ability to adapt to conventional big-data frameworks and tools with the use-cases required by the project Ability to communicate with research and development teams and data scientists, finding bottlenecks and resolving them Design and implement different architectural models for our scalable data processing, as well as scalable data storage Build tools for proper data ingestion from multiple heterogeneous sources
Skills
1) 6+ years of experience in design and implementation in an environment with hundreds of terabytes of data
2) 6+ years of experience with large data processing tools such as: Dataflow, GKE, BQ, Beam
3) 6+ years of experience with Java
4) Can-do attitude on problem solving, quality and ability to execute
5) Excellent inter-personal and teamwork skills

Headquartered in New Jersey (U.S), Cygnus Professionals Inc. is a next-generation global information technology solutions and consulting company, powered by a strong management and leadership team with over 30 person-years of experience. Cygnus has a presence in more than 4 countries with over 25 satisfied customers. We aim to expand across industries and geographies with our industry-focused business excellence. Cygnus Professionals Inc. has been recognized by the US Pan Asian American Chamber of Commerce Education Foundation (USPAACC) as one of the “Fast 100 Asian American Businesses” for its rapid revenue growth over the past two years. Job Description Greetings from Cygnus Professionals, We are currently seeking candidates for the following position. If you are interested, please submit your updated profile, contact details, and rate. Duration: Potential Contract to Hire Mode of Interview: Skype after phone screening Eligibility: Must be a Green Card holder, US Citizen, or hold an EAD. Candidates with other statuses cannot be considered at this time. Position Responsibilities: Experience with Java, Spark, and Hadoop Minimum of 2 years experience with distributed systems Knowledge in distributed system design, data pipelining, and implementation Knowledge of machine learning algorithms Experience building large-scale applications using software design patterns and OO principles Experience with distributed computing (Hadoop/Spark/Cloud) or parallel processing (CUDA/threads/MPI) Expertise in design patterns (UML diagrams) and data modeling for large-scale analytic systems Experience converting raw data into structured data suitable for productization Familiarity with data warehousing, distributed/parallel processing, Hadoop, Cloud technologies, HDFS, and Linux clusters Experience with Agile, Scrum, and SDLC methodologies Ability to work in a research-oriented, fast-paced, technical environment Strong communication and interpersonal skills, quick learner, collaborative attitude Additional Information **U.S. Citizens and authorized work candidates are encouraged to apply. We are unable to sponsor at this time. **All information will be kept confidential according to EEO guidelines. #J-18808-Ljbffr

Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the future of work. To support the active project demands and skills gaps, our staffing experts can help you find the best job for you.

Role:
Location:
Duration:
Required Skills:
Job Description:

For more information and other jobs available please contact our recruitment team at To view all the jobs available in the USA and Asia please visit our website at

Company: Qualcomm India Private Limited Job Area: Engineering Group, Engineering Group > Software Engineering General Summary: As a leading technology innovator, Qualcomm pushes the boundaries of what's possible to enable next-generation experiences and drives digital transformation to help create a smarter, connected future for all. As a Qualcomm Software Engineer, you will design, develop, create, modify, and validate embedded and cloud edge software, applications, and/or specialized utility programs that launch cutting-edge, world class products that meet and exceed customer needs. Qualcomm Software Engineers collaborate with systems, hardware, architecture, test engineers, and other teams to design system-level software solutions and obtain information on performance requirements and interfaces. Minimum Qualifications: Bachelor's degree in Engineering, Information Systems, Computer Science, or related field and 2+ years of Software Engineering or related work experience. OR Master's degree in Engineering, Information Systems, Computer Science, or related field and 1+ year of Software Engineering or related work experience. OR PhD in Engineering, Information Systems, Computer Science, or related field. 2+ years of academic or work experience with Programming Languages such as C, C++, Java, Python, etc. Preferred Qualifications: 3+ years of experience as a Data Engineer or in a similar role Experience with data modeling, data warehousing, and building ETL pipelines Solid working experience with Python, AWS analytical technologies and related resources (Glue, Athena, QuickSight, SageMaker, etc.) Experience with Big Data tools , platforms and architecture with solid working experience with SQL Experience working in a very large data warehousing environment, Distributed System. Solid understanding of various data exchange formats and complexities Industry experience in software development, data engineering, business intelligence, data science, or related field with a track record of manipulating, processing, and extracting value from large datasets Strong data visualization skills Basic understanding of Machine Learning; Prior experience in ML Engineering a plus Ability to manage on-premises data and make it inter-operate with AWS based pipelines Ability to interface with Wireless Systems/SW engineers and understand the Wireless ML domain; Prior experience in Wireless (5G) domain a plus Education: Bachelor's degree in computer science, engineering, mathematics, or a related technical discipline Preferred Qualifications: Masters in CS/ECE with a Data Science / ML Specialization Principal Duties and Responsibilities: Completes assigned coding tasks to specifications on time without significant errors or bugs. Adapts to changes and setbacks in order to manage pressure and meet deadlines. Collaborates with others inside project team to accomplish project objectives. Communicates with project lead to provide status and information about impending obstacles. Quickly resolves complex software issues and bugs. Gathers, integrates, and interprets information specific to a module or sub-block of code from a variety of sources in order to troubleshoot issues and find solutions. Seeks others' opinions and shares own opinions with others about ways in which a problem can be addressed differently. Participates in technical conversations with tech leads/managers. Anticipates and communicates issues with project team to maintain open communication. Makes decisions based on incomplete or changing specifications and obtains adequate resources needed to complete assigned tasks. Prioritizes project deadlines and deliverables with minimal supervision. Resolves straightforward technical issues and escalates more complex technical issues to an appropriate party (e.g., project lead, colleagues). Writes readable code for large features or significant bug fixes to support collaboration with other engineers. Determines which work tasks are most important for self and junior engineers, stays focused, and deals with setbacks in a timely manner. Unit tests own code to verify the stability and functionality of a feature. #J-18808-Ljbffr

1. Strong experience in Big Data , Cloudera Distribution 7.x, RDBMS development
2. 4-5 years of programming experience in Python , Java, Scala, and SQL is must.
3. Strong experience building data pipelines using Hadoop components Sqoop , Hive , SOLR , MR , Impala , Spark , Spark SQL , HBase .
4. Strong experience with REST API development using Python frameworks (Django , Flask, Fast API etc.) and Java Springboot frameworks
5. Project experience in AI/Machine Learning and NLP development.

Objective & Deliverables The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high level analysis, professional reports and presentations, and mentoring, support and training. Technical Knowledge and Skills: 4+ years of hands-on Development, Deployment and production Support experience in Big Data environment. 4-5 years of programming experience in Java, Scala, Python. Proficient in SQL and relational database design and methods for data retrieval. Knowledge of NoSQL systems like HBase or Cassandra Hands-on experience in Cloudera Distribution 6.x Hands-on experience in creating, indexing Solr collections in Solr Cloud environment. Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL. Must have experience with developing Hive QL, UDF's for analyzing semi structured/structured datasets. Must have experience with Spring framework, Web Services, REST API's and MicroServices. Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc. Must have working experience in the data warehousing and Business Intelligence systems. Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs. Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process. Experience in building Client models using MLLib or any Client tools. Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm Experience with Graph Databases like Neo4J, Tiger Graph, Orient DB Agile development methodologies. Top Must Haves: 5+ years of strong hands-on Design and Development experience in Hadoop/Big Data Environment. Strong Hands-on coding experience in Java, Scala and Python languages Strong Hands-on experience with Spring framework, Web Services, REST API's and MicroServices. 3+ years of Hands-on experience building data pipelines using Hadoop components Sqoop, Kafka, Hive, SOLR, Map Reduce, Spark, Spark SQL, HBase etc. #J-18808-Ljbffr

We offer a hybrid work environment. Most US-based positions can alsobeperformed remotely (any exceptions will be noted in the Minimum Qualifications below.)

Our Mission:

To actively connect people to their next great opportunity.

Who We Are:

ZipRecruiter is a leading online employment marketplace. Powered by AI-driven smart matching technology, the company actively connects millions of all-sized businesses and job seekers through innovative mobile, web, and email services, as well as through partnerships with the best job boards on the web. ZipRecruiter has the #1 rated job search app on iOS & Android.

Summary :

Our team has a unique opportunity to work on applications and data at scale, serving millions of jobseekers and tens of thousands of customers. We're working on building an efficient marketplace of jobseekers and employers and need generalist software engineers to build fast, scalable, and effective applications, stream and batch data processing, ML infrastructure and a variety of other systems all to help connect people to their next job. We provide an essential service and have a thriving business as a result.

Our stack is complex and we're looking for engineers who know how to write evolvable, properly instrumented, and efficient code as part of a growing distributed system. We're working on data driven systems and applications and need people who can build fast services and processes to power all of the intelligence we use to help change people's lives.

Responsibilities:

• Build data processing and exploration pipelines along with ML infrastructure to power our intelligence
• Deploy a range of cloud-based technologies for critical projects
• Write, test, instrument, and deploy code to our Kubernetes environment
• Help drive the innovation and evolution of ZipRecruiter

• 5+ years of professional software development experience with a focus on big data technologies
• Experience with Hadoop, Spark, Hive, and/or other big data technologies
• Comprehensive computer science fundamentals in coding, object-oriented programming, data structures, and algorithms
• Experience with containerization technologies like Docker and/or Kubernetes

• 8+ years of professional software development experience, with a focus on big data technologies
• BS/MS/PhD in Mathematics, Computer Science, Physics, related technical field or equivalent practical experience
• Experience with data integration tools like Apache Kafka, Flume, and NiFi
• Familiarity with data storage technologies such as Delta Lake, HBase, Cassandra, and MongoDB
• Familiarity with Apache Hudi, Apache Beam, Apache Flink, Google Cloud Dataflow, Amazon Kinesis Data Analytics, and/or Azure Databricks

• Competitive compensation
• Exceptional benefits package
• Flexible Vacation & Paid Time Off
• Employer-matched 401(k) plan

Data Engineering & Big Data Architect Technology Architect - US Company ITL USA Requisition ID 135930BR Infosys is seeking a highly experienced Data Engineering & Big Data Architect to join our team and perform Technical Product management/Data analyst tasks and own the development of a Tiered data architecture for our data platform . In this role, they would be responsible for crafting and building a comprehensive data architecture that will enable seamless data integration and enable the delivery of high-quality insights to our leadership and business stakeholders. You will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions, lead, and participate in sales and pursuits focused on our clients' business needs. You will be part of a learning culture, where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued. Candidate must be located within commuting distance of Austin, TX/ Sunnyvale, CA or be willing to relocate to the area. This position may require travel to project locations Required Qualifications: Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education. At least 7 years of experience in Information Technology. Experience with Hadoop distributed frameworks while handling large amount of data using Spark or PYSpark and Hadoop Ecosystems. Proven experience in data engineering, data architecture, or a related field Experience with Spark or PySpark is required. Strong understanding of data modeling, data warehousing, and ETL concepts Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop or Spark Experience with Scala or Python is required. Preferred Qualifications: Experience in understanding of Design Patterns, ability to discuss tradeoffs between RDBMS vs Distributed Storage At least 5 years of Experience with Spark or PySpark, Python, Scala and Data Engineering Experience in design and implementing a tiered data architecture that integrates analytics data from multiple sources in an efficient and effective manner. Experience with data orchestration tools like Airflow is a nice to have. Excellent problem-solving and analytical skills, and the ability to work well under tight deadlines. Excellent interpersonal skills and the ability to collaborate effectively with cross-functional teams. Experience in developing data models and mapping rules to transform raw data into actionable insights and reports. Experience in collaborating with the analytics and business teams to understand their requirements, with cross-functional teams to define and implement data governance policies and standards. Experience in developing data validation and reconciliation processes to ensure data quality and accuracy is met. Experience with development and maintenance of user documentation, including data models, mapping rules, and data dictionaries. Estimated annual compensation range for candidate based in the below locations will be: Sunnyvale, CA- $104079 -$75310 Along with competitive pay, as a full-time Infosys employee you are also eligible for the following benefits :- Long-term/Short-term Disability Health and Dependent Care Reimbursement Accounts Insurance (Accident, Critical Illness , Hospital Indemnity, Legal) 401(k) plan and contributions dependent on salary level The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements. Job description Infosys is seeking a highly experienced Data Engineering & Big Data Architect to join our team and perform Technical Product management/Data analyst tasks and own the development of a Tiered data architecture for our data platform . In this role, they would be responsible for crafting and building a comprehensive data architecture that will enable seamless data integration and enable the delivery of high-quality insights to our leadership and business stakeholders. You will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions, lead, and participate in sales and pursuits focused on our clients' business needs. You will be part of a learning culture, where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued. Candidate must be located within commuting distance of Austin, TX/ Sunnyvale, CA or be willing to relocate to the area. This position may require travel to project locations Required Qualifications: Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education. At least 7 years of experience in Information Technology. Experience with Hadoop distributed frameworks while handling large amount of data using Spark or PYSpark and Hadoop Ecosystems. Proven experience in data engineering, data architecture, or a related field Experience with Spark or PySpark is required. Strong understanding of data modeling, data warehousing, and ETL concepts Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop or Spark Experience with Scala or Python is required. Preferred Qualifications: Experience in understanding of Design Patterns, ability to discuss tradeoffs between RDBMS vs Distributed Storage At least 5 years of Experience with Spark or PySpark, Python, Scala and Data Engineering Experience in design and implementing a tiered data architecture that integrates analytics data from multiple sources in an efficient and effective manner. Experience with data orchestration tools like Airflow is a nice to have. Excellent problem-solving and analytical skills, and the ability to work well under tight deadlines. Excellent interpersonal skills and the ability to collaborate effectively with cross-functional teams. Experience in developing data models and mapping rules to transform raw data into actionable insights and reports. Experience in collaborating with the analytics and business teams to understand their requirements, with cross-functional teams to define and implement data governance policies and standards. Experience in developing data validation and reconciliation processes to ensure data quality and accuracy is met. Experience with development and maintenance of user documentation, including data models, mapping rules, and data dictionaries. Estimated annual compensation range for candidate based in the below locations will be: Sunnyvale, CA- $1 4079 - 175310 Along with competitive pay, as a full-time Infosys employee you are also eligible for the following benefits :- Medical/Dental/Vision/Life Insurance Long-term/Short-term Disability Health and Dependent Care Reimbursement Accounts Insurance (Accident, Critical Illness , Hospital Indemnity, Legal) 401(k) plan and contributions dependent on salary level Paid holidays plus Paid Time Off The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements. About Us Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem. Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. #J-18808-Ljbffr

Palo Alto Networks is the next-generation security company maintaining trust in the digital age by helping tens of thousands of organizations worldwide prevent cyber breaches. With our deep cybersecurity expertise, commitment to innovation, and game-changing Next-Generation Security Platform, customers can confidently pursue a digital-first strategy and embark on new technology initiatives, such as cloud and mobility. We have achieved this by having devices on the network, agents on endpoint devices, and services in our cloud to detect and prevent malicious activities. The Threat Intelligence Cloud Team is looking for a SENIOR-LEVEL BIG DATA SOFTWARE ENGINEER responsible for developing and operating this big data distributed system platform. We are looking for an experienced hands-on Developer to be responsible for developing Palo Alto Network's AutoFocus. This is our contextual threat intelligence service that accelerates analysis, correlation, and prevention workflows. Unique, targeted attacks are automatically prioritized with full context, allowing security teams to respond to critical attacks faster, without additional IT security resources. If your passion is Big Data and you want to join us in building our next-generation security platform, then we want to hear from you! KEYS TO SUCCESS FOR THIS POSITION: Experience working on Big Data computing systems like Hadoop MapReduce, Spark, etc. Real-time streaming like Kafka, Spark Streaming. No-SQL databases like HBase, Cassandra, etc., and working with ElasticSearch is a big plus. A successful candidate must have: 8+ years overall Software Development experience and 2+ years in developing large scale big data projects and/or distributed systems Hands-on experience with object-oriented languages like Java or C++ Ability to contribute technically to design and development Experience partnering with dev-ops for all operational aspects of the system Great communication skills and ability to work smoothly with cross-functional teams Hands-on experience working with large scale data ingestion, processing, and storage Master’s degree in computer science or related fields Preferred skills: Hands-on experience with the Hadoop ecosystem, Spark, non-relational databases (NoSQL, MongoDB, Cassandra), messaging systems (Kafka, RabbitMQ). Comfortable building automation systems for quality control, builds, monitoring, and alerting Learn more about Palo Alto Networks HERE and check out our FAST FACTS #LI-PALSAAS #J-18808-Ljbffr