21,610 AI Infrastructure jobs in the United States

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.

Headquartered in Chicago with offices throughout the U.S., Canada, Europe, and Asia, we trade a variety of asset classes including Fixed Income, ETFs, Equities, FX, Commodities and Energy across all major global markets. We have also leveraged our expertise and technology to expand into three non-traditional strategies: real estate, venture capital and cryptoassets.

We operate with respect, curiosity and open minds. The people who thrive here share our belief that its not just what we do that mattersit's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus.

As anAI Infrastructure Engineer at DRW, you will be an integral member of a collaborative research team solving the financial markets using machine learning. Youll work on high-impact machine learning (ML) and artificial intelligence (AI) projects central to our core business. In this role, you will build, maintain and optimize training and inference infrastructure to support researcher to build AI models for financial markets and discover innovative methods to challenging data and machine learning technical problems.

Key Responsibilities:

• Drive end-to-end development of data and AI infrastructure: from initial proof-of-concept to production deployment and ongoing maintenance.
• Provide technical leadership in selecting, integrating, and optimizing AI and ML frameworks, libraries, and tools across diverse hardware and software environments.
• Maintain, and optimize training infra stack, including data pipeline, GPU utilization, monitoring, and observability.
• Proactively troubleshoot performance bottlenecks, conduct root-cause analyses, and implement solutions to optimize GPU or CPU resource usage for both training and inference.
• Design and implement strategies for efficient data movement between storage and GPUs, ensuring high throughput and low latency.
• Develop and maintain high-performance data loading and preprocessing pipelines that maximize GPU utilization.
• Optimize data access patterns and memory management to improve the efficiency of large dataset processing.
• Architect solutions for handling vast volumes of data, ensuring scalability and performance.

Qualifications:

• 3+ years with demonstrated experience in optimizing data movement and processing for GPU-based systems.
• Expertise in GPU memory management and data transfer optimization.
• Experience with GPU-accelerated libraries like RAPIDS
• Skills in developing high-performance data loading and preprocessing pipelines with tools like DALI.
• Skills in profiling and optimizing GPU code using tools like NVIDIA Nsight and nvprof.
• Knowledge of distributed computing frameworks and multi-GPU setups.
• Knowledge of distributed training frameworks like DeepSpeed. Prior experience in scaling neural network training and multi-GPU experiments is preferred.
• Some proficiency in CUDA/Triton programming and CUDA kernels optimization is preferred.
• Proficient in problem-solving and analytical reasoning.
• Exceptional communication and collaboration skills.

The annual base salary range for this position is $130,000 to $200,000, depending on the candidates experience, qualifications, and relevant skill set. The position is also eligible for an annual discretionary bonus. In addition, DRW offers a comprehensive suite of employee benefits including group medical, pharmacy, dental and vision insurance, 401k (with discretionary employer match), short and long-term disability, life and AD&D insurance, health savings accounts, and flexible spending accounts.

For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at .

• Design, build, and maintain the AI infrastructure and systems needed to support our AI applications. Examples include
• AI workflow scheduling system to improve GPU efficiency and throughput of our batch inference systems
• Model optimization to improve inference performance
• Auto Train systems to power our avatar models
• Large scale model evaluation systems
• Online model serving systems
• Collaborate with data scientists and machine learning engineers to understand their computational and data needs and provide efficient solutions.
• Stay up-to-date with the latest industry trends in AI infrastructure technologies and advocate for best practices and continuous improvement.
• Assist in budget planning and management of cloud resources and other infrastructure expenses.

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 5+ years of experience
• Proven experience in managing infrastructure for large-scale AI or machine learning projects
• Excellent problem-solving skills and the ability to work independently or as part of a team.
• Proficiency in Python and C++
• Experience with GPU computing and optimizing computational workflows
• Familiarity with AI and machine learning frameworks like TensorFlow or PyTorch.

• Experience with CUDA
• Experience optimizing large deep learning model performance
• Experience building large scale batch inference system
• Prior experience in a startup or fast-paced tech environment.

• Competitive salary and benefits package.
• Dynamic and inclusive work environment.
• Opportunities for professional growth and advancement.
• Collaborative culture that values innovation and creativity.
• Access to the latest technologies and tools.

As a Software Engineer on the ML Infrastructure team, you will design and build the platform for our agent sandboxing platform: the secure, high-performance code execution layer powering our agentic workflows. This system underpins critical applications and research initiatives, and is deployed across both internal and customer-managed environments.

This position requires deep expertise in systems engineering: operating systems, virtualization, networking, containers, and performance optimization. Your work will directly enable agents to execute untrusted or user-submitted code safely, efficiently, and repeatedly, and with fast startup times, strong isolation guarantees, and support for snapshotting and inspection.
You will:

• Design and build the sandboxing platform for code execution across containerized and virtualized environments.
• Ensure strong isolation, security, and reproducibility of execution across user sessions and workloads.
• Optimize for cold-start latency, memory footprint, and resource utilization at scale.
• Collaborate across security, infra, and product teams to support both internal research use cases and enterprise customer deployments.
• Lead architecture reviews and own projects from design through deployment in fast-paced, cross-functional settings.

• 3+ years of experience building high-performance systems software (e.g. OS, container runtime, VMM, networking stack).
• Deep understanding of Linux internals, process isolation, memory management, cgroups, namespaces, etc.
• Experience with containerization and virtualization technologies (e.g., Docker, Firecracker, gVisor, QEMU, Kata Containers).
• Proficiency in a systems programming language such as Go, Rust, or C/C++.
• Familiarity with networking, security hardening, sandboxing techniques, and kernel-level performance tuning.
• Comfort working across infrastructure layers, from kernel modules to orchestration frameworks (e.g., Kubernetes).
• Strong debugging skills and the ability to make performance/security tradeoffs in production systems.

• Familiarity with LLM agents and agent frameworks (e.g., OpenHands, Agent2Agent, MCP).
• Experience running secure workloads in multi-tenant or untrusted environments (e.g., FaaS, CI sandboxes, remote notebooks).
• Exposure to snapshotting and restore techniques (e.g., CRIU, VM snapshots, overlayfs).

• Design and build training, inference, and evaluation infrastructure to support our current autonomy stack development, orchestrating massive GPU clusters to process petabytes of multimodal sensor data
• Optimize multimodal data ingestion and preprocessing pipelines (LiDAR, camera, radar, map priors) to support cutting-edge perception and planning model development
• Work across cloud environments to support high-throughput distributed training
• Collaborate closely with the AI research team and autonomy teams
• Technologies: Pytorch, CUDA, Ray, Flyte, K8s

• Experience with building software components to address production, full-stack machine learning challenges.
• Opinions about building a company-wide platform for ML training, evaluation, and deployment
• Knowledge of the open source landscape with judgment on when to choose open source versus build in-house
• Excellent analytical and problem-solving skills

A stealth-stage AI infrastructure company is building a self-healing system for software that automates defect resolution and development. The platform is used by engineering and support teams to:

• Autonomously debug problems in production software
• Fix issues directly in the codebase
• Prevent recurring issues through intelligent root-cause automation

• Own mission-critical services end-to-end - from architecture and design reviews to deployment, observability, and service-level objectives.
• Scale LLM-driven systems : build RAG pipelines, vector indexes, and evaluation frameworks handling billions of events per day.
• Design data-heavy backends : streaming ETL, columnar storage, time-series analytics - all fueling the self-healing loop.
• Optimize for cost and latency across compute types (CPUs, GPUs, serverless); profile hot paths and squeeze out milliseconds.
• Drive reliability : implement automated testing, chaos engineering, and progressive rollout strategies for new models.
• Work cross-functionally with ML researchers, product engineers, and real customers to build infrastructure that actually matters.

• Have 2-5+ years of experience building scalable backend or infra systems in production environments
• Bring a builder mindset - you like owning projects end-to-end and thinking deeply about data, scale, and maintainability
• Have transitioned ML or data-heavy prototypes to production , balancing speed and robustness
• Are comfortable with data engineering workflows : parsing, transforming, indexing, and querying structured or unstructured data
• Have some exposure to search infrastructure or LLM-backed systems (e.g., document retrieval, RAG, semantic search)

• Experience with vector databases (e.g., pgvector, Pinecone, Weaviate) or inverted-index search (e.g., Elasticsearch, Lucene)
• Hands-on with GPU orchestration (Kubernetes, Ray, KServe) or model-parallel inference tuning
• Familiarity with Go / Rust (primary stack), with some TypeScript for light full-stack tasks
• Deep knowledge of observability tooling (OpenTelemetry, Grafana, Datadog) and profiling distributed systems
• Contributions to open-source ML or systems infrastructure projects

Get to know Okta

Okta is The World's Identity Company. We free everyone to safely use any technology, anywhere, on any device or app. Our flexible and neutral products, Okta Platform and Auth0 Platform, provide secure access, authentication, and automation, placing identity at the core of business security and growth.

At Okta, we celebrate a variety of perspectives and experiences. We are not looking for someone who checks every single box - we're looking for lifelong learners and people who can make us better with their unique experiences.

Join our team! We're building a world where Identity belongs to you.

About the Role

We are looking for a smart and versatile AI Infrastructure Architect to build and evolve the AI infrastructure and platform that powers our identity security solutions. Your work will enable internal teams and product groups to integrate AI capabilities safely, securely, and at scale-empowering Okta's mission to protect millions of digital identities worldwide. While your primary focus will be to architect scalable, secure, and resilient infrastructure supporting AI-driven tools, frameworks, and identity services, we value someone who isn't afraid to get hands-on when needed to help solve complex challenges and drive projects forward.

• Lead AI enablement initiatives, including proof-of-concepts for emerging AI infrastructure technologies and integration approaches.
• Collaborate cross-functionally with engineering, security, data science, and product teams to align AI platform architecture with business and security goals.
• Architect scalable, resilient, and secure AI infrastructure that supports AI-powered tools and features across Okta's Identity Platform.
• Lead infrastructure decisions across AWS, GCP, or hybrid environments with a focus on secure identity data handling
• Develop and maintain infrastructure-as-code frameworks (e.g., Terraform, Helm) to ensure consistent, reproducible deployment of AI services
• Champion security and compliance by embedding data privacy and identity protection standards directly into the AI platform and infrastructure design.
• Serve as the key advocate and strategist for AI-driven efficiency initiatives across infrastructure platform teams and pre-production systems.
• Implement robust MLOps practices, such as model evaluation, rollback strategies, and A/B testing, to guarantee the reliability and governance of AI in production.
• Drive continuous innovation by staying current with AI and cloud infrastructure trends and evangelizing best practices internally.

Required

• 10+ years in infrastructure or software engineering, with 2 years building AI/ML systems
• Exceptional systems level thinking and a track record in architecting and building enterprise grade infrastructure
• Deep expertise in cloud platforms (AWS, GCP), distributed systems, and container orchestration (Kubernetes)
• Expected to be very hands-on in order to create, review, and contribute large chunks of quality code

Preferred

• Experience in identity, security, fraud, or risk analytics domains.
• Experience operationalizing large language models or foundation models in production environments.
• Contributions to MLOps or infrastructure open-source projects.

• Opportunity to lead infrastructure shaping AI systems that protect millions of identity transactions.
• Be at the core of building efficient and AI powered enterprise grade solutions that touch internal and external customers alike.
  
  

#LI-TM

#LI-Hybrid

Below is the annual base salary range for candidates located in California. Your actual base salary will depend on factors such as your skills, qualifications, experience, and work location. In addition, Okta offers equity (where applicable), bonus, and benefits, including health, dental and vision insurance, 401(k), flexible spending account, and paid leave (including PTO and parental leave) in accordance with our applicable plans and policies. To learn more about our Total Rewards program please visit:

What you can look forward to as a Full-Time Okta employee!

• Amazing Benefits
• Making Social Impact
• Developing Talent and Fostering Connection + Community at Okta

Okta cultivates a dynamic work environment, providing the best tools, technology and benefits to empower our employees to work productively in a setting that best and uniquely suits their needs. Each organization is unique in the degree of flexibility and mobility in which they work so that all employees are enabled to be their most creative and successful versions of themselves, regardless of where they live. Find your place at Okta today!

Some roles may require travel to one of our office locations for in-person onboarding.

Okta is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, marital status, age, physical or mental disability, or status as a protected veteran. We also consider for employment qualified applicants with arrest and convictions records, consistent with applicable laws.

If reasonable accommodation is needed to complete any part of the job application, interview process, or onboarding please use this Form to request an accommodation.

Okta is committed to complying with applicable data privacy and security laws and regulations. For more information, please see our Privacy Policy at

• Design, build, and manage scalable and efficient cloud-based (AWS, Azure, GCP) and on-premise infrastructure for AI/ML model training and deployment.
• Optimize hardware and software configurations for AI workloads, including GPU clusters, high-performance computing (HPC) environments, and distributed storage solutions.
• Implement and maintain MLOps pipelines, CI/CD processes for AI models, and automated deployment strategies.
• Monitor system performance, identify bottlenecks, and implement solutions to enhance efficiency and reduce latency.
• Ensure the security and compliance of AI infrastructure, implementing best practices for data protection and access control.
• Collaborate with data scientists, ML engineers, and software developers to understand their infrastructure needs and provide tailored solutions.
• Stay abreast of the latest advancements in AI infrastructure technologies, tools, and methodologies.
• Develop and maintain comprehensive documentation for infrastructure architecture, configurations, and operational procedures.
• Troubleshoot complex infrastructure issues, providing timely resolution to minimize downtime.
• Manage containerization technologies (Docker, Kubernetes) for AI model deployment and orchestration.

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Proven experience in designing, deploying, and managing large-scale infrastructure, with a focus on AI/ML workloads.
• Deep understanding of cloud platforms (AWS, Azure, GCP) and their services relevant to AI/ML.
• Expertise in infrastructure as code (IaC) tools such as Terraform or Ansible.
• Proficiency in containerization and orchestration technologies (Docker, Kubernetes).
• Strong knowledge of networking principles, storage solutions, and security best practices.
• Experience with MLOps tools and frameworks (e.g., Kubeflow, MLflow).
• Proficiency in scripting languages like Python or Bash.
• Excellent analytical and problem-solving skills.
• Strong communication and collaboration abilities, with the capacity to explain complex technical concepts to diverse audiences.

Meet the Team

We are an innovation team on a mission to transform how enterprises harness AI. Operating with the agility of a startup and the focus of an incubator, we're building a tight-knit group of AI and infrastructure experts driven by bold ideas and a shared goal: to rethink systems from the ground up and deliver breakthrough solutions that redefine what's possible - faster, leaner, and smarter.
We thrive in a fast-paced, experimentation-rich environment where new technologies aren't just welcome - they're expected. Here, you'll work side-by-side with seasoned engineers, architects, and thinkers to craft the kind of iconic products that can reshape industries and unlock entirely new models of operation for the enterprise.
If you're energized by the challenge of solving hard problems, love working at the edge of what's possible, and want to help shape the future of AI infrastructure - we'd love to meet you.

Your Impact

As an AI Infrastructure Abstraction Engineer , you will help shape the next generation of AI compute platforms by designing systems that abstract away hardware complexity and expose logical, scalable, and secure interfaces for AI workloads. Your work will enable multi-tenancy, resource isolation, and dynamic scheduling of GPUs and accelerators at scale - making infrastructure programmable, elastic, and developer-friendly.

You will bridge the gap between raw compute resources and AI/ML frameworks, allowing infrastructure teams and model developers to consume shared GPU resources with the performance and reliability of bare metal, but with the flexibility of cloud-native systems. Your contributions will empower internal and external users to run AI workloads securely, efficiently, and predictably - regardless of the underlying hardware topology.

This role is critical to enabling AI infrastructure that is multi-tenant by design, scalable in practice, and abstracted for portability across diverse platforms.

KEY RESPONSIBILITIES

• Design and implement infrastructure abstractions that cleanly separate logical compute units (vGPUs, GPU pods, AI queues) from physical hardware (nodes, devices, interconnects) .
• Develop runtime services, APIs, and control planes to expose GPU and accelerator resources to users and frameworks with multi-tenant isolation and QoS guarantees .
• Architect systems for secure GPU sharing , including time-slicing, memory partitioning, and namespace isolation across tenants or jobs.
• Collaborate with platform, orchestration, and scheduling teams to map logical resources to physical devices based on utilization, priority, and topology.
• Define and enforce resource usage policies , including fair sharing, quota management, and oversubscription strategies.
• Integrate with model training and serving frameworks (e.g., PyTorch, TensorFlow, Triton) to ensure smooth and predictable resource consumption.
• Build observability and telemetry pipelines to trace logical-to-physical mappings, usage patterns, and performance anomalies.
• Partner with infrastructure security teams to ensure secure onboarding, access control, and workload isolation in shared environments.
• Support internal developers in adopting abstraction APIs, ensuring high performance while abstracting away low-level details.
• Contribute to the evolution of internal compute platform architecture, with a focus on abstraction, modularity, and scalability.

Minimum Qualifications:

• Bachelors + 15 years of related experience, or Masters + 12 years of related experience, or PhD + 8 years of related experience
• Experience building scalable, production-grade infrastructure components or control planes using Go, Python, and C++ ,
• Experience with Kubernetes, Docker or Kubevirt for virtualization, containerization , and orchestration frameworks
• Experience designing or implementing logical resource abstractions for compute, storage, or networking with a focus in multi-tenant environments .
• Experience integrating with AI/ML platforms or pipelines (e.g., PyTorch, TensorFlow, Triton Inference Server, MLFlow).

Preferred Qualifications:

• Experience with GPU sharing, scheduling, or isolation techniques (e.g., MPS, MIG, time-slicing, device plugin frameworks, or vGPU technologies).
• Solid grasp of resource management concepts including quotas, fairness, prioritization, and elasticity.

#WeAreCisco

#WeAreCisco where every individual brings their unique skills and perspectives together to pursue our purpose of powering an inclusive future for all.

Our passion is connection-we celebrate our employees' diverse set of backgrounds and focus on unlocking potential. Cisconians often experience one company, many careers where learning and development are encouraged and supported at every stage. Our technology, tools, and culture pioneered hybrid work trends, allowing all to not only give their best, but be their best.

We understand our outstanding opportunity to bring communities together and at the heart of that is our people. One-third of Cisconians collaborate in our 30 employee resource organizations, called Inclusive Communities, to connect, foster belonging, learn to be informed allies, and make a difference. Dedicated paid time off to volunteer-80 hours each year-allows us to give back to causes we are passionate about, and nearly 86% do!

Our purpose, driven by our people, is what makes us the worldwide leader in technology that powers the internet. Helping our customers reimagine their applications, secure their enterprise, transform their infrastructure, and meet their sustainability goals is what we do best. We ensure that every step we take is a step towards a more inclusive future for all. Take your next step and be you, with us!

AI Infrastructure Support Engineer

At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global financial system, we influence nearly 20% of the world's investible assets. Every day, our teams harness cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative solutions that redefine industries and uplift communities worldwide.

Recognized as a top destination for innovators and champions of inclusion, BNY is where bold ideas meet advanced technology and exceptional talent. Together, we power the future of finance - and this is what #LifeAtBNY is all about. Join us and be part of something extraordinary.

We're seeking a future team member for the role of AI Infrastructure Support Engineer to join our AIHUB team. This role is located in Pittsburgh, PA (4 days in office required).

In this role, you'll make an impact in the following ways:

• Provide operational support and issue resolution for AI infrastructure and platform services.
• Manage AI deployments and monitor workloads on distributed systems using Kubernetes and Docker.
• Develop and maintain scripts and automation using Python and Shell scripting.
• Extensively use infrastructure ticketing systems such as JIRA and ServiceNow.
• Collaborate with cross-functional teams and be well versed in DevOps practices including CI/CD (GitLab), containerization (Docker), and secure access control (Azure AD).
• Assist with documenting infrastructure processes, runbooks, and architectural components.
• Troubleshoot networking and storage-related issues across AI workloads.
• Participate in periodic on-call rotations, including occasional weekend coverage.

To be successful in this role, we're seeking the following:

• Bachelor's degree in computer science or a related discipline or equivalent work experience required. 6 - 8 years of application or infrastructure related experience required; experience in the securities or financial services industry is a plus
• 4+ years of experience in infrastructure engineering, DevOps, or site reliability roles.
• Hands-on experience with Kubernetes, Docker, HELM, and GitLab CI/CD.
• Proficiency in Python and Bash/Shell scripting for automation and operational tooling.
• Familiarity with enterprise tools such as JIRA, ServiceNow, and Azure AD.
• Working knowledge of networking fundamentals and storage systems, especially as they apply to distributed AI systems.
• Strong documentation and collaboration skills in a cross-functional, fast-paced environment.
• Willingness to participate in weekend or after-hours support rotations as needed.

At BNY, our culture speaks for itself, check out the latest BNY news at:
BNY Newsroom

BNY LinkedIn

Here's a few of our recent awards:

* America's Most Innovative Companies, Fortune, 2025

* World's Most Admired Companies, Fortune 2025

* "Most Just Companies", Just Capital and CNBC, 2025

Our Benefits and Rewards:

BNY offers highly competitive compensation, benefits, and wellbeing programs rooted in a strong culture of excellence and our pay-for-performance philosophy. We provide access to flexible global resources and tools for your life's journey. Focus on your health, foster your personal resilience, and reach your financial goals as a valued member of our team, along with generous paid leaves, including paid volunteer time, that can support you and your family through moments that matter.

BNY is an Equal Employment Opportunity/Affirmative Action Employer - Underrepresented racial and ethnic groups/Females/Individuals with Disabilities/Protected Veterans.