Senior Manager, LLMOps & AI Integration Engineer

ROLE SUMMARY

As Senior Manager, LLMOps & AI Integration Engineer (Individual Contributor), you will lead the model integration, agent/tooling, and LLM operations strategy for our pip‑installable AI Capabilities SDK. You will design and maintain clean, extensible interfaces for LLMs, embeddings, retrieval, tool execution, and agentic workflows; build runtime guards and evaluation hooks; and optimize latency, reliability, and cost—so teams can embed AI safely and predictably.

You will collaborate closely with:

• Embedded AI Architecture and Solution Design to prioritize capabilities and define interfaces aligned to real-world solution patterns.

• AI Engineering to deliver PI increments, integrate external technologies, and codify model/agent failure modes and mitigations.

• Digital Creation Centers and Forward Impact Teams (FITs) as consumers of the SDK, capturing feedback to refine ergonomics and performance.

• DevOps, Evaluation & QA, and Developer Experience engineers to wire guardrails, evaluation, and developer-ready assets into CI/CD and docs.

Why this Role Matters

LLMOps sits at the intersection of capability, safety, and operability. By turning complex model/agent behaviors into well-abstracted, evaluable, and observable SDK modules, you enable teams to deliver AI features fast with safety—reducing bespoke implementations, ensuring reproducibility, and controlling latency/cost at scale. Your work is foundational for platform services that rely on the SDK as their backbone.

ROLE RESPONSIBILITIES

1) Model & Agent Integration Architecture

• Design Python SDK interfaces for LLM clients, embeddings, tokenization, and structured outputs (e.g., Pydantic/JSON schemas).

• Implement function/tool calling abstractions, agent orchestration patterns (ReAct-like, planner/executor), and MCP adaptors for interoperable tools.

• Provide configuration and secrets integration points that downstream services can adopt consistently (env vars, config schemas, key management hooks).

2) Retrieval & RAG Components

• Build RAG modules (document loaders, chunking/segmentation, embeddings, retrievers, rerankers) with pluggable backends (vector stores, search indices).

• Standardize connectors and interface contracts for common data sources; ensure reproducible pipelines and SDK↔service parity of behaviors.

• Optimize retrieval quality via evaluation hooks (precision/recall, MRR, hit rate) and tunable parameters (top‑k, thresholds).

3) Guardrails, Safety & Policy Integration

• Codify safety guardrails: prompt hygiene, jailbreak resistance, content filters, sensitive topic/policy checks, red‑team/adversarial probes.

• Integrate privacy/security controls: data minimization, PII detection flags, safe logging, and deterministic truncation strategies.

• Provide policy‑as‑code hooks and pre/post‑processing middleware to enforce runtime constraints within SDK flows.

4) Performance, Reliability & Cost Engineering

• Implement caching, batching, retry/backoff, circuit breakers, and fall‑back strategies across model/tool calls.

• Profile and tune latency, throughput, and token usage/cost; expose configuration knobs and budget guards (token and cost caps).

• Maintain benchmark suites and regression baselines for model and retrieval components; block releases on significant degradations.

5) Observability‑by‑Design for Libraries

• Embed telemetry hooks (structured logs, metrics, traces) suitable for a library so that services consuming the SDK can attach enterprise observability.

• Define SLIs relevant to LLMOps (latency, error rates, cache hit ratio, token usage, cost per call) and document runbook guidance for downstream teams.

• Standardize error models and return types to ensure predictable handling across consuming services.

6) Evaluation & CI/CD Integration

• Partner with Evaluation & QA Engineers to build test harnesses (unit/integration/contract/fuzz/adversarial) and benchmark suites for agents/tools, RAG, and model endpoints.

• Work with DevOps Engineers to wire quality gates into CI/CD: coverage thresholds, performance budgets, safety checks, SBOM/signing for release artifacts.

• Provide fixtures and simulators to enable deterministic tests across backends (vector stores, providers).

7) Developer Experience & Enablement

• Collaborate with Developer Experience and Technical Writers to deliver quickstarts, sample apps, notebooks, and code recipes for common patterns (RAG, agents, tool use, structured outputs).

• Contribute to cookiecutters/CLI scaffolds that prewire configuration, tests, telemetry, and guardrails for new modules or integrations.

• Support community contribution workflows (feature branches, PR checks, doc acceptance criteria) and mentor contributors on best practices.

8) Collaboration & Continuous Improvement

• Close feedback loops with Embedded Architecture, Solution Design, AI Engineering, Creation Centers, and FITs to refine APIs, defaults, and ergonomics.

• Participate in post‑release retrospectives and incident reviews to strengthen guardrails, performance tuning, and developer experience.

• Track and improve LLMOps KPIs (latency, cost per call, error rates, cache hit ratios, evaluation pass rates).

MEASURES OF SUCCESS

• Performance & cost: Latency and token/cost budgets consistently met; cache hit ratios improved; fewer performance regressions.

• Safety & reliability: Guardrail tests pass; reduced policy violations and incident rates; predictable error handling.

• Adoption & DX: Faster time‑to‑first‑success for consuming teams; positive developer feedback on APIs and examples.

• Evaluation coverage: Benchmarks and adversarial tests integrated; high pass rates; clear baselines with trend reporting.

• SDK↔Consumer parity: Consistent behavior across downstream services; reduced bespoke integrations and rework.

QUALIFICATIONS

Basic Qualifications

• 7+ years in software/AI engineering with significant experience in LLMOps or model/tool integration for Python libraries.
• Deep proficiency in Python and building extensible SDKs (interfaces, packaging, semantic versioning, backward compatibility).
• Hands‑on with GenAI/LLM ecosystems (e.g., tool/function calling, structured outputs, prompt orchestration) and RAG components (embeddings, retrievers, rerankers, vector stores).
• Strong grasp of performance engineering (profiling, caching, batching), resilience patterns (retry/backoff, circuit breakers), and cost control (token budgeting).
• Experience embedding observability hooks and integrating evaluation/benchmarks into CI/CD pipelines.

Preferred Qualifications

• Familiarity with agent frameworks, MCP adaptors, and multi‑provider model client abstractions.

• Experience with adversarial/safety testing, privacy/PII considerations, and policy‑as‑code guardrails.

• Background in open-source workflows (feature branches, PR reviews) and multi‑backend integration (vector DBs, search, storage).

• Exposure to regulated environments (healthcare/pharma/finance) and audit‑ready evidence capture.

• Comfortable collaborating across architecture, DevOps, QA, DX, platform, and product teams in matrixed organizations.

 
Work Location Assignment: Hybrid