Senior Manager, LLMOps & AI Integration Engineer
- Greece-Thessaloniki Chortiatis
ROLE SUMMARY
As Senior Manager, LLMOps & AI Integration Engineer (Individual Contributor), you will lead the model integration, agent/tooling, and LLM operations strategy for our pip‑installable AI Capabilities SDK. You will design and maintain clean, extensible interfaces for LLMs, embeddings, retrieval, tool execution, and agentic workflows; build runtime guards and evaluation hooks; and optimize latency, reliability, and cost—so teams can embed AI safely and predictably.
You will collaborate closely with:
Embedded AI Architecture and Solution Design to prioritize capabilities and define interfaces aligned to real-world solution patterns.
AI Engineering to deliver PI increments, integrate external technologies, and codify model/agent failure modes and mitigations.
Digital Creation Centers and Forward Impact Teams (FITs) as consumers of the SDK, capturing feedback to refine ergonomics and performance.
DevOps, Evaluation & QA, and Developer Experience engineers to wire guardrails, evaluation, and developer-ready assets into CI/CD and docs.
Why this Role Matters
LLMOps sits at the intersection of capability, safety, and operability. By turning complex model/agent behaviors into well-abstracted, evaluable, and observable SDK modules, you enable teams to deliver AI features fast with safety—reducing bespoke implementations, ensuring reproducibility, and controlling latency/cost at scale. Your work is foundational for platform services that rely on the SDK as their backbone.
ROLE RESPONSIBILITIES
1) Model & Agent Integration Architecture
Design Python SDK interfaces for LLM clients, embeddings, tokenization, and structured outputs (e.g., Pydantic/JSON schemas).
Implement function/tool calling abstractions, agent orchestration patterns (ReAct-like, planner/executor), and MCP adaptors for interoperable tools.
Provide configuration and secrets integration points that downstream services can adopt consistently (env vars, config schemas, key management hooks).
2) Retrieval & RAG Components
Build RAG modules (document loaders, chunking/segmentation, embeddings, retrievers, rerankers) with pluggable backends (vector stores, search indices).
Standardize connectors and interface contracts for common data sources; ensure reproducible pipelines and SDK↔service parity of behaviors.
Optimize retrieval quality via evaluation hooks (precision/recall, MRR, hit rate) and tunable parameters (top‑k, thresholds).
3) Guardrails, Safety & Policy Integration
Codify safety guardrails: prompt hygiene, jailbreak resistance, content filters, sensitive topic/policy checks, red‑team/adversarial probes.
Integrate privacy/security controls: data minimization, PII detection flags, safe logging, and deterministic truncation strategies.
Provide policy‑as‑code hooks and pre/post‑processing middleware to enforce runtime constraints within SDK flows.
4) Performance, Reliability & Cost Engineering
Implement caching, batching, retry/backoff, circuit breakers, and fall‑back strategies across model/tool calls.
Profile and tune latency, throughput, and token usage/cost; expose configuration knobs and budget guards (token and cost caps).
Maintain benchmark suites and regression baselines for model and retrieval components; block releases on significant degradations.
5) Observability‑by‑Design for Libraries
Embed telemetry hooks (structured logs, metrics, traces) suitable for a library so that services consuming the SDK can attach enterprise observability.
Define SLIs relevant to LLMOps (latency, error rates, cache hit ratio, token usage, cost per call) and document runbook guidance for downstream teams.
Standardize error models and return types to ensure predictable handling across consuming services.
6) Evaluation & CI/CD Integration
Partner with Evaluation & QA Engineers to build test harnesses (unit/integration/contract/fuzz/adversarial) and benchmark suites for agents/tools, RAG, and model endpoints.
Work with DevOps Engineers to wire quality gates into CI/CD: coverage thresholds, performance budgets, safety checks, SBOM/signing for release artifacts.
Provide fixtures and simulators to enable deterministic tests across backends (vector stores, providers).
7) Developer Experience & Enablement
Collaborate with Developer Experience and Technical Writers to deliver quickstarts, sample apps, notebooks, and code recipes for common patterns (RAG, agents, tool use, structured outputs).
Contribute to cookiecutters/CLI scaffolds that prewire configuration, tests, telemetry, and guardrails for new modules or integrations.
Support community contribution workflows (feature branches, PR checks, doc acceptance criteria) and mentor contributors on best practices.
8) Collaboration & Continuous Improvement
Close feedback loops with Embedded Architecture, Solution Design, AI Engineering, Creation Centers, and FITs to refine APIs, defaults, and ergonomics.
Participate in post‑release retrospectives and incident reviews to strengthen guardrails, performance tuning, and developer experience.
Track and improve LLMOps KPIs (latency, cost per call, error rates, cache hit ratios, evaluation pass rates).
MEASURES OF SUCCESS
Performance & cost: Latency and token/cost budgets consistently met; cache hit ratios improved; fewer performance regressions.
Safety & reliability: Guardrail tests pass; reduced policy violations and incident rates; predictable error handling.
Adoption & DX: Faster time‑to‑first‑success for consuming teams; positive developer feedback on APIs and examples.
Evaluation coverage: Benchmarks and adversarial tests integrated; high pass rates; clear baselines with trend reporting.
SDK↔Consumer parity: Consistent behavior across downstream services; reduced bespoke integrations and rework.
QUALIFICATIONS
Basic Qualifications
- 7+ years in software/AI engineering with significant experience in LLMOps or model/tool integration for Python libraries.
- Deep proficiency in Python and building extensible SDKs (interfaces, packaging, semantic versioning, backward compatibility).
- Hands‑on with GenAI/LLM ecosystems (e.g., tool/function calling, structured outputs, prompt orchestration) and RAG components (embeddings, retrievers, rerankers, vector stores).
- Strong grasp of performance engineering (profiling, caching, batching), resilience patterns (retry/backoff, circuit breakers), and cost control (token budgeting).
- Experience embedding observability hooks and integrating evaluation/benchmarks into CI/CD pipelines.
Preferred Qualifications
Familiarity with agent frameworks, MCP adaptors, and multi‑provider model client abstractions.
Experience with adversarial/safety testing, privacy/PII considerations, and policy‑as‑code guardrails.
Background in open-source workflows (feature branches, PR reviews) and multi‑backend integration (vector DBs, search, storage).
Exposure to regulated environments (healthcare/pharma/finance) and audit‑ready evidence capture.
Comfortable collaborating across architecture, DevOps, QA, DX, platform, and product teams in matrixed organizations.
Work Location Assignment: Hybrid
Purpose
Breakthroughs that change patients' lives... At Pfizer we are a patient centric company, guided by our four values: courage, joy, equity and excellence. Our breakthrough culture lends itself to our dedication to transforming millions of lives.
Digital Transformation Strategy
One bold way we are achieving our purpose is through our company wide digital transformation strategy. We are leading the way in adopting new data, modelling and automated solutions to further digitize and accelerate drug discovery and development with the aim of enhancing health outcomes and the patient experience.
Flexibility
We aim to create a trusting, flexible workplace culture which encourages employees to achieve work life harmony, attracts talent and enables everyone to be their best working self. Let’s start the conversation!
Equal Employment Opportunity
We believe that a diverse and inclusive workforce is crucial to building a successful business. As an employer, Pfizer is committed to celebrating this, in all its forms – allowing for us to be as diverse as the patients and communities we serve. Together, we continue to build a culture that encourages, supports and empowers our employees.
Disability Inclusion
Our mission is unleashing the power of all our people and we are proud to be a disability inclusive employer, ensuring equal employment opportunities for all candidates. We encourage you to put your best self forward with the knowledge and trust that we will make any reasonable adjustments to support your application and future career. Your journey with Pfizer starts here!
Breakthroughs that change patients’ lives
Research confirms what intuition tells us: that purpose-driven companies perform better, are more innovative, attract and retain the best people, and know how to unleash the power of those people. Pfizer’s purpose—Breakthroughs that change patients' lives—fuels everything we do and reflects our passion for building on our legacy as one of the greatest contributors of good to the world.
Each word in our purpose has meaning and reflects the value we strive to bring to patients and society:
“Breakthroughs” - These are the innovations, scientific and commercial, that we seek to deliver every day. All colleagues, regardless of role, level or location, strive for breakthroughs every day.
“Change” - We want to do more than simply improve patients’ medical conditions; we want to dramatically change their lives for the better.
“Patients’ lives” - We consider not only patients, but everyone they touch—including their families, caregivers, and friends—and everything they love to do. It’s an intentionally holistic view.
Our purpose ensures that patients remain at the center of all we do. We live our purpose by sourcing the best science in the world; partnering with others in the healthcare system to improve access to our medicines; using digital technologies to enhance our drug discovery and development, as well as patient outcomes; and leading the conversation to advocate for pro-innovation/pro-patient policies.
Every decision we make and every action we take is done with the patient in mind—and to nurture an environment where breakthroughs can thrive.
Our Values
To fully realize Pfizer’s purpose, we have established a clear set of expectations regarding “what” we need to achieve for patients and “how” we will go about achieving those goals.
The “how” is represented by four simple, powerful values—courage, excellence, equity, and joy—that define our company and our culture.
Courage: Breakthroughs start by challenging convention, especially in the face of uncertainty or adversity. This happens when we think big, speak up, and are decisive.
Excellence: We can only change patients’ lives when we perform at our best together. This happens when we focus on what matters, agree who does what, and measure our outcomes.
Equity: We believe that every person deserves to be seen, heard, and cared for. This happens when we are inclusive, act with integrity, and reduce healthcare disparities.
Joy: We give ourselves to our work, but it also gives to us. We find joy when we take pride, recognize one another, and have fun.
Pfizer Digital
Pfizer Digital takes immense pride in being at the forefront of innovation, harnessing cutting-edge smart technology that profoundly impacts the lives of our patients.
With the transformative potential of digital solutions, we revolutionize how we discover, develop, and deliver medicines, enabling us to achieve breakthroughs faster and more efficiently.
By streamlining and automating transactional processes, we create valuable time and resources, empowering our talented team to focus on meaningful work that truly makes a difference in healthcare and beyond.
Benefits at Pfizer
Pfizer offers competitive compensation and benefits programs designed to meet the diverse needs of our colleagues. Our Pay for Performance Philosophy and Practices reward colleagues based on the contributions they make to our business.
Our Competitive Benefits Programs help our colleagues by:
- Promoting Health and Wellness to help colleagues maintain and improve their physical and mental wellbeing. Pfizer offerings include health and disability insurance, preventative health programs, medical screenings, free or reduced-cost vaccinations, discounts on Pfizer products, mental health support, nutrition and fitness counseling and more.
- Strengthening Colleagues’ Financial Security by providing company contributions to retirement plans, life insurance and financial planning education to help colleagues achieve their financial goals.
- Providing Benefits and Time off for the Moments that Matter to ensure Pfizer colleagues have the time away from the office to recharge, recover and return to work as the best version of themselves. Vacation, Holiday time and Sick time are just the beginning, with a 12 week parental leave policy for both parents of a new born, 10 day of caregiver leave for those times when your family needs you whether it’s a child, spouse or parent, bereavement leave, and additional country-specific programs.
- And so much more, depending on your country and site, Pfizer offers childcare facilities or discount programs, on-site health and/or fitness centers, movement and mindfulness solutions, “Log in for your Day” work flexibility and so much more.
For U.S. based jobs, view an overview of Pfizer’s U.S. benefits program (opens in a new window)