AI Architect

About the job

Responsibilities

• Lead the architecture and delivery of Generative AI solutions, guiding development teams on relevant tools, frameworks, and platforms.
• Translate business requirements into robust application, service, and target-state architectures, including reusable RAG, agent orchestration, prompt, and evaluation patterns.
• Own scalability, performance, latency, observability, security, and cost optimisation across AI solutions.
• Lead technical design and code reviews and guide delivery against client needs and quality standards.
• Define the right adaptation approach for each use case—prompting, RAG, fine-tuning, or distillation—including when Generative AI is not appropriate.
• Own evaluation strategy, including golden datasets, offline and online evaluation, human-calibrated LLM-as-judge, and CI regression testing.
• Set retrieval quality standards for chunking, hybrid search, reranking, query rewriting, and permission-aware retrieval, using measured recall and precision.
• Define AI security, privacy, Responsible AI, and governance controls, including prompt injection, data exfiltration, jailbreak, sandboxing, least-privilege identity, and supply-chain risks.
• Own solution economics through model routing, caching, batching, capacity planning, and defensible cost-per-transaction estimates.
• Support enterprise architecture standards and help assess technology solutions.
• Supervise vendor developers and mentor engineers and junior architects.
• Support business development by identifying and researching opportunities with new and existing clients.
• Improve internal design and development practices and maintain documentation.
• Build internal relationships, strengthen the firm's reputation, and develop your skills in line with company and team priorities.
• Take on additional responsibilities relevant to the role and agreed with your manager.

Requirements

• 5+ years of experience in agile product delivery.
• Hands-on experience designing and delivering production software solutions at scale.
• Demonstrated expertise in Generative AI solution architecture.
• Personally delivered at least one Generative AI system to production and can explain its architecture, failure modes, operating costs, and lessons learned.
• Bachelor's or Master's degree in Computer Science or Engineering, or equivalent practical experience.
• Relevant Microsoft Azure certification, such as Azure Solutions Architect Expert or Azure Developer Associate, is preferred.
• Strong hands-on experience with Generative AI architecture is essential.
• Experience across several relevant technologies is expected; expertise in every named tool is not required.
• Experience with Large language models such as GPT-4, Gemini, and Llama.
• Experience with RAG, agent orchestration, and Microsoft Copilots using platforms or frameworks such as Azure OpenAI Service, Semantic Kernel, LangChain, AWS Bedrock, or Google Vertex AI.
• Experience with Vector search and retrieval engineering, including chunking, hybrid search, reranking, query rewriting, metadata filtering, and permission-aware retrieval.
• Experience with Production AI agents, including tool calls, state and memory management, durable execution, approval controls, retries, idempotency, and cost safeguards.
• Experience with Structured generation, including schema-enforced outputs, validation and repair, deterministic fallbacks, and graceful degradation.
• Experience with Cloud-native Azure architecture, including Azure Machine Learning, Azure AI services, and appropriate infrastructure and platform services.
• Experience with Service-oriented, event-driven, and microservices architectures.
• Experience with Relational and NoSQL data stores, such as Microsoft SQL Server and MongoDB, with sound technology-selection judgement.
• Experience with Containers and orchestration using Docker and/or Kubernetes.
• Experience with DevOps and delivery practices, including CI/CD, automation, environment management, and tools such as Azure DevOps, GitHub, or Jira.
• Working knowledge of user-centred product practices, including user stories, personas, and prototyping.
• Experience with AI evaluation using golden datasets, LLM-as-judge, and prompt and model regression testing.
• Experience with LLMOps and observability, including tracing, token and cost attribution, prompt and model versioning, trace replay, and quality-drift monitoring, using tools such as LangSmith, Langfuse, or Azure Monitor.
• Experience with Model economics, including routing, cascading, caching, batching, and provisioned-versus-consumption capacity planning.
• Experience with AI security, including prompt injection, data exfiltration, jailbreak resistance, sandboxing, least-privilege identities, supply-chain risk, and the OWASP Top 10 for LLM Applications.
• Experience with Model Context Protocol and agent interoperability standards.
• Experience with Model adaptation beyond prompting and RAG, including fine-tuning, LoRA/PEFT, and distillation.
• Experience with Multimodal AI.