RA
AI Engineer (AI System Calibration & Optimization)
full-time•Seattle
About this role
<p><span data-contrast="auto">Robots & Pencils is seeking an outcome-oriented AI Engineer to partner with a strategic client on a high-impact AI system calibration and optimization engagement. You'll embed directly with the client's AI and product engineering teams to improve the accuracy, reliability, and transparency of their Azure-hosted, fine-tuned GPT model through systematic prompt optimization and RAG calibration.</span><span data-ccp-props="{"201341983":0,"335559740":240}"> </span></p>
<p><span data-contrast="auto">As an AI Engineer, you'll serve as technical thought partner, actively coding and leveraging your software engineering experience to build calibration pipelines, optimize prompts using prompt optimization frameworks, and establish repeatable improvement workflows. You'll work on-site with the client, driving measurable outcomes that maximize their AI system performance.</span><span data-ccp-props="{"201341983":0,"335559740":240}"> </span></p>
<p><strong><span data-contrast="none"><span data-ccp-parastyle="heading 3">Key Responsibilities</span></span></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":281,"335559739":281,"335559740":279}"> </span></p>
<p><strong><em><span data-contrast="none"><span data-ccp-parastyle="heading 4">Client Engagement </span><span data-ccp-parastyle="heading 4">& </span><span data-ccp-parastyle="heading 4">Solution </span><span data-ccp-parastyle="heading 4">Development</span></span></em></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":319,"335559739":319,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">Embed with strategic client as their technical partner for AI system calibration and prompt optimization.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">Build production-grade calibration systems using Python within the client's Azure environment.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">Implement DSPy framework and GEPA optimizer to systematically improve prompt quality and retrieval performance.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><span data-contrast="auto">Design and develop Golden Dataset curation workflows using Azure Data Labeling, establishing gold/silver data tier schemas.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><span data-contrast="auto">Create evaluation frameworks to measure model accuracy, precision/recall, latency, and hallucination rates.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1"><span data-contrast="auto">Architect prompt optimization pipelines for retrieval, context synthesis, and answer generation tailored to client needs.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1"><span data-contrast="auto">Own the path to production - evaluation pipelines, Azure ML workflows, KPI dashboards, and optimization automation.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="8" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1"><span data-contrast="auto">Iterate rapidly based on client feedback and KPI results, translate business goals into technical calibration improvements.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><em><span data-contrast="none"><span data-ccp-parastyle="heading 4">Outcome Ownership & Business Impact</span></span></em></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":319,"335559739":319,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">Own end-to-end delivery of calibration systems from initial baseline to production-ready optimization workflows.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">Establish measurable KPIs and demonstrate accuracy improvements, latency reduction, and hallucination mitigation.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">Provide strategic guidance on RAG architecture improvements and retrieval parameter optimization.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><span data-contrast="auto">Accelerate client time-to-value through hands-on development and comprehensive knowledge transfer.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><span data-contrast="auto">Deliver operational playbooks and documentation enabling the client team to maintain calibration systems independently.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><em><span data-contrast="none"><span data-ccp-parastyle="heading 4">Engineering Leadership & </span><span data-ccp-parastyle="heading 4">In-Field </span><span data-ccp-parastyle="heading 4">Delivery</span><span data-ccp-parastyle="heading 4"> Excellence</span></span></em></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":319,"335559739":319,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1"><span data-contrast="auto">Lead complex, multi-stakeholder calibration initiatives on-site and remotely; drive clarity, remove blockers, and keep execution on track.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1"><span data-contrast="auto">Set coding standards and architectural patterns for calibration components; write clear docs, runbooks, and technical specifications.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1"><span data-contrast="auto">Mentor client engineers through code reviews, pairing sessions, and technical workshops on DSPy, GEPA, and evaluation best practices.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="9" data-aria-level="1"><span data-contrast="auto">Make sound tradeoffs under real-world constraints - Azure cost optimization, data quality, performance requirements, and security.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="10" data-aria-level="1"><span data-contrast="auto">Align delivery with Robots & Pencils' responsible AI practices and client governance requirements.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><em><span data-contrast="none"><span data-ccp-parastyle="heading 4">Cross-Functional Collaboration</span></span></em></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":319,"335559739":319,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">Work closely with client's AI SMEs and product engineering teams to understand product catalog structure and validation workflows.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">Collaborate with internal R&P product, engineering, and delivery teams on calibration methodology and best practices.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">Share insights from client engagement to improve R&P's prompt optimization frameworks and tooling.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><span data-contrast="auto">Contribute reusable patterns, evaluation frameworks, and documentation back to R&P's core platform.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><span data-contrast="auto">Collaborate across time zones with distributed teams.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><span data-contrast="none"><span data-ccp-parastyle="heading 3">Required Skills & Qualifications</span></span></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":281,"335559739":281,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">Bachelor's degree in computer science, Engineering, or equivalent experience.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">7+ years of professional software development with significant ownership of architecture and delivery.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">3+ years of Python in ML/AI systems with a strong focus on data processing and evaluation pipelines.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><span data-contrast="auto">2+ years building with Generative AI including hands-on prompt engineering and optimization work.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><span data-contrast="auto">Experience with prompt optimization frameworks - DSPy strongly preferred, or similar systematic approaches to prompt improvement.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1"><span data-contrast="auto">Deep understanding of RAG architectures - retrieval quality, latency/cost tuning, hallucination mitigation, and evaluation methods.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1"><span data-contrast="auto">Hands-on experience designing evaluation metrics and building assessment frameworks for LLM systems.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1"><span data-contrast="auto">Knowledge of systematic experimentation methods - A/B testing, parameter tuning, performance benchmarking.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="9" data-aria-level="1"><span data-contrast="auto">Experience with data curation, labeling workflows, and dataset quality management for AI systems.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="10" data-aria-level="1"><span data-contrast="auto">Strong Azure cloud experience with focus on AI/ML services - Azure Machine Learning, Azure AI Search, Azure OpenAI Service.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="11" data-aria-level="1"><span data-contrast="auto">Experience with Azure Data Labeling, Azure Blob Storage, and Azure infrastructure fundamentals.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="12" data-aria-level="1"><span data-contrast="auto">Understanding vector search platforms and retrieval optimization (Azure AI Search, Weaviate, Qdrant, Pinecone).</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="13" data-aria-level="1"><span data-contrast="auto">Strong IaC background (Terraform or ARM templates) plus containerization and distributed systems knowledge.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="14" data-aria-level="1"><span data-contrast="auto">Solid SDLC practices - testing strategies, CI/CD, code reviews, observability, and operational excellence.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="15" data-aria-level="1"><span data-contrast="auto">Upper-intermediate English for client communication.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="16" data-aria-level="1"><span data-contrast="auto">Experience leading complex technical projects with multiple stakeholders.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="17" data-aria-level="1"><span data-contrast="auto">Strong communication skills for technical and executive audiences.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="18" data-aria-level="1"><span data-contrast="auto">Ability to context-switch and adapt to client environments.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="19" data-aria-level="1"><span data-contrast="auto">Willingness to travel to client sites.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><span data-contrast="none"><span data-ccp-parastyle="heading 3">Nice to Have</span></span></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":281,"335559739":281,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><span data-contrast="auto">Direct hands-on experience with DSPy framework and GEPA optimizer.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><span data-contrast="auto">Understanding systematic optimization principles: evolutionary algorithms, Bayesian optimization, multi-objective optimization, and Pareto efficiency concepts.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><span data-contrast="auto">Familiarity with prompt optimization frameworks and methods - experience with any of: MIPROv2, TextGrad, EvoPrompt, AutoPrompt, or reinforcement learning approaches (GRPO, PPO).</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><span data-contrast="auto">Experience with LLM-as-judge patterns and automated evaluation pipelines.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><span data-contrast="auto">Knowledge of advanced RAG patterns - Adaptive RAG, Self-RAG, Corrective RAG - and retrieval evaluation methods (MRR, NDCG, precision@k).</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1"><span data-contrast="auto">Understanding of agentic AI patterns - ReAct, Chain-of-Thought, Tool Use - and their application in RAG systems.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="7" data-aria-level="1"><span data-contrast="auto">Experience building evaluation dashboards with Azure Monitor, Application Insights, or similar observability tools.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="8" data-aria-level="1"><span data-contrast="auto">Familiarity with MLOps practices - model versioning, experiment tracking, metric logging for evaluation systems.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="9" data-aria-level="1"><span data-contrast="auto">Experience with AWS or GCP AI/ML platforms (Bedrock, SageMaker, Vertex AI) and cross-cloud architecture patterns.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="10" data-aria-level="1"><span data-contrast="auto">Experience with product catalog systems, cross-reference matching, or e-commerce search optimization.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="11" data-aria-level="1"><span data-contrast="auto">Background in manufacturing, industrial equipment, or technical specification systems.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="2" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="12" data-aria-level="1"><span data-contrast="auto">Prior consulting or professional services experience with enterprise clients.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><strong><span data-contrast="none"><span data-ccp-parastyle="heading 3">Personal Competencies</span></span></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":281,"335559739":281,"335559740":279}"> </span></p>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="1" data-aria-level="1"><strong><span data-contrast="auto">Accountability</span></strong><span data-contrast="auto"> – Owns full client engagement cycle with quality, reliability, and attention to detail.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="2" data-aria-level="1"><strong><span data-contrast="auto">Adaptability</span></strong><span data-contrast="auto"> – Thrives in dynamic, fast-paced client environments.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="3" data-aria-level="1"><strong><span data-contrast="auto">Collaboration</span></strong><span data-contrast="auto"> – Builds strong partnerships across teams and time zones.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="4" data-aria-level="1"><strong><span data-contrast="auto">Execution-Focused</span></strong><span data-contrast="auto"> – Delivers maintainable, scalable solutions without overengineering.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="5" data-aria-level="1"><strong><span data-contrast="auto">Innovation-Minded</span></strong><span data-contrast="auto"> – Brings curiosity and experimentation to technology decisions.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<ul>
<li data-leveltext="" data-font="Symbol" data-listid="1" data-list-defn-props="{"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"hybridMultilevel"}" data-aria-posinset="6" data-aria-level="1"><strong><span data-contrast="auto">Craftsmanship</span></strong><span data-contrast="auto"> – Cares deeply about documentation and code quality, architecture, and user experience.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></li>
</ul>
<p><span data-ccp-props="{"201341983":0,"335559740":279}"> </span></p>
<p><strong><span data-contrast="none"><span data-ccp-parastyle="heading 3">Why Join Robots & Pencils?</span></span></strong><span data-ccp-props="{"134245418":true,"134245529":true,"201341983":0,"335559738":281,"335559739":281,"335559740":279}"> </span></p>
<p><span data-contrast="auto">We don’t just ship features, we build digital-first products that matter. As a Senior Forward Deploy Engineer, you’ll join a team that values deep craft, cross-functional collaboration, and relentless focus on quality. You’ll work on impactful agentic AI applications using modern technologies, while influencing engineering culture and best practices across the organization.</span><span data-ccp-props="{"201341983":0,"335559738":240,"335559739":240,"335559740":279}"> </span></p>
What you'll do
- The AI Engineer will partner with a strategic client to improve the accuracy and reliability of their AI systems through calibration and optimization. Responsibilities include building calibration systems, optimizing prompts, and establishing workflows for continuous improvement.
About Robots and Pencils
Robots & Pencils develops digital strategies and products that deliver exponential impact to our clients. We design and build solutions that unlock data and insights, infuse intelligent automation, and accelerate product innovation across the organization.
Everything we do starts by blending the sciences with the humanities, the Robots with the Pencils. Our top-tier talent fuses creativity + technology to help brands transform their businesses, deliver delightful customer and employee experiences, and maintain a competitive edge amidst a constantly changing industry landscape.
Ready to join Robots and Pencils?
Take the next step in your career journey
Frequently Asked Questions
What does a AI Engineer (AI System Calibration & Optimization) do at Robots and Pencils?
As a AI Engineer (AI System Calibration & Optimization) at Robots and Pencils, you will: the AI Engineer will partner with a strategic client to improve the accuracy and reliability of their AI systems through calibration and optimization. Responsibilities include building calibration systems, optimizing prompts, and establishing workflows for continuous improvement..
Is the AI Engineer (AI System Calibration & Optimization) position at Robots and Pencils remote?
The AI Engineer (AI System Calibration & Optimization) position at Robots and Pencils is based in Seattle, Washington, United States. Contact the company through Clera for specific work arrangement details.
How do I apply for the AI Engineer (AI System Calibration & Optimization) position at Robots and Pencils?
You can apply for the AI Engineer (AI System Calibration & Optimization) position at Robots and Pencils directly through Clera. Click the "Apply Now" button above to start your application. Clera's AI-powered platform will help match your profile with this opportunity and guide you through the application process.