AI and LLM Engineer

We are seeking a hands-on AI & LLM Engineer to join the AWS AI/GenAI Solutions Team within the IRS Advanced Analytics Program (AAP). This role focuses on the practical development and deployment of large language models (LLMs) and GenAI solutions using AWS services such as SageMaker, Bedrock, and open-source frameworks.

The engineer will be directly responsible for coding pipelines, fine-tuning models, building inference endpoints, and integrating GenAI workflows into production. By working closely with data engineers, architects, and Trustworthy AI specialists, this role ensures that GenAI capabilities are secure, scalable, and aligned with IRS mission needs.

Key Responsibilities:

• Build end-to-end LLM pipelines: data preparation, training, fine-tuning, and evaluation of models using SageMaker and Bedrock.
• Develop prompt engineering strategies, chaining pipelines, and custom evaluation scripts to validate LLM behavior.
• Implement RAG (retrieval-augmented generation) workflows by integrating LLMs with IRS data sources.
• Code and deploy inference endpoints, APIs, and integration layers for mission teams to consume LLM services.
• Optimize model performance, latency, and cost through benchmarking, hyperparameter tuning, and scaling strategies.
• Embed bias detection, fairness, and explainability checks in model pipelines, in partnership with Trustworthy AI specialists.
• Contribute to CI/CD automation for LLM deployments, including rollback and retraining workflows.
• Write production-grade Python code and leverage frameworks such as Hugging Face Transformers, LangChain, PyTorch, or TensorFlow.
• Document workflows and create reusable templates/accelerators for faster onboarding of new GenAI use cases.
• Participate in hands-on troubleshooting and debugging of pipelines, deployments, and model behavior.

Required Qualifications:

• Bachelor’s or master’s degree in computer science, Data Science, or related field.
• 5+ years of hands-on AI/ML engineering experience, including direct model training, fine-tuning, and deployment.
• Strong expertise in Python programming and ML/LLM frameworks (Hugging Face, LangChain, PyTorch, TensorFlow).
• Experience with AWS AI services (SageMaker, Bedrock, S3, Lambda, Step Functions) in production workflows.
• Proven ability to build and deploy inference endpoints and APIs for AI/ML workloads.
• Familiarity with CI/CD pipelines and IaC (Terraform, CloudFormation) for model deployment.
• Practical understanding of LLM evaluation methods (prompt testing, bias/toxicity detection, response consistency).

Desired Skills:

• Certifications: AWS Certified Machine Learning Specialty or equivalent.
• Experience implementing RAG pipelines or multi-model orchestration for enterprise use cases.
• Familiarity with federal compliance frameworks (FedRAMP, NIST 800-53) and secure AI/ML operations.
• Knowledge of Trustworthy AI principles (auditability, explainability, fairness) in LLM contexts.
• Strong problem-solving skills and ability to debug real-world AI/LLM issues in production.