The Conversational Shopping team is looking for a Language Engineer to drive efficiencies and innovation in its efforts to deliver a seamless, fluent, and engaging experience for AI-assisted shopping. This is an opportun…
Skills: Computational Linguistics, Large Language Models, Python, Natural Language Processing, Data Annotation
If you are a current associate, you will need to apply through our internal career site. Please log into Workday and click on the Jobs Hub app or search for Browse Jobs. Veterinary Assistant Careers at Banfield Pet Hospi…
If you are a current associate, you will need to apply through our internal career site. Please log into Workday and click on the Jobs Hub app or search for Browse Jobs. Client Service Coordinator SUMMARY OF JOB PURPOSE …
Skills: Customer Focus, Communication Skills, Client Service Skills, Time Management, Priority Setting
Interested in working on an innovative and high-impact business that is changing the way customers think about grocery? Amazon is looking for a talented Vendor Manager (VM) responsible for managing vendor relationships a…
Now Brewing – senior executive assistant to evp - #tobeapartner From the beginning, Starbucks set out to be a different kind of company. One that not only celebrated coffee and the rich tradition, but that also brought a…
Compensation TypeHourly Highgate Hotels Highgate is a leading real estate investment and hospitality management company with over $15 billion of assets under management and a global portfolio of more than 400 hotels span…
Premier Client Relationship Consultant 5 (Banker) - Magnolia - Seattle, WA
Seattle, Washington, United States · On-site
$37/hr–$45/hr
Senior
At U.S. Bank, we’re on a journey to do our best. Helping the customers and businesses we serve to make better and smarter financial decisions and enabling the communities we support to grow and succeed. We believe it tak…
InnVentures Overview Driven by an innovative, entrepreneurial spirit, InnVentures has a culture of building teams that produce winning results. We operate over 75 hotels in 20+ states for some of the Nation’s largest REI…
Skills: Direct Sales, Team Leadership, Negotiation, Account Management, Market Analysis
Job Description UW Medicine Digital Health Office has an outstanding opportunity for a Program Manager, External eConsults/Virtual Care WORK SCHEDULE 100% FTE Weekdays Onsite as business needs require DEPARTMENT DESCRIPT…
Loot Labs is on a mission to make collecting fun. We believe that every collectible tells a story: of passion, nostalgia, and discovery. Through our flagship product, Boxed.gg we are reimagining how the world collects by…
Senior Technical Program Manager, Elastic Block Store
Seattle, Washington, United States · On-site
$149k–$201k/yr
Senior$35B raised
Join Elastic Block Store (EBS), the solid-state drive for the Internet, which stores exabytes of data for millions of customers around the world running GenAI training and inference, large-scale data processing, database…
Skills: Technical Program Management, Project Management, Block Storage, Network Protocols, Linux OS
LOCATION: 600 Pine Street, Seattle, WA 98101 (Pacific Place) COMPENSATION: $23.25/hour The base salary or hourly wage range for this role will vary based on multiple factors, including but not limited to prior experience…
Multi-Site Maintenance Technician III - Arrive First Hill
Seattle, Washington, United States · On-site
$26/hr–$29/hr
Mid level
About Trinity Property Consultants At Trinity Property Consultants, your expertise fuels our mission. Guided by our values of innovation, teamwork, excellence, and empowerment, we’ve been a leader in the multifamil…
Skills: HVAC Troubleshooting, Electrical Systems Repair, Plumbing Systems Repair, Project Oversight, Vendor Coordination
Overview Responsible for providing E&S Underwriters with information necessary to make sound underwriting decisions (i.e., marketing, customer service and administrative support affecting brokers/insureds from initial su…
Are you passionate about shaping growth strategy for one of Amazon's most innovative businesses? Amazon Business is a fast-growing, $40+ billion global business revolutionizing procurement for organizations worldwide. We…
Skills: Financial Planning, ROI Assessment, Financial Modeling, P&L Controllership, Data Mining
About BRINC: At BRINC, we are redefining public safety with an innovative ecosystem of life-saving tools. Our journey started with the development of drones and ruggedized throw phones, designed to access unsafe areas an…
National Electrical Distributor Sales Account Executive
Seattle, Washington, United States · Hybrid
$105k–$140k/yr
Senior
Here at King Electric, we build smart, reliable, and efficient electric heaters with unparalleled user experience. We’re looking for people who enjoy working with a team, are committed to delivering outstanding customer …
Overview SP+, a Metropolis company, is an artificial intelligence company for the real world. We use computer vision to enable checkout-free parking experiences. So there’s no fumbling with tickets, machines, apps, or cr…
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
$83k–$132k/yr
Full-time
postgraduate degree
Health Insurance, Medical Insurance, Dental Insurance, Vision Insurance, Prescription Insurance, Basic Life & AD&D Insurance
Posted 47d ago
~40 hrs/week
Responsibilities
Develop and automate LLM-assisted evaluation tools and workflows to improve the AI-assisted shopping experience. Collaborate with cross-functional teams to define quality metrics and lead the production of high-quality editorial data.
Requirements
Requires a Master's degree or higher in Computational Linguistics and experience executing language data collection projects. Preferred candidates have 2+ years of experience in computational linguistics and a strong understanding of LLM fundamentals.
Full job description
The Conversational Shopping team is looking for a Language Engineer to drive efficiencies and innovation in its efforts to deliver a seamless, fluent, and engaging experience for AI-assisted shopping. This is an opportunity to join the high-performing team behind Amazon’s Generative AI shopping initiatives such as Rufus AI, Amazon’s Conversational Shopping assistant. Our objective is to make it easy for customers worldwide to find and discover the best products, meet their personalized needs with product research, providing comparisons and recommendations, answering specific product questions, and more. This role is inherently high-visibility and highly cross-functional, requiring collaboration and influence across global product, design, science, and engineering teams.
We are looking for candidates who are passionate about the intersection of language and technology and who are keen to use their technical abilities to develop automated, scalable solutions to questions in the Large Language Model (LLM) space. Applying a combination of expertise in LLMs, coding and linguistics (i.e., semantics, syntax, pragmatics), they will overcome complex problems in natural language processing (NLP), language understanding and automated AI evaluations.
In this role within the Editorial team, they will act as one of the driving forces behind our evaluation-driven product development strategy. They will design processes to facilitate the production of high quality editorial data which will allow us to evaluate and improve the Shopping AI experience in different languages. To do so, they will be tasked with the creation and development of LLM-assisted editorial tools, automated verification scripts and automated annotations (e.g. LLM-as-a-judge) to support the humans-in-the-loop (HITL) work of the broader Editorial team. They will lead and drive the requirements behind data annotation tasks and tooling, writing intuitive annotation guidelines and guiding the creation of the tools adapted to these workflows. They will employ their data processing and analysis skills to track team productivity and measure output quality. They will work in close collaboration with other Language Engineers, AI Editors, Product Managers, Applied Scientists and Software Engineers on initiatives that drive editorial quality, speed and consistency. By creating and synthesizing quality metrics, they will also guide Conversational Shopping teams in delivering both internal stakeholder requirements and achieve the desired Amazon customer outcomes.
This role requires strong analytical and technical skills as well as experience in language technology to help us measure, analyze and solve complex problems. They should have experience in creating technical solutions for automating and processing data workflows at scale and have the ability to do so while upholding the highest linguistic quality standards. They should also have exceptional writing and communication skills with the ability to interface between both technical and non-technical teams.
Key job responsibilities Produce, process and manipulate different types of language data, analyze, and provide efficient solutions Automate operations and perform data analysis using coding/scripting language (e.g. Python) Develop LLM-assisted workflows and annotations solutions (e.g. LLM-as-a-judge) to support Human-in-the-loop evaluations Design and lead editorial data production/collection by defining scope with internal customer teams Define clear editorial workflows (SOPs) to meet or exceed the quality bar Adopt and design control mechanisms, metrics and methodologies for editorial and annotation quality Maximize productivity, process efficiency and quality through streamlined workflows, process standardization, documentation, audits and investigations on a periodic basis. Collaborate with editors, applied scientists, engineers, and product managers to deliver the optimal customer experience and define metrics, guidelines, and workflows to continue doing so Establish processes and mechanisms to onboard and train editors on an ongoing basis. Handle work prioritization and deliver based on business priorities. * Be flexible in changes to conventions deployed in response to customers’ requests and change workflows accordingly.
A day in the life - Build an LLM-powered judge that automatically scores thousands of AI responses - Write a clear evaluation guideline, then pair with a PM-T and Scientist to validate it captures what "good" actually looks like - Debug a Python pipeline that processes multilingual annotation data, spot a pattern in the errors, and ship a fix - Join a cross-functional sync to align on quality metrics for an upcoming feature launch - Analyze evaluation results to surface insights that shape what the product team prioritizes next
You'll split your time between hands-on technical work (code, prompts, data) and collaborative problem-solving with editors, engineers, and PMs.
About the team We're the language technology team within Amazon's conversational shopping organization. We build and operate LLM-as-a-Judge systems that automatically measure response quality across every customer experience, develop agentic evaluation architectures that resolve cases single-hop judges can't, and create the tooling and automation that let a small team evaluate millions of AI responses at scale. You'll work alongside language engineers, AI editors, data scientists, and product managers, collaborating cross-functionally with applied scientists and software engineers to ship judges, define quality standards, and turn evaluation data into product decisions for customers around the world.
Basic Qualifications:
- Master's degree or above in Computational Linguistics - Experience owning and executing language data collection projects, including guidelines, labelset and annotation workflow development
Preferred Qualifications:
- 2+ years of computational linguistics, language data processing, semantics, philosophy of language experience - Experience in development in the last 3 years, or experience with Machine Learning and Large Language Model fundamentals, including architecture, training/inference lifecycles, and optimization of model execution
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, WA, Seattle - 82,700.00 - 131,600.00 USD annually
Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. We are driven by the excitement of building technologies, inventing products, and providing services that change lives. We embrace new ways of doing things, make decisions quickly, and are not afraid to fail. We have the scope and capabilities of a large company, and the spirit and heart of a small one.
Together, Amazonians research and develop new technologies from Amazon Web Services to Alexa on behalf of our customers: shoppers, sellers, content creators, and developers around the world.
Our mission is to be Earth's most customer-centric company. Our actions, goals, projects, programs, and inventions begin and end with the customer top of mind.
You'll also hear us say that at Amazon, it's always "Day 1." What do we mean? That our approach remains the same as it was on Amazon's very first day - to make smart, fast decisions, stay nimble, invent, and focus on delighting our customers.
Offices: 2127 7th Ave., Seattle, WA 98109, US · 12900 Worldgate Dr, Herndon, VA 20170, US · 7200 Discovery Dr, Chattanooga, TN 37416, US · 1100 Enterprise Way, Sunnyvale, CA 94089, US · 2010 Broening Hwy, Baltimore, MD 21224, US