Software Jobs in Arlington, VA (Now Hiring) — 1,189 open — Page 24

How do you measure what makes a great leader? How do you evaluate a development program when outcomes take years to materialize and clean experimental conditions are rarely available? How do you take a scientific methodology that a researcher validated carefully in one context and turn it into a system that any HR team across a company of over a million employees can run on their own? These are the kinds of questions the Senior Talent and Transformation Science team works on inside Amazon's People eXperience and Technology organization, and they are questions that matter: the systems this team builds shape how Amazon identifies, develops, and invests in its most senior leaders.

As an Applied Scientist on this team you are the person who closes the gap between a validated scientific methodology and a system that runs in production without a scientist standing next to it. The architectural decisions about how scientific methods get encoded into software, the engineering quality bar for the code that implements them, and the reliability of the pipelines that other teams depend on are yours to own. You will work alongside Senior and Principal Research Scientists, an Amazon Scholar, Product Management, and a Senior Applied Scientist who bring deep expertise in behavioral science, psychometrics, and causal inference, and you will be the driving force behind turning that expertise into working, deployable systems for our Amazon executives.

The problems you will be building for are genuinely hard and largely unsolved. Scoring a simulation-based leadership assessment with an LLM requires both measurement rigor and a production system that behaves consistently at scale. Estimating the effect of a talent program on leader outcomes requires both a defensible identification strategy and an analytical pipeline someone else can run and trust. Building a self-serve tool that lets a PXT team evaluate a new feature without calling a scientist requires both sound methodology and software that is robust enough to operate without expert supervision. If you want to do work that is technically demanding, scientifically cutting edge, and consequential for real leaders in a large organization, this is that role.

Key job responsibilities
• Own the production implementation of the team's scientific systems from end to end. When the team validates a new assessment methodology, evaluation framework, or causal identification strategy, you are the scientist who translates it into code that runs reliably, scales, and does not require a scientist standing next to it to operate.
• Make the architectural and tooling decisions that determine how scientific methods get encoded into software on this team, choosing abstractions, data structures, and system designs that make the team's scientific components testable, maintainable, and extensible over time.
• Define and hold the engineering quality bar for scientific code across the team, establishing and modeling best practices for testing, documentation, reproducibility, and peer review of code in a research team that does not have dedicated software development engineers.
• Build the LLM-powered pipelines that operationalize the team's people science, including prompt orchestration, retrieval grounding, automated scoring, and LLM-as-judge evaluation harnesses, writing the implementation yourself and owning the quality and reliability of those systems once deployed.
• Extend and adapt scientific techniques at the product level when established approaches fall short. When scoring a simulation-based assessment, estimating a program effect under unusual identification constraints, or evaluating a novel AI feature requires a methodological contribution that does not yet exist, you devise and implement that solution.
• Partner with the Research Scientists during methodology design to surface implementation feasibility and trade-offs early, contributing your own scientific judgment on what can be built rigorously within real production constraints before design decisions become expensive to reverse.
• Build reusable scientific components, services, and templates that encode methodology once and allow downstream teams to run it without scientist involvement, making the team's research operational infrastructure rather than a bespoke consulting engagement.
• Contribute to the design and execution of quasi-experimental evaluations of people programs, owning the analytical implementation and the code pipelines that produce defensible causal evidence from observational and field data.
• Mentor scientists on the team on software engineering practices and applied implementation, and participate actively in peer review of experiment designs, analytical approaches, and scientific code written by others.
• Communicate implementation trade-offs and system design decisions clearly to product and HR partners in written documents that connect technical choices to business outcomes.

A day in the life
Your day is anchored in building and testing. You might spend the morning working through a thorny implementation problem, figuring out how to encode a psychometric scoring model into a pipeline that holds up under the messiness of real production data, debugging an LLM evaluation harness that is behaving inconsistently across assessment scenarios, or refactoring a causal estimation component so that another team can run it without calling you first. In the afternoon a Research Scientist might pull you into a methodology design conversation, and your job in that room is not just to follow along but to push back on approaches that would be difficult or brittle to implement, and to propose alternatives that preserve scientific rigor while actually being buildable. You might then shift to reviewing a colleague's code, writing documentation that makes a deployed pipeline understandable to someone who was not in the room when it was designed, or working through a data pipeline problem that is blocking the team's ability to evaluate a new product feature. At the end of most days something that was not working is now working, and the science the team does is a little more durable and a little more independent of any one person than it was in the morning. Basic Qualifications: - Experience leading the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems, or experience building complex software systems that have been successfully delivered to customers
- PhD in industrial-organizational psychology, organizational behavior, economics, statistics, computer science, or a related quantitative discipline
- 5+ years of applied research experience after the PhD, with a demonstrable track record of delivering scientifically complex solutions into production systems that other teams depend on
- Strong software engineering skills in Python, including the ability to design, build, test, and maintain production pipelines independently without dedicated software development engineering support
- Deep scientific expertise in at least one of the following areas and enough working knowledge in the others to contribute meaningfully across the team's full research portfolio: psychometric measurement and validation, causal inference with observational and quasi-experimental data, or applied LLM systems including prompt orchestration and evaluation Preferred Qualifications: - Experience serving as the primary or sole implementer of scientific systems on a research team, where engineering quality and production reliability were your responsibility rather than a dedicated engineer's
- Hands-on experience building LLM pipelines including retrieval-augmented generation, automated scoring, and LLM-as-judge evaluation harnesses, with direct ownership of those systems in production
- Experience designing or validating simulation-based, work-sample, or structured assessment instruments in an applied organizational context, including familiarity with psychometric validation standards relevant to high-stakes talent decisions
- Applied experience with quasi-experimental methods such as difference-in-differences, regression discontinuity, matching, or synthetic control in field settings where identification strategy required genuine methodological judgment rather than textbook application
- Experience establishing and modeling software engineering best practices, such as testing, documentation, and code review, for colleagues who are strong scientists but not trained software engineers
- Publications or presentations at venues such as SIOP, AOM, NeurIPS, EMNLP, or peer-reviewed journals in measurement, causal inference, or machine learning

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.

USA, NY, New York - 183,800.00 - 248,700.00 USD annually
USA, VA, Arlington - 167,100.00 - 226,100.00 USD annually
USA, WA, Seattle - 167,100.00 - 226,100.00 USD annually