Machine Learning Engineer

Replay turns sensitive internal enterprise data into de-identified datasets without destroying the structure and meaning that make the data valuable. The data does not arrive in one clean modality. It spans messages, documents, tables, files, images, metadata, and provider-specific structures, with important context distributed across all of them.

You will improve how well our system understands and protects that data. Your initial scope will be a prioritized subset of named-entity recognition, entity and identity resolution, structured extraction, classification, semantic review, or other model-backed parts of the de-identification pipeline. We do not expect one person to be an expert in every modality. The goal is measurable improvement in the areas you own: better precision, recall, F1, high-risk coverage, and preserved data utility across the failure modes that matter.

This is an applied, production-facing ML role. You will study errors, form hypotheses, build datasets and experiments, improve or replace models, and ship the result into a live pipeline. Evaluation, reproducibility, observability, and safe releases matter because they let us identify, ship, and verify meaningful model improvements in production.

What You'll Do

• Own and improve NER, entity resolution, structured or tabular detection, document understanding, semantic review, or related de-identification systems
• Transform model failures and capability ceilings into a prioritized improvement roadmap
• Design active-learning loops that combine model sweeps, LLM-assisted review, clustering, and uncertainty signals to identify the examples most worth hand-labeling
• Build representative datasets and benchmarks, and use decision-relevant metrics to reveal strengths, weaknesses, uncertainty, and failure costs
• Choose and combine deterministic rules, classical ML, fine-tuning, embeddings, multimodal models, and LLM-based approaches based on the problem and evidence
• Design experiments, tune thresholds, analyze precision-recall and utility tradeoffs, and explain which changes are real, uncertain, or limited to particular conditions
• Productionize improvements with reproducible artifacts, evaluation evidence, runtime instrumentation, and safe rollout
• Optimize inference cost, latency, and throughput without hiding regressions in quality or high-risk recall
• Build high-fidelity evaluation environments with seeded failure modes and programmatic verifiers that expose subtle regressions
• Build reliable model- or agent-based harnesses with bounded behavior and explicit output verification when the problem calls for them
• Partner with Applied Science on measurement and calibration, Data and Product Engineering on pipeline and review systems, and Security and Quality on acceptable risk
• Use AI engineering tools deeply to accelerate research, implementation, error analysis, and evaluation while verifying their output

What Success Looks Like

• Model improvements generalize beyond the examples used to develop them and hold up in replay, shadow, and production evidence
• Priority modalities and entity classes show credible improvements in precision, recall, F1, or other decision-relevant quality measures
• High-risk misses decline without unacceptable over-redaction or loss of useful structure
• New formats and modalities can be covered without relying on brittle one-off fixes
• Improvements reduce meaningful delivery risk, review or rework burden, or loss of data utility rather than moving only an isolated benchmark
• The team can explain why a model changed, where it improved or regressed across consequential failure modes and data segments, and whether the change should ship
• The path from error discovery to a trustworthy production improvement becomes faster and more repeatable
• Quality gains remain inside acceptable inference-cost, latency, and operational constraints

This Role May Not Be for You If

• You want to focus on research novelty without owning measurable production improvement
• You prefer optimizing one aggregate benchmark without investigating consequential failure modes, data segments, and failure costs
• You want data preparation, evaluation, deployment, and production diagnosis to belong entirely to other teams
• You reach for a larger model before understanding the errors, constraints, and simpler alternatives
• You do not want AI tools to be part of your daily engineering and research workflow