Machine Learning / Data Engineer
Salary not listed • Remote • Colombia, Huila, Colombia; São Paulo, Brazil • Full-time
Posted 24d ago
Job Location
Colombia, Huila, Colombia; São Paulo, Brazil
Tech Stack
Remote Work Policy
Fully remote
Employment Type
Full-time
Categories
Machine Learning Engineer
About the job
This role focuses on ensuring the quality and sensitive data compliance of enterprise data as it flows through our systems. You will be responsible for building machine learning models that detect and replace sensitive information like PII, PHI, and company-identifiable data in text and image formats. This involves developing detection models, validation infrastructure, adversarial test sets, and audit processes to guarantee data integrity and de-identification performance. You will work closely with a senior ML lead, focusing on hands-on engineering with a strong QA mindset, ensuring data remains useful and secure for downstream applications.
Responsibilities
- Assess data quality by performing deep dives into enterprise data, evaluating topic coherence, domain depth, completeness, and consistency.
- Design and automate validation suites for data pipelines, including schema checks, completeness, drift detection, and reconciliation across different stages.
- Surface and characterize data quality issues for engineering and product teams.
- Design, train, and evaluate ML models (e.g., NER) to detect sensitive entities in text and image data.
- Build replacement pipelines to substitute detected sensitive entities with consistent alternatives, maintaining data utility.
- Run de-identification algorithms on large datasets for downstream agentic task building.
- Develop adversarial test sets to challenge de-identification across various sensitive data classes and formats.
- Specifically address company identifiable information detection, including organization names, domains, internal project names, and relationships.
- Cover financial data detection, including account numbers, revenue figures, and transaction records.
- Measure and report de-identification performance, including entity-level precision and recall, leak rates, and replacement consistency.
- Implement regression gates in CI/CD to ensure data quality and sensitive-data checks pass before pipeline changes are deployed.
- Conduct sampling-based human-in-the-loop audits and maintain audit trails for compliance evidence.
- Partner with engineering to analyze inconsistencies or leaks and drive resolutions.
Requirements
- Approximately 4-5 years of hands-on machine learning experience with ML as the primary background.
- Proficiency in Python for ML development and data validation using tools like pytest or Great Expectations.
- Solid SQL skills and experience validating data across pipeline stages.
- Familiarity with sensitive data categories and relevant standards (e.g., HIPAA, GDPR, PCI DSS).
- Experience building and testing ML-based detection systems (e.g., NER), including creating labeled evaluation sets and handling non-determinism.
- Understanding of re-identification risk and how data points can combine to reveal sensitive information.
- Comfort working with ambiguity in a fast-moving environment.
- A detail-oriented and skeptical approach to problem-solving.