Responsible ML Data Science Engineer

Responsibilities

• Partner with the Salesforce AI Research Science team, as well as distributed Data Science and Engineering teams, to measure and remediate potential bias in data and models.
• Build datasets for testing models, both generative and predictive, in collaboration with partner teams.
• Participate in technical discussions, and drive engineering investments in responsible AI to improve model fairness and safety prior to launch.
• Create notebooks using SQL or Python, and other tools, to empower other members of OEHU with advanced analytics capabilities.
• Compare between different tools and libraries options (both Salesforce-built and open-source) to identify the most appropriate one for the task at hand.
• Develop quantitative metrics to help surface insights from structured datasets.
• Plan and conduct end-to-end experiments and analyses, from gathering requirements through processing and modeling the data, to identifying emerging risks.
• Monitor and analyze logs data for harms/ threats and insights.
• Validate any synthetic data generation, crowdsource, or contractor jobs for labeling or data creation.
• Participate in labeling test data in partnership with OEHU and partner teams.
• Contribute to the creation and use of libraries and tools that perform testing, evaluation, and mitigation of content safety issues and algorithmic bias.
• Collaborate with industry leaders in similar positions in peer organizations on ways to improve the state of responsible AI development.
• Extract logs and data from various parts of the tech stack for analysis.

Skillset

• Practical experience in machine learning.
• MS in a quantitative discipline with 8+ years of experience.
• Fluent in building/prototyping machine learning models and algorithms and wrangling large datasets.
• Knowledgeable about standard machine learning approaches (Regression, Cross-Validation, Boosting, Matrix-Factorization, Decision Trees, Clustering, CNNs, RNNs, Transformers, GANs).
• Proficient in using Python and common machine learning frameworks (e.g., TensorFlow, Pandas, PyTorch, SciPy, scikit-learn, JAX) to implement models and algorithms.
• Proficient in SQL, shell scripting, and Unix/Linux command-line tools.
• Experience with designing and building micro-services, familiar with Kubernetes/containerization/RESTful API/gRPC, etc.
• Grasp of the evolving understanding of fairness and ability to meet both state-of-the-art and global standards for fairness evaluation, particularly in generative AI.
• Experience working across teams of engineers, data scientists, and researchers.
• Strong communication skills. Comfortable presenting ideas to diverse teams and individuals in multiple formats, from slide decks to informal chats.
• Earns trust in relationships both internally and externally, and at all levels of the organization. Challenges the status quo to improve the productivity, effectiveness, and culture of a team – without burning bridges.
• Ability to creatively prioritize, stage, and sequence solutions to challenging/complex problems.