Sr. Responsible AI Analyst

The Sr. Responsible AI Analyst / Responsible AI Analyst will be responsible for supporting the design, development, evaluation, and implementation of Responsible AI solutions across model lifecycle stages. The role involves conducting technical assessments, building evaluation datasets, defining metrics, supporting model testing, contributing to guardrail design, assisting in deploying Responsible AI workflows, and collaborating with engineering, product, governance, ISG, and DPO teams to ensure that AI systems adhere to ethical, safety, fairness, transparency, and regulatory expectations.

Responsibilities

1. Execute Responsible AI Evaluations:

Conduct structured assessments on fairness, safety, robustness, explainability, and hallucination risks for models under development or deployment.

2. Design Evaluation Datasets and Metrics:

Create curated, adversarial, and edge-case datasets and define quantitative metrics for evaluating fairness, toxicity, reliability, and ethical alignment.

3. Support Guardrail and Control Implementation:

Contribute to implementing technical guardrails, safety filters, explainability modules, and automated checks within AI pipelines.

4. Analyze Ethical and Technical Risks:

Review datasets, model outputs, and system behavior to identify fairness gaps, robustness issues, transparency deficiencies, and other Responsible AI risks.

5. Participate in Red Teaming and Stress Testing:

Assist with scenario-based adversarial evaluations, prompt safety checks, robustness tests, and model vulnerability analysis.

6. Support Deployment of Responsible AI Workflows:

Assist in implementing lifecycle governance workflows, templates, and processes—such as via IBM OpenPages or equivalent governance tooling.

7. Prepare Transparency and Governance Documentation:

Develop model cards, system cards, evaluation reports, risk logs, and supporting documentation required for governance reviews and audit readiness.

8. Assist in Continuous Monitoring:

Support creation of dashboards, metrics, and monitoring signals to track fairness drift, hallucination patterns, model instability, and safety deviations.

9. Collaborate Across Engineering and Governance Functions:

Work closely with AI engineers, data scientists, product teams, legal, ISG, DPO, and governance bodies to ensure Responsible AI requirements are consistently applied.

10. Assist in Training and Knowledge Enablement:

Help develop training content, guides, and resources to educate internal teams on Responsible AI evaluation methods, guardrails, and governance expectations.

Technical requirements

Proficiency in Python and ML/DL frameworks (PyTorch, TensorFlow) for evaluation and experimentation.

Understanding of fairness libraries (Fairlearn, AIF360) and ability to compute ethics related evaluation metrics.

Familiarity with explainability tools (SHAP, LIME, Captum, Integrated Gradients).

Exposure to red teaming concepts, prompt safety evaluation, and data integrity checks.

Experience with MLOps basics including evaluation pipelines, experiment tracking, and CI workflows.

Understanding of ML algorithms, generative models, and supervised/unsupervised learning techniques.

Familiarity with NLP, vision, speech, and structured data domains.

Knowledge of datasets, benchmark suites, and third party model ecosystems.

Education

Bachelor of Engineering