Senior Data Scientist, AI

JOB SUMMARY

• The Senior Data Scientist, AI executes data science, machine learning, statistical modeling, generative AI, agentic AI, evaluation, and advanced analytics work required to power BSWH's priority AI use cases and workflow-enabled solutions.
• This role partners closely with AI Product Managers, Data Product Managers, AI Engineers, Software Engineering, Ontology and Knowledge teams, Quality Engineering, Clinical, Operations, Digital, Journey, Privacy, Security, and Governance stakeholders to translate business and clinical questions into rigorous data science solutions.
• The role develops predictive and statistical models, LLM- and agent-based capabilities, analytical frameworks, evaluation approaches, outcome measures, and reusable AI/data science assets that help AI-enabled products improve customer experience, operational efficiency, clinical workflows, personalization, decision support, and enterprise performance.
• This role is expected to combine strong statistical and machine learning skills with hands-on generative AI and agent engineering capabilities, practical product judgment, healthcare data fluency, and the ability to translate ambiguous problems into deployed solutions that work in real operational workflows.
• The Senior Data Scientist, AI reports to the Vice President, Data Science and may operate as either a senior individual contributor or a small-team technical lead, depending on assigned domain and portfolio needs.

ESSENTIAL FUNCTIONS OF THE ROLE

AI Use Case Data Science Delivery

• Translate AI use-case objectives into data science problem statements, modeling approaches, analytical plans, success metrics, and execution roadmaps.
• Develop predictive models, classification models, segmentation approaches, personalization methods, statistical analyses, causal inference approaches, optimization models, simulations, or other traditional data science methods, and combine them with LLM- and agent-based approaches when appropriate to support AI use cases.
• Partner with product, engineering, data, ontology, and clinical/operational stakeholders to ensure data science work is aligned to real workflow needs and measurable outcomes.
• Rapidly prototype, validate, iterate, and productionalize data science, generative AI, and agentic methods in partnership with engineering and platform teams, with a bias toward delivering working capabilities into real user workflows.

Model Development, Validation & Evaluation

• Build, evaluate, tune, and monitor data science and machine learning models using appropriate statistical, ML, and AI methods.
• Define model success criteria, comparison benchmarks, holdout strategies, validation methodologies, error analysis approaches, and ongoing monitoring expectations.
• Develop evaluation approaches for AI-enabled products, including performance measurement, quality thresholds, subgroup analysis, drift detection, fairness considerations, and operational impact tracking.
• Ensure modeling outputs are interpretable, reliable, well-documented, and appropriate for the risk and decision context of the use case.

Generative AI, Agentic AI & Workflow Solution Delivery

• Design, build, evaluate, and deploy LLM-powered applications and agentic solutions, including retrieval-augmented generation (RAG), tool/function calling, structured outputs, reasoning workflows, and multi-step agent patterns.
• Build and orchestrate agents that combine enterprise data, analytical models, APIs, business rules, knowledge/ontology assets, and software tools to perform useful work across analytics, decision support, and operational workflows.
• Select and apply appropriate agent architecture patterns, including single-agent and multi-agent approaches, routing, planning, delegation, memory/context management, human-in-the-loop controls, and deterministic workflow components where reliability requires them.
• Develop and maintain agent tools, prompts/instructions, context and retrieval strategies, orchestration logic, guardrails, and interfaces that enable LLMs and agents to interact safely and reliably with enterprise systems and workflows.
• Create evaluation suites for LLM and agent behavior, including task success, groundedness, factuality, tool-use correctness, workflow completion, latency, cost, safety, robustness, and regression testing; use evaluation results to iteratively improve deployed capabilities.
• Work directly with product, engineering, clinical, and operational partners to understand real-world processes, rapidly prototype solutions in context, and evolve prototypes into production capabilities that measurably improve analytics or workflow execution.

Healthcare Data, Feature Engineering & Analytical Assets

• Work with enterprise healthcare datasets, including clinical, operational, customer, scheduling, claims, financial, digital, journey, or other data domains required to support AI use cases.
• Partner with Data Product Managers and data engineering teams to define required data elements, feature sets, data quality expectations, latency/freshness requirements, lineage needs, and remediation priorities.
• Create reusable features, cohorts, labels, datasets, metrics, and analytical assets that can support multiple AI use cases and reduce rework across teams.
• Evaluate data quality, missingness, representativeness, bias, leakage, measurement error, and other data limitations that affect model validity and business interpretation.

Experimentation, Measurement & Business Impact

• Design analytical frameworks to measure the value of AI-enabled products, including adoption, user behavior, customer experience, clinical or operational outcomes, efficiency gains, and financial impact.
• Support A/B testing, quasi-experimental design, causal inference, uplift modeling, attribution, cohort analysis, and longitudinal measurement where appropriate.
• Develop dashboards, monitoring views, analytical narratives, and executive-ready insights that help stakeholders understand whether AI use cases are working as intended.
• Partner with product and operational leaders to translate analytical results into prioritization decisions, product improvements, workflow changes, or model refinements.

MLOps, Reuse & Production Partnership

• Partner with AI Engineers, ML engineers, software engineers, and platform teams to move validated statistical/ML models, LLM applications, agents, and orchestration workflows from notebook or prototype into reliable production capabilities.
• Support model and agent documentation, versioning, reproducibility, observability, monitoring, automated validation/evaluation, performance checks, and operational runbooks required for production-grade AI solutions.
• Contribute to reusable data science patterns, libraries, templates, evaluation assets, and standards that improve speed and consistency across AI use-case teams.
• Work with Quality Engineering, Governance, Privacy, Security, and Responsible AI stakeholders to ensure data science methods are safe, auditable, compliant, and fit for purpose.

Cross-Functional Collaboration & Communication

• Communicate complex analytical, statistical, and machine learning concepts clearly to executives, product managers, engineers, clinicians, operational leaders, and other stakeholders.
• Collaborate with clinical and operational subject matter experts to ensure assumptions, data definitions, model outputs, and interpretation are grounded in real-world workflows.
• Mentor peers and contribute to the growth of BSWH's data science practice through code reviews, methodology reviews, best-practice sharing, and reusable standards.
• Stay current on developments in healthcare data science, machine learning, generative AI evaluation, model monitoring, responsible AI, and applied AI methods.

KEY SUCCESS FACTORS

• Proven ability to use both traditional data science/machine learning methods and modern LLM/agentic AI methods to solve ambiguous business, clinical, operational, customer, analytics, or workflow problems.
• Strong statistical and analytical judgment, including the ability to select methods appropriate to the data, decision context, and risk profile of the use case.
• Ability to work with large, complex, imperfect healthcare or enterprise datasets and identify data quality, bias, leakage, missingness, and interpretability concerns.
• Strong partnership with product, engineering, data, ontology, clinical, operational, governance, privacy, and security stakeholders to deliver usable and trusted AI-enabled capabilities.
• Ability to turn analytical findings into clear recommendations, measurable outcomes, product improvements, and executive-ready narratives.
• Strong coding, reproducibility, documentation, and production handoff discipline so models and analytical assets can be maintained and reused.
• Ability to move fluidly between analysis, modeling, software-oriented prototyping, and workflow implementation - choosing traditional ML, LLMs, agents, deterministic logic, or a hybrid architecture based on the problem rather than the technology.
• Comfort operating in a regulated or high-trust environment where privacy, safety, compliance, auditability, and responsible AI practices are essential.

BENEFITS

Our competitive benefits package includes the following

• Immediate eligibility for health and welfare benefits
• 401(k) savings plan with dollar-for-dollar match up to 5%
• Tuition Reimbursement
• PTO accrual beginning Day 1

Note: Benefits may vary based upon position type and/or level

MINIMUM REQUIREMENTS

Education

• Bachelor's degree in Data Science, Statistics, Biostatistics, Computer Science, Mathematics, Engineering, Econometrics, Health Informatics, or related quantitative field.
• Master's degree preferred; PhD preferred for candidates focused on advanced modeling, clinical AI evaluation, or complex machine learning methods.

Experience

• 5-8+ years of experience in data science, machine learning, advanced analytics, applied statistics, product analytics, health analytics, or a related quantitative role.
• Experience developing models, analytical frameworks, datasets, features, metrics, or decision-support methods that influence product, operational, clinical, or business outcomes.
• Experience working with cross-functional teams including product, engineering, data engineering, analytics, operations, clinical/business stakeholders, or governance teams.
• Experience using Python, SQL, statistical modeling, machine learning, experimentation, data visualization, generative AI/LLMs, and modern analytics, ML, or AI development tooling.
• Hands-on experience building and deploying LLM-powered or agentic capabilities that use enterprise data and tools to complete multi-step analytical or workflow tasks; experience integrating agents with APIs, data platforms, applications, or operational systems.
• Healthcare, life sciences, payer, provider, digital health, financial services, or other regulated-industry experience preferred but not required.

Required Technical Expertise

• Strong programming skills in Python and SQL, with the ability to work across modern data science and AI engineering environments, build production-oriented prototypes, integrate APIs/tools, and collaborate effectively with software and platform engineering teams.
• Strong knowledge of statistics, machine learning, experiment design, causal inference, forecasting, classification, clustering, segmentation, optimization, or other relevant data science methods.
• Experience with common data science libraries and tools such as pandas, NumPy, scikit-learn, statsmodels, PyTorch, TensorFlow, Spark, Databricks, Snowflake, or comparable platforms.
• Ability to design validation approaches, evaluate model performance, conduct error analysis, monitor drift, assess subgroup performance, and communicate model limitations.
• Ability to work with structured and unstructured data sources, including healthcare, customer, operational, digital, or enterprise data products.
• Hands-on expertise with generative AI and LLM application development, including prompt/instruction design, RAG and semantic retrieval, embeddings/vector search, structured outputs, tool/function calling, context management, model selection, and LLM evaluation.
• Experience building agents and agentic workflows using one or more modern orchestration frameworks or platform-native agent capabilities; ability to design tool-enabled, multi-step workflows and determine when single-agent, multi-agent, or deterministic orchestration is appropriate.
• Ability to design and implement agent tools and integrations that connect AI capabilities to enterprise data, APIs, analytical models, knowledge assets, and workflow systems, with appropriate authentication, permissions, guardrails, observability, and human review.
• Ability to evaluate and monitor non-deterministic AI systems using repeatable test cases and metrics for quality, groundedness, task completion, tool use, safety, latency, cost, and regression performance.
• Working knowledge of software engineering practices relevant to production AI, including source control, testing, CI/CD, API integration, containers or cloud runtime patterns, and operational troubleshooting.

PREFERRED QUALIFICATIONS

• Experience building or evaluating data science methods for AI-enabled products, digital health experiences, care navigation, scheduling, personalization, capacity, throughput, engagement, or operational workflow use cases.
• Experience with healthcare data and standards such as EHR data, claims, clinical documentation, encounters, medications, labs, ICD-10, CPT, SNOMED CT, LOINC, RxNorm, HL7, or FHIR.
• Experience creating reusable features, analytical datasets, model evaluation assets, golden datasets, quality checks, or data science templates that can be used across teams.
• Experience with model and agent observability, MLflow or similar MLOps/LLMOps tooling, data/version control, automated model and LLM evaluation, runbooks, and production handoff practices.
• Experience with agent/orchestration frameworks and AI platform capabilities (for example, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI Agents SDK, cloud-native agent platforms, or comparable frameworks), and with protocols/patterns for securely exposing tools and context to agents.
• Experience applying data science to regulated or high-trust decision contexts where safety, privacy, explainability, fairness, or auditability are important.
• Experience mentoring junior data scientists, leading technical workstreams, or serving as a technical lead for cross-functional data science projects.