We are sharing a full-time opportunity for experienced medical and healthcare researchers with deep expertise in clinical reasoning, evidence synthesis, healthcare research, benchmarking, and AI evaluation to contribute to advanced research on trustworthy healthcare-focused AI systems.
Selected professionals will design rigorous evaluation frameworks, develop benchmark datasets and validation environments, analyse medical reasoning and failure patterns, and collaborate with clinicians, researchers, and engineers to improve AI systems supporting healthcare, biomedical, and clinical workflows.
Key Responsibilities
Healthcare AI Evaluation Frameworks
- Design evaluation frameworks for AI systems operating in medical and healthcare domains
- Develop benchmark structures, scoring methodologies, and clinical quality rubrics
- Define validation protocols for healthcare-oriented AI systems
- Establish consistent standards for measuring medical reasoning and decision-support quality
- Refine evaluation criteria as model capabilities and healthcare use cases evolve
Medical Research & Evidence Synthesis
- Conduct applied research on medical reasoning and evidence-based decision-making
- Analyse clinical workflows and healthcare information-synthesis processes
- Evaluate the quality and relevance of medical and biomedical evidence
- Apply research-design principles to healthcare-focused AI evaluation
- Translate complex medical findings into structured research conclusions
Benchmark & Evaluation Environment Development
- Develop benchmark datasets based on realistic healthcare and biomedical scenarios
- Create evaluation environments reflecting practical clinical and research workflows
- Design tasks that test reasoning, evidence integration, and healthcare decision support
- Support development of agentic medical and healthcare evaluation workflows
- Ensure benchmark design reflects meaningful real-world healthcare challenges
Safety, Reliability & Failure Analysis
- Analyse model performance across medical and healthcare use cases
- Identify reasoning weaknesses, failure patterns, and safety considerations
- Evaluate reliability across clinical, biomedical, and healthcare decision-support scenarios
- Investigate where AI-generated outputs diverge from professional standards
- Produce structured recommendations for improving model quality and trustworthiness
Research Collaboration & Standards Development
- Collaborate with clinicians, researchers, engineers, and multidisciplinary stakeholders
- Produce research reports, evaluation methodologies, and best-practice guidelines
- Contribute to healthcare AI benchmarking and reliability standards
- Monitor developments in medicine, healthcare delivery, biomedical research, and AI
- Help shape rigorous industry practices for evaluating healthcare-focused AI systems
Ideal Profile
- Advanced degree in Medicine, Public Health, Nursing, Pharmacy, Biomedical Sciences, Epidemiology, Health Economics, or a related healthcare field
- Relevant credentials may include MD, DO, PhD, MPH, PharmD, or equivalent advanced qualifications
- Deep expertise in clinical practice, healthcare operations, biomedical research, public health, healthcare analytics, or evidence-based medicine
- Demonstrated experience conducting medical, clinical, biomedical, or healthcare research
- Strong understanding of research design and evidence evaluation
- Strong knowledge of healthcare decision-making processes
- Excellent analytical reasoning and scientific communication skills
- Experience working across multidisciplinary research, healthcare, technology, or policy environments
- Familiarity with digital health or clinical decision-support systems is advantageous
- Experience with healthcare analytics or health technology assessment is beneficial
- Knowledge of healthcare quality measurement, clinical guidelines, evidence synthesis, or benchmarking is valuable
- Published medical, clinical, healthcare, or biomedical research is advantageous
- Experience with clinical studies, systematic reviews, or healthcare evaluation initiatives is beneficial
- Experience with AI evaluation or benchmarking is useful but not required
Engagement Details
- Full-time engagement
- Fully remote
- Compensation: $400,000–$800,000/year
- Work will involve healthcare AI evaluation, medical research, benchmark development, evidence synthesis, and reliability analysis
- Assignments may span clinical reasoning, healthcare decision support, biomedical research, healthcare analytics, and agentic medical workflows
- Strong research-design, evidence-evaluation, and healthcare-domain expertise is central to this role
- Collaboration will involve researchers, clinicians, engineers, and other multidisciplinary contributors
- Equity compensation and performance-based incentives may be available depending on role and applicable policies
- Benefits may include health-insurance support, paid time off, retirement-plan contributions, and other employment benefits
- Project scope, research priorities, benchmarks, and evaluation standards may evolve over time
- Work must be completed without using confidential, proprietary, patient-identifiable, or unpublished information belonging to any employer, healthcare organisation, research institution, client, or other third party
About the Platform
This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.
By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy