We are sharing a specialised part-time consulting opportunity for experienced software engineering professionals with strong expertise in technical analysis, architecture, documentation, quality assurance, prompt development, and AI-output evaluation to contribute to an advanced AI training and software-engineering evaluation project.
Selected professionals will review AI-generated technical content, design and refine prompts, evaluate outputs against structured rubrics, and contribute high-quality engineering analysis and documentation. The work is suited to Software Engineers, Staff Engineers, Technical Leads, Engineering Managers, Solutions Architects, and similar professionals with strong technical judgement and communication skills. No prior experience in AI is required.
Key Responsibilities
Technical & Architecture Evaluation
- Review AI-generated software-engineering content for accuracy, completeness, and practical relevance
- Evaluate architecture, system-design, and implementation decisions
- Identify incorrect assumptions, weak reasoning, or maintainability, scalability, reliability, and performance concerns
- Review RFCs, design specifications, engineering proposals, and related technical materials
- Provide structured feedback grounded in professional engineering practice
Prompt Development & AI Evaluation
- Design and refine prompts for complex software-engineering scenarios
- Create tasks testing technical reasoning and engineering judgement
- Evaluate model outputs using detailed rubrics for correctness, relevance, and technical quality
- Identify ambiguous or edge-case outputs requiring deeper analysis
- Provide clear written rationales supporting evaluation decisions
Technical Writing & Quality Assurance
- Draft and edit technical documentation, reports, and analytical summaries
- Conduct research and fact checking to validate technical claims
- Review and annotate engineering content according to project standards
- Support consistency and calibration across evaluation workflows
- Translate complex engineering concepts for technical and non-technical audiences
Ideal Profile
- 3+ years of professional software-engineering experience
- Background as a Software Engineer, Senior Software Engineer, Staff Engineer, Technical Lead, Engineering Manager, Solutions Architect, or comparable professional
- Strong analytical reasoning and technical problem-solving skills
- Experience authoring or reviewing technical documentation
- Familiarity with architecture documents, RFCs, design specifications, engineering proposals, postmortems, or code reviews
- Strong understanding of software architecture and engineering trade-offs
- Excellent written communication and technical-editing skills
- Strong attention to detail and commitment to quality
- Experience with prompt authoring, content evaluation, data annotation, or rubric-based assessment is advantageous
- Familiarity with AI coding tools, model evaluation, RLHF, or prompt engineering is beneficial but not required
- Advanced academic qualifications are valued but not required
- No prior AI-training experience is required
Engagement Details
- Part-time independent contractor engagement
- Fully remote
- Compensation: $100–$200/hour
- Work will involve software-engineering analysis, technical-content review, prompt development, rubric-based evaluation, technical documentation, architecture analysis, research, fact checking, and AI-output assessment
- Assignments may involve architecture documents, RFCs, design specifications, engineering proposals, technical reports, code-review materials, model outputs, and evaluation rubrics
- Collaboration will primarily occur asynchronously with project leads and other subject-matter experts
- Prior experience with AI tools or model evaluation is helpful but not required
- Project scope, workload, technical topics, and evaluation standards may evolve depending on project requirements
- Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party
About the Platform
This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.
By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy