Real-Time Computer Vision Architect

The Video Computer Vision and Video Engineering teams are centralized applied research and engineering organizations responsible for developing real-time on-device Computer Vision, Machine Perception, and Image Quality technologies for Apple products. We balance research and product development to deliver Apple quality, state-of-the-art experiences. We innovate across the full stack and partner with hardware and software teams to influence SW architecture, sensors, silicon, and PD/ID roadmaps that bring our vision to life. The Architecture team within our organization is responsible for bringing together computer vision algorithms, firmware, and low-level software teams to drive some of the most exciting programs across many Apple products! We do this by enabling the co-design of computer vision algorithms and systems through a deep understanding of the complexities and trade-offs between algorithm performance, imaging pipeline, system resources, and real-time constraints. To be successful in this role, you not only have to be an excellent engineer, but also a phenomenal collaborator-comfortable communicating with a wide range of experts and leaders across many different domains, from firmware and OS to feature and experience. Are you ready to be a part of the next big thing at Apple?Break down high-level product goals into system, module, and component requirements, including firmware, ISP, and OS constraints. Find creative solutions within system constraints that unlock new perception capabilities and maximize user value. Develop and optimize computer vision perception algorithms and ISP pipelines for performance, power, and memory on embedded platforms. Work with firmware, ISP, mechanical and electrical engineering, and optical science teams to evaluate sensing tradeoffs across cameras, depth sensors, and IMUs. Debug and optimize performance at the firmware or OS level, and tune ISP behavior to match hardware capability with algorithmic needs. Communicate technical tradeoffs and requirements to partners ranging from firmware and OS engineers to feature and experience teams.BS in Computer Science, Electrical Engineering, or other relevant fields with a minimum of 10 years relevant, proven experience. Validated ability to work in a highly multi-functional environment at the intersection of Computer Vision, Embedded Systems, ISP, Mechanical and Electrical Engineering, Optical Science, Human Factors or similar. Proven track record in developing and improving CV perception algorithms and ISP pipelines. Demonstrated experience in optimizing computer vision algorithms for performance, power, and memory on embedded platforms. Familiarity with firmware integration, operating system constraints, power and memory constraints, and low-level software development for image signal processing (ISP) in real-time applications. Strong understanding of perception sensing technologies including color and monochrome cameras, depth sensors, and IMUs. Ability to understand and articulate trade-offs between multiple sensing solutions and their impact on both high-level algorithms and low-level system performance. Depth of expertise in one or more computer vision specialty areas relevant to this role. These areas may include SLAM, tracking, classification, calibration, PnP-based pose estimation, scene understanding, human representation, and image enhancement.Breadth of knowledge in some areas of computer vision including either classical methods or ML. These areas may include SLAM, tracking, classification, calibration, PnP-based pose estimation, scene understanding, human representation, and image enhancement. Experience in translating ambiguous, high-level product goals into detailed feature-level architectures and system requirements. Proven ability to derive defensible end-to-end requirements at the system level and decompose them into module/component-specific requirements that include firmware, ISP, and OS constraints. Strong leadership skills to drive large multi-functional efforts, resolve conflicts, and present results to the highest levels of Apple leadership. Demonstrated ability to bridge the gap between high-level algorithm design and low-level system optimization-including the challenges of optimizing ISP performance under tight power and latency constraints. Experience in optimizing and fine-tuning ISP pipelines, including tasks like noise reduction, color correction, demosaicing, and dynamic range enhancement. Ideal candidates will also have exposure to modern innovative techniques including vision transformers, foundation models, and vision-language models, with a focus on integrating these methods into resource-constrained environments. Proficiency in debugging and optimizing performance at the firmware or OS level, with specific expertise in tuning ISP performance to ensure tight integration between hardware capabilities and algorithmic performance.