Senior Data Engineer

NIRA is seeking a highly experienced Senior Data Engineer to support the U.S. Department of Justice Digital Evidence Review Platform program. The Data Engineer will design, develop, test, operate, and optimize high-volume data ingestion and processing pipelines for digital evidence and associated metadata. The platform must process large and diverse datasets while preserving integrity, provenance, auditability, access controls, and chain-of-custody information throughout the evidence lifecycle.

Key Responsibilities

• Design, develop, test, operate, and optimize high-volume data ingestion, transformation, integration, and processing pipelines.
• Ingest, normalize, enrich, validate, store, search, export, archive, and restore structured and unstructured data.
• Process sources such as forensic extraction packages, documents, messages, multimedia, call records, geolocation data, cloud-provider returns, metadata, and outputs from Government-furnished parsers and external systems.
• Implement validation, hashing, error handling, retry, reconciliation, reprocessing, and data-quality controls.
• Preserve source identifiers, processing history, lineage, provenance, and chain-of-custody events.
• Support OCR, transcription, translation, deduplication, metadata extraction, entity extraction, and enrichment workflows.
• Develop schemas, canonical data models, metadata structures, and open, portable export formats.
• Support search, analytics, and AI/ML-enabled processing while maintaining human oversight and auditability.
• Automate deployment and testing through CI/CD and configuration-management practices.
• Tune pipeline performance and troubleshoot failures across development, test, UAT, production, and disaster-recovery environments.
• Maintain technical documentation and collaborate with architects, software engineers, cloud integrators, cybersecurity personnel, and product specialists.

Minimum Qualifications

• Bachelor''s degree in engineering, mathematics, or science.
• At least 15 years of relevant data engineering, data integration, data architecture, software engineering, or large-scale data-processing experience.
• A master''s degree may substitute for two years of required experience; a PhD may substitute for four years.
• Extensive experience designing, developing, and maintaining enterprise data ingestion, transformation, integration, and processing pipelines.
• Experience processing structured and unstructured data from multiple sources and formats.
• Experience with data modeling, metadata, schema design, data lineage, data quality, error handling, reconciliation, and audit logging.
• Experience developing in Python, Java, Scala, SQL, or comparable data-engineering languages.
• Experience with cloud-based data platforms, distributed processing, object storage, databases, search platforms, or analytics environments.
• Experience supporting CI/CD, automated testing, configuration management, performance tuning, and production troubleshooting.
• S. citizenship and ability to meet DOJ residency and personnel-security requirements.
• Ability to obtain and maintain a High-Risk Public Trust investigation and PIV credential.

Preferred Qualifications

• Experience with digital evidence, forensic data, eDiscovery, law-enforcement data, case-management data, or other high-integrity federal datasets.
• Experience with AWS GovCloud, Azure Government, FedRAMP-authorized SaaS, or hybrid federal cloud environments.
• Experience processing PDFs, text, messages, multimedia, call records, geolocation data, forensic extraction files, or large document collections.
• Experience with OCR, natural-language processing, entity extraction, transcription, translation, deduplication, AI/ML enrichment, or LLM/RAG pipelines.
• Experience with Python-based data frameworks, Spark, Databricks, Kafka, Airflow, cloud-native data services, or comparable technologies.
• Experience implementing hashing, immutable audit records, provenance, lineage, retention, archival, and data-restoration controls.
• Experience designing open-format exports, APIs, canonical schemas, or data-exchange interfaces that reduce vendor lock-in.