New York, NY | Jersey City, NJ | Charlotte, NC, NY
7 days ago
$70
Want to know if you’re a fit? Upload your resume and let our AI show you.
Skills
Agile Programming Methodologiesunmatched
Application Programming Interface (API)unmatched
Artificial Intelligence (AI)unmatched
Communication Skillsunmatched
Cross-Functionalunmatched
Data Lakeunmatched
Data Managementunmatched
Data Processingunmatched
Data Qualityunmatched
Engineeringunmatched
Enterprise Data Integrationunmatched
Information/Data Security (InfoSec)unmatched
Internet Securityunmatched
Microsoft Windows Azureunmatched
Multiplatform/Cross-Platformunmatched
NoSQLunmatched
Performance Tuning/Optimizationunmatched
Python Programming/Scripting Languageunmatched
REST (Representational State Transfer)unmatched
Regulatory Complianceunmatched
Relational Databases (RDBMS)unmatched
SQL (Structured Query Language)unmatched
Scalable System Developmentunmatched
Security Architectureunmatched
Structured Dataunmatched
Team Playerunmatched
Unstructured Dataunmatched
Description
A leading enterprise organization is seeking an AI Databricks Engineer to support the buildout of a centralized data lakehouse environment focused on cybersecurity and enterprise data integration. This role will play a key part in designing and developing scalable data pipelines that ingest and transform large volumes of structured and unstructured data.
You will work closely with cross-functional teams including data owners, security engineering, and architecture to ensure reliable, high-performance data delivery while supporting modernization initiatives across the platform.
RESPONSIBILITIES
Design and build automated data pipelines using Azure and Databricks
Develop ingestion frameworks for API-based and external data sources
Optimize and tune pipeline performance for large-scale data processing
Implement and maintain data lake and lakehouse architectures
Collaborate with stakeholders across engineering and security teams
Ensure data quality, reliability, and compliance with internal standards
Support agile delivery cycles and sprint-based development
QUALIFICATIONS
3 5 years of data engineering experience
Strong expertise in Python, PySpark, and SQL
Hands-on experience with Azure Databricks
Experience with Azure services (Data Lake Gen2, Data Factory, APIs)
Proven experience building pipelines using REST API data sources
Experience with performance tuning and optimization
Strong understanding of relational and/or NoSQL databases
Experience building APIs using Python frameworks (e.G., FastAPI)
Strong communication and stakeholder collaboration skills
PREFERRED EXPERIENCE
Experience in cybersecurity or security data environments
Familiarity with Azure Synapse or Microsoft Fabric
Experience with enterprise-scale data lakehouse platforms
Azure certifications
How We Work:
Agile, sprint-based development environment
Highly collaborative across technical teams
Focus on scalable, production-ready solutions
Emphasis on ownership and accountability
Numbers & Facts
Location
New York, NY | Jersey City, NJ | Charlotte, NC, NY