DevOps/Data Engineer

• Drive the infrastructure strategy alongside the rest of the DevOps team
• Be responsible for building, managing, and automating our AWS infrastructure
• Oversee the maintenance and growth of our data lakehouse infrastructure and implementation
• Champion security by adopting necessary tools and processes
• Provide leadership to the entire engineering team on various infrastructure and big data topics
• Identify and adopt high-level technologies, processes, and services that impact the organization on a system-wide scope
• Participate in the L2 rotation for engineering support

Requirements

• 8+ years of experience as a DevOps Engineer
• 5+ years of experience with the AWS cloud platform, including proficiency with AWS SDK, AWS CLI/UI, and services such as VPC, Route53, Private and Public subnets, route tables, IGW, EC2 Instances, CloudFront, API Gateway, IAM, ELB, Autoscaling, CloudWatch, EFS, NFS, EBS, S3, RDS, Lambda, SQS/SNS, Kafka, Security groups, etc.
• 5+ years of experience with Terraform/Terragrunt
• 5+ years of experience with event processing infrastructure
• 2+ years of experience as a Data Engineer with Databricks, including data pipelines, Debezium, Spark, and data modeling
• Expert proficiency with Python
• Model serving experience
• Expert proficiency in deploying, maintaining, and scaling applications
• Proficiency with Docker, Kubernetes, and Serverless functions
• Expert proficiency in setting up CI/CD pipelines (tools/processes/governance)
• Experience creating and maintaining fully automated CI/CD pipelines for code deployment using tools such as Git, CircleCI, AWS Code build, and Code Deploy / Code pipeline
• Proficient with both Relational and Non-Relational databases
• Experience with Security vulnerability scanning tools (Guard Duty, Inspector)
• Experience building Monitoring dashboards and leveraging tools such as Cloudwatch, Datadog, and Sumologic