Data Engineer

Description

• Attitude – Optimistic, hopeful, and forward-thinking
• Compassionate Teamwork – Respectful collaboration through self-awareness and mutual understanding
• Integrity – Doing the right thing, in the right way, for the right reasons
• Versatility – Adaptive and anticipatory problem-solving
• Excellence – Dedicated to raising the bar of quality and performance

Key Responsibilities

• Build and optimize ETL data pipelines using Databricks and Apache Spark
• Develop scalable data models leveraging Spark Data Frames and Spark SQL for efficient querying, aggregation, and integration against numerous diverse data sources
• Utilize Python and Regular Expressions for advanced text parsing, pattern matching, and data cleansing to prepare structured datasets for downstream [CM1] ingestion
• Use APIs for seamless data exchange and automation across systems
• Manage SharePoint data (lists, libraries, files, metadata) programmatically using Python and Microsoft Graph API[CM2]
• Leverage Microsoft 365 tools for documentation and collaboration
• Develop and maintain Amazon Web Services (AWS) including S3
• Fully utilize the Microsoft 365 environment for cross-functional data sharing and productivity
• Use Jira to track and manage issues, tasks, and project workflows
• Collaborate with cross-functional teams and technical stakeholders to define data transformations of legacy systems to modern schemas
• Leverage SME collaboration to interpret business rules and build business logic into data extracts and transformation processes
• Build and maintain strong working relationships with SME’s, helping to identify requirements for data transformations based upon iterative product feedback

Skills, Knowledge and Expertise

• Bachelor’s degree in computer science, engineering, or a related field

• Databricks, Apache Spark, and cloud-based data engineering and modeling (2-3 years preferred)
• SQL for querying and managing relational data (4 years preferred)
• AWS cloud services, including S3 (3 years preferred)
• Python for data analysis, cleansing, and visualization (4 years preferred)
• SharePoint lists, document libraries, and Microsoft Graph
• Microsoft 365 apps and integration workflows
• Developing and consuming APIs for data exchange between different systems
• Jira or other agile workflow management systems for issue and project tracking

• Proven ability to build trust and maintain strong partnerships with customers and stakeholders
• Takes initiative and drives projects forward with minimal supervision, anticipating challenges before they arise
• Meticulous in managing complex tasks and deadlines, ensuring accuracy and consistency across deliverables
• Thrives in cross-functional environments, fostering open communication and teamwork across diverse groups
• Rapidly assimilates new technologies and processes, adapting to evolving requirements in dynamic settings
• Ability to work collaboratively with a diverse team of contractors and government personnel
• Ability to work collaboratively with a diverse team of contractors, government personnel, and stakeholders from multiple organizations
• Skilled in building strong customer relationships and engaging with Subject Matter Experts (SMEs) to derive business logic for data transformation methods and rules

• AWS Certified Solutions Architect 
• Databricks Certified Data Engineer
• U.S federal government security clearance 

Benefits

• 401(k)
• 401(k) matching
• Dental insurance
• Flexible spending account
• Health insurance
• Health savings account
• Life insurance
• Paid time off
• Referral program
• Tuition reimbursement
• Vision insurance