Data Engineer

Summary

The Data Engineer is responsible for developing, designing, and maintaining data platform solutions. The measures of an ideal candidate include communication skills, technical proficiency, collaboration, and willingness to learn new things. This new position will be based in our Arlington, VA headquarters and report to the Director of Business Intelligence. This position is structured within IT under the Vice President of Applications.

The position will be located in Arlington, VA.

General Description Duties & Responsibilities

• Build complex data solutions in an Azure Cloud environment.
• Maintain the Azure components and services.
• Develop architecture changes using infrastructure as code.
• Create data architectures diagrams for complex data modeling solutions.
• Create and maintain data dictionaries for contextualizing data from disparate sources.
• Enable end-to-end automation for deployment using continuous integration and continuous delivery.
• Implement data pipelines to ingest data to the platform, standardize the data, and transform the data into business facing datasets.
• Perform unit and integration testing of data pipelines.
• Perform data integration at scale with Azure Data Factory.
• Utilize Databricks to transform, curate, and organize data for use with business intelligence reporting and data analytics.
• Work with BI developers and business SMEs to translate business requirements into curated datasets suitable for analytics solutions.
• Document data pipelines for maintainability.
• Work with source system owners and business owners to incorporate business changes into data pipelines.
• Design data architecture for future platform growth including data warehousing, machine learning, streaming analytics, and data visualization.
• Assist in testing, governance, data quality, training, and documentation efforts.
• Actively engage in business stakeholder requirement workshops to understand, interpret, and translate requirements into effective technical solutions.

Qualifications

Required experience

• Bachelor's degree in Computer Science, Data Analytics, Engineering, Mathematics, Business, or related field of study.
• 2+ years in cloud architecture and data engineering
• Extensive experience in implementing data storage solutions, managing, and developing data processing, and optimizing data solutions (Azure)
• Expertise in integrating, transforming, and consolidating data from structured and unstructured data systems suitable for analytics solutions.
• Understanding of parallel processing and data architecture patterns.
• Knowledgeable in data processing languages such as SQL, Python, or Spark.
• Experience in building secure data processing pipelines using Azure data services.
• Ability to self-manage and make your own decisions.
• Excellent interpersonal and communications skills, with strong critical thinking and attention to detail.
• Strong work ethic with ability to effectively prioritize, meet deadlines, adapt to changing priorities and business needs, and succeed in a fast-paced environment.
• Excellent attention to detail and the ability to efficiently summarize and prioritize information.

Preferred Experience

• Proficient in Microsoft Azure Storage Explorer, Azure DevOps, Azure Data Factory, Azure Databricks, Azure Data Lake Gen 2, Azure DevOps, Unity Catalog and Git
• Experience in coding languages primarily Python and Spark. Experience in agile development and sprint planning. Strong technical writing skills.