Position Summary:
The IT Division is responsible for the development and operations of information systems for the State and Federal agencies doing business related to or using information from the administration of motor vehicles and driver licenses.
The Data Engineer position has core responsibilities for the design, development, maintenance, optimization, and operational support of components which move and ingest data from sources into the analytics systems.
Essential Duties and Responsibilities:
You will work with cutting-edge technologies to build and maintain our data platform, focusing on batch and stream processing workflows. You will leverage the Microsoft Azure ecosystem, particularly Azure Synapse Analytics, Microsoft Fabric, Apache Spark, and Delta Lake, to create efficient and reliable data solutions that power our analytics and reporting capabilities.
As a detail-oriented professional, you have a strong track record of independently managing projects and driving them to successful completion. Your exceptional business acumen and communication skills enable you to collaborate effectively with stakeholders, identify vital business questions, and make informed, data-driven decisions.
Key responsibilities include:
Designing and implementing scalable data pipelines using Synapse, Fabric, and Apache SparkBuilding and maintaining streaming and batch solutions using Azure Event HubsDeveloping and maintaining Python applications for data processing and integrationImplementing Delta Lake architecture for reliable data storage and processingLeveraging Microsoft platform capabilities for end-to-end data analytics solutionsCreating reports and data visualizations to support business intelligence needsOptimizing data workflows for performance and cost efficiencyCollaborating with cross-functional teams to understand data requirements and deliver solutionsDirect Reports:
None
QUALIFICATIONS
Formal Education:
Bachelor’s degree in computer science, Data Science, or related field. Equivalent work experience may be substituted.
Knowledge, Skills, and Abilities:
Basic Qualifications
3-5 years of hands-on experience in data engineeringStrong proficiency with Fabric, Azure Synapse Analytics and Apache SparkHands-on experience with Azure Event Hubs and stream processingExperience building and maintaining data pipelines using Delta LakeStrong understanding of relational database designProficiency in Python and SQLExperience with version control systems (Git) and CI/CD practicesStrong problem-solving skills and attention to detailPreferred Qualifications
Knowledge of data modeling and dimensional modeling conceptsFamiliarity with operations and monitoring of mission critical data platformExperience with CI/CD deployment of data pipelinesExperience with statistics, machine learning, time-series, anomaly detectionExperience with Power BI or similar visualization toolsExperience creating reports and dashboards for business stakeholdersAzure or Databricks certificationsTechnical Stack
Core Technologies: Fabric, Azure Synapse Analytics, Apache Spark, Delta Lake, Azure Event HubsLanguages: Python, SQL, PySpark, C#T-SQL, Stored Procedures, SQL Server Availability GroupsAzure Functions, Azure Data Factory, Azure Key VaultOptional: Databricks, Snowflake, Lakehouse ArchitectureVisualization: Power BIDevelopment: Azure DevOps, CICD
Disclaimer Statement: The preceding job description has been written to reflect management’s assignment of essential functions. It does not prescribe or restrict the tasks that may be assigned.
AAMVA is an Equal Opportunity Employer/Veterans/Disabled