Job Description

Job Title: Senior Azure Data Engineer

Location: Redmond, WA 98052

Work Setup: Hybrid

Duration: 12 Months

Experience: 8-10 Years


Job Description


We are looking for an experienced Senior Azure Data Engineer with 8-10 years of experience in data engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks .


The candidate will be responsible for developing and supporting scalable data pipelines, transforming large datasets, and optimizing data processing solutions using Azure cloud technologies.


Must Have Skills

• Strong experience with Python and SQL
• Hands-on experience with Apache Spark, PySpark, and Spark SQL
• Strong experience with Azure Databricks , including:
  • Notebooks
  • Jobs/Workflows
  • Clusters
• Experience with Azure Synapse
• Strong knowledge of Delta Lake
• Experience working with Parquet
• Hands-on experience with ADLS Gen2
• Strong understanding of data engineering and ETL/ELT concepts

Nice to Have Skills

• Experience with Scala
• ETL/ELT development
• Experience with MERGE/Upsert operations
• Knowledge of data partitioning and partition pruning
• Strong Spark performance tuning experience
• Experience handling data skew, shuffles, caching, broadcast joins, and partition optimization
• Experience with Git
• Experience with Azure DevOps and CI/CD

Key Responsibilities

• Develop and maintain scalable ETL/ELT data pipelines
• Extract, transform, clean, and load data from various source systems
• Build data solutions using Azure Databricks, Spark, Delta Lake, Synapse, and ADLS Gen2
• Implement MERGE/Upsert logic using Delta Lake
• Design effective partitioning strategies for large datasets
• Optimize Spark jobs for performance and cost
• Manage data skew, shuffles, caching, partitions, and cluster resources
• Collaborate with development and data teams to deliver reliable data solutions
• Follow best practices for Git, Azure DevOps, and CI/CD where applicable

Job Tags

Similar Jobs