Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

RStBbW9OT0MxMDFpcG1tRzhITFJTbGVBWmc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

Insynch Health PLLC

Physician Assistant Job at Insynch Health PLLC

 ...Looking For? InSynch Health is on a continued path for growth. We are offering a new full-time opportunity for an experienced Physician Assistant (PA) focusing on psychiatry to join our inpatient psychiatric care hospital. The ideal candidate will be dedicated to... 

Labor Finders

Construction Worker Job at Labor Finders

 ...is good at performing a variety of tasks? Can you be ready to work at a different worksite from one day to the next? Do you prioritize...  ...other specialized contractors. Controlling traffic at road construction sites, setting up markers, cones and barricades as well as... 

Royal Caribbean Cruises, Inc.

Stage & Production Manager Job at Royal Caribbean Cruises, Inc.

Stage & Production ManagerStage & Production Managers are responsible for managing, operating, and maintaining all entertainment and entertainment technical disciplines of their assigned entertainment venue, and any auxiliary venues around the ship where Sound & Light... 

Missoula Family YMCA

PT-Weekend-Afternoon-Custodian Job at Missoula Family YMCA

 ...POSITION SUMMARY: : Under the direction of the Maintenance Supervisor/Tech, the Custodian insures that the YMCA facilities are cleaned according to Y directions and standards and that the facilities and property are clean and sanitized for all members, program participants... 

Compass Group

GUEST SERVICES SENIOR AGENT (FULL TIME) Job at Compass Group

 ...Solutions, Laundry and Linen Services, Ambulatory, and Sterile Processing. Job Summary Summary: Checks guests in and/or out of the hotel. Resolves guest challenges throughout their stay. Promotes hotel services, amenities and upsells products to the guests. Supervises...