Fractal

Fractal

Lead Architect

New York · Full-time

Sponsorship not specified$150k-$200kDetected 4 days ago
PythonGitSQLNoSQLMongoDBElasticsearchSnowflakeDatabricksAzureCloud PlatformsCI/CDDevOpsKafkaMachine LearningSparkData EngineeringAI OrchestrationAgileHadoop

About the role

  • We are looking for a Lead Architect - Azure & AI to join our growing data engineering practice.
  • In this role, you will architect and deliver large-scale, cloud-native data solutions on Azure and Snowflake, enabling advanced analytics, ML/AI workloads, and enterprise data governance.
  • You will work closely with cross-functional stakeholders to design resilient, scalable data platforms that drive business impact for Fortune 500 clients.

Responsibilities

  • Design, build, and optimize end-to-end data pipelines leveraging Azure Data Factory, Databricks, and Snowflake for large-scale data ingestion, transformation, and loading.
  • Develop and maintain ETL/ELT workflows for structured, semi-structured (JSON, XML, Parquet), and unstructured data from on-premises systems, cloud platforms, APIs, flat files, RDBMS, and streaming sources.
  • Architect scalable Snowflake solutions including schema design, Virtual Warehouse configuration, query performance tuning, and cost-efficient data storage and retrieval strategies.
  • Partner with business and technology stakeholders to design DataMart and Data Lake solutions supporting analytics, reporting, and ML/AI use cases.
  • Implement data quality frameworks, governance policies, lineage tracking, and metadata management across multiple environments.
  • Optimize Spark workloads on Databricks for distributed processing at scale
  • deploy and manage reusable Notebooks and Jobs in production.
  • Drive DevOps best practices including CI/CD pipelines, source control (Git), and automated deployments via Azure DevOps and Jenkins.

Requirements

  • 8+ years of experience as a Data Engineer with deep expertise in Azure Data Services (ADF, Synapse, Databricks) and Snowflake.
  • Strong background in the Insurance domain with hands-on Azure Data Engineering experience (ADB + PySpark/Python + Snowflake).
  • Proficiency in SQL (complex queries, stored procedures, functions, views) and Python/PySpark programming.
  • Proven experience designing and implementing ETL/ELT pipelines with Azure Data Factory and orchestrating complex, multi-step workflows.
  • All applications must be made through posted job openings

Nice to have

  • Solid understanding of Spark architecture with hands-on experience in performance tuning, data partitioning, and distributed processing.
  • Experience integrating data from RDBMS, SFTP, APIs, Kafka, Event Hub, and big-data platforms (Hadoop, Hive).
  • Knowledge of ML/AI data preparation workflows
  • ability to refactor traditional ML code for distributed execution on Spark/Databricks.
  • Strong command of Agile methodologies, DevOps practices, and CI/CD deployment frameworks.
  • Exposure to data search and indexing technologies such as Elasticsearch or Azure Cognitive Search.
  • Azure Data Engineer Associate certification (DP-200/201/203) or SnowPro Core/Advanced Architect certification.
  • Experience working on data platforms serving analytics and AI use cases in large enterprise environments.

Compensation

  • The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs.
  • At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case.
  • A reasonable estimate of the current range is: $150,000 - $200,000.

Benefits

  • You will be eligible for benefits on the first day of employment with the Company.
  • The Company provides for 11 paid holidays and 12 weeks of Parental Leave.
  • We also follow a "free time" PTO policy, allowing you the flexibility to take time needed for either sick time or vacation.
Apply directly at Fractal →Create a free account for alerts like thisView Fractal immigration profile

This listing is sourced directly from Fractal's careers page and normalized into a canonical job model.