Chainalysis Government Solutions
Senior Data Engineer, Data Cloud
New York · Senior
Sponsorship not specifiedDetected 37 days ago
PythonNoSQLSnowflakeDatabricksAWSCloud PlatformsTerraformCI/CDKafkaSparkAirflowdbtData EngineeringMentoring
About the role
- We're a global organization that thrives on challenging work and doing it alongside exceptionally talented teammates.
- Our data and solutions have been used to solve some of the world's most high-profile criminal cases and grow consumer access to cryptocurrency safely.
- You'll be a key leader in creating and optimizing our petabyte-scale data storage, data processing, and data querying platforms.
Responsibilities
- Design, implement, and optimize our high-performance, scalable data serving platform that enables data querying and consumption across the organization and external facing data products
- Design, implement, and optimize our high-performance, scalable data storage and transformation platform that enables both batch and stream processing with 100+ million updates per day on datasets > 100 billion rows
- Build seamless integrations between Data Cloud and various relational and noSQL OLTP databases
- Build batch and streaming data pipelines for core blockchain datasets widely used across the company
- Deploy cloud infrastructure at scale with enterprise-grade reliability, implement and maintain infrastructure automation and self-service, and create robust CI/CD pipelines
- Establish and maintain observability, security, and data governance solutions to ensure high quality, efficiency, and reliability of data pipelines
- Experience in building and maintaining both batch and streaming data pipelines using Apache Spark/Apache Flink, as well as deep understanding of data modeling best practices
- Experience building tools and frameworks to accelerate the development of data pipelines, and familiarity with data governance, data quality, and observability best practices
Requirements
- 6+ years of experience as a Data Engineer or Software Engineer or Infrastructure Engineer, with hands-on expertise in streaming data at scale.
- Expertise with AWS services, cloud architecture, fault-tolerant distributed data systems, and proficiency with Terraform for provisioning and managing cloud infrastructure
Nice to have
- Exposure to or interest in the cryptocurrency technology ecosystem
- Experience working with different blockchain technologies
- Technologies we use:
- Data Lakehouse: Kafka, Flink, Spark, Databricks, DBT, Debezium, StarRocks/Clickhouse
- AWS Services: MSK, EC2, VPC, IAM, S3, SQS, Managed Flink, EKS, etc.
- One that turns instructions into work done, and helps us move faster than the threats we're built to counter, and we expect our employees to take ownership of the output and ensure quality.
- AI fluency is tied directly to how we measure performance and how we plan to win.
Skills
- MSK, EC2, VPC, IAM, S3, SQS, Managed Flink, EKS, etc.
- AI at Chainalysis
- AI is not a feature at Chainalysis - it is a new way of working.
- There is no substitute for your own curiosity.
- About Chainalysis
- Chainalysis is the blockchain data platform, making it easy to connect the movement of digital assets to real-world services.
Benefits
- Passion for leading/contributing towards the technical vision of the team/org, strong ownership of mission critical systems, dedication to honing their craft while mentoring others
- Help define the technical vision of the team/org, articulate how our data platform and architecture could evolve
Apply directly at Chainalysis Government Solutions →Create a free account for alerts like thisView Chainalysis Government Solutions immigration profile
This listing is sourced directly from Chainalysis Government Solutions's careers page and normalized into a canonical job model.