Persona Identities

Senior Software Engineer, Data Platform

San Francisco · Senior · Contract

Sponsorship not specifiedDetected 22 days ago

PythonGoDistributed SystemsMySQLMongoDBGCPKubernetesRESTKafkaSparkData EngineeringData ScienceCommunicationCollaborationMentoring

About the role

You'll work the full depth of that stack, from raw ingestion through storage layout and compute all the way to the query layer - owning the performance, reliability, and cost of each.
This is a high-ownership role on a small team, working closely with the rest of engineering, data science, and infrastructure to make the platform fast, reliable, and cost-efficient at hundreds of billions of records per day.
Operate the low-latency serving layer for real-time, high-concurrency access to platform data.

Build and scale our datalake - the pipelines that feed it and the storage design that keeps it fast and efficient.
Develop high-throughput data-movement and ingestion services that bring data into the platform reliably and at scale.
Own streaming and batch ingestion and the change-data-capture pipelines that absorb billions of records and terabytes of data per day.
Build new query and analytics capabilities directly on the datalake - for use cases that don't fit a traditional warehouse.
Partner with infrastructure and product teams to provision, secure, and scale the platform.
Experience building and operating scalable data pipelines across structured and unstructured data.

5+ years of experience in software engineering, with deep expertise in data infrastructure or large-scale distributed systems.
Proficiency in Python - our primary language across pipelines.
Hands-on experience with big-data or streaming technologies such as Spark, Kafka, or Flink, and interest in lakehouse table formats like Iceberg.

ClickHouse, DuckDB, Google Cloud (GCP), Kubernetes, MySQL, MongoDB, Snowflake.
This step is used solely to confirm that candidates are who they say they are, and will have no impact on hiring decisions.
Nice to have: ClickHouse, DuckDB, Google Cloud (GCP), Kubernetes, MySQL, MongoDB, Snowflake.

Excellent communication and collaboration skills across engineering, data science, and infrastructure teams.

As part of our interview process, all candidates will be asked to verify their identity with Persona.

This listing is sourced directly from Persona Identities's careers page and normalized into a canonical job model.