Basis Research
Data Engineer, Platform
New York Office · Full-time
Sponsorship not specifiedDetected 241 days ago
PythonSQLSparkAirflowData EngineeringResearch
About the role
- You will work on both platform-specific data needs and cross-project data coordination, preventing duplicate work and facilitating shared datasets.
- We are looking for people who are technically excellent and treat data quality as a first-class concern.
- We seek individuals who aspire to do rigorous, high-quality, robust data engineering, but are not afraid to iterate, learn from real usage, and explore different approaches to achieve excellence.
Responsibilities
- Design and build data pipelines for training and evaluation across Basis research projects and platform offerings, ensuring reliability, performance, and scalability.
- Implement data quality frameworks including validation rules, quality gates, anomaly detection, and monitoring that catch data issues before they impact research or production systems.
- Develop and maintain feature stores or equivalent systems that enable consistent feature access across training and serving environments, preventing train-serve skew.
- Optimize data infrastructure for scale as compute grows, including cost optimization, performance tuning, caching strategies, and efficient data access patterns.
- Collaborate with research and engineering teams to understand data needs, translate requirements into technical solutions, and provide consultation on data architecture and best practices.
Requirements
- The second is to advance society's ability to solve intractable problems.
- This means expanding the scale, complexity, and breadth of problems that we can solve today, and even more importantly, accelerating our ability to solve problems in the future.
- Experience with feature stores (Tecton, Feast) or building feature platforms.
- Experience with data lineage tools (Apache Atlas, DataHub, Monte Carlo) and metadata management.
- Knowledge of vector databases and embedding pipelines for modern AI applications.
- Possess strong proficiency in data technologies including SQL (expert level), Python for data processing, distributed computing frameworks (Spark, Dask), and workflow orchestration tools (Airflow, Dagster, Prefect).
Skills
- Creating data quality frameworks and implementing governance systems
Compensation
- Non-Discrimination Notice
- By submitting your application, you grant Basis permission to use your materials for both hiring evaluation and recruitment-related research and development purposes.
- Your information may be processed in different countries, including the US.
- You retain copyright while providing Basis a license to use these materials for the stated purposes.
- Competitive salary.
Company info
- We are in the office four days a week.
- Be prepared to attend multi-day Basis-wide in-person events.
- Basis is a collaborative effort, both internally and with our external partners; we are looking for people who enjoy building data foundations for problems larger than ones they can tackle alone.
- Implement data governance policies ensuring compliance with privacy regulations, security requirements, and responsible AI practices as Basis serves external customers.
- In-person Policy: We are in the office four days a week. Be prepared to attend multi-day Basis-wide in-person events.
Apply directly at Basis Research →Create a free account for alerts like thisView Basis Research immigration profile
This listing is sourced directly from Basis Research's careers page and normalized into a canonical job model.