Inception
Member of Technical Staff, Data
San Mateo, USA · Staff+
Sponsorship not specifiedDetected 134 days ago
PythonSQLNoSQLBigQueryVector DatabasesMachine LearningTensorFlowPyTorchSparkAirflowData EngineeringLLMsResearch
About the role
- We seek experienced engineers and scientists to shape how we collect, process, and curate the datasets that power our models.
Responsibilities
- Develop data mixes for training LLMs, including by leveraging open-source datasets, synthetically generated data, and curated human feedback.
- Design and implement data pipelines for processing petabyte-scale datasets.
- Build systems for web crawling, data ingestion, and real-time data processing to support model training.
- Develop tools and frameworks for efficient data storage, retrieval, and versioning across distributed systems.
- Create evaluation frameworks to measure data diversity, quality, and representativeness.
Requirements
- Experience with distributed computing and large-scale data storage systems (HDFS, S3, BigQuery).
Nice to have
- The Role We seek experienced engineers and scientists to shape how we collect, process, and curate the datasets that power our models.
- Ensure data collection adheres to privacy regulations.
- 3+ years of experience building data processing pipelines at scale, particularly with AI/ML applications.
- Strong proficiency in Python and experience with data processing frameworks (Apache Spark, Beam, Airflow).
- Familiarity with synthetic data generation techniques and data augmentation strategies.
- Familiarity with web scraping, crawling technologies, and Common Crawl datasets.
- Experience with SQL and NoSQL databases for managing structured and unstructured data.
- Preferred Skills Experience with large language models and understanding of tokenization, embeddings, and model architectures.
Benefits
- Qualifications BS/MS/PhD in Computer Science, Machine Learning, or a related field (or equivalent experience).
- Solid understanding of machine learning fundamentals and experience with ML frameworks (PyTorch, TensorFlow).
Apply directly at Inception →Create a free account for alerts like thisView Inception immigration profile
This listing is sourced directly from Inception's careers page and normalized into a canonical job model.