Thinking Machines Lab
Research Engineer, Web Crawling
San Francisco · Senior
Stay score
odds of building a lasting career here
Sponsors, but it's cap-subject — you still face the weighted lottery (~61% per draw at Level IV). Good if you win; have a cap-exempt backup on your list.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Files H-1B transfers
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- The ideal candidate has built and scaled a web crawler or large-scale data-acquisition systems.
- You'll work closely with our pretraining and data teams to understand what's actually moving model quality, but this is fundamentally an engineering role, not a research one.
Responsibilities
- Design and scale the web crawler and ingestion infrastructure that sources Inkling's pretraining data
- Build pipelines for large-scale extraction, deduplication, and data quality filtering
- Build specialized crawlers for high-value or hard-to-reach data sources
- 8+ years designing, building, and scaling web crawlers, scrapers, or large-scale distributed data-acquisition systems
- We believe the future worth building is human, and we're hiring people who want to build it.
Requirements
- A track record of owning crawler or data-acquisition infrastructure at internet scale
- Strong software engineering skills in a language such as Python, Go, or Rust, with real experience in distributed systems
- Working knowledge of the practical and legal considerations of large-scale web data collection (robots.txt, rate limiting, licensing)
Nice to have
- Experience setting technical direction for a crawling, data acquisition, or search infrastructure team, whether or not that was your formal title
- Experience designing systems for petabyte-scale storage and processing
- Track record of open-source contributions to crawling, scraping, or data infrastructure tools
- Background at a search engine (crawling, indexing) or a frontier AI lab's data acquisition team
- Visa sponsorship: We sponsor visas.
- While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
- As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.
Compensation
- Depending on background, skills and experience, the expected annual salary range for this position is $350,000-$475,000 USD.
Benefits
- Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
- Experience applying machine learning to crawl selection, extraction, or data quality classification at internet scale
Company info
- The mission of Thinking Machines is to build AI that extends human will and judgment.
- We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication.
Visa & Work Authorization
- We sponsor visas.
- While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
This listing is sourced directly from Thinking Machines Lab's careers page and normalized into a canonical job model.