Mlabs

Web Scraping Specialist

New York, New York, United States · Full-time

No sponsorship$75k-$100kDetected 26 days ago

JavaScriptPythonHTMLCSSAlgorithmsNoSQLMongoDBCassandraAWSGCPCloud PlatformsMachine Learning

About the role

This role is ideal for a lean, technical builder who thrives in a fast-paced environment without bureaucratic red tape.
Code Development: Write, test, and refine high-performance code to extract data from various online sources, ensuring maximum reliability and efficiency.
Data Quality: Clean and format extracted data to ensure it meets rigorous quality standards for downstream analysis and processing.

Data Retrieval: Manage complex data retrieval tasks, including handling pagination and dynamic content loaded via AJAX.
Database Management: Store and manage scraped data in appropriate databases, optimizing for both access speed and long-term data integrity.
Web Fundamentals: In-depth knowledge of HTML, CSS, JavaScript, and the Document Object Model (DOM).
Your data may be shared only with clients and trusted partners where necessary for recruitment purposes.
Manage complex data retrieval tasks, including handling pagination and dynamic content loaded via AJAX.
$75K - $100K We are hiring on behalf of our client who is seeking a Web Scraping Specialist to join a specialized technical team focused on building the infrastructure that delivers massive amounts of web data for the training of advanced AI models.
This organization operates a massive distributed crawler and manages complex pipelines for ingesting, segmenting, and annotating billions of data points, including videos, transcripts, and audio files.
The successful candidate will lead efforts to gather and analyze data, optimize scraping processes, and support the scaling of high-quality public web data accessibility.

Demonstrated ability to extract data from complex websites with minimal supervision, supported by a portfolio of past projects.
Strong knowledge of asynchronous programming, multithreading, and distributed scraping architectures.
Experience with NoSQL databases (e.g., MongoDB, Cassandra), including the ability to design efficient storage solutions.
If you do not hear back from us within 4 weeks of your application, please assume that you have not been successful on this occasion.
Technical Proficiency: Advanced skills in Python or JavaScript, specifically with libraries and frameworks such as BeautifulSoup, Scrapy, or Selenium.
Data Storage: Experience with NoSQL databases (e.g., MongoDB, Cassandra), including the ability to design efficient storage solutions.

Experience deploying and managing large-scale scraping jobs using cloud services such as AWS, Google Cloud, or Azure.

A highly competitive salary ranging from $75,000 to $100,000, complemented by a comprehensive benefits and equity package.

Preferred Skills: Ability to apply machine learning algorithms for data cleaning, categorization, or predictive analysis; active participation in relevant open-source projects.

A professional environment that prioritizes low ego, technical autonomy, and rapid execution.
At MLabs, we are committed to offer equal opportunities to all candidates.
We ensure no discrimination, accessible job adverts, and providing information in accessible formats.
Our goal is to foster a diverse, inclusive workplace with equal opportunities for all.
If you need any reasonable adjustments during any part of the hiring process or you would like to see the job-advert in an accessible format please let us know at the earliest opportunity by emailing human-resources@mlabs.city.
MLabs Ltd collects and processes the personal information you provide such as your contact details, work history, resume, and other relevant data for recruitment purposes only.
You may request the deletion of your data or withdraw your consent at any time by contacting legal@mlabs.city.
Due to the high volume of applications we anticipate, we regret that we are unable to provide individual feedback to all candidates.
Commitment to Equality and Accessibility: At MLabs, we are committed to offer equal opportunities to all candidates.

This listing is sourced directly from Mlabs's careers page and normalized into a canonical job model.