Allocate.co careers

Allocate.co careers

Sr. Data Engineer

Palo Alto Office (Hybrid) · Senior · Full-time

Sponsorship not specified$145k-$220kDetected 14 days ago
TypeScriptPythonC#Vue.jsNode.js.NETCode ReviewSQLPostgreSQLSnowflakeRedshiftDatabricksVector DatabasesAWSDockerKubernetesTerraformCI/CDDevOpsKafkaMachine LearningPandasAirflowdbt

About the role

  • Through a single, data-rich digital experience, clients access top-tier opportunities across venture capital, private equity, private credit, and other private asset classes-backed by powerful tracking, analytics, and administration tools.
  • This includes preparing training datasets, setting up feature stores, and orchestrating workflows that feed LLM-based agents with the context they need (e.g. retrieving relevant data via vector similarity search).

Responsibilities

  • Build and Extend Data Architecture: Build on and extend Allocate's data lakehouse on AWS, combining data lake storage and warehouse technologies to store diverse financial datasets.
  • Develop Data Pipelines: Create robust ETL/ELT pipelines to ingest, clean, and transform data from various sources (internal application data and third-party APIs).
  • Ensure both batch processing and real-time data streaming are handled to support up-to-date analytics and recommendations.
  • Build pipelines with an eye on scalability (able to handle increasing data volume and complexity) and reliability (proper error handling and monitoring).
  • Engineering Excellence and Collaboration: Partner with our data lead and the broader engineering team to deliver data and AI infrastructure.
  • Challenge conventions and innovate: we encourage rethinking how things are done as we push to build a world-class, intelligent platform.
  • Strong Data Engineering Experience: 5+ years of hands-on experience in data engineering (or related fields), including designing and building large-scale data pipelines and storage solutions.
  • You should have taken projects through the full lifecycle from design to production deployment.
  • Strong Analytical and Problem-Solving Skills: Ability to analyze complex data problems, debug pipeline issues, and optimize system performance.
  • Compliance and Security: working in a regulated SEC environment, handling sensitive investor/financial data, building with auditability, least-privilege, and data governance in mind.

Requirements

  • Python is commonly used for data pipelines, and pandas/PySpark experience is valuable.
  • We also value experience with TypeScript/Node.js in data contexts, since our stack leans toward modern web technologies.
  • The ideal candidate can work across languages, for example writing a data API in C# or Node.js to interface with our backend while also crafting Python scripts for data processing.
  • Bachelor's degree in Computer Science, similar technical field of study, or equivalent practical experience
  • We believe open, intellectually curious conversations are required to consistently arrive at the best decisions.
  • Civil Discourse is embraced: We believe open, intellectually curious conversations are required to consistently arrive at the best decisions.
  • Database and Data Modeling Skills: Proficiency in SQL and relational database design.
  • Familiarity with graph databases (Neo4j, AWS Neptune, etc.) and knowledge graph schemas will help you hit the ground running.
  • Programming Expertise: Fluency in at least one major programming language used in data engineering.
  • If you are a hands-on engineer who wants to do high-impact data work in a collaborative startup environment, we want to hear from you.

Nice to have

  • Strong experience working with AWS cloud services for data.
  • You should be comfortable with tools like S3, EC2, ECS, EKS, Athena, Redshift, Glue, and Step Functions.
  • Experience setting up infrastructure-as-code (Terraform/CloudFormation) for these services is a plus.
  • Experience preparing datasets for training, working with feature stores, or integrating ML model outputs into applications is important.
  • Knowledge of vector embeddings and experience with vector databases (Postgres pgvector, Chroma, Pinecone, etc.) is a big plus, as our AI features rely on semantic search.
  • Cloud Proficiency (AWS): Strong experience working with AWS cloud services for data.

Skills

  • Proficiency in SQL and relational database design.
  • Solid understanding of containerization and deployment.
  • Experience using Docker to package data applications and Kubernetes (or AWS EKS) to run distributed jobs/services.
  • Experience with workflow managers (Airflow, Prefect, dbt, or similar) is beneficial.
  • AI-Native Mindset: treating AI as core to the workflow, fluency with agentic tools and LLM-assisted dev, pushing the frontier of AI tooling.
  • DevOps and DataOps

Compensation

  • Total compensation may also include a discretionary performance-based bonus.
  • The expected base salary range for this role is $145,000 to $220,000.
  • Actual compensation will be determined based on the candidate's primary work location and other job-related factors including skills, experience, qualifications, interview performance, internal equity, and market data.
  • This range reflects base salary only and does not include bonus, equity, benefits or other forms of compensation that may be offered.

Benefits

  • Medical, dental, and vision.
  • 401(k), and responsible vacation time (PTO)
  • Monitor the health and performance of our data platforms (setting up alerts, dashboards) and be ready to troubleshoot and resolve issues in production to ensure uptime of critical data and AI services.
  • Salary: Total compensation may also include a discretionary performance-based bonus.
  • This range reflects base salary only and does not include bonus, equity, benefits or other forms of compensation that may be offered.
  • Enable AI/ML Capabilities: Work closely with our data science and engineering team to provision the data and infrastructure needed for machine learning models and AI features.
  • AI/ML Familiarity: While this is not a pure ML researcher role, you should understand how machine learning models consume data.
  • Comfort mentoring peers and driving technical projects to completion is important, as is a positive attitude toward continuous learning and improvement.
  • Actual compensation will be determined based on the candidate's primary work location and other job-related factors including skills, experience, qualifications, interview performance, internal equity, and market data.

Company info

  • Providing our clients with a world-class experience is our number one priority. We obsessively search for ways to improve the experience for our clients and partners. This requires extraordinary response times, proactivity, and ensuring that everything we do, from product strategy to offline communications is a top-tier client experience.
  • Respect is paramount in our dealings with one another, but our mission is always to get the right answer collectively, not to be right.
  • Our data lead has established the foundational architecture and strategy, and we are now looking for a strong senior engineer to help extend, scale, and harden it.
Apply directly at Allocate.co careers →Create a free account for alerts like thisView Allocate.co careers immigration profile

This listing is sourced directly from Allocate.co careers's careers page and normalized into a canonical job model.