Etchedai
Machine Learning Research Engineer
Cupertino, CA
Sponsorship not specifiedDetected 118 days ago
PythonRustAlgorithmsRESTMachine LearningPyTorchNLPLLMsResearch
About the role
- We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.
Responsibilities
- Develop deep architectural knowledge informing best-in-the-world software performance on Sohu HW, collaborating with HW architects and designers.
- Co-design and finetune emerging model architectures for highest efficiency on Sohu
- Propose and implement a novel test time compute algorithm that leverages Sohu's unique capabilities to unlock a product could never be achieved on a typical GPU
- Implement diffusion models on Sohu to achieve GPU-impossible latencies that allow for real-time image generation
- Optimize model instructions and scheduling algorithms to optimize for utilization, latency, throughput, and/or a mix of these metrics.
- Implement model-specific inference-time acceleration techniques such as speculative decoding, tree search, KV cache sharing, priority scheduling, etc by interacting with the rest of the inference serving stack.
- An ML Research background with interests in HW co-design
- ML Systems Research and HW Co-design backgrounds
- Relocation support for those moving to Cupertino
- We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware.
Requirements
- Experience with Python, Pytorch, and / or JAX
- Familiarity with transformer model architectures and/or inference serving stacks (vLLM, SGLang, etc.) and/or experience working in distributed inference/training environments
- Experience working cross-functionally in diverse software and hardware organizations
- Experience with Rust
- Familiarity with GPU kernels, the CUDA compilation stack and related tools, or other hardware accelerators
- You may be a good fit if you have
Benefits
- Full medical, dental, and vision packages, with 100% of premium covered
Company info
- We are a fully in-person team in Cupertino, and greatly value engineering skills.
Apply directly at Etchedai →Create a free account for alerts like thisView Etchedai immigration profile
This listing is sourced directly from Etchedai's careers page and normalized into a canonical job model.