Samsung Semiconductor
Technical Director, Large-Scale AI Model Inferencing
San Jose, California, United States · Director
Sponsorship not specifiedDetected 4 days ago
Full-Stack DevelopmentPyTorchNLPLLMsRAGStatisticsRecruitingSystems EngineeringAvidLeadershipCommunication
Stay score
odds of building a lasting career here
42Risky
Cap-exempt (no lottery)0
Sponsors this role100
Entry-level history0
PERM / green-card track0
Lottery odds40
Fits your clock70
Thin sponsorship signal and lottery-bound. A low-probability bet with your clock running. Prioritize cap-exempt roles and proven entry-level sponsors first.
Lottery odds assume a STEM candidate.
Personalize to your clock →Employer immigration record
from this employer's Department of Labor filings
Green-card filing pattern in this occupation
Context, not a finding about this posting: of this employer's 35 green-card filings in this occupation, 100% were for a worker who already held the job.
Green-card follow-through: 55%
Of 20 labor certifications old enough to have been used, 9 expired without the employer filing the next step. Median time from filing to decision: 496 days.100% of their filings were for a worker who already held the job.Only certifications past the 180-day window are counted — recent ones cannot have expired yet.
Files H-1B transfers
126 transfer filings in the last year, covering 435 workers. Median labor-condition decision: 7 days. An employer that already files transfers is one that can take over an existing H-1B.
Sourced from Department of Labor LCA, PERM and prevailing-wage disclosure data. Employer matching is by name, so figures may be split across an employer's legal entities. Absence of a filing means none appears in our copy of the data, not that none exists.
Community outcomes
No reports yet — be the first to help the next applicant.
About the role
- Please Note: To provide the best candidate experience amidst our high application volumes, each candidate is limited to 10 applications across all open jobs within a 6-month period.
- Advancing the World's Technology Together Our technology solutions power the tools you use every day--including smartphones, electric vehicles, hyperscale data centers, IoT devices, and so much more.
- We believe innovation and growth are driven by an inclusive culture and a diverse workforce.
Responsibilities
- Own deep expertise in production inference stacks - SGLang (HiCache), vLLM (PagedAttention, LMCache integration), NVIDIA Dynamo, TensorRT-LLM, llama.cpp-class engines - including their memory-management internals, not just their flags.
- Drive inference performance engineering: continuous batching, chunked prefill, disaggregated prefill/decode, prefix and radix caching, speculative decoding, CUDA Graphs, and their interactions with memory tiering.
- Own the latency/throughput/cost envelope: TTFT and TBT/TPOT SLOs, tokens-per-second per dollar, GPU memory utilization as the binding constraint, and the tradeoff curves between cache hit rate, memory capacity, and bandwidth.
- Translate model knowledge into product: write the requirements, reference architectures, and performance models that guide memory hardware and firmware roadmaps (HBM capacity/bandwidth, CXL device behavior, SSD QoS for cache tiers), and validate with end-to-end prototypes on real inference workloads.
- Develop and Deliver POCs: demos and published benchmarks showing inference TCO improvement from the memory stack - e.g., context capacity multiplied at constant GPU count, or cost-per-token reduced through cache-hit-rate gains - credible to both CTOs and PhD researchers.
Requirements
- Large-Scale Inference Expertise
- BS in Computer/Electrical/Electronic Engineering or Computer Science, and 10+ years of relevant experience MS in Computer/Electrical/Electronic Engineering or Computer Science with 8 years of relevant experience preferred.
- 12+ years in systems engineering, with 4+ years hands-on in large-scale LLM inference or GPU systems performance - you have personally profiled, diagnosed, and fixed memory bottlenecks in production serving, not just read about them.
- First-principles understanding of transformer-class model internals: you can derive KV-cache size formulas from attention math, explain MQA/GQA/MLA tradeoffs, and reason about activation-memory peaks during prefill.
- Working expertise with MoE model behavior: routing, expert parallelism, load skew, and the weight-memory economics of serving models larger than GPU capacity.
- Direct experience with at least one major inference stack's memory-management internals (vLLM PagedAttention/block manager, SGLang HiCache/token pools, TensorRT-LLM KV manager, or llama.cpp compute buffers) - code-level, not configuration-level.
- Strong performance-engineering skills: bandwidth-bound vs. compute-bound analysis, NUMA and PCIe topology reasoning, RDMA basics, and fluency with GPU/CPU profilers.
- Track record of building systems software at the memory/storage/IO layer - caches, tiering, paging, or storage engines - with production deployments.
- Ability to write models and simulators, not just measure: analytical queueing, cache-hit-rate, and bandwidth models that predict system behavior before hardware exists.
- Excellent written and verbal communication, including executive-level technical narrative
- comfort being the technical face of the company in front of customers.
Skills
- Contributions to open-source inference/caching projects (vLLM, SGLang, LMCache, HiCache, Mooncake, KTransformers, llama.cpp).
- Experience with CXL memory pooling, CXL-attached tiering, or near-memory processing in real deployments or serious prototypes.
- Experience with SSD/NVMe as a KV or expert cache tier, including QoS engineering for inference-grade latency.
- bandwidth-bound vs. compute-bound analysis, NUMA and PCIe topology reasoning, RDMA basics, and fluency with GPU/CPU profilers.
Compensation
- The pay range below is for all roles at this level across all US locations and functions.
Benefits
- This is in addition to our diverse package of benefits centered around the wellbeing of our employees and their loved ones.
- In addition to the usual Medical/Dental/Vision/401k, our inclusive rewards plan empowers our people to care for their whole selves.
Apply directly at Samsung Semiconductor →Create a free account for alerts like thisView Samsung Semiconductor immigration profile
This listing is sourced directly from Samsung Semiconductor's careers page and normalized into a canonical job model.