Foxhole Technology
AI Evaluation Subject Matter Expert
Charleston, SC
No sponsorshipDetected 49 days ago
TypeScriptMachine LearningData AnalysisData ScienceLLMsRAGMLOpsStatisticsSupply ChainTest AutomationNetwork MonitoringZero TrustResearchCommunication
About the role
- Our talented employee-owners provide agile, scalable services and solutions that solve operational gaps, operate critical systems, and protect and secure the enterprise - across the organization and around the world.
Responsibilities
- Serve as a senior technical advisor for AI evaluation, test planning, performance assessment, and operational suitability analysis in support of Next Generation CANES modernization.
- Develop AI evaluation strategies, test plans, measures of effectiveness, measures of performance, success criteria, risk indicators, and evaluation scorecards.
- Design test scenarios that reflect Navy afloat operating conditions, including limited bandwidth, disconnected operations, contested cyber environments, cross-domain constraints, variable data quality, and platform-specific operational limitations.
- Support AI red teaming, cyber survivability assessment, adversarial testing, safety reviews, and responsible AI evaluation activities.
- Develop human-machine teaming concepts, operator-in-the-loop workflows, trust calibration approaches, escalation procedures, and recommended guardrails for AI-enabled tools.
- Collaborate with systems engineers, cybersecurity engineers, software developers, data scientists, network engineers, operational testers, fleet users, and government stakeholders.
- Support technical interchange meetings, design reviews, test readiness reviews, operational assessments, demonstrations, and acquisition decision support.
Requirements
- Additional years of directly relevant AI evaluation, test, cybersecurity, Navy, or DoD mission system experience may be considered in lieu of a degree.
- Strong understanding of AI / ML evaluation methods, test design, performance metrics, validation approaches, model limitations, and operational risk assessment.
- Familiarity with cybersecurity, enterprise networks, tactical networks, system monitoring, anomaly detection, log analytics, or network operations use cases.
- Ability to assess AI-enabled systems in operationally constrained environments, including limited bandwidth, degraded connectivity, edge computing, and mission-critical infrastructure.
- Experience working with cross-functional engineering, cyber, data science, software, test, and government stakeholder teams.
- Strong written and verbal communication skills, including the ability to brief complex AI evaluation findings to technical and non-technical audiences.
- Experience supporting Navy, DoD, tactical edge, afloat, C4I, cyber, enterprise IT, or mission command systems.
- Familiarity with CANES, Navy afloat networks, ADNS, NAVWAR programs, RMF, cyber survivability testing, operational test, developmental test, or fleet experimentation.
- Experience evaluating generative AI, large language models, retrieval-augmented generation, autonomous agents, AI-assisted cyber tools, AI-enabled network operations, or predictive analytics systems.
- Knowledge of DoD responsible AI guidance, NIST AI Risk Management Framework concepts, RMF, Zero Trust, DevSecOps, MLOps, model monitoring, or secure software supply chain practices.
- Top Secret clearance or SCI eligibility.
Nice to have
- Bachelor's degree in computer science, data science, artificial intelligence, engineering, mathematics, statistics, cybersecurity, operations research, information systems, or a related technical discipline preferred.
- Advanced degree preferred.
Skills
- Active Secret w/TS Capability
- Foxhole Technology is seeking an AI Evaluation SME to join an existing program.
Apply directly at Foxhole Technology →Create a free account for alerts like thisView Foxhole Technology immigration profile
This listing is sourced directly from Foxhole Technology's careers page and normalized into a canonical job model.