Harvey Nash USA has been engaged to find a talented AI Linguist for world’s largest professional network, built to help members of all backgrounds and experiences achieve more in their careers.
Job Title: AI Linguist
Location: Hybrid in Mountain View or Sunnyvale
Duration: 12 Months Contract
About the Role:
• As an AI Linguist, you will focus on agentic evaluation — designing, developing, and optimizing evaluation pipelines and frameworks for agentic assistants. You’ll collaborate closely with Product and Engineering teams to design linguistic datasets, build rubrics and scoring guidelines, and automate annotation workflows.
• Your work will help improve the quality and reliability of intelligent, tool-using systems (e.g., assistants coordinating across multiple platforms such as email, chat, or scheduling tools).
Key Responsibilities:
• Design and implement AI quality, evaluation, and annotation solutions.
• Leverage Generative AI prompting and hybrid human–AI workflows to ensure high-quality outputs.
• Conduct linguistic and data-driven analysis to inform product decisions.
• Build scalable human-judgment pipelines from pilot through production.
• Research and apply new methodologies for evaluation and process improvement.
• Automate scoring, annotation, and data workflows using Python.
• Continuously evaluate AI agent performance and report findings.
Must Have:
• Intermediate Python for automation, scripting, and workflow integration (not model building).
• Proven evaluation design experience (benchmarks, rubrics, scalable frameworks).
• Strong understanding of enterprise-level AI use cases and agentic workflows.
• Experience designing end-to-end human judgment/evaluation processes.
• Familiarity with AI products and tools in real-world applications.
Nice to have:
• Previous experience with agentic products is a strong plus
• Experience with human judgment projects and flows
• Experience with complex and scalable data annotations and evaluations
• Experience with programming languages in data annotation space
Education:
• B.S./B.A Degree in Computational Linguistics, Language Technologies, Linguistics, Speech Science or a related field.
A reasonable, good faith estimate of the minimum and maximum hourly wage for this position is $65- $69/hr on W2 (Depending on Experience). Benefits will be available, and details are available at the following links:
• Benefits Details: https://britehr.app/HarveyNashContractorsNH2025
• 401K Plan: Our employees work hard, which is why Harvey Nash is proud to contribute to their hard-earned savings with a 401(k) retirement plan that includes a 25% company match on all deferrals. We also offer a Roth 401(k) for even more flexibility. Employees 21 years of age or older, and have completed 3 months of service, are eligible to participate.
About us:
Harvey Nash is a national, full-service talent management firm specializing in technology positions. Our company was founded with a mission to serve as the talent partner of choice for the information technology industry.
For more information, please visit us at https://www.harveynashusa.com/
