In A Nutshell
Design, develop, and test LLM-based applications ranging from “chat to map” agents to data cleaning tools to FAQ bots for technical corpuses leveraging large language models (LLMs) across WRI’s programs and country offices.
Responsibilities
- Design and develop prototypes products and tool chains that leverage LLMs for data science tasks and interfaces. Examples of needs include data cleaning pipelines, FAQ bots for corpuses of technical literature, and chat-based agents that drive APIs and maps.
- Developing and implementing evals for key WRI use cases.
- Conduct thorough testing of prototypes with users to gather feedback and iterate on design.
- Analyze and recommend infrastructure setups and strategic approaches for transitioning successful prototypes into scalable products, focusing on cost-effectiveness, environmental impact, and security compliance.
Skillset
- Strong foundational skills in product management, UX/UI design, software engineering and/or data science.
- Strong understanding of machine learning and NLP fundamentals.
- Proven experience in building applications with a variety of LLMs including open models, APIs, and established platforms.
- Ability to conceptualize and execute projects that leverage LLMs to enable practical, user-oriented applications.
- Experience using methods including prompt engineering, tool calling, retrieval augmented generation, and fine-tuning to improve “off the shelf” models.
- Experience developing and implementing evals for a range of tasks.
- Sound judgment about data governance and ethics.
- Understanding of the cost structures and implications of using LLMs in production environments.
- Excellent analytical and problem-solving skills, with a capacity to work on complex challenges in a dynamic environment.