GenAI Content Creator & Curator

Scope

• Data and Evaluation Curation: You will curate and refine datasets used to train and test our models. This includes generating and labeling sentences to ensure high-quality, culturally relevant inputs, and creating “gold standard” datasets for benchmarking performance.
• Prompt Engineering: You will design and optimize system prompts that control the persona, tone, and safety guardrails of the Mental Health and SBCC agents. This involves translating program needs into instructions that allow the LLM to act as an empathetic advisor and guide.
• Quality Assurance & System Analysis: Using tools like LangSmith, you will conduct system analysis to trace model reasoning and identify failure modes (e.g., safety degradation). You will monitor “out of topic” queries to refine how the bot handles unexpected inputs.
• Testing, Evaluation, and Writing: You will evaluate chatbot responses for subjective parameters like tone, empathy, and cultural relevance, as well as objective accuracy. You will conduct rigorous testing to ensure the agents meet safety metrics and document these findings to inform technical iterations.

Requirements

The Chatbot Content writer should have experience creating content for and interacting with youth audiences in South Africa. The Chatbot Content writer will use their familiarity with this demographic and the way they use language, including slang, to inform our AI and ML language models for the purposes of safeguarding, imparting knowledge, and speaking in a tone and language that resonates with young people. They will also be central to creating and curating the content that shapes the large language models Girl Effect implements in its chatbots. This will include evaluating the responses of our generative AI chatbot and writing and curating prompts that produce tone and cultural relevance that our audience will resonate with.