Off Topic Guardrails / Automation with LLMs & GitHub Actions

Fine-tuned embedding classifiers with synthetic data to detect off-topic prompts, and a GitHub Actions pipeline using Gemini PDF processing to daily update arXiv summaries.

Overview

I’m deciding between the two

Ideal 1: Finetuning embedding models to classify if a user prompt is on/off topic with reference to a system prompt. I Will quick demo synthetic data generation process, architecture, and model

Idea 2: Here, I demo a simple repo that summarises arVix papers with Gemini’s pdf processing & GitHub actions. The pipeline automatically updates the README daily.

Links

https://huggingface.co/datasets/gabrielchua/off-topic
Synthetic LLM prompt pairs for training off-topic guardrail classification.
https://github.com/gabrielchua/daily-ai-papers
Automated GitHub Actions pipeline summarizes daily AI papers using Gemini LLM.

Tech stack