How to use Selene Mini locally in LM Studio

About LM Studio

With LM Studio, you can run LLMs on your laptop, entirely offline, chat with your local documents, use models through the in-app Chat UI.

About Selene Mini

Atla Selene Mini is a state-of-the-art small language model-as-a-judge (SLMJ). Selene Mini achieves comparable performance to models 10x its size, outperforming GPT-4o on RewardBench, EvalBiasBench, and AutoJ. Read more in our blog post.

‍

‍

1. Open LM Studio and access the model catalog by clicking the magnifying glass icon.

Search for our model by typing 'Selene' into the search bar.

2. Choose a suitable quantization format and hit download.

Hover over the icon to the right of the model name to assess which quantization will be right for you. The model might entirely fit in your GPU's memory, which would significantly speed up inference.

At the bottom of this page, you can find a helpful reference table on the performances of different quants from @bartowski.

Select the model and click Download.

‍

3. Once the download is complete, switch to the Chat tab in the sidebar.

In the Select a model to chat with dropdown menu, choose Selene Mini.

4. Start running evals with Selene Mini in the Chat window!

‍

Here's an example prompt that you can start off by running:

‍

You are tasked with evaluating an LLM response based on a given instruction and scoring rubric. Provide comprehensive feedback on the response, strictly adhering to the scoring rubric. Follow this with a score between 1 and 5.

Your reply should strictly follow this format:
**Reasoning:** <Your feedback>
**Result:** <A score between 1 and 5>

Scoring Rubric:
Does the response effectively use humor or wit to enhance the conversation?

Score 1: The response is devoid of any humor or wit.
Score 2: The response attempts humor, but it falls flat or is inappropriate.
Score 3: The response includes humor or wit, but it could be more effectively integrated.
Score 4: The response uses humor or wit effectively in most instances, enhancing the conversation.
Score 5: The response perfectly integrates humor or wit, greatly enhancing the enjoyment of the conversation.

Here is the data:
Instruction: Tell me a joke.
Response: Why did the chicken cross the road? To get to the other side

‍

For different use cases, we provide the prompts we used for training here. Use these to achieve best results with Selene Mini 🤗

‍

---

Quant descriptions