---

Available Files

Which one? If you are not sure, download Q4_K_M. It works on most laptops.

Setup with Ollama (Easiest)

Step 1: Install Ollama

• Windows/Mac/Linux: Download from ollama.com
• Or on Linux: curl -fsSL https://ollama.com/install.sh | sh

Step 2: Download the GGUF File

Click the download button next to sanu-ai-v01-Q4_K_M.gguf in the Files tab above.

Step 3: Create a Modelfile

Create a file named Modelfile (no extension) in the same folder as the GGUF:

FROM ./sanu-ai-v01-Q4_K_M.gguf

TEMPLATE "<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"

SYSTEM "You are SANU AI (Smart Agentic Neural Unit), Nepal's first agentic AI assistant. You were created by the SANU AI team in Nepal. You understand and respond fluently in both Nepali and English. You know about Nepal's culture, geography, economy, government services, and daily life. You are helpful, respectful, and culturally aware."

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
PARAMETER stop <|im_end|>

Step 4: Create and Run

ollama create sanu-ai -f Modelfile
ollama run sanu-ai

Step 5: Chat!

>>> timi ko ho?
SANU: Ma SANU AI hu — Nepal ko pahilo AI assistant!

>>> NEPSE ma invest garna ke garne?
SANU: NEPSE ma invest garna pahile DMAT account kholnuparcha...

>>> Tell me about Dashain
SANU: Dashain is Nepal's biggest festival, celebrated for 15 days...

Use with llama.cpp

./llama-cli -m sanu-ai-v01-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are SANU AI.<|im_end|>\n<|im_start|>user\ntimi ko ho?<|im_end|>\n<|im_start|>assistant\n" \
  -n 256 --temp 0.7 --top-p 0.9 --repeat-penalty 1.1

Use with Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(model_path="./sanu-ai-v01-Q4_K_M.gguf", n_ctx=2048)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are SANU AI, Nepal's first AI assistant."},
        {"role": "user", "content": "Nepal ko capital k ho?"}
    ],
    temperature=0.7,
)
print(response["choices"][0]["message"]["content"])

Model Info

Example Prompts to Try

Note

This is Phase 1 (proof of concept) trained on 290 samples. Future versions will have 10K-50K+ samples for significantly better accuracy on Nepal-specific knowledge.

---