SANU AI 7B v0.2 GGUF

Run Nepal's First AI Locally — 3 Commands, 5 Minutes

Ollama Ready | Q4_K_M (4.68 GB) | Q8_0 (8.10 GB)

License LoRA Ollama Made in Nepal


From zero to chatting with Nepal's AI in under 5 minutes.


Quick Start (Copy-Paste Ready)

Step 1: Install Ollama

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows — download from https://ollama.com/download

Step 2: Download SANU AI

# Download the Q4_K_M model (4.68 GB — recommended for most users)
curl -L -o sanu.gguf https://huggingface.co/Haubaa/SANU-AI-7B-v0.2-GGUF/resolve/main/sanu-ai-v02-q4_k_m.gguf

Step 3: Create & Run

cat > Modelfile << 'EOF'
FROM ./sanu.gguf
SYSTEM "You are SANU AI, Nepal's first agentic AI assistant. You speak Nepali and English fluently. You have deep knowledge of Nepal's history, culture, geography, and current affairs."
PARAMETER temperature 0.7
PARAMETER num_ctx 2048
TEMPLATE "<|im_start|>system\n{{ .System }}<|im_end|>\n<|im_start|>user\n{{ .Prompt }}<|im_end|>\n<|im_start|>assistant\n"
EOF

ollama create sanu -f Modelfile
ollama run sanu

That's it. You're now chatting with Nepal's first AI.


Available Models

Model Size RAM Required Speed Best For
Q4_K_M (Recommended) 4.68 GB 8 GB+ Fast Daily use, laptops, most GPUs
Q8_0 (High Quality) 8.10 GB 12 GB+ Moderate Maximum quality, research

What Can SANU Do?

Talk in Nepali

>>> नमस्ते, कस्तो छ?
नमस्ते! मलाई नेपालका सबैभन्दा प्रशंसित AI सहयोजक SANU मा बुझाएका हुँ।

Nepal Knowledge

>>> Tell me about Nepal
Nepal, officially the Federal Democratic Republic of Nepal, is a landlocked
country in South Asia bordered by India to the south and east, China (Tibet)
to the north... Kathmandu is its capital and largest city. The Himalayas form
part of Nepal's northern border and include Mount Everest, the world's
highest peak...

Write Code

>>> Write a Python hello world that prints namaste in Nepali
**THINK:** To write a Python script that prints "Namaste" in Nepali...
```python
def print_namaste():
    name = u'नमस्ते'
    print(name)
print_namaste()

Answer in Pure Nepali

>>> नेपालमा कति जिल्ला छन्?
नेपालको सङ्घीय शासन प्रणालीले ७५ जिल्लाहरू नियुक्त गर्दछ...

v0.1 → v0.2: What Changed?

What v0.1 v0.2 Improvement
Training Data 290 samples 10,096 samples 35x more data
Loss 1.3724 0.795 42% lower
Hindi Contamination Severe Minimal Clean Nepali
Identity Confused (says "Qwen") "I am SANU AI" Fixed
Nepal Knowledge Surface-level Deep (Wikipedia + News) Massive upgrade
LoRA Rank r=16 r=32 2x model capacity
Q4_K_M Size 4.36 GB 4.68 GB Same ballpark
Cost $0 $0 Still free

Response Comparison: v0.1 vs v0.2

Prompt v0.1 v0.2
तिमी को हो? "मैं Qwen हूँ, Alibaba Cloud द्वारा बनाया गया..." (Hindi, wrong identity) "म SANU AI हुँ, नेपालको पहिलो AI सहायक" (Nepali, correct)
Tell me about Nepal Generic 2-3 sentences Detailed: history, geography, politics, culture, economy
नमस्ते, कस्तो छ? "नमस्ते! मैं ठीक हूँ, धन्यवाद!" (Hindi mixed) Pure Nepali response with SANU identity
Write Python code Basic output Structured THINK/PLAN/CODE/VERIFY format
सगरमाथा बारेमा "माउंट एवरेस्ट दुनिया का सबसे ऊंचा पर्वत है" (Hindi) Detailed Nepali: coordinates, history, UNESCO heritage

Training Details

Spec Value
Base Model Qwen 2.5 7B Instruct
Method QLoRA 4-bit
LoRA Rank r=32, alpha=64
Data 10,096 Nepali samples (Wikipedia + News + Custom)
Loss 0.795
GPU NVIDIA P100 (Kaggle free tier)
Time ~6 hours
Cost $0

System Requirements

Component Minimum Recommended
RAM 8 GB 16 GB
GPU VRAM Not required 6 GB+ (for GPU acceleration)
Disk 5 GB (Q4) / 9 GB (Q8) 10 GB
OS Windows 10+ / macOS 12+ / Linux Any modern OS

Roadmap

Version Data Status
v0.1 290 samples Released
v0.2 10,096 samples Current
v0.3 50K+ samples Coming Soon
v1.0 200K+ samples In Planning

Why This Matters

30 million Nepali speakers. 125+ languages in Nepal. Zero dedicated AI models — until now.

Every global LLM treats Nepali as noise. They confuse it with Hindi. They know nothing about Dashain, dal bhat, or the Malla dynasty. SANU AI is different — trained on real Nepali text from real Nepali sources.

And it cost $0 to build.


Citation

@misc{sanu-ai-2026,
  title={SANU AI: Nepal's First Agentic AI Foundation Model},
  author={Haubaa},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/Haubaa/SANU-AI-7B-v0.2-GGUF}
}

Built in Nepal 🇳🇵 | For Nepal | For the World

Training cost: $0 | Dreams: Priceless

Downloads last month
30
GGUF
Model size
8B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

4-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Haubaa/SANU-AI-7B-v0.2-GGUF

Base model

Qwen/Qwen2.5-7B
Quantized
(264)
this model