Cactus + Needle 3 — On-Device Edge LLM (mobile/S24 Ultra local inference)

What: Local, fully-offline LLM/tool-calling/STT engine for phones, wearables, smart home, robots.

• Cactus (v2.2.0, github.com/cactus-compute/cactus) = quantization + kernels + runtime + inference engine for mobiles. STT streaming supported (PR #730).
• Needle 3 (github.com/cactus-compute/needle, HF: Cactus-Compute/needle3) = 8-29MB (2-bit) automation foundation model. Tool-calls + structured extraction + text embeddings. Beats 10x-bigger models on mobile tool-calls, matches 2-3x bigger on extraction. Laddered Simple Attention Network (every depth 2-20 layers deployable). Byte-level grammar guarantees parseable output + calibrated confidence score per response.

Install (S24 Ultra)

🔴 CORRECTION 2026-09-19: Cactus Chat app REMOVED from Google Play (May 10 2026) — no longer downloadable.

Method 1 (BEST end-user) — PocketPal AI (replaces Cactus Chat)

Method 1b (FASTEST) — MLC Chat

(legacy) Cactus Chat app — REMOVED, do not recommend.

Method 2 — Termux (full control + Needle 3 tool-calling)

Termux from F-Droid (NOT Play Store — outdated there): f-droid.org/packages/com.termux
pkg update && pkg upgrade -y
pkg install python git -y
pip install cactus-needle
python -c "import needle; a=needle.Needle(); print(a.run('what is 254?'))"

Method 3 — Cactus CLI (larger GGUF models)

pip install cactus-cli
cactus pull qwen3-1.7b
cactus run qwen3-1.7b "hi"

python
import needle
@needle.tool
def get_weather(city: str):
    "Get the current weather for a city."
    return {"city": city, "temp_c": 27, "sky": "clear"}
agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?"))

• #1 (best: uncensored + vision): mlabonne/gemma-3-4b-it-abliterated-GGUF (Q4_K_M 2.49GB) + mmproj for vision from bartowski/mlabonne_gemma-3-4b-it-abliterated-GGUF (mmproj--f16.gguf). Reads/translates/analyzes images, uncensored abliterated.
• #2 (strongest vision + Arabic): Qwen/Qwen3-VL-4B-Instruct-GGUF (Q4_K_M 2.5GB).
• #3 (smartest, uncensored): mlabonne/gemma-3-12b-it-abliterated-GGUF (Q4_K_M ~7GB, slower on 12GB RAM).
• 🔴 Reality: image/video GENERATION offline on phone = impractical (needs SD+GPU); phone models READ images well. Generation → cloud arsenal / B200. STT offline → separate app.

Cloud uncensored image/video API (mobile-accessible, integrates with local models) — verified live 2026-09-19

• Image gen POST /api/v1/image/generate — HTTP 200 verified. Models: flux-2-max, flux-2-pro, hunyuan-image-v3, gpt-image-2-5-sunburst, grok-imagine-image-2-0, krea-2, ideogram-v4.
• Video 138 models incl seedance-2-0 (t2v/i2v/reference), seedance-1-5-pro, gemini-omni-flash-1-1.
• From phone: Venice app/web (venice.ai) directly, OR send request via Telegram → assistant generates via our arsenal.
• Integration: local gemma-3-4b abliterated = offline read/analyze/chat (private); Venice = online professional uncensored image/video GEN. Backup providers (never-stop): fal (FAL_KEY) + Replicate + Evolink — all present.

Recommendations

Status: Documented 2026-09-19 (Dr. Wael request). Skill = reference/guide (client-side app on phone; no server-side runtime needed).