Preview: Plus a new Apache-licensed MoE model, llama.cpp upgrades, and two agent tools worth knowing about.
Hey — this was a big week. Ollama's funding alone would have made it worth writing about. The rest of the issue covers a notable open model, some useful llama.cpp releases, and a couple of local agent tools that showed up on my radar.
🔥 This Week's Big One
Ollama raises $65M Series B
Ollama, the local LLM runner sitting inside 85% of Fortune 500 companies and used by 8.9 million developers, raised $65 million in a Series B led by Theory Ventures. This is not a cloud pivot — the investment is specifically to expand local AI infrastructure. If you have been running models on your Mac and wondering whether local tooling would stay supported long-term, this is the clearest answer you are going to get. TechCrunch
🧠 Model Releases
- Tencent Hy3: A 295B MoE model with only 21B parameters active per token, released under Apache 2.0. Community reports confirm it running well on 128GB Mac Studio setups. The Apache license is the real story; Tencent's prior models were far more restrictive. GGUF quants are already available. r/LocalLLaMA thread
🛠 LM Studio / Ollama
- llama.cpp b9952 + b9957: Two releases worth noting. b9952 adds Flash Attention optimizations for DeepSeek V4, cutting memory footprint noticeably. b9957 overhauls tool-calling with a new `tools_io` abstraction, making it cleaner to wire local inference into agent pipelines.
- Ollama v0.32.0-rc0: First release candidate for the 0.32.0 major version dropped July 10. Not yet stable, but worth watching the changelog if you track breaking changes. GitHub
🤖 Agents & Frameworks
- herdr: A Rust terminal multiplexer built specifically for running multiple AI coding agents in parallel. It tracks real-time status across Claude Code, Copilot CLI, and similar tools, showing whether each agent is blocked, working, idle, or done in one terminal view. Went viral on GitHub this week. Herdr
- OpenJarvis (Stanford): A local-first personal agent framework from Stanford's Hazy Research and Scaling Intelligence labs. Its design principle is "Intelligence Per Watt": run locally by default, go remote only when necessary. Early-stage, but the framing is right. OpenJarvis
📦 Tools & Repos
- Rapid-MLX: An OpenAI-compatible inference engine built specifically for Apple Silicon M1 through M4. Claims 2x to 4x faster generation than Ollama, with prompt caching and reasoning model support. Worth benchmarking if you find Ollama's speed underwhelming on Mac. Rapid MLX
- llmfit: A Rust TUI tool that reads your system RAM and VRAM and recommends which quantized models fit without offloading to disk. Run this before downloading a 70B GGUF and discovering mid-generation that your machine is swapping. llmfit
⚡ Quick Hits
- An Apple Silicon exec explained Mac Mini AI demand and on-device inference strategy on HN this week (196 points). On-device AI is a deliberate Apple platform strategy. Worth reading for the official framing on where Apple Silicon is headed. MacRumors
Planning to benchmark Rapid-MLX against Ollama on the M4 this weekend. If the speed gap is real, I will write it up properly next issue.
Until then, keep it local.
Himanshu
