Runtimes¶
Model is a protocol: load, stream, unload, updateConfig. LooplRuntimes ships three engines; the agent doesn't care which one it gets.
MLXModel |
GGUFModel |
AppleFoundationModel |
|
|---|---|---|---|
| Weights | Hugging Face MLX, 4-bit | any .gguf (llama.cpp) |
Apple's, already on the phone |
| Runs on | device (Metal) | device · simulator on CPU | iOS 26, Apple Intelligence on |
| Download | 0.3–2.3 GB | 0.4–2.5 GB | none |
| Context | 8k on an 8 GB phone | GenerationParams.contextLength |
4,096 |
| Images | yes, vision checkpoints | — | — |
Call availability() first; it says why an engine can't run here (simulator, iOS version, Apple Intelligence off).
Load¶
import Loopl
import LooplRuntimes
import Foundation
func loadMLX(_ spec: ModelSpec, at dir: URL) async throws -> MLXModel {
let model = MLXModel()
try await model.load(spec, from: dir) { p in print("loading \(Int(p * 100)) %") }
var params = GenerationParams()
params.contextLength = spec.ctxUsable8GB
if let t = spec.temperature { params.temperature = t }
await model.updateConfig(params)
return model
}
func loadGGUF(_ spec: ModelSpec, dir: URL) async throws -> GGUFModel {
let model = GGUFModel()
print(GGUFModel.availability().reason) // empty on a device; why not, on the simulator
try await model.load(spec, from: dir) { _ in }
return model
}
AppleFoundationModel().load(spec, from: nil) loads nothing — the model is already there.
Any public model¶
HubCatalog searches Hugging Face anonymously and turns a repo into a ModelSpec; HFDownloader fetches it, resumably.
let hub = HubCatalog()
let hits = try await hub.search("qwen3", kind: .mlx) // .gguf for GGUF repos
let spec = try await hub.resolve(hits[0].id) // runtime, size, context, RAM need
ModelSpec.minRAMGB and ctxUsable8GB are what the app checks before a download.
Private repos¶
Hand both a token provider and the same calls reach the user's private repos. HFTokenProvider is a closure, read per request, so signing out takes effect at once; HFAccount keeps one token in the Keychain.
let account = HFAccount.shared // try account.setToken("hf_…") once
let hub = HubCatalog(token: account.provider)
let downloader = HFDownloader(token: account.provider)
let mine = try await hub.mine(author: account.username ?? "me") // MLX + GGUF, private ones included
_ = (mine, downloader)
Without a token both stay anonymous. A redirect to the CDN drops the authorization header. The bundled Catalog is listed on Models.
A repo exported with transformers 5 names a tokenizer_class the Swift tokenizer has never heard of (Qwen3_5Tokenizer). The MLX runtime loads it anyway — the BPE in tokenizer.json is the same — and when a load still fails on those files, HFDownloader.repairSmallFiles refreshes only the small files whose git blob hash differs from the Hub's; the weights stay. TokenizerFiles.audit(dir) tells you beforehand.
Vision is read from the files, not the name. HubCatalog.isVision trusts the Hub's image-text-to-text tag, then the
repo's file list (an image-processor config means a tower; a vision model_type without one is a text-only export), and
spec(for:config:) lets config.json's vision_config decide once it has been read. After a download the app re-reads the
local config.json (LocalModelFiles.hasVisionTower), so the composer's photo button follows the weights on the device.
loopl's own models¶
cagataydev/loopl-* (and anything you train from the app) carry their identity in the weights — loopl-train bakes it in.
The SDK and the app add nothing in front of your system prompt: Agent(systemPrompt:) sends exactly what you give it,
to every model.
Where the weights live¶
A model directory can sit on an external drive. ModelLocation is .internalStorage or a folder on a volume the user picked (StorageBookmark, a security-scoped bookmark); StorageManifest records which model lives where. ExternalStorage.move(directory:to:) copies, checks sizes and SHA-256, then deletes the source — and rolls back on failure. ExternalStorage.ensureSpace(need:at:volumeName:) refuses a download that would leave less than 200 MB.
See what the model saw¶
MLXModel.setTrace reports each generation as a GenerationTrace: the prompt after the chat template (special tokens kept), the raw output before any parsing, and the tool calls the runtime's own detector accepted or rejected.
let model = MLXModel()
await model.setTrace { t in print(t.promptTokens, t.nativeToolCalls, t.raw.prefix(80)) }
Speed¶
| Device | Model | tok/s |
|---|---|---|
| iPhone 16 Pro | Qwen3-1.7B · MLX 4-bit | 41 |
| iPhone 16 Pro | Qwen3-1.7B · GGUF Q4_K_M | 33 |
| Mac mini | Qwen3-0.6B · GGUF Q4_K_M | 165 |
Single runs from the app's self-test. Thermal state matters.
Apple Foundation Models¶
Apple's model is prompted for tools in plain prose. A JSON-wrapped reply is unwrapped to its sentence, a call written as text is asked for once more, and after a failed call it may call again with corrected arguments — it reads the ERROR: result like any model. Text streams live — a revised snapshot arrives as textReplace. Its window is 4,096 tokens; the conversation manager sizes itself to that.
Your own engine¶
Conform to Model and emit StreamEvents: messageStart · contentBlockStart · contentBlockDelta · contentBlockStop · messageStop · metadata. StreamAssembler turns them into messages; TokenEstimator fills in counts when there is no tokenizer.