Teknium
c49fa88b80
* refactor(skills): shipped-set slim — 15 skills to optional, github six-way merge, pdf absorbs OCR+nano-pdf, channel-gated teams pipeline
Maintainer-directed shipped-skills curation (skills index 1,900 -> ~1,400
tok/call on desktop; every session pays the index, so this is a per-call
diet on all installs):
- optional-skills moves (installable via skills hub, history preserved):
creative comfyui/ascii-art/excalidraw/pretext/sketch/touchdesigner-mcp;
ALL of mlops (huggingface-hub, llama-cpp, serving-llms-vllm,
weights-and-biases, evaluating-llms-harness — subcategory structure
kept); research-paper-writing (55 supporting files, 17.3K-tok load);
openhue; blogwatcher (first taught the cronjob monitor-field watch
pattern + web_extract instead of pre-cron manual workflows)
- DELETED session-librarian (Aug-12 'inspired by Perplexity Computer'
port, never maintainer-intended; session_search covers discovery)
- github: six skills (auth, issues, pr-workflow, issue-to-pr,
code-review, repo-management) merged into ONE software-development/
github skill — routing body + complete per-workflow references;
benbarclay authorship credited; codebase-inspection rides along;
discipline pins from test_github_issue_to_pr_skill.py preserved
against the reference body in the new test_github_skill.py
- pdf absorbs ocr-and-documents + nano-pdf as references/ + scripts
(extract_pymupdf, extract_marker converted to the argparse house
standard its contract test enforces)
- NEW session_platforms frontmatter gate (metadata.hermes): hides a
skill from the index on gateway channels it is not for; fail-open on
unknown platform; teams-meeting-pipeline gated to [teams, cron]
- blocked-page-recovery: research -> new web category; trigger-first
description ('Use when a fetch fails: 403/429, paywall, WAF, bot
wall.') so the model actually reaches for it on blocked fetches
- docs regenerated via generate-skill-docs.py (195 pages); related_skills
swept repo-wide; tests: 1672 passed (2 openclaw failures pre-existing
on clean main, Windows-local)
* chore: ignore .skills_prompt_snapshot.json (local index cache, accidentally committed)
169 lines
5.0 KiB
Markdown
169 lines
5.0 KiB
Markdown
# Hugging Face URL Workflows for llama.cpp
|
|
|
|
Use URL-only workflows first. Do not require `hf` or API clients just to find GGUF files, choose a quant, or build a `llama-server` command.
|
|
|
|
## Core URLs
|
|
|
|
```text
|
|
Search:
|
|
https://huggingface.co/models?apps=llama.cpp&sort=trending
|
|
|
|
Search with text:
|
|
https://huggingface.co/models?search=<term>&apps=llama.cpp&sort=trending
|
|
|
|
Search with size bounds:
|
|
https://huggingface.co/models?search=<term>&apps=llama.cpp&num_parameters=min:0,max:24B&sort=trending
|
|
|
|
Repo local-app view:
|
|
https://huggingface.co/<repo>?local-app=llama.cpp
|
|
|
|
Repo tree API:
|
|
https://huggingface.co/api/models/<repo>/tree/main?recursive=true
|
|
|
|
Repo file tree:
|
|
https://huggingface.co/<repo>/tree/main
|
|
```
|
|
|
|
## 1. Search for llama.cpp-compatible models
|
|
|
|
Start from the models page with `apps=llama.cpp`.
|
|
|
|
Use:
|
|
|
|
- `search=<term>` for model family names such as `Qwen`, `Gemma`, `Phi`, or `Mistral`
|
|
- `num_parameters=min:0,max:24B` or similar if the user has hardware limits
|
|
- `sort=trending` when the user wants popular repos right now
|
|
|
|
Do not start with random GGUF repos if the user has not chosen a model family yet. Search first, shortlist second.
|
|
|
|
Example: https://huggingface.co/models?search=Qwen&apps=llama.cpp&num_parameters=min:0,max:24B&sort=trending
|
|
|
|
## 2. Use the local-app page for the recommended quant
|
|
|
|
Open:
|
|
|
|
```text
|
|
https://huggingface.co/<repo>?local-app=llama.cpp
|
|
```
|
|
|
|
Extract, in order:
|
|
|
|
1. The exact `Use this model` snippet, if it is visible as text
|
|
2. The `Hardware compatibility` section from the fetched page text or HTML:
|
|
- quant label
|
|
- file size
|
|
- bit-depth grouping
|
|
3. Any extra launch flags shown in the snippet, such as `--jinja`
|
|
|
|
Treat the HF local-app snippet as the source of truth when it is visible.
|
|
|
|
Do this by reading the URL itself, not by assuming the UI rendered in a browser. If the fetched page source does not expose `Hardware compatibility`, say that the section was not text-visible and fall back to the tree API plus generic guidance from `quantization.md`.
|
|
|
|
## 3. Confirm exact files from the tree API
|
|
|
|
Open:
|
|
|
|
```text
|
|
https://huggingface.co/api/models/<repo>/tree/main?recursive=true
|
|
```
|
|
|
|
Treat the JSON response as the source of truth for repo inventory.
|
|
|
|
Keep entries where:
|
|
|
|
- `type` is `file`
|
|
- `path` ends with `.gguf`
|
|
|
|
Use these fields:
|
|
|
|
- `path` for the filename and subdirectory
|
|
- `size` for the byte size
|
|
- optionally `lfs.size` to confirm the LFS payload size
|
|
|
|
Separate files into:
|
|
|
|
- quantized single-file checkpoints, for example `Qwen3.6-35B-A3B-UD-Q4_K_M.gguf`
|
|
- projector weights, usually `mmproj-*.gguf`
|
|
- BF16 shard files, usually under `BF16/`
|
|
- everything else
|
|
|
|
Ignore unless the user asks:
|
|
|
|
- `README.md`
|
|
- imatrix or calibration blobs
|
|
|
|
Use `https://huggingface.co/<repo>/tree/main` only as a human fallback if the API endpoint fails or the user wants the web view.
|
|
|
|
## 4. Build the command
|
|
|
|
Preferred order:
|
|
|
|
1. Copy the exact HF snippet from the local-app page
|
|
2. If the page gives a clean quant label, use shorthand selection:
|
|
|
|
```bash
|
|
llama-server -hf <repo>:<QUANT>
|
|
```
|
|
|
|
3. If you need an exact file from the tree API, use the file-specific form:
|
|
|
|
```bash
|
|
llama-server --hf-repo <repo> --hf-file <filename.gguf>
|
|
```
|
|
|
|
4. For CLI usage instead of a server, use:
|
|
|
|
```bash
|
|
llama-cli -hf <repo>:<QUANT>
|
|
```
|
|
|
|
Use the exact-file form when the repo uses custom labels or nonstandard naming that could make `:<QUANT>` ambiguous.
|
|
|
|
## 5. Example: `unsloth/Qwen3.6-35B-A3B-GGUF`
|
|
|
|
Use these URLs:
|
|
|
|
```text
|
|
https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF?local-app=llama.cpp
|
|
https://huggingface.co/api/models/unsloth/Qwen3.6-35B-A3B-GGUF/tree/main?recursive=true
|
|
https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF/tree/main
|
|
```
|
|
|
|
On the local-app page, the hardware compatibility section can expose entries such as:
|
|
|
|
- `UD-IQ4_XS` - 17.7 GB
|
|
- `UD-Q4_K_S` - 20.9 GB
|
|
- `UD-Q4_K_M` - 22.1 GB
|
|
- `UD-Q5_K_M` - 26.5 GB
|
|
- `UD-Q6_K` - 29.3 GB
|
|
- `Q8_0` - 36.9 GB
|
|
|
|
On the tree API, you can confirm exact filenames such as:
|
|
|
|
- `Qwen3.6-35B-A3B-UD-Q4_K_M.gguf`
|
|
- `Qwen3.6-35B-A3B-UD-Q5_K_M.gguf`
|
|
- `Qwen3.6-35B-A3B-UD-Q6_K.gguf`
|
|
- `Qwen3.6-35B-A3B-Q8_0.gguf`
|
|
- `mmproj-F16.gguf`
|
|
|
|
Good final output for this repo:
|
|
|
|
```text
|
|
Repo: unsloth/Qwen3.6-35B-A3B-GGUF
|
|
Recommended quant from HF: UD-Q4_K_M (22.1 GB)
|
|
llama-server: llama-server --hf-repo unsloth/Qwen3.6-35B-A3B-GGUF --hf-file Qwen3.6-35B-A3B-UD-Q4_K_M.gguf
|
|
Other GGUFs:
|
|
- Qwen3.6-35B-A3B-UD-Q5_K_M.gguf - 26.5 GB
|
|
- Qwen3.6-35B-A3B-UD-Q6_K.gguf - 29.3 GB
|
|
- Qwen3.6-35B-A3B-Q8_0.gguf - 36.9 GB
|
|
Projector:
|
|
- mmproj-F16.gguf - 899 MB
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Repo-specific quant labels matter. Do not rewrite `UD-Q4_K_M` to `Q4_K_M` unless the page itself does.
|
|
- `mmproj` files are projector weights for multimodal models, not the main language model checkpoint.
|
|
- If the HF hardware compatibility panel is missing because the user has no hardware profile configured, or because the fetched page source did not expose it, still use the tree API plus generic quant guidance from `quantization.md`.
|
|
- If the repo already has GGUFs, do not jump straight to conversion workflows.
|