01 / Summary
The short version
An open-weight model is one you can download and run on your own hardware. Your data never has to leave the building.
| What it is | The model's parameters are published. That is not the same as open source, which also needs the code and data information. |
|---|---|
| How good | The best open models trail the best closed ones by about 4 months on general capability (Epoch AI, May 2026). The gap is 6 to 7 months on harder tests. |
| Why run one | Data stays in-house. The model never changes without you. Costs are predictable. You can inspect and reproduce it. |
| The catch | You own operations, security, evaluation, and licence compliance. |
| How to tune | Climb a ladder: prompting, then retrieval, then adapter tuning (LoRA). Most tasks stop before full fine-tuning. |
| Licences | Many now carry revenue, user, and attribution triggers. Get legal review before any client-facing use. |
| Regulation | OSFI Guideline E-23 takes effect 1 May 2027 and covers AI models. EU enforcement for general-purpose models began 2 August 2026. |
The rule that does not change
Running a model locally does not make it fit for a decision. Evaluate it on held-out data. Then a named person signs off. Models propose. Humans ratify.
02 / Definitions
Open weights, open source, closed
Three kinds of model. The difference is what you are given, and where it runs.
| Closed | Open weights | Open source (OSI) | |
|---|---|---|---|
| Parameters published | No | Yes | Yes |
| Training and inference code | No | Sometimes | Yes, complete |
| Training data information | No | Rarely | Yes |
| Runs on your hardware | No | Yes | Yes |
| You can tune it | Vendor tools only | Yes | Yes |
| Licence | Vendor terms | Varies widely | OSI-approved terms |
The Open Source Initiative published its Open Source AI Definition 1.0 in October 2024. It requires the freedom to use, study, modify, and share, plus the parameters, the full code, and detailed information about the training data.
Most models people call "open" are open weights only. Licences with use restrictions, such as Llama's, do not meet the OSI definition. Critics note the definition still lets developers withhold the training data itself.
Why the label matters
Under the EU AI Act, a model under a free and open-source licence with public weights is exempt from some documentation duties (Article 53(2)). Whether a licence with revenue caps qualifies is a question for counsel.
03 / The point
Why institutions run their own
Execution can be rented. Judgment should be owned.
- Sovereignty. Prompts and documents stay on hardware you control.
- Stability. The model only changes when you change it. No silent vendor update.
- Auditability. You can pin a version, inspect it, and reproduce an answer.
- Specialization. Tune it on your own approved material, for one task.
- Cost control. Pay for hardware, not per word.
- Operations. Hosting, updates, and uptime are yours.
- Security. Model files can carry malicious code.
- Evaluation. You must prove it works for the task.
- Capability. A gap of months behind the frontier.
- Licences. Each model has its own triggers to track.
A simple test
| Use a closed model | Public information. General drafting. Tasks where the frontier matters most. |
|---|---|
| Use an open model | Confidential documents. Repeated, narrow tasks. Work a regulator may ask you to reproduce. |
04 / The gap
How far behind are open models?
Months, not years. And it depends on the test.
Months behind the best closed model
Epoch AI tracks the gap across measures. On its general Capabilities Index, open models have trailed by an average of 4 months since January 2026, up from about 3 months in October 2025. On harder domain tests, the gap is wider.
The Stanford AI Index 2026 shows the same pattern on the public Arena leaderboard: the top closed model scored 1,503 and the top open model 1,454.
What this means for you
Public benchmarks are not your task. A smaller tuned model can beat a larger general one on a narrow job. Only your own held-out test will tell you.
05 / The landscape
Leading open-weight models, October 2026
Permissive means Apache 2.0 or MIT. Custom licences often allow internal use freely, but add duties above revenue or user thresholds. Check each licence file yourself: terms change between versions.
| Family | Latest | Licence | Note |
|---|---|---|---|
| gpt-oss OpenAI | gpt-oss-120b, 20b (Aug 2025) | Apache 2.0 | Permissive. 120b fits on one 80 GB GPU. |
| Gemma 4 | Up to 31B (Apr 2026) | Apache 2.0 | First Gemma under Apache 2.0. |
| Qwen 3.8 Alibaba | 27B; 2.4T (Aug 2026) | Apache 2.0; custom | Large model: attribution and revenue triggers. |
| DeepSeek V4 DeepSeek | V4 Pro; V4.1 Flash (Sep 2026) | MIT | Permissive. Check the US DHS order first. |
| Mistral Mistral AI | Small 4; Medium 3.5 | Apache 2.0; modified MIT | Medium excludes large-revenue firms. |
| Phi-4 Microsoft | reasoning-vision 15B (Mar 2026) | MIT | Small and permissive. |
| Granite 4.2 IBM | 3B, 8B, 30B (Aug 2026) | Apache 2.0 | Small and permissive. |
| Nemotron 3.5 NVIDIA | Super VL 120B (Oct 2026) | OpenMDW-1.1 | Commercial use allowed. Training data partly public. |
| Command A+ Cohere | 218B (May 2026) | Card says Apache 2.0 | Confirm the licence file before use. |
| GLM-5.3 Z.ai | 753B (Aug 2026) | Custom | Top open model on Epoch's cyber index. |
| Kimi K3 Moonshot AI | 2.8T total | Custom | Attribution and revenue triggers. |
| Llama 4 Meta | Apr 2025, latest Llama | Llama 4 licence | 700M user cap. Naming and attribution rules. |
| Muse Spark Open Meta | 816B (Oct 2026) | Muse Model License | Free up to $20M revenue. Use restrictions. |
06 / Tuning
The tuning ladder
Start at rung 01. Move down only when a test shows you need to.
| Method | What it does | What it costs or gives | Source |
|---|---|---|---|
| Prompting | Instructions and examples. No weights change. | Cheapest. Easiest to audit. | |
| Retrieval (RAG) | Pulls passages from your documents at answer time. | Keeps sources current and citable. | Lewis et al., 2020 |
| LoRA | Trains small low-rank adapters. The base model stays frozen. | Up to 10,000 times fewer trainable parameters. About 3 times less GPU memory. | Hu et al., 2021 |
| QLoRA | LoRA on a 4-bit compressed base model. | Fine-tuned a 65B model on one 48 GB GPU. | Dettmers et al., 2023 |
| Full fine-tuning | Updates every weight. | Highest cost. Multi-GPU at 70B. | |
| Preference tuning (DPO) | Teaches which of two answers is better. | No separate reward model needed. | Rafailov et al., 2023 |
| Distillation | A large model teaches a smaller one. | Cheaper to run once trained. | Hinton et al., 2015 |
| Quantization | Stores weights in fewer bits (GGUF, MLX). | Smaller and faster. Some quality loss. |
07 / Hardware and tools
What it takes to tune
Adapter tuning fits on one machine. Full fine-tuning at 70B does not.
Minimum GPU memory to tune (Unsloth)
| Model size | QLoRA (4-bit) | LoRA (16-bit) |
|---|---|---|
| 7B | 5 GB | 19 GB |
| 8B | 6 GB | 22 GB |
| 14B | 8.5 GB | 33 GB |
| 32B | 26 GB | 76 GB |
| 70B | 41 GB | 164 GB |
Common local tools
| Tool | Use | Latest (Oct 2026) |
|---|---|---|
| mlx-lm | Run and tune models on Apple silicon. LoRA, QLoRA, DoRA, full fine-tuning. | 0.32.2 |
| Ollama | Run models locally with one command. | 0.40.9 |
| llama.cpp | Fast local inference. Reads GGUF files. | 0.7.0 |
| vLLM | High-throughput serving on GPU servers. | 0.32.0 |
| Hugging Face PEFT | LoRA and other adapter methods. | 0.21.2 |
| Hugging Face TRL | Fine-tuning and preference tuning (DPO). | 1.16.0 |
| Unsloth | Faster, lower-memory fine-tuning. | 0.1.911 beta |
08 / Method
A governed tuning project, step by step
Eight steps. The order is the control.
Write down the one task the model supports, and who is accountable for its output.
Use approved data only. Record its source, licence, and any personal information. Fix the train and test split so it can be reproduced.
Check the licence for revenue, user, and attribution triggers. Download weights in safetensors from the maker's official page.
Measure prompting and retrieval before tuning anything. Many tasks stop here.
Tune with LoRA or QLoRA on your own hardware. Keep the base model unchanged so you can roll back.
Score it on examples it never saw in training. Compare against the cheap rungs and a closed model.
A named person approves the model for that task only. Running locally is not approval on its own.
Re-test on a schedule. Retire the adapter when the task, the data, or a better base model changes.
09 / Regulation
The dates that matter
| Canada: OSFI E-23 | Model risk management for federally regulated financial institutions, effective 1 May 2027. Covers AI and vendor models. No longer applies to federally regulated pension plans. |
|---|---|
| Canada: AI strategy | The June 2026 national strategy proposes targeted laws, not one comprehensive AI act. |
| EU AI Act | General-purpose model duties apply. Systemic risk is presumed above 10^25 training FLOPs, and those models get no open-source exemption. High-risk rules for credit scoring and insurance pricing are deferred to 2 December 2027. |
| United States | SR 26-2 replaces SR 11-7 for banks, but excludes generative and agentic AI. The NIST AI Risk Management Framework is being revised. |
| Standard | ISO/IEC 42001:2023 sets out an AI management system an auditor can certify. |
General information, not legal advice. Confirm how each rule applies to your institution with counsel.
10 / Security
The model file is code
Treat a downloaded model like any other software you install.
| The risk | Older "pickle" model files can run arbitrary code when loaded. Hugging Face says its own scanner is not foolproof. |
|---|---|
| Seen in the wild | JFrog found about 100 malicious models on Hugging Face, roughly 95 percent using pickle. Three scanner bypasses were disclosed in December 2025 (Cloud Security Alliance, May 2026). |
| The fix | Use safetensors files. A 2023 Trail of Bits audit found no critical code-execution flaws in the format. |
Before any model touches firm data
- Download only from the maker's official page. Record the exact file hash.
- Prefer safetensors. Refuse pickle files.
- Log the licence and its triggers (revenue, users, attribution, naming).
- Check for sanctions or procurement orders that cover the maker, such as the US DHS order on DeepSeek (published October 2026).
- Run the model with no network access until it passes your tests.
11 / sh4pe
DAP's local model studio
sh4pe is a local model studio for training, evaluating, and serving private models on institution-controlled hardware.
How a model earns its place
A candidate model is assessed on a defined task before it can take a place in a governed workflow. Local deployment alone does not establish that authority.
| Built for | Teams that handle sensitive institutional knowledge and need explicit data boundaries. |
|---|---|
| How it fits | Model quality and permitted tasks are evaluated separately from the decision to run locally. Deployment scope follows those results. |
| Connected work | Antechamber (qualification architecture). Overture (governed runtime). |
Talk to DAP
Discuss sh4pe, and read the related research, at dap.solutions/solutions/systems/sh4pe
12 / Sources
Where every fact comes from
- OSI Open Source AI Definition 1.0
https://opensource.org/ai/open-source-ai-definition - InfoWorld: OSI unveils Open Source AI Definition 1.0
https://www.infoworld.com/article/3593266/osi-unveils-open-source-ai-definition-1-0.html - Epoch AI: open vs closed ECI gap (May 2026)
https://epoch.ai/data-insights/open-closed-eci-gap - Epoch AI: open models topic page
https://epoch.ai/topics/open-models - Stanford AI Index 2026: technical performance
https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance - OpenAI gpt-oss-120b model card
https://huggingface.co/openai/gpt-oss-120b - Google: Gemma 4 under Apache 2.0
https://opensource.googleblog.com/2026/03/gemma-4-expanding-the-gemmaverse-with-apache-20.html - Qwen3.8-27B model card
https://huggingface.co/Qwen/Qwen3.8-27B - Qwen3.8-2.4T licence
https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/blob/main/LICENSE - DeepSeek V4 Pro model card
https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro - DeepSeek V4.1 Flash model card
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash - Mistral Small 4 model card
https://huggingface.co/mistralai/Mistral-Small-4-119B-2603 - Mistral Medium 3.5 model card
https://huggingface.co/mistralai/Mistral-Medium-3.5-128B - Microsoft Phi-4-reasoning-vision model card
https://huggingface.co/microsoft/Phi-4-reasoning-vision-15B - IBM Granite 4.2 model card
https://huggingface.co/ibm-granite/granite-4.2-8b - NVIDIA Nemotron 3.5 Super VL model card
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 - Cohere Command A+ model card
https://huggingface.co/CohereLabs/command-a-plus-05-2026-bf16 - Z.ai GLM-5.3 licence
https://huggingface.co/zai-org/GLM-5.3/blob/main/LICENSE - Moonshot Kimi K3 licence
https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE - Meta Llama 4 licence
https://dev.meta.ai/llama/llama4/license - The New Stack: Meta Muse Spark Open (Oct 2026)
https://thenewstack.io/meta-muse-spark-open/ - Lewis et al. 2020, Retrieval-Augmented Generation
https://arxiv.org/abs/2005.11401 - Hu et al. 2021, LoRA
https://arxiv.org/abs/2106.09685 - Dettmers et al. 2023, QLoRA
https://arxiv.org/abs/2305.14314 - Rafailov et al. 2023, Direct Preference Optimization
https://arxiv.org/abs/2305.18290 - Hinton et al. 2015, Distilling the Knowledge in a Neural Network
https://arxiv.org/abs/1503.02531 - Hugging Face: GGUF
https://huggingface.co/docs/hub/gguf - mlx-lm LoRA guide
https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LORA.md - Unsloth hardware requirements
https://unsloth.ai/docs/get-started/fine-tuning-for-beginners/unsloth-requirements.md - mlx-lm on PyPI
https://pypi.org/project/mlx-lm/ - Ollama releases
https://github.com/ollama/ollama/releases/latest - llama.cpp releases
https://github.com/ggml-org/llama.cpp/releases/latest - vLLM releases
https://github.com/vllm-project/vllm/releases/latest - PEFT releases
https://github.com/huggingface/peft/releases/latest - TRL releases
https://github.com/huggingface/trl/releases/latest - Unsloth releases
https://github.com/unslothai/unsloth/releases/latest - European Commission: guidelines for GPAI providers
https://digital-strategy.ec.europa.eu/en/policies/guidelines-gpai-providers - EU AI Act Article 53
https://artificialintelligenceact.eu/article/53/ - EU AI Act Article 51
https://artificialintelligenceact.eu/article/51/ - EU AI Act Annex III
https://artificialintelligenceact.eu/annex/3/ - National Law Review: EU Digital Omnibus on AI in force
https://natlawreview.com/article/eu-digital-omnibus-ai-enters-force - OSFI Guideline E-23 (2027)
https://www.osfi-bsif.gc.ca/en/guidance/guidance-library/guideline-e-23-model-risk-management-2027 - Baker McKenzie: Canada's refreshed AI strategy (Jun 2026)
https://www.bakermckenzie.com/en/insight/publications/2026/06/canada-federal-government-releases-refreshed-national-ai-strategy - Federal Reserve SR 26-2
https://www.federalreserve.gov/supervisionreg/srletters/SR2602.htm - Orrick: model risk guidance overhaul (Apr 2026)
https://www.orrick.com/en/Insights/2026/04/Agencies-Overhaul-Model-Risk-Management-Guidance-for-Banks-Heres-What-Changed - NIST AI Risk Management Framework
https://www.nist.gov/itl/ai-risk-management-framework - ISO/IEC 42001:2023
https://webstore.iec.ch/publication/90574 - Hugging Face: pickle security
https://huggingface.co/docs/hub/security-pickle - Hugging Face: safetensors
https://huggingface.co/docs/safetensors/index - Cloud Security Alliance: malicious model repositories (May 2026)
https://labs.cloudsecurityalliance.org/research/csa-research-note-malicious-ai-model-repositories-attack-sur/ - Federal Register: DeepSeek exclusion order (20 Oct 2026)
https://www.federalregister.gov/documents/2026/10/20/2026-21447/notice-of-order-regarding-exclusion-and-removal-of-deepseek-products-from-federal-civilian-executive - DAP: sh4pe
https://dap.solutions/solutions/systems/sh4pe/
Settings checked on 29 October 2026. Platforms change their menus often. Follow the source links before relying on a path. This guide is general information. It is not legal, compliance, or investment advice.