Revision of Qwen38-MTP-NInfer-Install.md

ZZ revisou este gist 1787164268. Ir para a revisão

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -57,7 +57,7 @@ time cmake --build build-sm89 --config Release --target ninfer ninfer-serve ninf
57	57
58	58		#### Run NInfer Server
59	59		```sh
60		-	time ./build-sm89/apps/Release/ninfer-serve.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 4 --lm-head-draft --max-context 220000 --preserve-thinking --host 0.0.0.0 -p 8080
	60	+	time ./build-sm89/apps/Release/ninfer-serve.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 4 --lm-head-draft --max-context 220000 --preserve-thinking --host 0.0.0.0 --port 8080
61	61		```
62	62
63	63		Open port 8080 for LAN access to ninfer-serve.exe

ZZ revisou este gist 1787145746. Ir para a revisão

Sem alterações

ZZ revisou este gist 1787145693. Ir para a revisão

Sem alterações

ZZ revisou este gist 1787122729. Ir para a revisão

1 file changed, 2 insertions, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -99,8 +99,9 @@ time aider $aider_files \
99	99		}
100	100		# ---
101	101		```
102		-	```sh
	102	+
103	103		NInfer.exe
	104	+	```sh
104	105		./build-sm89/apps/Release/ninfer.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" \
105	106		--prompt "Explain prefill and decode in three sentences." \
106	107		--max-context 16384 \

ZZ revisou este gist 1787122710. Ir para a revisão

1 file changed, 10 insertions, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -99,4 +99,13 @@ time aider $aider_files \
99	99		}
100	100		# ---
101	101		```
102		-
	102	+	```sh
	103	+	NInfer.exe
	104	+	./build-sm89/apps/Release/ninfer.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" \
	105	+	--prompt "Explain prefill and decode in three sentences." \
	106	+	--max-context 16384 \
	107	+	--max-new 256 \
	108	+	--spec mtp --draft-tokens 3 \
	109	+	--lm-head-draft
	110	+
	111	+	```

ZZ revisou este gist 1787121883. Ir para a revisão

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -92,7 +92,7 @@ time aider $aider_files \
92	92		--no-show-model-warnings \
93	93		--model "openai/$ai_model" \
94	94		--openai-api-base "$ollama_api_base_aider" \
95		-	--openai-api-key "ollama" \
	95	+	--openai-api-key "$OPENAI_API_KEY" \
96	96		--dark-mode --code-theme solarized-dark \
97	97		--notifications --notifications-command "ssh ai minecraft.sound" --watch-files
98	98

ZZ revisou este gist 1787121264. Ir para a revisão

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -1,5 +1,5 @@
1	1
2		-	## Install NInfer Server for Fast Qwen3.8-27B inference
	2	+	## Install NInfer Server for fast Qwen3.8-27B inference
3	3
4	4		NInfer installs are tied to a specific GPU.
5	5		- 5090 - Neroued/ninfer - High-performance single-GPU inference for selected model checkpoints and GPUs. - https://github.com/Neroued/ninfer#build

ZZ revisou este gist 1787092753. Ir para a revisão

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -13,7 +13,7 @@ NInfer installs are tied to a specific GPU.
13	13		- Compile and install curl from vcpkg
14	14		- Install FFMPEG from Chocolatey
15	15		- Configure Ninfer
16		-	- Build Ninfer
	16	+	- Build NInfer
17	17		- Run `ninfer-serve.exe`
18	18		- Query `ninfer-serve.exe`
19	19

ZZ revisou este gist 1787091186. Ir para a revisão

1 file changed, 6 insertions, 6 deletions

Qwen38-MTP-Ninfer-Install.md

			@@ -1,13 +1,13 @@
1	1
2		-	## Install Ninfer Server for Fast Qwen3.8-27B inference
	2	+	## Install NInfer Server for Fast Qwen3.8-27B inference
3	3
4		-	Ninfer installs are tied to a specific GPU.
	4	+	NInfer installs are tied to a specific GPU.
5	5		- 5090 - Neroued/ninfer - High-performance single-GPU inference for selected model checkpoints and GPUs. - https://github.com/Neroued/ninfer#build
6	6		- 4090 - UDPSendToFailed/ninfer-4090 - NInfer port for Qwen 3.8 27B on RTX 4090 - https://github.com/UDPSendToFailed/ninfer-4090/tree/feat/rtx-4090-sm89-native
7	7		- 4090 Linux Docker - sergiuszm/ninfer-4090 - Qwen3.8-27B on one RTX 4090 - full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots - https://github.com/sergiuszm/ninfer-4090#quick-start-linux- 3090 -
8	8		- 3090 - Don-Chad/ninfer-3090 - Fast Qwen3.8-27B inference on one RTX 3090 - ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux builds. - https://github.com/Don-Chad/ninfer-3090
9	9
10		-	### Ninfer install
	10	+	### NInfer install
11	11		- Setup file structure
12	12		- Compile and install vcpkg
13	13		- Compile and install curl from vcpkg
			@@ -50,12 +50,12 @@ time cmake -B build-sm89 -G "Visual Studio 17 2022" -A x64 \
50	50		-DCOMPILE_DEFINITIONS="CURL_STATICLIB"
51	51		```
52	52
53		-	Build Ninfer ~20mins
	53	+	Build NInfer ~20mins
54	54		```sh
55	55		time cmake --build build-sm89 --config Release --target ninfer ninfer-serve ninfer_bench --parallel 32
56	56		```
57	57
58		-	#### Run Ninfer Server
	58	+	#### Run NInfer Server
59	59		```sh
60	60		time ./build-sm89/apps/Release/ninfer-serve.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 4 --lm-head-draft --max-context 220000 --preserve-thinking --host 0.0.0.0 -p 8080
61	61		```
			@@ -65,7 +65,7 @@ Open port 8080 for LAN access to ninfer-serve.exe
65	65		New-NetFirewallRule -DisplayName "NInfer Server (8080)" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8080
66	66		```
67	67
68		-	### Query Ninfer Server
	68	+	### Query NInfer Server
69	69
70	70		Curl
71	71		```sh

ZZ revisou este gist 1787080979. Ir para a revisão

1 file changed, 2 insertions, 2 deletions

Qwen38-MTP-Ninfer-Install.md

			@@ -92,8 +92,8 @@ time aider $aider_files \
92	92		--no-show-model-warnings \
93	93		--model "openai/$ai_model" \
94	94		--openai-api-base "$ollama_api_base_aider" \
95		-	-openai-api-key "ollama" \
96		-	--dark-mode --code-theme solarized-dark \
	95	+	--openai-api-key "ollama" \
	96	+	--dark-mode --code-theme solarized-dark \
97	97		--notifications --notifications-command "ssh ai minecraft.sound" --watch-files
98	98
99	99		}