Qwen38-MTP-NInfer-Install.md düzenlemesi

Değişiklik yok

Değişiklik yok

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -57,7 +57,7 @@ time cmake --build build-sm89 --config Release --target ninfer ninfer-serve ninf
57	57
58	58		#### Run NInfer Server
59	59		```sh
60		-	time ./build-sm89/apps/Release/ninfer-serve.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 4 --lm-head-draft --max-context 220000 --preserve-thinking --host 0.0.0.0 -p 8080
	60	+	time ./build-sm89/apps/Release/ninfer-serve.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" --kv-dtype rk4v4-e8 --spec mtp --draft-tokens 4 --lm-head-draft --max-context 220000 --preserve-thinking --host 0.0.0.0 --port 8080
61	61		```
62	62
63	63		Open port 8080 for LAN access to ninfer-serve.exe

Değişiklik yok

Değişiklik yok

1 file changed, 2 insertions, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -99,8 +99,9 @@ time aider $aider_files \
99	99		}
100	100		# ---
101	101		```
102		-	```sh
	102	+
103	103		NInfer.exe
	104	+	```sh
104	105		./build-sm89/apps/Release/ninfer.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" \
105	106		--prompt "Explain prefill and decode in three sentences." \
106	107		--max-context 16384 \

1 file changed, 10 insertions, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -99,4 +99,13 @@ time aider $aider_files \
99	99		}
100	100		# ---
101	101		```
102		-
	102	+	```sh
	103	+	NInfer.exe
	104	+	./build-sm89/apps/Release/ninfer.exe "H:/AI/Code/Model/qwen3_8_27b.ninfer" \
	105	+	--prompt "Explain prefill and decode in three sentences." \
	106	+	--max-context 16384 \
	107	+	--max-new 256 \
	108	+	--spec mtp --draft-tokens 3 \
	109	+	--lm-head-draft
	110	+
	111	+	```

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -92,7 +92,7 @@ time aider $aider_files \
92	92		--no-show-model-warnings \
93	93		--model "openai/$ai_model" \
94	94		--openai-api-base "$ollama_api_base_aider" \
95		-	--openai-api-key "ollama" \
	95	+	--openai-api-key "$OPENAI_API_KEY" \
96	96		--dark-mode --code-theme solarized-dark \
97	97		--notifications --notifications-command "ssh ai minecraft.sound" --watch-files
98	98

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -1,5 +1,5 @@
1	1
2		-	## Install NInfer Server for Fast Qwen3.8-27B inference
	2	+	## Install NInfer Server for fast Qwen3.8-27B inference
3	3
4	4		NInfer installs are tied to a specific GPU.
5	5		- 5090 - Neroued/ninfer - High-performance single-GPU inference for selected model checkpoints and GPUs. - https://github.com/Neroued/ninfer#build

1 file changed, 1 insertion, 1 deletion

Qwen38-MTP-Ninfer-Install.md

			@@ -13,7 +13,7 @@ NInfer installs are tied to a specific GPU.
13	13		- Compile and install curl from vcpkg
14	14		- Install FFMPEG from Chocolatey
15	15		- Configure Ninfer
16		-	- Build Ninfer
	16	+	- Build NInfer
17	17		- Run `ninfer-serve.exe`
18	18		- Query `ninfer-serve.exe`
19	19