The download is free, but we offer some paid services. Support us by subscribing
Delete ads and speed up browsing the library.
The download starts with the click of a button without waiting for the book to be ready.
No limits for download times.
You can upload unlimited books in the library.
Enable readers to download your books without waiting.
Delete ads on the books that you publish.
No problems with download links for your uploaded books.
| Author: | ENG Ahmed Saeed |
| Category: | Artificial Intelligence And Machine Learning [Edit] |
| Language: | English |
| Pages: | 42 |
| File Size: | 1.01 MB |
| Extension: | |
| Creation Date: | 03 Nov 2026 |
| Rank: | 898,216 No 1 most popular |
| Short link: | Copy |
| More books like this book | |
A complete, reproducible inference benchmark of Qwen3.8-27B (Unsloth Dynamic NVFP4) served with vLLM 0.33.0 on a single node with four NVIDIA GeForce RTX 5090 GPUs. The report compares five serving configurations behind one OpenAI-compatible API: single GPU (TP=1), tensor parallel across 2 and 4 GPUs (TP=2, TP=4), data parallel replicas (DP=4), and a hybrid DP=2 × TP=2 layout.
Measured results cover single-user decode speed, short-context throughput sweeps from 1 to 256 concurrent requests, 128K-context concurrency limits, oversubscription and queueing behaviour, time-to-first-token on 128K prompts, maximum bootable context (up to the model’s native 262,144 tokens), needle-in-a-haystack retrieval at ~120K tokens, and a GSM8K accuracy sanity check. A dedicated section analyses multi-token-prediction (MTP) speculative decoding with a depth sweep (k=0 to k=4) and an MTP on/off comparison, showing where speculative decoding helps latency and where it costs throughput.
Key findings include a 3,167 tok/s peak throughput with DP=2 × TP=2, 186.7 tok/s single-user decode with TP=4 (245 tok/s greedy), 17 simultaneous 128K requests on TP=4 with zero preemptions, and the impact of missing GPU peer-to-peer on PCIe-only RTX 5090 cards. Every launch command, server flag, KV-cache size, GPU telemetry sample (utilisation, VRAM, power, temperature), and workaround for FlashInfer FP4 autotune and OOM issues is documented, so the results can be reproduced or adapted to other Blackwell consumer-GPU deployments.
Ideal for ML engineers, MLOps teams, and self-hosters sizing local LLM inference clusters, choosing between tensor and data parallelism, or planning long-context agent workloads on consumer GPUs.
Be the first one to Rate, Review and Quote from the book
Be the first one to Rate, Review and Quote from the book
E-books are complementary and supportive of paper books and never cancel it. With the click of a button, the e-book reaches anyone, anywhere in the world.
E-books may weaken your eyesight due to the glare of the screen. Support the book publisher by purchasing his original paper book. If you can access it and get it, do not hesitate to buy it.
Publish your book now for free
We require cookies for this site to function. Please enable them to continue.
نحن نظهر لك هذه الرسالة لأننا نحترم خصوصيتك.
By using this website, you consent to us collecting cookies to provide you with a better user experience,
more details.
You cannot browse the site since you refused the use of cookies, as the site relies primarily on them to work.
Intellectual property is reserved for the authors mentioned on the books and the library is not responsible for the ideas of the authors
Old and forgotten books that have become past to preserve Arab and Islamic heritage are published,
and books that their authors are accepted to published.
The Universal Declaration of Human Rights states: "Everyone has the right freely to participate in the cultural life of the community, to enjoy the arts and to share in scientific advancement and its benefits.Everyone has the right to the protection of the moral and material interests resulting from any scientific, literary or artistic production of which he is the author".