Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 2-2
Phase 8: Compiling and Starting the Web UI Now I can compile and start the web UI. First, I switch...
Read More
Author: Maker
Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 1-2
After showing in my previous posts how to install Ollama, Open WebUI, LLaMA Factory, vLLM, and...
Read More
Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 3-3
Download and Run Additional Models Locally Since you already downloaded models locally in Phase 4,...
Read More
Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 2-3
Configuring vLLM for Production Deployment (More Complex) For production use, I want to run the...
Read More
Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 1-3
After showing you how to install Ollama, Open WebUI, ComfyUI, and LLaMA Factory on the Gigabyte AI...
Read More- NeMo Agent Toolkit – Multi-Agent Supervisor-Pattern local
After my last post, where I dissected the ReAct loop in detail and built the first custom GPU status tool, the next logical step follows: multi-agent orchestration with the supervisor pattern. Several specialized ReAct agents, each with its own toolset and its own identity, coordinated by an overarching supervisor agent. This is the point where… Read more: NeMo Agent Toolkit – Multi-Agent Supervisor-Pattern local - NeMo Agent Toolkit – GenAI Agent Orchestrierung lokal
Agent-Orchestrierung ist konzeptionell genau der Sprung, der aus „LLM-Inferenz” tatsächlich „intelligente Anwendungen” macht. Ein funktionierendes NAT-Setup mit Ollama habe ich mir wie in meinem Blogbeitrag hier “NeMo Agent Toolkit auf der RTX A6000 Ada – vom Inferenz-Layer zum Orchestrator-Layer” beschrieben bereits aufgesetzt. Jetzt möchte ich schrittweise von der Basis bis zu komplexen Multi-Agent-Mustern das Vorgehen… Read more: NeMo Agent Toolkit – GenAI Agent Orchestrierung lokal - NeMo Agent Toolkit on the RTX A6000 Ada – From Inference Layer to Orchestrator Layer
In my four-part TensorRT-LLM series I showed how I optimize inference performance on the RTX A6000 Ada — 251 tokens/sec with Qwen-2.5-7B in FP8, deployable .engine files, all cleanly reproducible. But in doing so, I had only built one part of the stack: the inference layer. Inspired by the ever-present GenAI agent publications, it became… Read more: NeMo Agent Toolkit on the RTX A6000 Ada – From Inference Layer to Orchestrator Layer - Adding a new board to ESP-Claw: Board adaptation for the Guition JC1060P470
After I showed you in Part 2 of my ESP-Claw series how I compiled ESP-Claw for the first time, today we move on to what is probably the trickiest part of the whole project: How do I get ESP-Claw to recognize my Guition JC1060P470 as a supported board? Anyone who read Part 2 knows: my… Read more: Adding a new board to ESP-Claw: Board adaptation for the Guition JC1060P470 - Preparing an Ubuntu 24.04 Server for AI Inference: CUDA, Docker, NVIDIA Container Toolkit
Whether I later want to run TensorRT-LLM, Ollama, vLLM, or any other container-based inference framework on my server, the base installation is always the same: an up-to-date Ubuntu, the matching NVIDIA driver, Docker, and the NVIDIA Container Toolkit so that containers can even access the GPU in the first place. In this post I’ll show… Read more: Preparing an Ubuntu 24.04 Server for AI Inference: CUDA, Docker, NVIDIA Container Toolkit
The tutorial offers a clear and practical guide for setting up and running the Tensorflow Object Detection Training Suite. Could…
This works using an very old laptop with old GPU >>> print(torch.cuda.is_available()) True >>> print(torch.version.cuda) 12.6 >>> print(torch.cuda.device_count()) 1 >>>…
Hello Valentin, I will not share anything related to my work on detecting mines or UXO's. Best regards, Maker
Hello, We are a group of students at ESILV working on a project that aim to prove the availability of…