Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 3-3
Download and Run Additional Models Locally Since you already downloaded models locally in Phase 4,...
Read More
Tag: LLM Inference
Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 2-3
Configuring vLLM for Production Deployment (More Complex) For production use, I want to run the...
Read More
Install vLLM on Gigabyte AI TOP ATOM: High-Performance LLM Inference with OpenAI-Compatible API – Part 1-3
After showing you how to install Ollama, Open WebUI, ComfyUI, and LLaMA Factory on the Gigabyte AI...
Read More
- Install LibreChat on Gigabyte AI TOP ATOM: Professional Chat Interface for Local LLM Models
After showing in my previous posts how to install Ollama, Open WebUI, ComfyUI, LLaMA Factory, vLLM, and LM Studio on the Gigabyte AI TOP ATOM, here comes another interesting alternative for everyone looking for a professional chat interface with advanced features like RAG (Retrieval-Augmented Generation), multi-user support, and a plugin system: LibreChat is an open-source… Read more: Install LibreChat on Gigabyte AI TOP ATOM: Professional Chat Interface for Local LLM Models - Installing LM Studio on Gigabyte AI TOP ATOM: User-friendly GUI with OpenAI-compatible API for local LLMs
After showing in my previous posts how to install Ollama, Open WebUI, LLaMA Factory, vLLM, ComfyUI, and the AI Toolkit on the Gigabyte AI TOP ATOM, here is another interesting alternative for everyone looking for a user-friendly GUI interface for local Large Language Models and who specifically does not want to use Ollama: LM Studio… Read more: Installing LM Studio on Gigabyte AI TOP ATOM: User-friendly GUI with OpenAI-compatible API for local LLMs - Installing RAGFlow on Ubuntu Server: Setting up a RAG system with two NVIDIA RTX A6000 GPUs
Anyone working with Large Language Models who also needs to analyze large volumes of documents knows the problem: simple chat interfaces are not enough when it comes to extracting and understanding specific information from PDFs, Word documents, or other file formats. For me, the solution was clear: I use RAGFlow on my X86 Ubuntu Server… Read more: Installing RAGFlow on Ubuntu Server: Setting up a RAG system with two NVIDIA RTX A6000 GPUs - Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 2-2
Phase 8: Compiling and Starting the Web UI Now I can compile and start the web UI. First, I switch to the UI directory of the AI Toolkit: Command: cd ui If you haven’t cloned the AI Toolkit repository yet, you need to do that first. The UI directory should be located in the root… Read more: Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 2-2 - Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 1-2
After showing in my previous posts how to install Ollama, Open WebUI, LLaMA Factory, vLLM, and ComfyUI on the Gigabyte AI TOP ATOM, here is something for everyone who wants to train their own Stable Diffusion AI models: AI Toolkit by Ostris – a comprehensive solution for training Stable Diffusion models with a user-friendly web… Read more: Install AI Toolkit on Gigabyte AI TOP ATOM – Step-by-Step Guide – Part 1-2
The tutorial offers a clear and practical guide for setting up and running the Tensorflow Object Detection Training Suite. Could…
This works using an very old laptop with old GPU >>> print(torch.cuda.is_available()) True >>> print(torch.version.cuda) 12.6 >>> print(torch.cuda.device_count()) 1 >>>…
Hello Valentin, I will not share anything related to my work on detecting mines or UXO's. Best regards, Maker
Hello, We are a group of students at ESILV working on a project that aim to prove the availability of…