{"id":948,"date":"2023-08-01T15:22:52","date_gmt":"2023-08-01T15:22:52","guid":{"rendered":"https:\/\/ai-box.eu\/?p=948"},"modified":"2023-08-08T11:31:39","modified_gmt":"2023-08-08T11:31:39","slug":"oobabooga-text-generation-webui-mit-llama-2-local-installation","status":"publish","type":"post","link":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/","title":{"rendered":"oobabooga &#8211; text-generation-webui mit Llama 2 &#8211; local Installation"},"content":{"rendered":"<p>I always wanted to try a Llama model of META and with the release of Llama 2 it was then so far that I went on the search how I can set up this on my computer locally. There are several guides but the best framework I could find was oobabooga which is also opensource. After I decided on the framework I had to decide which Llama 2 model to test. I decided to use the model Llama-2-13B-Cha, because it fits into the memory of my NVIDIA A6000 graphics card. I signed up with META to get access to the current Llama 2 models. Because that way I could then download them from Hugging Face. In the end I chose the model TheBloke_Llama-2-13B-Chat-fp16 which I could easily download via the web interface of the oobabooga framework.<\/p>\n<p>But now enough introduction and let&#8217;s start with the description of the installation of the necessary components.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Installation_oobabooga\" >Installation oobabooga<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Create_Anaconda_environment\" >Create Anaconda environment<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Install_Pytorch\" >Install Pytorch<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Download_oobabooga_framework\" >Download oobabooga framework<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#bitsandbytes_installation\" >bitsandbytes installation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Starting_the_oobabooga_web_interface\" >Starting the oobabooga web interface<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Download_LLM_Models\" >Download LLM Models<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Starting_the_web_interface_%E2%80%93_extended_call\" >Starting the web interface &#8211; extended call<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Video\" >Video<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Update_the_installation\" >Update the installation<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#Summary\" >Summary<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Installation_oobabooga\"><\/span>Installation oobabooga<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I have installed the oobabooga framework on an Ubuntu system and use as written in the beginning a NVIDIA A6000 graphics card in which I can load the Llama 2 model.<\/p>\n<p>Here is the GitHub page that leads you to the framework.<\/p>\n<p style=\"padding-left: 40px;\"><strong>GitHub:<\/strong> <a href=\"https:\/\/github.com\/oobabooga\/text-generation-webui\" target=\"_blank\" rel=\"noopener\">https:\/\/github.com\/oobabooga\/text-generation-webui<\/a><\/p>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"Create_Anaconda_environment\"><\/span>Create Anaconda environment<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The installation of the oobabooga framework is done in a separate Anaconda environment. So we can separate this installation and the necessary packages from the other installations. Now please execute the two following commands.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>conda create -n Llama_2_textgen python=3.10.9<\/code><\/p>\n<p>Now the Anaconda environment with the name Llama_2_textgen is created and can be started as shown below.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>conda activate Llama_2_textgen<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Hint:<\/strong> If Anaconda is not yet installed on your system you can read <a href=\"https:\/\/ai-box.eu\/stable_diffusion\/stable-diffusion-automatic1111-ubuntu-installation-teil-1-2\/546\/\" target=\"_blank\" rel=\"noopener\">here<\/a> how to install Anaconda under Ubuntu.<\/p>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"Install_Pytorch\"><\/span>Install Pytorch<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>For everything to run smoothly you need to install torch and a few packages. You do this with the following command.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip3 install torch torchvision torchaudio<\/code><\/p>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"Download_oobabooga_framework\"><\/span>Download oobabooga framework<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>git clone https:\/\/github.com\/oobabooga\/text-generation-webui<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>cd text-generation-webui<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong>\u00a0<code>pip install -r requirements.txt<\/code><\/p>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"bitsandbytes_installation\"><\/span>bitsandbytes installation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install bitsandbytes==0.38.1<\/code><\/p>\n<p>I also had to install the following additional modules in my CONDA environment.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install idna<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install certifi<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install pytz<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install six<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install pyyaml<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install pyparsing<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command<\/strong><\/p>\n<p>After that, I was actually able to launch the application so the web UI.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Starting_the_oobabooga_web_interface\"><\/span>Starting the oobabooga web interface<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>conda activate Llama_2_textgen<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>cd text-generation-webui<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command: <\/strong><code>python server.py<\/code><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Download_LLM_Models\"><\/span>Download LLM Models<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>On the oobabooga web interface, in the Model tab, enter the name on the right in the Download custom model or LoRA item and tap download. Now the corresponding model of Hugging Face will be downloaded and set up. The following model TheBloke\/Llama-2-13B-Chat-fp16 I could load locally with my NVIDIA A6000 without any problems.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Model Name: <\/strong><code>TheBloke\/Llama-2-13B-Chat-fp16<\/code><\/p>\n<div id=\"attachment_945\" style=\"width: 1034px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-1024x424.jpg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-945\" class=\"wp-image-945 size-large\" src=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-1024x424.jpg\" alt=\"oobabooga model download\" width=\"1024\" height=\"424\" srcset=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-1024x424.jpg 1024w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-300x124.jpg 300w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-768x318.jpg 768w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download-1080x447.jpg 1080w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_model_download.jpg 1395w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><p id=\"caption-attachment-945\" class=\"wp-caption-text\">oobabooga model download<\/p><\/div>\n<p>The model occupies about 29 GB of memory on the graphics card. But that probably also means that the 30B model, when it is available, probably can&#8217;t be loaded without further ado.<\/p>\n<p>If you want to calculate which model fits maximum in the memory of your graphics card then I would like to explain below how this calculation works on the example of a 70B model.<\/p>\n<p style=\"padding-left: 40px;\">Number of parameters = 70.000.000.000<\/p>\n<p style=\"padding-left: 40px;\">32 bit of information = 32 bit<\/p>\n<p style=\"padding-left: 40px;\">1 byte = 8 bit<\/p>\n<p style=\"padding-left: 40px;\">Calculation: (70.000.000 * 32) \/ 8 = 280.000.000 byte = 280 GB<\/p>\n<p>With this we would need a video RAM of a size of 280GB GPU RAM to load the 70B model.<\/p>\n<p>If you don&#8217;t want to do the calculation yourself, you can use this parameter calculator to do it.<\/p>\n<p style=\"padding-left: 40px;\"><strong>URL:<\/strong> <a href=\"https:\/\/kdcreer.com\/parameter_calculator\" target=\"_blank\" rel=\"noopener\">https:\/\/kdcreer.com\/parameter_calculator<\/a><\/p>\n<p>I tried to load the 70B model partly into the video RAM of my graphics card and partly onto a fast M2 SSD but the response time of the network was unbearably slow for a human.<\/p>\n<p>The significantly large model with TheBloke\/Llama-2-70B-Chat-fp16 did not fit into the memory of my NVIDIA A6000 which is simply too small with 48GB RAM.<\/p>\n<p><strong>Model name: <\/strong><code>TheBloke\/Llama-2-70B-Chat-fp16 <\/code><\/p>\n<h3><span class=\"ez-toc-section\" id=\"Starting_the_web_interface_%E2%80%93_extended_call\"><\/span>Starting the web interface &#8211; extended call<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>If you want oobabooga to start automatically and start with some preset parameters like the model to load then the command looks like this. How to start the whole thing as a cron job I have explained on my blog here a few times and here you can find an explanation that you can use from the principle.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>python server.py --chat --multi-user --share --model TheBloke_Llama-2-13B-Chat-fp16 --gradio-auth &lt;User-Name&gt;:&lt;Password&gt;<\/code><\/p>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"Video\"><\/span>Video<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>I also recorded a small video that shows how fast the model responds on my computer. I found it very impressive and was thrilled.<\/p>\n<div id=\"attachment_942\" style=\"width: 1034px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/youtu.be\/7FHSNaQkEHY\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-942\" class=\"wp-image-942 size-large\" src=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-1024x576.jpg\" alt=\"oobabooga Video llama 2\" width=\"1024\" height=\"576\" srcset=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-1024x576.jpg 1024w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-300x169.jpg 300w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-768x432.jpg 768w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-1080x607.jpg 1080w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2-1280x721.jpg 1280w, https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg 1282w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/a><p id=\"caption-attachment-942\" class=\"wp-caption-text\">oobabooga Video llama 2<\/p><\/div>\n<h3 dir=\"auto\" tabindex=\"-1\"><span class=\"ez-toc-section\" id=\"Update_the_installation\"><\/span>Update the installation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>If you want to update your installation of oobabooga you have to execute the following commands.<\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>conda activate Llama_2_textgen<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong>\u00a0<code>cd text-generation-webui<\/code><\/p>\n<p style=\"padding-left: 40px;\"><strong>Command:<\/strong> <code>pip install -r requirements.txt --upgrade<\/code><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I thought that it is more difficult to get Llama 2 running locally on my computer. But the oobabooga framework does a lot of work for me. I had some difficulties to get the framework installed. But in the end it all worked out and now it runs just great and I can test the model a bit and think about what I could use it for. On the whole I am very satisfied and will go deeper into the topic of large language models.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I always wanted to try a Llama model of META and with the release of Llama 2 it was then so far that I went on the search how I can set up this on my computer locally. There are several guides but the best framework I could find was oobabooga which is also opensource. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":943,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[162,51,50],"tags":[107,88,89,109],"class_list":["post-948","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-large-language-models-en","category-software-en","category-top-story-en","tag-anleitung-en","tag-howto-en-2","tag-installation-en","tag-lokal-en","et-has-post-format-content","et_post_format-et-post-format-standard"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box\" \/>\n<meta property=\"og:description\" content=\"I always wanted to try a Llama model of META and with the release of Llama 2 it was then so far that I went on the search how I can set up this on my computer locally. There are several guides but the best framework I could find was oobabooga which is also opensource. [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/\" \/>\n<meta property=\"og:site_name\" content=\"Exploring the Future: Inside the AI Box\" \/>\n<meta property=\"article:published_time\" content=\"2023-08-01T15:22:52+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-08-08T11:31:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1282\" \/>\n\t<meta property=\"og:image:height\" content=\"721\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Maker\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Ingmar_Stapel\" \/>\n<meta name=\"twitter:site\" content=\"@Ingmar_Stapel\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Maker\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/\"},\"author\":{\"name\":\"Maker\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/#\\\/schema\\\/person\\\/cc91d08618b3feeef6926591b465eab1\"},\"headline\":\"oobabooga &#8211; text-generation-webui mit Llama 2 &#8211; local Installation\",\"datePublished\":\"2023-08-01T15:22:52+00:00\",\"dateModified\":\"2023-08-08T11:31:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/\"},\"wordCount\":833,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ai-box.eu\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/oobabooga_video_llama_2.jpg\",\"keywords\":[\"Anleitung\",\"HowTo\",\"Installation\",\"lokal\"],\"articleSection\":[\"Large Language Models\",\"Software\",\"Top story\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/\",\"url\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/\",\"name\":\"oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ai-box.eu\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/oobabooga_video_llama_2.jpg\",\"datePublished\":\"2023-08-01T15:22:52+00:00\",\"dateModified\":\"2023-08-08T11:31:39+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/#\\\/schema\\\/person\\\/cc91d08618b3feeef6926591b465eab1\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ai-box.eu\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/oobabooga_video_llama_2.jpg\",\"contentUrl\":\"https:\\\/\\\/ai-box.eu\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/oobabooga_video_llama_2.jpg\",\"width\":1282,\"height\":721,\"caption\":\"oobabooga Video llama 2\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/software-en\\\/oobabooga-text-generation-webui-mit-llama-2-local-installation\\\/948\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Start\",\"item\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"oobabooga &#8211; text-generation-webui mit Llama 2 &#8211; local Installation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/\",\"name\":\"Exploring the Future: Inside the AI Box\",\"description\":\"Inside the AI Box, we share our experiences and discoveries in the world of artificial intelligence.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/#\\\/schema\\\/person\\\/cc91d08618b3feeef6926591b465eab1\",\"name\":\"Maker\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g\",\"caption\":\"Maker\"},\"description\":\"I live in Bavaria near Munich. In my head I always have many topics and try out especially in the field of Internet new media much in my spare time. I write on the blog because it makes me fun to report about the things that inspire me. I am happy about every comment, about suggestion and very about questions.\",\"sameAs\":[\"https:\\\/\\\/ai-box.eu\"],\"url\":\"https:\\\/\\\/ai-box.eu\\\/en\\\/author\\\/ingmars\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/","og_locale":"en_US","og_type":"article","og_title":"oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box","og_description":"I always wanted to try a Llama model of META and with the release of Llama 2 it was then so far that I went on the search how I can set up this on my computer locally. There are several guides but the best framework I could find was oobabooga which is also opensource. [&hellip;]","og_url":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/","og_site_name":"Exploring the Future: Inside the AI Box","article_published_time":"2023-08-01T15:22:52+00:00","article_modified_time":"2023-08-08T11:31:39+00:00","og_image":[{"width":1282,"height":721,"url":"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg","type":"image\/jpeg"}],"author":"Maker","twitter_card":"summary_large_image","twitter_creator":"@Ingmar_Stapel","twitter_site":"@Ingmar_Stapel","twitter_misc":{"Written by":"Maker","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#article","isPartOf":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/"},"author":{"name":"Maker","@id":"https:\/\/ai-box.eu\/en\/#\/schema\/person\/cc91d08618b3feeef6926591b465eab1"},"headline":"oobabooga &#8211; text-generation-webui mit Llama 2 &#8211; local Installation","datePublished":"2023-08-01T15:22:52+00:00","dateModified":"2023-08-08T11:31:39+00:00","mainEntityOfPage":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/"},"wordCount":833,"commentCount":0,"image":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#primaryimage"},"thumbnailUrl":"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg","keywords":["Anleitung","HowTo","Installation","lokal"],"articleSection":["Large Language Models","Software","Top story"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/","url":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/","name":"oobabooga - text-generation-webui mit Llama 2 - local Installation - Exploring the Future: Inside the AI Box","isPartOf":{"@id":"https:\/\/ai-box.eu\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#primaryimage"},"image":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#primaryimage"},"thumbnailUrl":"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg","datePublished":"2023-08-01T15:22:52+00:00","dateModified":"2023-08-08T11:31:39+00:00","author":{"@id":"https:\/\/ai-box.eu\/en\/#\/schema\/person\/cc91d08618b3feeef6926591b465eab1"},"breadcrumb":{"@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#primaryimage","url":"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg","contentUrl":"https:\/\/ai-box.eu\/wp-content\/uploads\/2023\/08\/oobabooga_video_llama_2.jpg","width":1282,"height":721,"caption":"oobabooga Video llama 2"},{"@type":"BreadcrumbList","@id":"https:\/\/ai-box.eu\/en\/software-en\/oobabooga-text-generation-webui-mit-llama-2-local-installation\/948\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Start","item":"https:\/\/ai-box.eu\/en\/"},{"@type":"ListItem","position":2,"name":"oobabooga &#8211; text-generation-webui mit Llama 2 &#8211; local Installation"}]},{"@type":"WebSite","@id":"https:\/\/ai-box.eu\/en\/#website","url":"https:\/\/ai-box.eu\/en\/","name":"Exploring the Future: Inside the AI Box","description":"Inside the AI Box, we share our experiences and discoveries in the world of artificial intelligence.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/ai-box.eu\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/ai-box.eu\/en\/#\/schema\/person\/cc91d08618b3feeef6926591b465eab1","name":"Maker","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e96b93fc3c7e50c1f21c5c6b1f146dc4867936141360830b328947b32cacf93a?s=96&d=mm&r=g","caption":"Maker"},"description":"I live in Bavaria near Munich. In my head I always have many topics and try out especially in the field of Internet new media much in my spare time. I write on the blog because it makes me fun to report about the things that inspire me. I am happy about every comment, about suggestion and very about questions.","sameAs":["https:\/\/ai-box.eu"],"url":"https:\/\/ai-box.eu\/en\/author\/ingmars\/"}]}},"_links":{"self":[{"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/posts\/948","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/comments?post=948"}],"version-history":[{"count":4,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/posts\/948\/revisions"}],"predecessor-version":[{"id":969,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/posts\/948\/revisions\/969"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/media\/943"}],"wp:attachment":[{"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/media?parent=948"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/categories?post=948"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ai-box.eu\/en\/wp-json\/wp\/v2\/tags?post=948"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}