AI Pipeline - Hardware Example Configurations - Exploring the Future: Inside the AI Box

With the first part around the topic hardware now follows the second part which presents concrete solutions. Here I go on the one hand on ready built systems but also on my private system that I have built from scratch. The question of what I need at all for which task has already been discussed extensively and therefore it now continues with concrete solutions from which can be selected for a suitable or this article is to be understood so that one is given ideas for systems to hand.

Table of Contents

Machine Learning Laptop PC

If you want to get started with machine learning and are currently thinking about buying a laptop for your studies or work, you might like the following recommendation.

If it is to be a laptop, then it must be clear that the training of neural networks can quickly take several days. Therefore, a laptop is certainly not ideal, but very interesting for an introduction to the topic of AI. If the laptop works under full load for several days, then the waste heat must also be removed from the device. Therefore, it is recommended to buy at least a 17″ device so that the packing density of the individual components is not so high and there is enough space for heat sinks, heat pipes and fans.

Since the training of neural networks is not very exciting in terms of graphics, you can save money on the display. Here, a distinction can be made between gamer displays with a high refresh rate beyond 200 Hz and the group of displays with a high resolution. Generally, a simple HD resolution would suffice in this laptop with a normal refresh rate of 100Hz. The money saved in this way can then be spent on hardware for the CPU or just for the graphics card.

If Deep Learning or Machine Learning applications are to be executed on the laptop, one should do without the mobile version of the GPU if possible and buy a device with a full desktop GPU. However, this bears the risk that the computer will become even louder when it is under full load and has to be cooled. However, a mobile version of a GPU is still faster than a typical mobile version of a CPU. This statement may not be true for the current M1 MAX processors, which I have not been able to test myself.

A laptop that would meet these requirements would be, for example, the following model from the company MIFCOM, which can also be customized according to customer requirements.

Workstation Laptop R9 5900HX – RTX 3070 (17.3″)

Hint: This laptop model has not yet been tested by me and was only suggested here as a starting model for an individual configuration based on its performance data from the configurator.

URL to the configurator: https://www.mifcom.de/workstation-laptop-r9-5900hx-rtx-3070-17-3-id15460?configurator

Deep Learning Workstation PC

Für alle die nicht nur einen Computer für das Gaming von anspruchsvollen Speilen anschaffen möchten sondern auch den Einstieg in das Thema künstliche Intelligenz wagen möchten ist ein Desktop PC zu empfehlen. Der Vorteil eines Desktop Rechner ist die Möglichkeit zu haben viel Rechenleistung bei geringer Geräuschentwicklung in einen Workstation Rechner zu verbauen was bei einem Laptop so schwierig ist aufgrund der kleinen schnell drehenden Lüfter.

Hint Cooling: A water-cooled CPU and ideally also a water-cooled GPU are highly recommended to keep the noise as low as possible. Personally, I only water cooled the CPU in my custom workstation. Converting the GPU, in my case an NVIDIA A6000 privately to water cooling was technically too risky for me. When my workstation is running under full load for a few days, the office is actually only usable with hearing protection according to DIN EN 352-1:2002 and light clothing for this time. Thanks to the home office that the colleagues in online meetings in this case have only asked whether you currently have his workstation in production because of the noise and switched off camera …

One of the great advantages of a deep learning workstation computer is that, depending on the budget, components can always be replaced or expanded, which is only possible to a limited extent in the case of a laptop.

Following the recommendations mentioned at the beginning of this chapter, a configuration like the following would be recommended for an ambitious start. Possibly a large SSD hard drive would make sense for the administration and storage of the training data. But this could always be added later when the acute need arises. Depending on the computer’s location, whether office, home office, possibly a separate room, and the available budget, the use of water cooling for the CPU and GPU should be considered.

Hint noise emission: Privately for my own workstation and in the business environment, the case Define 7 XL from Fractal Design was often installed. Privately, I appreciate the case very much because it offers a lot of volume in the case itself for cooling and thanks to the mats for sound insulation is hardly noticeable when the GPU is not running under load. It is also very expandable and many large, slowly rotating and therefore quiet fans can be installed.

The following computer from the company MIFCOM would meet the requirements of an entry-level model in the field of deep learning or artificial intelligence, except for the water cooling.

Workstation Ryzen 9 5950X – RTX 3080 Dual

Hint: This workstation model has not yet been tested and has only been suggested here as a starting model for an individual configuration based on its performance data.

URL to the configurator: https://www.mifcom.de/workstation-ryzen-9-5950x-rtx-3080-dual-id14873?configurator

Self-build suggestion Deep Learning Workstation PC

The own construction of a computer is aimed at private individuals who trust themselves to build such a computer. Since I did not know at that time where I could buy a computer especially for Deep Learning and how professional such a computer would be, I decided to build it myself.

Hint: Personally, I would no longer build a computer myself because I simply lack the experience and practice alone in the subject of cabling as this is done once properly and airflow optimized.

The following picture shows a professionally built workstation where the cables have been neatly routed inside the case.

MIFCOM 8TB storage capacity production

Nevertheless, I would like to list the components I have installed here for all those who would like to build a Deep Learning system themselves. So you get an impression what you need and how my own computer is technically configured.

When thinking about the hardware I decided to at least use a water cooled CPU. So I have a very quiet computer when the graphics card is not training a neural network.

All other components like the fans that were already installed in the Fractal Design case run very slowly and therefore quietly.

Component overview:

The following components were purchased in the spring of 2021 and the system was expanded throughout 2021 with additional Samsung 980 Pro drives, for example.

Anzahl:	Beschreibung:
1	AMD Ryzen™ 9 3950X
1	GIGABYTE X570 AORUS ULTRA
1	iCUE H150i ELITE CAPELLIX CPU-Liquid cooler- Corsair
2	Crucial DIMM 32 GB DDR4-4400 Kit
1	NVIDIA A6000 Graphics card
1	GIGABYTE P1000GM PSU GA-P1000GM
3	SAMSUNG 980 PRO 2 TB
1	ARCTIC thermally conductive pasteMX-4 4G ACTCP00002B
2	Kingston A400 480 GB, SSD – Operating system drives
1	Fractal Design – Delfine XL – case
2	SATA cable

With these components I have personally built my own computer with which I am still very satisfied.

The following picture shows the inside of my self-built Deep Learning system with water-cooled CPU and NVIDIA A6000 graphics card.

No Code AI Pipeline Deep Learning Computer setup

The biggest drawback for me is the noise development of the NVIDIA A6000, where a water-cooled version would be much more pleasant. The water cooling should always be considered if the GPU is not only driven to its limit every now and then and the computer is operated in the vicinity of people.

Since a lot of air is moved in such a computer, I would like to point out that the Fractal Design case already has dust filters installed. These also regularly fill up with a mat of fine dust. I can’t imagine what would have happened if all that dust had found its way through the GPU fan.

Hint hardware prices: The prices that are currently called for hardware make a self-build increasingly unattractive. Here, please calculate exactly, compare and estimate what you technically trust yourself and what a finished system costs.

Purchase proposal smaller companies and research

The setup suggested here has not yet been tested 1:1 by me personally, but it corresponds approximately to the configuration that I could already use extensively at work. Very pleasant here is the built-in water cooling that cools all components ideally when the computer is running under full load and the noise is kept within limits. Thus, it is possible to operate the computer under the desk and still work directly next to the computer with these models from MIFCOM. A prerequisite, however, is an air conditioning system in the office or a shady and generally cool office.

With the following suggestion, it should be considered whether the CPU and memory could be saved depending on the use of the system. However, depending on the number of users and the way the system is used, the CPU may well be under load and the memory used. This is because even if fewer neural networks are trained over time, they are used by a large number of users. Then the CPU and the memory are definitely under load and in use.

Hint: This workstation model has not yet been tested by me in this configuration and has only been suggested here as a starting model for an individual configuration based on its performance data and experience.

URL to the configurator: https://www.mifcom.de/boss-ryzen-tr-pro-3975wx-rtx-3090-dual-id15278?configurator