When using ollama or lmstudio, does it automatically use zenbook GPU?

Leon Chase

• 3 min read

When using tools like Ollama or LM Studio to run large language models (LLMs) locally, whether your ASUS ZenBook's GPU (in this case, AMD Radeon Graphics) is utilized depends on several factors. Let’s break it down:

1. Does Ollama Use the GPU?

General Behavior

  • Ollama is designed to simplify the process of running LLMs locally. By default, it uses the CPU for inference unless explicitly configured to use the GPU.
  • If you want Ollama to leverage your GPU, you need to ensure that:
    • Your GPU is supported by the underlying framework (e.g., PyTorch, TensorFlow).
    • The necessary GPU drivers and libraries (e.g., ROCm for AMD GPUs) are installed and properly configured.

AMD GPU Support

  • Ollama relies on frameworks like PyTorch or other backends to handle GPU acceleration. For AMD GPUs, support typically comes through ROCm (Radeon Open Compute), which is AMD's open software platform for GPU computing.
  • However, ROCm support is not as widespread or mature as NVIDIA's CUDA ecosystem, so GPU acceleration with AMD GPUs can be more challenging to set up.
  • As of now, Ollama primarily focuses on NVIDIA GPUs (via CUDA), and official support for AMD GPUs may be limited or require additional configuration.

Steps to Check GPU Usage

  1. Run Ollama and monitor system resources using a tool like Task Manager (Windows) or htop/nvidia-smi (Linux).
  2. If you see high CPU usage but no GPU activity, Ollama is likely running on the CPU.

2. Does LM Studio Use the GPU?

General Behavior

  • LM Studio is another popular tool for running LLMs locally. It provides better GPU support compared to Ollama, especially for NVIDIA GPUs (via CUDA).
  • For AMD GPUs, LM Studio also relies on ROCm or similar frameworks for GPU acceleration. However, AMD GPU support is less common and may require manual setup.

AMD GPU Support

  • LM Studio may work with AMD GPUs if:
    • You have ROCm installed and configured.
    • The specific LLM you're running supports AMD GPU acceleration.
  • Unfortunately, many precompiled models and frameworks are optimized for NVIDIA GPUs, so AMD users may face compatibility issues.

Steps to Check GPU Usage

  1. Open LM Studio and check the settings to see if GPU acceleration is enabled.
  2. Monitor GPU usage using tools like GPU-Z (Windows) or rocm-smi (Linux).
  3. If GPU usage remains at 0%, LM Studio is likely running on the CPU.

3. Why Might the GPU Not Be Used?

Even if you have an AMD Radeon GPU, there are several reasons why it might not be used by Ollama or LM Studio:

a. Lack of Native Support

  • Many LLM frameworks (like PyTorch and TensorFlow) prioritize NVIDIA GPUs due to CUDA's widespread adoption. AMD GPU support via ROCm is still catching up.

b. Missing Drivers or Libraries

  • To use your GPU, you need to install the correct drivers and libraries:
    • For AMD GPUs: Install the latest AMD drivers and ROCm (if supported on your system).
    • Ensure the version of ROCm matches your GPU model and operating system.

c. Model Compatibility

  • Some LLMs are precompiled for specific hardware (e.g., NVIDIA GPUs). If the model isn't compatible with AMD GPUs, it will default to CPU execution.

d. Software Limitations

  • Tools like Ollama and LM Studio may not fully support AMD GPUs out of the box. NVIDIA GPUs are often the primary focus for GPU acceleration.

4. How to Enable GPU Acceleration (If Possible)

For Ollama

  1. Check Documentation: Review Ollama's official documentation to see if AMD GPU support is mentioned.
  2. Install ROCm: If supported, install ROCm on your system:
  3. Test GPU Usage: Run Ollama and monitor GPU activity to confirm whether it’s being utilized.

For LM Studio

  1. Enable GPU in Settings:
    • Open LM Studio and navigate to the settings menu.
    • Look for an option to enable GPU acceleration.
  2. Install ROCm: Follow the same steps as above to install ROCm.
  3. Test with Compatible Models: Try running models that explicitly support AMD GPUs.

5. Alternative Solutions

If you find that your AMD GPU isn’t being utilized effectively, here are some alternatives:

a. Use the CPU

  • Both Ollama and LM Studio can run models on the CPU, though performance will be slower compared to GPU acceleration.
  • For smaller models (e.g., 7B or 13B parameter models), CPU execution may still be feasible.

b. Switch to NVIDIA GPU

  • If GPU acceleration is critical for your workflow, consider using a system with an NVIDIA GPU. NVIDIA’s CUDA ecosystem is well-supported by most LLM frameworks.

c. Use Cloud-Based Solutions

  • If local GPU acceleration is not an option, consider using cloud-based services like:
    • Google Colab
    • Hugging Face Spaces
    • RunPod
      These platforms provide access to powerful GPUs without requiring local hardware.

6. Conclusion

By default, tools like Ollama and LM Studio are more likely to use the CPU rather than your AMD Radeon GPU unless you explicitly configure them for GPU acceleration. While AMD GPUs can theoretically be used via ROCm, support is limited, and setup can be complex. If GPU acceleration is essential for your workflow, you may need to explore alternative solutions, such as switching to an NVIDIA GPU or using cloud-based platforms.

If you’re determined to use your AMD GPU, start by installing ROCm and checking the documentation for Ollama or LM Studio to see if AMD GPU support is available.