Что лучше cuda или opencl

What are we actually doing on CUDA and OpenCL?

Nowadays, quite often we hear about high-performance GPGPU (general purpose GPU) computations, OpenCL and CUDA. To discuss their pro and cons, it could be a good idea to start from more info about these technologies. Then we’ll take a look at common features and differences, discuss possible applications and future trends.

It’s a widely accepted opinion that GPU is intended to render monitor output at PC and for video games only. The scope of GPU-based applications is much more versatile and wide. In most cases, our daily life is closely connected with GPU computing, though we need some more insight to reveal that. For example, in most cases, fast video encoding, video editing, and rendering are done on GPU at real-time and this is significantly faster than the performance of CPU based solutions.

OpenCL and CUDA

CUDA and OpenCL are software frameworks which allow GPU to perform general purpose computations. It looks quite simple, but it wasn’t like that in the past. The first opportunity to use GPU for tasks other than rendering was the appearance of Shaders. But Shader languages and representations were not convenient for regular use. Historically, GPU was intended for graphics applications only, to ensure monitor output at each PC. They had highly specialized graphics processors which were able to draw polygons very fast. Developers had to program Pixel Shaders, but they weren’t intended for general purpose computations at all. CUDA and OpenCL have given that opportunity to developers.

Hardware for CUDA and OpenCL

Currently, there are three main GPU manufacturers: NVIDIA, AMD, and Intel. NVIDIA has the biggest market share, which is important for technology coverage.

NVIDIA is offering versatile solutions both for rendering and general purpose computations: mobile GPU (Tegra), discrete GPU for a laptop (GeForce GT), desktop (GeForce GTX) and server (Quadro and Tesla). This wide range of hardware can work both with CUDA and OpenCL, though performance for CUDA solutions is higher.

AMD is manufacturing Radeon GPUs for embedded (mobile) solutions, laptop, desktop, and server (Radeon Instinct).

Intel is offering GPUs which are built into modern CPUs. This is not the best choice for general purpose computations, but in most cases, it’s enough to process graphics data at laptops. Apart from GPUs, we can run OpenCL code on CPU and FPGA/ASIC and this is a trend which is vitally important for Intel, who is the main CPU manufacturer with recently acquired Altera, one of the biggest FPGA manufacturers. This is an important trend for OpenCL usage at embedded solutions.

Software for CUDA and OpenCL

NVIDIA puts lots of efforts to market and to develop CUDA framework. There are also significant activities in the creation of educational centers and organization of conferences worldwide. NVIDIA has developed the CUDA Toolkit, implemented MSVS integration, Visual Profiler, NPP, Video SDK, etc. There is a huge number of third-party toolkits and libraries for CUDA and this is a great help for developers. One should note that the latest features of GPU hardware are implemented at CUDA Toolkit very quickly.

In comparison with NVIDIA, activities of AMD in the field looks much more moderate and the results are good, but not so great. CodeXL Toolkit offers developers a full range of tools for OpenCL programming.

Differences between CUDA and OpenCL

To be more precise, CUDA is not a language or an API. CUDA is a platform for parallel computing and at the same time, it’s a programming model to utilize GPU to speed up general purpose computing. The developer still can write software at C or C++, and incorporate some extensions in the form of a few basic keywords. You can’t write your software on C++ at OpenCL, but the main idea is the same, you will be able to work directly with GPU on C language with some extensions.

CUDA is a proprietary technology from NVIDIA, though OpenCL is an open standard. Their approaches are different, but they are actually solving the same problem: how to organize general purpose GPU computations.

OpenCL is a cross-platform solution by definition. You can run OpenCL application almost on any OS, but also on most of the hardware, including CPU, FPGA, and ASIC. You can run CUDA on Windows, Linux, MacOS, but only at NVIDIA hardware.

CUDA vs OpenCL for a developer (Pros and Cons)

Applications for CUDA and OpenCL

This is a brief overview for widespread applications for general purpose computations on GPU. We would like not to perform any comparison here, but to offer a review of possible choices for GPU usage.

To get a better understanding of the real-life usage of GPU-based solutions, you can see the latest software catalog from NVIDIA for 500+ applications with GPU acceleration.

That catalog shows that GPU computing is a must for many well-known software packages in absolutely different fields and applications. Variety of choices is outstanding and this is an excellent proof that GPU can work almost anywhere, and great demand in computational power is universal.

You can also try our software for NVIDIA GPU and CUDA to see image processing benchmarks on GPU, how to accelerate encoding of your JPEG or JPEG2000 images, how to process CinemaDNG on GPU, etc.

Future Trends

Latest GPUs from NVIDIA and AMD are offering just incredible performance benchmarks. That computational power is a powerful driver for further progress in data science and many other applications. Currently, we clearly see great interest in Machine Learning, Deep Learning, AI, Convolutional Neural Networks, etc. Science and industry make use of modern GPUs and we could expect absolutely stunning results like auto-driving cars, a breakthrough in AI and medicine. OpenCL usage at embedded solutions is also a very promising trend as well.

To summarize, we do need GPU-based computations in various applications, both on CUDA and OpenCL. These technologies are evolving rapidly and current progress is really impressive.

CUDA vs OpenCL

CUDA serves as a platform for parallel computing, as well as a programming model.

CUDA was developed by NVIDIA for general-purpose computing on NVIDIA’s graphics processing unit (GPU) hardware.

With CUDA programming, developers can use the power of GPUs to parallelize calculations and speed up processing-intensive applications.

For GPU-accelerated applications, the sequential parts of the workload run single-threaded on the machine’s CPU, and the compute-intensive parts run in parallel on thousands of GPU cores.

Developers can use CUDA to write programs in popular languages (C, C++, Fortran, Python, MATLAB, etc.) and add parallelism to their code with a few basic keywords.

Related content: read our guide to NVIDIA CUDA

What is OpenCL?

Open Computing Language (OpenCL) serves as an independent, open standard for cross-platform parallel programming.

OpenCL is used to accelerate supercomputers, cloud servers, PCs, mobile devices, and embedded platforms.

OpenCL has dramatically improved the speed and flexibility of applications in various market categories, including professional development tools, scientific and medical software, imaging, education, and deep learning.

OpenCL uses a programming language similar to C. It provides an API that enables programs running on a host to load the OpenCL kernel on computing devices. You can also use the API to manage device memory separately from host memory.

OpenCL programs are designed to be compiled at run time, so applications that use OpenCL can be ported between different host devices.

OpenCL is not just for GPUs (like CUDA) but also for CPUs, FPGAs… In addition, OpenCL was developed by multiple companies, as opposed to NVIDIA’s CUDA.

In this article, you will learn:

CUDA vs OpenCL: What’s the Difference?

Hardware

There are three major manufacturers of graphic accelerators: NVIDIA, AMD and Intel.

NVIDIA currently dominates the market, holding the largest share. NVIDIA provides comprehensive computing and processing solutions for mobile graphics processors (Tegra), laptop GPUs (GeForce GT), desktops GPUs (GeForce GTX), and GPU servers (Quadro and Tesla).

This wide range of NVIDIA hardware can be used both with CUDA and OpenCL, but the performance of CUDA on NVIDIA is higher, because it was designed with NVIDIA hardware in mind.

Related content: read our in-depth guide about CUDA on NVIDIA

AMD creates Radeon GPUs for embedded solutions and mobile systems, laptops and desktops, and Radeon Instinct GPUs for servers. OpenCL is the primary language used to run graphics processing on AMD GPUs.

Intel offers GPUs integrated into its CPUs. OpenCL can run on these GPUs, but while sufficient for laptops, it does not perform competitive performance for general-purpose computations.

Besides GPU, you can run OpenCL code on CPU and FPGA / ASIC. This is a major trend when using OpenCL in integrated solutions.

Operating Systems

CUDA can run on Windows, Linux, and MacOS, but only on NVIDIA hardware.

OpenCL applications can run on almost any operating system, and on most types of hardware, including FPGAs and ASICs.

Software and Community

NVIDIA is committed to the commercialization and development of the CUDA platform. NVIDIA developed tools including the CUDA Toolkit, NVIDIA Performance Primitives (NPP), and Video SDK, and Visual Profiler, and built integrations with Microsoft Visual Studio and other popular platforms. CUDA has a broad ecosystem of third-party tools and libraries. The latest NVIDIA hardware features are quickly supported in the CUDA Toolkit.

AMD’s community activity is more limited. AMD built the CodeXL Toolkit, which provides a full range of OpenCL programming tools.

Programming Model

CUDA is not a language or an API. It is a platform and programming model for parallel computing, and it accelerates general-purpose computing using GPUs. Developers can still write software in C or C++ and include parallelization by using CUDA keywords.

OpenCL does not enable writing code in C++, but you can work in an environment resembling the C programming language, and work directly with GPU resources.

CUDA Advantages and Limitations

Advantages

There are several advantages that give CUDA an edge over traditional general purpose graphics processor (GPGPU) computers with graphics APIs:

• Unified memory (in CUDA 6.0 or later) and unified virtual memory (in CUDA 4.0 or later)
• Shared memory—provides a faster area of shared memory for CUDA threads. It can be used as a caching mechanism, and provides more bandwidth than texture lookups.
• Scattered reads: code can be read from any address in memory.
• Improved performance on downloads and reads, which works well from the GPU and to the GPU
• There is full support for bitwise and integer operations

Limitations

• CUDA source code is provided on host machines or GPU, as defined by C++ syntax rules. Older versions of CUDA used C syntax rules, meaning that updated CUDA source code may or may not work as expected.
• CUDA has one-way interoperability with rendering languages like OpenGL. OpenGL can access the CUDA registered memory, but CUDA cannot access OpenGL memory.
• Later versions of CUDA do not provide emulators or fallback support for older versions.
• CUDA only supports NVIDIA hardware.

OpenCL Advantages and Limitations

Advantages

• OpenCL provides abstract memory and portability, due to its run-time execution model.
• The OpenCL kernel can run on any supported software implementation.
• OpenCL supports a heterogeneous system architecture that enables efficient communication between the GPU and the processor using C++ 17 atomics.

Limitations

• Developers cannot directly implement proprietary hardware technologies like inline Parallel Thread Execution (PTX) on NVIDIA GPUs without sacrificing portability.
• A study that directly compared CUDA programs with OpenCL on NVIDIA GPUs showed that CUDA was 30% faster than OpenCL.
• OpenCL is rarely used for machine learning. As a result, the community is small, with few libraries and tutorials available.

Running CUDA and OpenCL at Scale with Run:AI

Run:AI automates resource management and orchestration for machine learning infrastructure. With Run:AI, you can automatically run as many compute intensive experiments as needed, incorporating CUDA and/or OpenCL.

Here are some of the capabilities you gain when using Run:AI:

• Advanced visibility—create an efficient pipeline of resource sharing by pooling GPU compute resources.
• No more bottlenecks—you can set up guaranteed quotas of GPU resources, to avoid bottlenecks and optimize billing.
• A higher level of control—Run:AI enables you to dynamically change resource allocation, ensuring each job gets the resources it needs at any given time.

Run:AI simplifies machine learning infrastructure pipelines, helping data scientists accelerate their productivity and the quality of their models.

CUDA vs OpenCL: Which should I use?

If you are looking to get into GPU programming, you are currently faced with an annoying choice:

• Should I base my work upon OpenCL or CUDA? I maintain two packages for accelerated computing in Python, PyCuda and PyOpenCL, so obviously I can’t decide either. Still, this is a common question, so this page compiles a number of facts to help you decide. Since the question is broad and difficult as it stands, this page will focus on the Python angle when there is any benefit in doing so.

Facts

Vendors

As of right now, there is one vendor of CUDA implementations, namely Nvidia Corporation.

The following vendors have OpenCL implementations available:

OpenCL 2.0 SDK for SSE3-supporting CPUs (Intel and AMD chips are supported) and AMD GPUs
• All Radeon 5xxx, 6xxx, 7xxx series, R9xxx series are supported
• All FirePro W-series (for workstation, active cooling) and FirePro S-series (for servers, passive heatsink)
• All boards with GCN 1.0 or newer are OpenCL 2.1 ready, pending AMD OpenCL 2.1 SDK availability
• All CPUs support 1.2 only
• Note: Drivers from 280.x onward self-report as supporting OpenCL 1.2. As of September 2016, there is still no timeframe for OpenCL 2.x support (MacOS X only)
• supports NVIDIA GeForce 8600M GT, GeForce 8800 GT, GeForce 8800 GTS, GeForce 9400M, GeForce 9600M GT, GeForce GT 120, GeForce GT 130, ATI Radeon 4850, Radeon 4870, likely more.
• supports host CPUs as compute devices . CPU, GPU, and «MIC» (Xeon Phi). . Deprecated since end 2012 — use POCL instead. . FPGAs, Stratix 5. You need to load binaries as compiling takes a long time. . FPGAs, Ultrascale and Virtex-7.

Portable OpenCL is an open-source, LLVM-based CL implementation for CPUs. The following groups are or may be producing CL implementations:

The open-source community (Mesa) is on it, OpenCL is already part of the mesa library, and the code is apparently already able to do some rudimentary things.

Other Free components

, a library of CL C built-in functions for LLVM

Code Portability

• While OpenCL can natively talk to a large range of devices, that doesn’t mean that your code will run optimally on all of them without any effort on your part. In fact, there’s no guarantee it will even run at all, given that different CL devices have very different feature sets. If you stick to the OpenCL spec and avoid vendor-specific extensions, your code should be portable, if not tuned for speed. For now, it is safe to assume that you are facing efforts on the scale of a rewrite for your kernel code when switching devices for nontrivial codes. Fortunately though, the host-side code stays the same across devices.

Capabilities

• OpenCL does not appear to support pinned host memory. This may cause a penalty of about a factor of two in host-device transfer rates.
• Note: Well it looks like pinned host memory exists in OpenCL, with the flag CL_MEM_ALLOC_HOST_PTR (see 3.1 in the NVIDIA OpenCL Guide) — JulianBilcke
  • Oh, funny. By its original purpose, CL_MEM_ALLOC_HOST_PTR allocates device memory that is mapped into the host address space (or the other way around?). Pinned host memory doesn’t necessarily have a device mapping. But quoting from the guide:
    • OpenCL applications do not have direct control over whether memory objects are allocated in pinned memory or not, but they can create objects using the CL_MEM_ALLOC_HOST_PTR flag and such objects are likely to be allocated in pinned memory by the driver for best performance. —AndreasKloeckner
    • Note, the design of using CL_MEM_ALLOC_HOST_PTR while allocating a buffer puts restrictions on the amount of memory can actually be pinned. For example, the NVidia OpenCL implementation seems to limit this to CL_DEVICE_MAX_MEM_ALLOC_SIZE. While CUDA lets you pin as much physical memory as you can get ahold of. —BrianCole
      • That’s arguably a bug in the Nvidia implementation, and should probably be reported. — AndreasKloeckner 2013-03-17 18:49:24

      Speed

      • If you’re addressing the same hardware, both frameworks should be able to achieve the same speeds. With the current beta drivers, this may not be the case, but any advantage should level out quickly. Some early implementations of special functions aside, this has been found to be the case.

      Maintenance

      • It is not likely that either OpenCL or CUDA will disappear in short order, given existing commitments. and PyOpenCL will be maintained for the foreseeable future.

      An Attempt at a Conclusion

      (Careful: While the above collection is supposed to consist of objective facts, this section is for personal opinion. Feel free to add yours.)

      Personally, I would like to see OpenCL succeed. It has the right ingredients as a standard—mainly run-time code generation and reasonable support of heterogeneous computing. On top of that, being in a multi-vendor marketplace is a good thing—also for Nvidia, although they might not immediately see it that way.

      If I was starting something new, I would likely go with OpenCL, unless I desperately needed one of the proprietary CUDA libraries. —AndreasKloeckner

      If you are on Mac OS X get started with PyOpencl because installing the CUDA Framework is painful right now (summer, 2010). OpenCL comes bundled with your OS and supports more cards so starting is a snap. I agree with Andreas that learning about GPU programming is similar for both frameworks. OpenGL interoperability helped me also since I knew some stuff about OpenGL. Holger

      Adding my own opinion here. I am a Game Designer from RIT. I have been using OpenCL for the last 2 months or so, and feel that I have a basic understanding of it, if not a moderate view. My boss told me to look into the development environment for CUDA, due to the fact that OpenCL is SOOOO hard to debug and get working properly. The errors sometimes do not even report the actual problem (i.e. «Out of resources exception» != «Out of bounds exception»).

      (To be fair, the same is true for CUDA—the corresponding error is «invalid context», and you’ll have to learn that means you caused something like a segfault. Also, nowadays, AMD offers CPU CL debugging using gdb and on-GPU debugging through gDEBUGGER, so this may no longer be true.) — AndreasKloeckner 2012-07-23 04:13:02 That being said, I also have to have a separate program to debug syntax in OpenCL. CUDA can be used straight through Visual Studio, and it has intellisense. CUDA can also use variables straight out of code, due to it being code. OpenCL is parsed as a string. The CUDA environment is much more user friendly. OpenCL has more «customizable» options for it, but this just leads to code refactoring between machines. CUDA seems to be able to port much more consistently, and its easier to work with Development Environments with CUDA. Overall, I have done OpenCL for 2 months, CUDA for 2 days, and I have had more success with CUDA.

      Сравнение OpenCL с CUDA, GLSL и OpenMP

      
      На хабре уже рассказали о том, что такое OpenCL и для чего он нужен, но этот стандарт сравнительно новый, поэтому интересно как соотносится производительность программ на нём с другими решениями.

      В этом топике приведено сравнение OpenCL с CUDA и шейдерами для GPU, а также с OpenMP для CPU.

      Тестирование проводилось на задаче N-тел. Она хорошо ложится на параллельную архитектуру, сложность задачи растёт как O(N 2 ), где N — число тел.

      Задача

      В качестве тестовой была выбрана задача симуляции эволюции системы частиц.
      На скриншотах (они кликабельны) видна задача N точечных зарядов в статическом магнитном поле. По вычислительной сложности она ничем не отличается от классической задачи N тел (разве что картинки не такие красивые).

      Во время проведения замеров вывод на экран был отключен, а FPS означает число итераций в секунду (каждая итерация — это следующий шаг в эволюции системы).

      Результаты

      Код на GLSL и CUDA для этой задачи был уже написан сотрудниками ННГУ.

      NVidia Quadro FX5600

      Версия драйвера 197.45
      

      CUDA обгоняет OpenCL приблизительно на 13%. При этом, если оценивать теоретически возможную производительность для этой задачи для данной архитектуры, реализация на CUDA достигает её.
      (В работе A Performance Comparison of CUDA and OpenCL говорится о том, что производительность ядра OpenCL проигрывает CUDA от 13% до 63% )
      Несмотря на то, что тесты проводились на карточке серии Quadro, понятно, что обычный GeForce 8800 GTS или GeForce 250 GTS дадут схожие результаты (все три карточки основаны на чипе G92).

      Radeon HD4890

      ATI Stream SDK версия 2.01
      

      OpenCL проигрывает шейдерам на карточках от AMD так как вычислительный блоки на них имеют архитектуру VLIW, на которую (после оптимизации) могут хорошо лечь многие шейдерные программы, но компилятор для кода OpenCL (который является частью драйвера) плохо справляется с оптимизацией.
      Также этот весьма скромный результат может быть вызван тем, что карточки от AMD не поддерживают локальную память на физическом уровне, а отображают область локальной памяти на глобальную.

      Код с использованием OpenMP был скомпилирован при помощи компиляторов от Intel и Microsoft.
      Компания Intel не выпустила своих драйверов для запуска кода OpenCL на центральном процессоре, поэтому был использован ATI Stream SDK.

      Intel Core2Duo E8200

      ATI Stream SDK версия 2.01
      

      Код на OpenMP, скомпилированный при помощи MS VC++ имеет практически идентичную производительность с OpenCL.
      Это ещё при том, что Intel не выпустил своего драйвера для интерпретации OpenCL, и используется драйвер от AMD.

      Компилятор от Intel поступил не совсем «честно» он полностью развернул основной цикл программы, повторив его где-то 8k раз (число частиц было задано константой в коде) и получив семикратный прирост производительности также благодаря использованию SSE инструкций. Но победителей, конечно, не судят.

      Что характерно, на моём стареньком AMD Athlon 3800+ код тоже запустился, но таких выдающихся результатов, как на Intel, конечно, ждать не приходится.