ProgrammingwithOpenCL1.2ByGastnHillar,August26,2014

Post a Comment

printfstyledebuggingandtheabilitytopartitioncomputingdevicesintosubdevicesmakeOpenCL1.2averyusefulupgrade.

OpenCL 1.2 is the latest stable OpenCL release with available drivers for both CPU and GPU support.It is an interesting and useful enhancement to OpenCL 1.1 that adds several useful features someof which are available extensions to earlier OpenCL versions. In this article, I cover the addition ofthe simplest debug tool ever, which makes it easier to debug OpenCL kernels, and the new "devicefission," which enables you to partition a single compute device into many subdevices.

The Simplest Debug Tool: printf

OpenCL 1.2 includes a built-in printf function to the OpenCL C programming language. In previousOpenCL versions, only a few specific vendors provided an equivalent function through an extension,such as the cl_intel_printf extension found in the Intel drivers. The built-inprintf function isvery similar to the printf defined in the C99 standard, but there are differences that you shouldreview by checking the documentation. For example, OpenCL Cprintf returns 0 for a successfulexecution and -1 otherwise, while C99 printf returns the number of printed characters for asuccessful execution. At the time of writing, it was possible to use printf with OpenCL 1.2 driversfor CPU targets and also for AMD GPUs.

The printf function buffers output until the kernel execution completes, then transfers the outputback to the host. Thus, you have to be extremely careful to send text to the standard output whenadding calls to printf in your kernels. As you might guess, the call to this function within amassively parallel execution has undesirable side effects, including a big impact on performance andmemory usage. You should use printf only for specific debugging purposes on reduced data setsand then remove calls to this function when you want to execute the kernels on the entire data set.

The following lines show the code for a very simple OpenCL kernel that computes the product of amatrix and a vector:

You can add the following line after the assignment to result[gid] to write data about thegenerated result index and value to the standard output:

This way, you can easily gather data from the kernel with a function that is familiar.

Working with Device Fission

One of the classic problems that arises when you target CPUs with OpenCL is that the execution of akernel uses all the available cores and doesn't leave any core free to execute other processes.The cl_ext_device_fission extension provides an interface for subdividing an OpenCL device intomultiple subdevices. OpenCL 1.2 has incorporated device fission in the specification, and all thefunctions that assigned work to devices now accept subdevices.

With the use of device fission, you can control which of the available compute units you want theOpenCL runtime to use in order to execute kernels. For example, if you have an eight-core CPU, you

123456

__kernelvoidmatrix_dot_vector(__globalconstfloat4*matrix,__globalconstfloat4*vector,__globalfloat*result){intgid=get_global_id(0);result[gid]=dot(matrix[gid],vector[0]);}

1 printf("result[%d]%f\n",gid,result[i]);

?

?

can use device fission to create a subdevice with six cores and leave two cores for the operatingsystem tasks and other processes that require CPU usage. As a developer, device fission is extremelyuseful because you can continue working with your IDE and other development tools while executingOpenCL kernels that target the CPU.

You can also create sets composed of one or more subdevices with their own command queue. Withthis feature, you can use device fission to control the queues and dispatch work to each set. Thefeature is useful when you have algorithms that benefit from distributing work among many sets ofsubdevices. You have a lot of different possibilities for partitioning tasks and work with advancedtask parallelism scenarios.

Obviously, the use of device fission requires that you have a good understanding of the details of theunderlying hardware. When you create subdevices and dispatch work to them, you must considermany things that might affect performance (if you're writing OpenCL kernels, it means you want toachieve the best performance). If you don't take into account the shared resources for differentsubdevices, such as a shared cache memory or Non-Uniform Memory Access (NUMA) nodes, you willlose performance. Luckily, device fission in OpenCL 1.2 provides many predefined partitioning typesand options that make it easy to specify the ways in which you want to split a device, and many ofthem allow you to consider the affinity of the compute units to share levels of cache hierarchy or aNUMA node. These predefined partitioning types and options allow you to make an efficient use ofshared hardware resources when generating subdevices.

Device fission allows you to partition subdevices. Thus, once you create subdevices, they can befurthered partitioned by creating new subdevices. The relationships of the different subdevices forma tree in which the subdevices have a parent device or subdevice. It is possible to use differentpartition types and options each time you request OpenCL to split the device or subdevices. Forexample, you can partition the CPU devices by affinity, then partition one of those subdevicesequally into eight subdevices. Obviously, the root device doesn't have a parent.

When you work with device fission for CPUs, you need to take into account that each compute unitis equivalent to a logical core or a hardware thread. So, when you work with Intel CPUs with Hyper-Threading technology enabled, two logical cores or hardware threads share one physical core.

The clGetDeviceInfo function has new associated device property IDs to retrieve information thatallows you to plan the partitioning scheme to create subdevices. Before creating subdevices, youcan retrieve the following information using the specified device property IDs:

The maximum number of subdevices that you can create for adevice:CL_DEVICE_PARTITION_MAX_SUB_DEVICES. For example, if a CPU has eight logical cores,the value will be 8.

The partition types that the device supports: CL_DEVICE_PARTITION_PROPERTIES. I'll dive deepon partition types later.

The affinity domains for partitioning the device that the devicesupports:CL_DEVICE_PARTITION_AFFINITY_DOMAIN. When you specify a partitioning by affinitydomain, you can use any of the affinity domains included in the returned list of supported values.

You can determine whether both the OpenCL implementation and the device support device fissionby checking the maximum number of subdevices that you can create for a device(CL_DEVICE_PARTITION_MAX_SUB_DEVICES). Once you are sure that you can create subdevices, youcan check the supported partition types and the affinity domains in cases where you want to workwith partitioning by affinity domain. With all this information, you can write code that can take intoaccount different hardware architectures and use different partitioning schemes based on theinformation gathered from the different clGetDeviceInfocalls.

After you make the call clGetDeviceIDs and the necessary calls to clGetDeviceInfo for theselected device, you can start creating the subdevices with calls to thenewclCreateSubDevices function. Note that you must create the subdevices before you create theOpenCL context. OpenCL 1.2 devices and subdevices have retain (clRetainDevice) and release

(clReleaseDevice) functions that allow you to increment and decrement the reference count as isdone on other OpenCL objects.

The following lines show the C declaration of the clCreateSubDevices function:

The function requires the following arguments:

in_device: Indicates the ID of the device (cl_device_id) that you want to split intosubdevices.properties: Provides a property list that starts with the desired partition type and then

provides additional values required by the selected partition type. The last value of the propertylist that indicates the end of the properties list mustbeCL_DEVICE_PARTITION_BY_COUNTS_LIST_END (0). As I explained before, in order to have asuccessful subdevices creation, you need to make sure that the partition type specified in thisproperty list is supported by the device by making calls to clGetDeviceInfo.num_devices: Specifies the size of the out_devices array.out_devices: Provides a buffer for the generated subdevices with a number of elements

specified by num_devices.num_devices_ret: Returns the number of subdevices that the device may be partitioned into

considering the partition type and the other values specified in the property list (properties).

12345

cl_intclCreateSubDevices(cl_device_idin_device,constcl_device_partition_property*properties,cl_uintnum_devices,cl_device_idout_devices,cl_uint*num_devices_ret);

?