Ray-traced sun shadows is a GPU-memory intensive feature. When there is not enough available GPU memory left, the feature will be disabled automatically. It can be re-enabled after restarting the renderer. However, you should make sure to free up some GPU memory before trying to enable it again. Let’s look at some suggestions you could try apart from upgrading the GPU to a model with more VRAM.
Optimize your scene
The memory requirement depends on the amount of geometry in the scene. You could try to simplify the scene by (temporarily) removing some geometry not required for the export (video/still image) you want to create.
Lower the quality level
Higher quality levels have a greater demand on GPU memory. Depending on your needs, you can reduce the quality level in Enscape. Ray traced sun shadows are available in all quality levels, so it might be an option to switch to a lower quality level to free GPU memory for ray traced sun shadows.
Rendering Quality Slider
Disable animated vegetation
Animated models potentially use a lot more GPU memory than non-animated ones. Switching off wind by reducing its intensity to zero allows Enscape to optimize that and handle animated vegetation the same way as non-animated models and potentially reducing the memory footprint.
Wind Intensity Slider
Lower the resolution of your rendering
Higher resolutions also increase the GPU usage. When the resolution is changed we not only scale the image that we are generating but also a lot of secondary resources stored on the GPU. These resources help with the lighting calculations or transparency, etc.
Resolution Settings
Free up GPU memory
GPU memory is a shared resource in a computer. All running applications share the same memory. If there are multiple applications running that use GPU memory, closing some of them might free up enough GPU memory to use the ray traced sun shadows.
Windows Task Manager Process Tab
For further considerations on performance, refer to this article.