Performance Optimizations ======================= Tracked optimizations applied to the library. Each entry records what was changed, why, the measured impact, and any portability considerations. The baseline for all measurements is the state before any optimization was applied (pipelines, shader modules, descriptor set layouts, and pipeline layouts created and destroyed on every operation call). ---- Optimization 1: Filter Pipeline Caching --------------------------------------- **Date:** 2026-07-26 **Status:** Applied **What changed:** Before this change, ``vc_filter`` created and destroyed all Vulkan pipeline state on every invocation: three shader modules (``vkCreateShaderModule`` / ``vkDestroyShaderModule``), three compute pipelines (``vkCreateComputePipelines`` / ``vkDestroyPipeline``), two descriptor set layouts, two pipeline layouts. This is a driver-level compilation step with non-trivial CPU cost. After: pipeline state is created once on the first ``vc_filter`` call and cached in a static ``VcFilterPipelineCache`` struct. Subsequent calls reuse the cached state. **Vulkan primitives affected:** - ``vkCreateShaderModule`` — 3 calls per invocation → 3 calls total - ``vkCreateComputePipelines`` — 3 calls per invocation → 3 calls total - ``vkCreateDescriptorSetLayout`` — 2 calls per invocation → 2 calls total - ``vkCreatePipelineLayout`` — 2 calls per invocation → 2 calls total **Portability:** Fully portable. Uses standard Vulkan 1.3 API. No vendor extensions required. ---- Optimization 2: GPU-Only Buffer Allocation (VMA Flags) ------------------------------------------------------ **Date:** 2026-07-26 **Status:** Applied **What changed:** ``prefixes_buf`` and ``output_buf`` in ``vc_filter`` switched from ``VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT`` to ``VMA_MEMORY_USAGE_AUTO`` (``flags=0``). Neither buffer is accessed by the CPU. Staging buffers (``totals_staging``, ``offsets_staging``) retain host access for the pass-2 CPU merge. **Why it matters:** BAR memory (``HOST_ACCESS_RANDOM``) is limited to ~256 MB on discrete GPUs and has lower bandwidth than device-local VRAM. Intermediate GPU-only buffers that exceed BAR cause the driver to spill into system RAM — a sudden performance cliff (GPU reads/writes at PCIe speed instead of HBM). Device-local allocation prevents this entirely. **Performance impact:** Not a flat percentage — a scalability guard. Below ~64 MB columns there is no difference. Above that, avoids a ≥10x slowdown when BAR is exhausted. **Portability:** Fully portable. ``VMA_MEMORY_USAGE_AUTO`` lets VMA pick the best memory type for each GPU. ---- Optimization 3: Reusable Command Buffer per Context --------------------------------------------------- **Date:** 2026-07-26 **Status:** Applied **What changed:** ``vc_filter`` previously allocated a command buffer via ``vkAllocateCommandBuffers`` and freed it via ``vkFreeCommandBuffers`` on every invocation. Now a single ``VkCommandBuffer`` is allocated once in ``vc_create_context`` (as ``VcContext::reusable_cmd``), freed in ``vc_destroy_context``, and reused across all operations via ``vkResetCommandBuffer``. **Why it matters:** Command buffer allocation hits the driver allocator on every call. The pattern also establishes a convention for all future operations — they should use ``ctx->reusable_cmd`` instead of allocating their own. **Performance impact:** Not a measurable speedup for large columns (kernel time dominates), but eliminates two driver allocator calls per invocation and makes repeated small-column filter calls faster. **Vulkan primitives affected:** - ``vkAllocateCommandBuffers`` — 1 call per ``vc_filter`` invocation → 0 - ``vkFreeCommandBuffers`` — 1 call per ``vc_filter`` invocation → 0 **Portability:** Fully portable. Uses standard ``vkResetCommandBuffer`` with ``VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT`` (already set on the context command pool). Vulkan 1.0. ---- Optimization 4: Subgroup Scan in Filter Pass 1 ---------------------------------------------- **Date:** 2026-07-26 **Status:** Applied **What changed:** ``filter_pass1.comp`` used a full shared-memory Blelloch scan (upsweep + downsweep, 8 barriers per workgroup). On GPUs with ``VK_KHR_shader_subgroup_arithmetic``, a variant shader (``filter_pass1_subgroup.comp``) now uses ``subgroupExclusiveAdd()`` for intra-wave scan — register-speed, 2 barriers — with shared memory only for the 4-way cross-wave merge. Selection at context creation via ``ctx->has_subgroup_arithmetic``; falls back to Blelloch otherwise. **Vulkan primitives affected:** - ``VK_KHR_shader_subgroup_arithmetic`` device extension - ``VkPhysicalDeviceSubgroupProperties::subgroupSize`` queried at ``vc_create_context`` **Estimated performance:** 20-30% faster scan phase (8 barriers → 2). Benchmarks pending — scan time is a fraction of end-to-end filter (dominated by PCIe transfers). **Portability:** ``VK_KHR_shader_subgroup_arithmetic`` is supported on AMD (Vulkan 1.1+), NVIDIA (Maxwell+, Vulkan 1.1+), Intel (Gen12+), and MoltenVK (Metal 3.0+). Fallback to Blelloch for others. ---- Optimization 5: Push Descriptors in vc_filter --------------------------------------------- **Date:** 2026-07-26 **Status:** Applied **What changed:** ``vc_filter`` allocated a descriptor pool and three descriptor sets on every call, then called ``vkUpdateDescriptorSets`` + ``vkCmdBindDescriptorSets`` for each pass. With ``VK_KHR_push_descriptor``, descriptors are written directly into the command buffer via ``vkCmdPushDescriptorSet`` — no pool, no sets, no allocate/update/bind/destroy calls. **Vulkan primitives affected:** - ``vkCreateDescriptorPool`` — 1 call per invocation -> 0 - ``vkAllocateDescriptorSets`` — 3 -> 0 - ``vkUpdateDescriptorSets`` — 3 -> 0 - ``vkCmdBindDescriptorSets`` — 3 -> 0 - ``vkDestroyDescriptorPool`` — 1 -> 0 - ``vkCmdPushDescriptorSet`` — 3 new calls (push path only) - ``VK_KHR_push_descriptor`` device extension **Performance impact:** Eliminates 11 driver calls per filter invocation. Not measurable for large columns (kernel time dominates) but reduces CPU overhead on repeated or small-column calls. **Portability:** Supported on AMD, NVIDIA, Intel, and MoltenVK (Vulkan 1.1+). Devices without the extension fall back to the traditional pool+set path automatically. Feature detection at ``vc_create_context`` stores ``ctx->has_push_descriptors``. ---- Optimization 6: Sort Pipeline Caching -------------------------------------- **Date:** 2026-07-26 **Status:** Applied **What changed:** ``vc_sort`` creates pipeline state (3 shader modules, 3 compute pipelines, 3 descriptor set layouts, 3 pipeline layouts) once on the first call and caches them in ``ctx->sort_cache``. This is the same context-scoped pattern used by ``vc_filter`` and ``vc_reduce``. **Vulkan primitives affected:** - ``vkCreateShaderModule`` — 3 calls per ``vc_sort`` invocation → 3 calls total - ``vkCreateComputePipelines`` — 3 per invocation → 3 total - ``vkCreateDescriptorSetLayout`` — 3 per invocation → 3 total - ``vkCreatePipelineLayout`` — 3 per invocation → 3 total **Portability:** Fully portable. Standard Vulkan 1.3. No vendor extensions. ---- Planned Optimizations --------------------- 7. **Subgroup-size-aware workgroup tuning.** Query ``VkPhysicalDeviceSubgroupProperties::subgroupSize`` at context creation. Use to enforce workgroup sizes that are multiples of the native wave/warp size (AMD=64, NVIDIA=32, Intel=32). Current 256 threads works for all vendors but may not be optimal for occupancy.