When programming elevated achievement applications, specified as video games, you study neat tricks for penning accelerated code that makes the most of constricted resources.
Memory allocation is a huge topic for achievement crucial code that needs a changeable amount of memory.
When you comprehend “stack allocator” you power think of “alloca” which lets you allocate recollection from the genuine program stack. That is excellent helpful in a pinch, but that is not what today’s article is about. Another name for the category of stack allocator we are talking concerning today is “bump allocator”.
The Stack Allocator
Around 2010 I was operating at Surreal Software (Midway Seattle) on an open earth equivalent “This is Vegas” which unfortunately was cancelled whenever Midway went out of business.
On that project, one of my duties was animation programming. Animation worked by having a province device for sets of bones, anywhere all province was a tree of animation blends. When states changed, they would blend out from the old province into the new province to maintain continuity.
There was a lot of blending going on, and it used a ample figure of energetic recollection allocations for these animations to be evaluated and blended on the CPU. These allocations were all transient although and lone needed to be for part of the frame. Furthermore, these allocations had to happen per frame, as things could alter drastically from one example to the next, as to which characters were being animated that example – according to the camera frustum for example. The allocator showed up as a hot place in profiling this division of code.
To resolve this problem, we implemented a “stack allocator” anywhere allocation is fair an entire figure add, and deallocation is equal cheaper.
To do this, you allocate and keep several amount of recollection that is the maximum amount you volition need, akin say 1MB. This is your stack. You additionally keep an entire figure “tos” changeable to cognize anywhere the top of the stack is.
When you desire to allocate memory, your recollection is the recollection at the top of the stack, and you add the figure of bytes you desire to allocate to the tos changeable so that the next allocation comes following yours:
In a genuine implementation, you would additionally desire to cognize how big the recollection was, so that you could grip the tos hitting the end of the recollection – by returning null, or asserting, or item else.
When you are done alongside the memory, you fair set the tos changeable to 0.
In our case, we set the tos to 0 following animation blending was completed and we had copied the results into the buffers that were heading to the vertex shader for skinning. If you were using a stack allocator additional mostly for all transient recollection allocations during a game’s frame, you may set it rear to 0 whenever the example starts.
The outcome was night and day. Allocation no longer took any measurable period in the profiler, since allocation was an entire figure addition, and freeing all recollection was a sole entire figure assigment. It doesn’t get faster than that.
One item to note is that as written, this allocator does not call destructors or destructors. Using placement new for constructors would be uncomplicated enough. Calling destructors would be harder, as it would have to be a manual process, which is error prone. You power be capable to use several RAII pattern, but stack allocators activity finest for simpler types.
Another awesome item concerning stack allocators is that they can activity in multi threaded environments. If the tos changeable uses nuclear addition, multiple threads can allocate from the identical stack allocator. This can be nice in a fork/join circumstance on the CPU, or for a complex algorithm operating on the GPU.
You can additionally do “partial deallocations” alongside stack allocators. For instance, if you had several recursive procedure anywhere all flat needed energetic allocation, you could recall anywhere the tos was whenever that flat started, and set it rear to that value whenever that flat of handling finished.
The Double Headed Stack Allocator
A few years later, I established myself at Blizzard on the StarCraft 2 team. My chief James Anhalt was (and motionless is) a really astute dude who introduced me to shader toy, and shared a bunch of really chill techniques that have been former blog posts. He told me concerning twice headed stack allocators as a resolution to a recollection fragmentation issue he had hit in the past.
Imagine that you had a streaming earth equivalent anywhere as the participant ran about the world, old flat geometry would deallocate, and new flat geometry would allocate and stream in from disk.
At the identical time, ideate the participant character could form change into distinct forms – and all period they changed forms, the old form was deallocated, and the new form was allocated and streamed in from disk.
A difficulty is, whenever you have these two systems allocating and deallocating random amounts of memory, on their own timelines, your recollection is going to get fragmented.
Lets stroll through a uncomplicated example. Let’s say you have 100 MB total recollection for the flat (red) and the character (green). The flat allocates 20MB, afterward the character allocates 30MB, afterward the flat allocates 20MB and the character allocates the final 30MB.
Now let’s say that the participant runs to a new section, so the old flat data recollection is freed. The new flat data lone needs to create a sole 30MB allocation. We do have 40MB recollection free, but it isn’t in one chunk! We lone have two blocks of 20MB. So equal although the new flat data is smaller, we can’t allocate area for it. This is recollection fragmentation, and it’s a big issue in recollection constrained environments.
So, James told me what they did in this category of circumstance was to have a chunk of recollection akin a normal stack allocator, but the flat data had a tos which grew upwards from 0, and the character data had a tos which grew downward from the end of the memory. You knew you were out of recollection whenever they traversed complete all another and the flat tos was greater than the character tos.
Let’s see how that works:
This made it so whenever the flat data needed to change, it could reset the flat tos rear to 0, and reallocate the recollection it needed, without any fragmentation whatsoever. Similarly, whenever the character freed its memory, it would reset its tos rear to the end of memory, and could afterward reallocate the recollection it needed, additionally without any fragmentation.
Once again, these allocators activity fine in multithreaded environments, equal whenever all thread power allocate from either side. Atomic add to the tos indices is all you need.
You power ask “why not have two stack allocators alternatively of a twice headed one?” The chief advantage is whenever you are recollection constrained and cognize that the size of one stack allocator is adequate to assist two systems, but the proportion of recollection that goes to one flank vs the another is flexible. In a equivalent circumstance akin described above, that method several levels could use additional memory, but in those situations, the character would need to have reduced recollection usage. A bit of a contrived example but hopefully you get what I mean.
Storing Variable Sized Objects In A Finite Region
There is a assortment of this whenever you are trying to shop a figure of changeable sized objects in a fixed amount of memory, but you desire random admission by index. An example may be that you desire to dispatch network packets of a particular size for effectiveness reason, and you desire to that recollection to encode a changeable figure of changeable sized network commands in it. You desire to be capable to effortlessly iterate through the messages, but you desire to assistance them having changeable sizes too.
What you do is have a twice headed stack allocator anywhere one stack is for the energetic allocations, and the another stack is for an gathering of integers that are an offset in the recollection to the energetic allocations.
Let me display you what I average alongside an example. Starting alongside an bare twice headed stack allocator:
- You desire to allocate 24 bytes. Stack 0 has a tos of 0, so that is anywhere you compose your 24 byte object. You add 24 to stack 0’s tos and it is now 24, anywhere the next liberated area of recollection is. You additionally allocate an entire figure from stack 1 and compose 0 there, as the offset for anywhere this archetypal item lives.
- Next you desire to allocate 14 bytes. Stack 0 has a tos of 24, so that is anywhere you compose your 12 byte object. You add 12 to stack 0’s tos and it is now 38. You additionally allocate an entire figure from stack 1 and compose 24 there, as the offset for anywhere this second item lives.
- and so on…
you additionally have to somehow keep track of how many objects there are. You could have a distinct entire figure keeping figure of how many offsets there are in the offset table, or perchance you could have a sentinel value in the offset array to signify anywhere the end of the offsets are. Perhaps ~0 (all 1s in binary) could be the sentinel value.
Just akin the another stack allocators we talked about, this one can additionally activity in a multi threaded environment. You power be tempted to create the two updates transactional – akin alongside a difference and toggle iteration – but it isn’t needed.
Lastly, this configuration allows you to eliminate items, or move them about in memory, which is another way of reducing fragmentation – you fair need to update the offset array to equivalent actuality whenever you do so.
Links
Another small compose up that includes this topic, from the AMD vulkan recollection allocator: https://gpuopen-librariesandsdks.github.io/VulkanMemoryAllocator/html/custom_memory_pools.html#linear_algorithm
-oOdUn61bDRSykk4H951SoDOHzqoOe1.png)
