A twisted tale of memory optimization
I was looking into reducing the allocation in a particular part of our code, and I ran into what was basically the following code (boiled down to the essentials):
As you can see, this does a lot of allocations. The actual method in question was a pretty good size, and all those operations happened in different locations and weren’t as obvious.
Take a moment to look at the code, how many allocations can you spot here?
The first one, obviously, is the string allocation, but there is another one, inside the call to GetBytes(), let’s fix that first by allocating the buffer once (I’m leaving aside the allocation of the reusable buffer, you can assume it is big enough to cover all our needs):
For that matter, we can also easily fix the second problem, by avoiding the string allocation:
That is a few minutes of work, and we are good to go. This method is called a lot, so we can expect a huge reduction in the amount of memory that we allocated.
Except… that didn’t happen. In fact, the amount of memory that we allocate remained pretty much the same. Digging into the details, we allocate roughly the same number of byte arrays (how!) and instead of allocating a lot of strings, we now allocate a lot of character arrays.
I broke the code apart into multiple lines, which made things a lot clearer. (In fact, I threw that into SharpLab, to be honest). Take a look:
This code: buffer[..len] is actually translated to:
char charBuffer= RuntimeHelpers.GetSubArray(buffer, Range.EndAt(len));
That will, of course, allocate. I had to change the code to be very explicit about the types that I wanted to use:
This will not allocate, but if you note the changes in the code, you can see that the use of var in this case really tripped me up. Because of the number of overloads and automatic coercion of types that didn’t happen.
For that matter, note that any slicing on arrays will generate a new array, including this code:
This makes perfect sense when you realize what is going on and can still be a big surprise, I looked at the code a lot before I figured out what was going on, and that was with a profiler output that pinpointed the fault.