[DUG] [computing] Re: D2007 to XE2 conversion notes

[Peter Ingham]

On 17/01/2012 1:47 p.m., Jolyon Smith wrote:
>  > On the other hand - it looks like the same test program uses a lot less
>  > memory than the old D2007 version.    The figures vary wildly
> depending on
>
> John, none of the utilities for measuring "memory use" are actually
> reporting truly "used" memory.  The only thing that can do that is some
> utility which can interpret the information in the process heap and work
> out which memory is actually in use by the process and which memory has
> been allocated just in case it may need to be used.

If you are using an OS-level tool to measure memory usage (e.g. Task 
Manager, Process Explorer), then you need to know the meaning of the 
particular "Memory" figure you are looking at.  Unfortunately many 
people don't know what they mean and get confused by the results.

It doesn't help that the default "memory" figure under Windows Task 
Manager (a.k.a "Working Set") is usually one of the worst that can be 
used by developer (it can be heavily influenced by other processes 
running on the system to the point of being meaningless in isolation).

A much better value to use is "Private Virtual Bytes" or "Commit Size".

At the OS memory management level (for a native application), there is 
no knowledge of the existence of a heap (or heaps).  Ignoring Virtual 
Memory issues,  Memory is either allocated to a process or not.  What 
the application (or its run time libraries) do with the memory is of 
little interest.  It gets more interesting when we stop ignoring Virtual 
Memory issues and the process asks to be allocated memory, but never (or 
infrequently) accesses it.  There is unlikely to be any OS level tools 
that will tell you anything about the use of Heap memory per se.

As most modern apps do a large number of memory allocation and 
deallocation operationg (e.g. Object Create/Destroy, Temp Strings etc) - 
and usually for relatively small amounts of memory, it is usual for some 
form of internal process memory manager to be used.  Going to the OS 
every time for small chunks of memory is very slow.  Most process memory 
managers will maintain pools of memory that have been allocated by the 
OS, but are not currently being used by anything in the process.


When we know what Memory Manager (or managers) are being used within the 
process, then we may be able to come up with more accurate ideas of how 
a process is using memory.  In the case of a Java Virtual Machine or a 
.Net VM,  there is explicit knowledge of memory usage derived for some 
parts of the process when Garbage Collection is done (however for 
efficiency, modern GC techniques do not come up with definitive 
results).  Some OS level tools understand how to externally query the VM 
memory manager inside the process to obtain usage information.

FastMM4 (full version) has API calls available to dump out all memory 
blocks allocated by it, so you can compare at a detailed level what is 
being used for what (if the FastMM4 debugging options are set you can 
get details of what class a block is containing and a stack trace taken 
at the time the block was allocated).


Cheers