I have a program that uses way too much memory for allocating numerous small objects on heap. So I would like to investigate into ways to optimize it. The program is compiled with Visual C++ 7.
我有一个程序使用太多的内存来分配堆上的许多小对象。所以我想研究一下优化它的方法。该程序使用Visual C ++ 7编译。
Is there a way to determine how much memory is actually allocated for a given object? I mean when I call new the heap allocates not less than the necessary amount. How can I find how much exactly has been allocated?
有没有办法确定给定对象实际分配了多少内存?我的意思是当我调用new时,堆分配的数量不少于必要的数量。我怎样才能找到确切分配的数量?
6 个解决方案
#1
9
There is no exact answer, because some heap managers may use different amount of memory for sequential allocations of the same size. Also, there is (usually) no direct way to measure number of bytes the particular allocation took.
没有确切的答案,因为一些堆管理器可能会使用不同数量的内存来进行相同大小的顺序分配。此外,(通常)没有直接的方法来测量特定分配所花费的字节数。
You can approximate this value by allocating a certain number of items of the same size (say, 4096) and noting the difference in memory used. Dividing the later by the former would give you the answer. Please note that this value changes from OS to OS, and from OS version to OS version and sometimes Debug builds of your application may enable extra heap tracking, which increases the overhead. On some OSes users can change heap policies (i.e. use one heap allocator vs. another as default). Example: Windows and pageheap.exe
您可以通过分配一定数量的相同大小的项目(例如,4096)并注意使用的内存差异来估算此值。将前者除以前者会给你答案。请注意,此值从操作系统更改为操作系统,从操作系统版本更改为操作系统版本,有时应用程序的调试版本可能会启用额外的堆跟踪,从而增加了开销。在某些操作系统上,用户可以更改堆策略(即默认使用一个堆分配器与另一个堆分配器)。示例:Windows和pageheap.exe
Just FYI, the default (not LFH) heap on Windows 32-bit takes up:
仅供参考,Windows 32位上的默认(非LFH)堆占用:
- sizeof(your_type), rounded up to DWORD boundary,
- 3 pointers for prev/next/size
- 2 pointers for security cookies
sizeof(your_type),四舍五入到DWORD边界,
prev / next / size的3个指针
2个安全cookie指针
#2
4
You're probably looking to move to a memory pool model (which is what the previous answer about "allocating a large pool" was describing. Because memory pools do not require overhead for each allocation, they offer space savings for large numbers of small objects. If the lifetime of a group of those small objects is short (i.e. you allocate a bunch of small objects then need to get rid of the lot), a memory pool is also much faster because you can just free the pool instead of each object.
您可能希望转移到内存池模型(这是之前关于“分配大型池”的答案所描述的内容。因为内存池不需要每次分配的开销,所以它们可以为大量小对象节省空间如果一组这些小对象的生命周期很短(即你分配了一堆小对象然后需要摆脱批次),内存池也要快得多,因为你可以释放池而不是每个对象。
Wikipedia has some info on the technique as well as a link to a few implementations:
*有关于该技术的一些信息以及一些实现的链接:
http://en.wikipedia.org/wiki/Memory_pool
you should be able to find other implementations with a simple web search.
您应该能够通过简单的Web搜索找到其他实现。
#3
3
Not for certain in a platform-independent way. I don't remember any details off-hand (except for an OS I'm pretty sure you're not using), but your OS might offer a way of testing the "size" of a malloc-ed allocation, and new might use malloc or equivalent. So you might be able to get what the memory allocator thinks of as the size of the allocation. This may or may not include any header preceding the allocation (probably not, I'd guess).
并非以平台无关的方式确定。我不记得任何细节(除了操作系统,我很确定你没有使用),但你的操作系统可能提供了一种测试malloc-ed分配的“大小”的方法,以及新的使用malloc或等价物。因此,您可能能够获得内存分配器所认为的分配大小。这可能包括也可能不包括分配之前的任何标题(可能不是,我猜)。
One option is to allocate a few million small objects, and look at how much memory your program is using. Then allocate a few million more and look again. The total will usually go up in chunks as the process is allocated RAM (or virtual address space) by the OS, but with a large number of objects the effect of this "rounding error" will normally tend towards 0 bytes per object.
一种选择是分配几百万个小对象,并查看程序使用的内存量。然后分配几百万再看一遍。由于操作系统为RAM(或虚拟地址空间)分配了进程,因此总数通常会以块为单位,但是对于大量对象,这种“舍入错误”的影响通常趋向于每个对象0个字节。
This should tell you what you probably want to know, which is the average memory overhead of numerous small heap objects. It will include any bookkeeping overhead by the memory allocator, such as headers immediately before the allocation, or external structures to track allocations and/or blocks.
这应该告诉你你可能想知道什么,这是许多小堆对象的平均内存开销。它将包括内存分配器的任何簿记开销,例如分配之前的标头,或跟踪分配和/或块的外部结构。
#4
1
If you know you're going to do lots of small allocations and you're worried about memory fragmentation, why not allocate a single large buffer and then map into that? You'll probably see some performance improvements as well if you're doing a lot of allocating / deallocating.
如果你知道你要做很多小的分配并且你担心内存碎片,为什么不分配一个大的缓冲区然后映射到那个呢?如果你正在进行大量的分配/解除分配,你可能会看到一些性能改进。
* has some useful posts pertaining to avoiding memory fragmentation which might be of some use.
*有一些有用的帖子,可以避免可能有用的内存碎片。
#5
1
Under Windows you can use the Heap32First and HEAPENTRY32 structures to determine the size of any given heap entry, assuming you are not using a customised heap manager. It is also worth pointing out that the size of an allocated block is liable to be larger in debug than release builds due to guard bytes. I don't see mention of Heap64 functions in MSDN so I guess they simply use the Heap32 name.
在Windows下,您可以使用Heap32First和HEAPENTRY32结构来确定任何给定堆条目的大小,假设您没有使用自定义堆管理器。还值得指出的是,由于保护字节,分配块的大小在调试中可能比发布版本大。我没有在MSDN中看到Heap64函数的提及,所以我猜他们只是使用Heap32名称。
#6
0
[EDIT] This answer is wrong! I'm leaving it as an example of what not to do. The sizeof
function
operator works at compile time.
Have you tried:
你有没有尝试过:
SomeClass* some_instance = new SomeClass;
printf("Size of SomeClass == %u", sizeof(*some_instance) );
I seem to recall that by passing in an instance of a class you should get the size that was allocated.
我似乎记得通过传入一个类的实例,你应该得到分配的大小。
#1
9
There is no exact answer, because some heap managers may use different amount of memory for sequential allocations of the same size. Also, there is (usually) no direct way to measure number of bytes the particular allocation took.
没有确切的答案,因为一些堆管理器可能会使用不同数量的内存来进行相同大小的顺序分配。此外,(通常)没有直接的方法来测量特定分配所花费的字节数。
You can approximate this value by allocating a certain number of items of the same size (say, 4096) and noting the difference in memory used. Dividing the later by the former would give you the answer. Please note that this value changes from OS to OS, and from OS version to OS version and sometimes Debug builds of your application may enable extra heap tracking, which increases the overhead. On some OSes users can change heap policies (i.e. use one heap allocator vs. another as default). Example: Windows and pageheap.exe
您可以通过分配一定数量的相同大小的项目(例如,4096)并注意使用的内存差异来估算此值。将前者除以前者会给你答案。请注意,此值从操作系统更改为操作系统,从操作系统版本更改为操作系统版本,有时应用程序的调试版本可能会启用额外的堆跟踪,从而增加了开销。在某些操作系统上,用户可以更改堆策略(即默认使用一个堆分配器与另一个堆分配器)。示例:Windows和pageheap.exe
Just FYI, the default (not LFH) heap on Windows 32-bit takes up:
仅供参考,Windows 32位上的默认(非LFH)堆占用:
- sizeof(your_type), rounded up to DWORD boundary,
- 3 pointers for prev/next/size
- 2 pointers for security cookies
sizeof(your_type),四舍五入到DWORD边界,
prev / next / size的3个指针
2个安全cookie指针
#2
4
You're probably looking to move to a memory pool model (which is what the previous answer about "allocating a large pool" was describing. Because memory pools do not require overhead for each allocation, they offer space savings for large numbers of small objects. If the lifetime of a group of those small objects is short (i.e. you allocate a bunch of small objects then need to get rid of the lot), a memory pool is also much faster because you can just free the pool instead of each object.
您可能希望转移到内存池模型(这是之前关于“分配大型池”的答案所描述的内容。因为内存池不需要每次分配的开销,所以它们可以为大量小对象节省空间如果一组这些小对象的生命周期很短(即你分配了一堆小对象然后需要摆脱批次),内存池也要快得多,因为你可以释放池而不是每个对象。
Wikipedia has some info on the technique as well as a link to a few implementations:
*有关于该技术的一些信息以及一些实现的链接:
http://en.wikipedia.org/wiki/Memory_pool
you should be able to find other implementations with a simple web search.
您应该能够通过简单的Web搜索找到其他实现。
#3
3
Not for certain in a platform-independent way. I don't remember any details off-hand (except for an OS I'm pretty sure you're not using), but your OS might offer a way of testing the "size" of a malloc-ed allocation, and new might use malloc or equivalent. So you might be able to get what the memory allocator thinks of as the size of the allocation. This may or may not include any header preceding the allocation (probably not, I'd guess).
并非以平台无关的方式确定。我不记得任何细节(除了操作系统,我很确定你没有使用),但你的操作系统可能提供了一种测试malloc-ed分配的“大小”的方法,以及新的使用malloc或等价物。因此,您可能能够获得内存分配器所认为的分配大小。这可能包括也可能不包括分配之前的任何标题(可能不是,我猜)。
One option is to allocate a few million small objects, and look at how much memory your program is using. Then allocate a few million more and look again. The total will usually go up in chunks as the process is allocated RAM (or virtual address space) by the OS, but with a large number of objects the effect of this "rounding error" will normally tend towards 0 bytes per object.
一种选择是分配几百万个小对象,并查看程序使用的内存量。然后分配几百万再看一遍。由于操作系统为RAM(或虚拟地址空间)分配了进程,因此总数通常会以块为单位,但是对于大量对象,这种“舍入错误”的影响通常趋向于每个对象0个字节。
This should tell you what you probably want to know, which is the average memory overhead of numerous small heap objects. It will include any bookkeeping overhead by the memory allocator, such as headers immediately before the allocation, or external structures to track allocations and/or blocks.
这应该告诉你你可能想知道什么,这是许多小堆对象的平均内存开销。它将包括内存分配器的任何簿记开销,例如分配之前的标头,或跟踪分配和/或块的外部结构。
#4
1
If you know you're going to do lots of small allocations and you're worried about memory fragmentation, why not allocate a single large buffer and then map into that? You'll probably see some performance improvements as well if you're doing a lot of allocating / deallocating.
如果你知道你要做很多小的分配并且你担心内存碎片,为什么不分配一个大的缓冲区然后映射到那个呢?如果你正在进行大量的分配/解除分配,你可能会看到一些性能改进。
* has some useful posts pertaining to avoiding memory fragmentation which might be of some use.
*有一些有用的帖子,可以避免可能有用的内存碎片。
#5
1
Under Windows you can use the Heap32First and HEAPENTRY32 structures to determine the size of any given heap entry, assuming you are not using a customised heap manager. It is also worth pointing out that the size of an allocated block is liable to be larger in debug than release builds due to guard bytes. I don't see mention of Heap64 functions in MSDN so I guess they simply use the Heap32 name.
在Windows下,您可以使用Heap32First和HEAPENTRY32结构来确定任何给定堆条目的大小,假设您没有使用自定义堆管理器。还值得指出的是,由于保护字节,分配块的大小在调试中可能比发布版本大。我没有在MSDN中看到Heap64函数的提及,所以我猜他们只是使用Heap32名称。
#6
0
[EDIT] This answer is wrong! I'm leaving it as an example of what not to do. The sizeof
function
operator works at compile time.
Have you tried:
你有没有尝试过:
SomeClass* some_instance = new SomeClass;
printf("Size of SomeClass == %u", sizeof(*some_instance) );
I seem to recall that by passing in an instance of a class you should get the size that was allocated.
我似乎记得通过传入一个类的实例,你应该得到分配的大小。