Hello,

can it be, that Alloc only can handle allocations up to ca. 2 MB?

If i allocate a larger block and write some data at the end of it, there is an error and the programm terminates.

For example:

Allocate 4 MB and use ZeroMemory.

Is it a known bug, because VirtualAlloc works well.



MfG Manuel.
Posted on 2004-02-21 13:50:46 by other
Alloc uses the IMalloc interface (and is sloppy code, heh) - anyway, the PlatformSDK description of IMalloc::Alloc doesn't mention any allocation size limits like this. Have you tried HeapAlloc instead?

If you're doing large allocations like this and don't need to resize the block, you're probably better off using VirtualAlloc anyway.
Posted on 2004-02-21 14:03:01 by f0dder
Do you check that the memory was actually allocated?
Besides, the Alloc function in M32Lib is pretty, well, silly. Instead of using CoGetMalloc(), it would be better to use CoTaskMemAlloc(). Also, notice that it doesn't Release() the interface it got.
Posted on 2004-02-21 14:09:59 by death

Alloc uses the IMalloc interface (and is sloppy code, heh) - anyway, the PlatformSDK description of IMalloc::Alloc doesn't mention any allocation size limits like this. Have you tried HeapAlloc instead?

If you're doing large allocations like this and don't need to resize the block, you're probably better off using VirtualAlloc anyway.


I use VirtualAlloc and there are no problems. But is use VA for a memory-lib, which will be used for a dynamic string-lib (it set his length dynamically).
Is it better to use HeapAlloc instead. Can i use the ProcessHead or is it better to Create a new Heap?


MfG Manuel.
Posted on 2004-02-21 14:28:22 by other
Dynamic string lib... I guess I would use HeapAlloc for that, probably with a secondary heap - this way fragmentation can be reduced. I'd implement some additional bookkeeping ontop of HeapAlloc though: a bunch of 'slots' for the most common string sizes, so that allocation/deallocation of those could be very fast (ie, mark them unused in the slot table instead of HeapFree+HeapAlloc) - this approach is similar to the one used in SysAllocString... I'd rather design my own than using SysAllocString, though, as this gives more control. Also, I'd have the resizes grow in 'chunks' - keep track of "stringsize" and "blocksize"... this way you have less calls to Heap(Re)Alloc, and less fragmentation.

You could get even better performance by using VirtualAlloc as the memory allocation 'primitive' instead of HeapAlloc, but this requires a whole lot of additional code... managing chunks of memory, etc. And if not done right, you could end up with worse performance. I'd say give HeapAlloc with an additional heap a go first, see if it meets your performance requirements. If not, a heavy-duty VirtualAlloc version could be implemented.
Posted on 2004-02-21 14:51:47 by f0dder

Dynamic string lib... I guess I would use HeapAlloc for that, probably with a secondary heap - this way fragmentation can be reduced. I'd implement some additional bookkeeping ontop of HeapAlloc though: a bunch of 'slots' for the most common string sizes, so that allocation/deallocation of those could be very fast (ie, mark them unused in the slot table instead of HeapFree+HeapAlloc) - this approach is similar to the one used in SysAllocString... I'd rather design my own than using SysAllocString, though, as this gives more control. Also, I'd have the resizes grow in 'chunks' - keep track of "stringsize" and "blocksize"... this way you have less calls to Heap(Re)Alloc, and less fragmentation.


In a similar way i have done this:



POINTER typedef DWORD
PPOINTER typedef DWORD

; Memory
; -12: Blocksize
; -8: can be used from caller ; Status, Len etc.

.const
BLOCKSIZE equ 1024
BLOCKMASK equ 0FFFFFFFFh - (BLOCKSIZE - 1)
STATUSSIZE equ 3 * SIZEOF DWORD

AllocMem proc nSize: DWORD

; align
MOV ECX, nSize
ADD ECX, BLOCKSIZE + STATUSSIZE - 1
AND ECX, BLOCKMASK
PUSH ECX

; alloc
; invoke Alloc, ECX
invoke VirtualAlloc, NULL, ECX, MEM_COMMIT or MEM_RESERVE, PAGE_READWRITE ; better will be a memory manager, which distribute memory blocks

; set size
POP ECX

MOV [EAX], ECX
LEA EAX, [EAX+STATUSSIZE]

Ret
AllocMem EndP


And then, when i ReAllocate Memory, i use the following code:



ReAllocMem proc p: PPOINTER, nSize: DWORD

MOV EDX, p
.if DWORD PTR [EDX] == NULL
invoke AllocMem, nSize
MOV EDX, p
MOV [EDX], EAX
.else
MOV EDX, [EDX]
MOV ECX, [EDX-STATUSSIZE]
.if ECX < nSize
{ allocate new, copy old buffer to new, deallocate mem-block }

...



Thats a small part of the string library:



StrNew proc nSize: DWORD
MOV ECX, nSize
LEA ECX, [ECX+4] ; 1 = #0; 3 = align
AND ECX, 0FFFFFFFCh ; align to 4 byte
invoke AllocMem, ECX
MOV ECX, nSize
MOV [EAX-4], ECX
Ret
StrNew EndP


StrReAlloc proc pasmstr: PASMSTRING, nSize: DWORD
MOV ECX, nSize
LEA ECX, [ECX+4] ; 1 = #0; 3 = align
AND ECX, 0FFFFFFFCh ; align to 4 byte
invoke ReAllocMem, pasmstr, ECX
MOV ECX, nSize
MOV [EAX-4], ECX
Ret
StrReAlloc EndP

SetLength proc pasmstr: PASMSTRING, nSize: DWORD
MOV EDX, pasmstr

; asmstr empty?
.if DWORD PTR [EDX] == NULL
invoke StrNew, nSize
MOV EDX, pasmstr
MOV [EDX], EAX
.else
MOV EDX, [EDX]
MOV ECX, [EDX-4]
.if ECX < nSize ; buffer to small
invoke StrReAlloc, pasmstr, nSize
.else
MOV ECX, nSize
MOV [EDX-4], ECX
MOV EAX, EDX
.endif
.endif
Ret
SetLength EndP

StrAdd proc asDest: PASMSTRING, asSrc: ASMSTRING
invoke StrLength, asSrc
.if EAX > 0
MOV ECX, EAX
PUSH ECX
MOV EDX, asDest
invoke StrLength, DWORD PTR [EDX]
PUSH EAX
ADD ECX, EAX
Invoke SetLength, asDest, ECX
POP EAX ; asDest
POP ECX ; asSrc
MOV EDX, asDest
MOV EDX, [EDX]
LEA EDX, [EDX+EAX]
PUSH EDX
PUSH ECX
invoke CopyMem, EDX, asSrc, ECX
POP ECX
POP EDX
MOV BYTE PTR [EDX+ECX], BYTE PTR 0
.endif

Ret
StrAdd EndP




You could get even better performance by using VirtualAlloc as the memory allocation 'primitive' instead of HeapAlloc, but this requires a whole lot of additional code... managing chunks of memory, etc. And if not done right, you could end up with worse performance. I'd say give HeapAlloc with an additional heap a go first, see if it meets your performance requirements. If not, a heavy-duty VirtualAlloc version could be implemented.


I consider to implement a memory manager which will be distribute the memory blocks.


Regards, Manuel.
Posted on 2004-02-21 16:15:32 by other