Kernel malloc/realloc?

[Bogdan Bejan]

How can you alocate then realocate a memory buffer in kernel mode (W2k+) ?

[Maxim S. Shatskih]

I never heard about realloc() in kernel, but ExAllocatePoolWithTag is used
instead of malloc().

[David J. Craig]

Should you be writing drivers? Look at the source code for realloc that
comes with Visual Studio and adapt it to kernel mode.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode (W2k+) ?

[Bogdan Bejan]

Post by David J. Craig
Should you be writing drivers? Look at the source code for realloc that

Let me guess, realloc is written using system calls like GlobalRealloc, to
which i don't have the souce code, do I?
Why I should not write drivers? When i was learning about virtual vs.
hardware memory the top of the lince CPU was 386...soo..
Besides, I know how realloc works on x86 CPUs, what i don't know is the
windows kernel API to get the same thing.

Post by David J. Craig
comes with Visual Studio and adapt it to kernel mode.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+) ?

[Don Burn]

No, you are talking about Win32 user space API's not kernel API's. The
reason people are asking is if you haven't learned the basics, and you
aren't working to get the knowledge yourself, you will be comming back here
an awful lot with stupid questions. I recomend as a minimum getting a good
book on driver writing such as "Programming the Windows Driver Model Second
Edition" by Walter Oney, or better yet taking a seminar in driver writing.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by Bogdan Bejan

Post by David J. Craig
Should you be writing drivers? Look at the source code for realloc that

Let me guess, realloc is written using system calls like GlobalRealloc, to
which i don't have the souce code, do I?
Why I should not write drivers? When i was learning about virtual vs.
hardware memory the top of the lince CPU was 386...soo..
Besides, I know how realloc works on x86 CPUs, what i don't know is the
windows kernel API to get the same thing.

Post by David J. Craig
comes with Visual Studio and adapt it to kernel mode.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+) ?

[Bogdan Bejan]

Post by Don Burn
No, you are talking about Win32 user space API's not kernel API's. The
reason people are asking is if you haven't learned the basics, and you
aren't working to get the knowledge yourself, you will be comming back here
an awful lot with stupid questions. I recomend as a minimum getting a good
book on driver writing such as "Programming the Windows Driver Model Second
Edition" by Walter Oney, or better yet taking a seminar in driver writing.

Do you think I should attend a seminar to find out if there is or not a
kernel mode API function to do what GlobalRealloc/realloc function does?

The fact is that you and David did not understood my question, even wors you
started to assume what I know and what I don't know and gave generic
unhelpfull answers, advertised some book etc..etc..

As i sed, I do know what is behind GlobalRealloc/realloc/etc.. from the x86
point of view (do you?), what I didn't know was if there was a kernel mode
function to facilitate it.

Aniway, my question was answered, thanx again Dorono..

Post by Don Burn
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by Bogdan Bejan

Post by David J. Craig
Should you be writing drivers? Look at the source code for realloc that

Let me guess, realloc is written using system calls like GlobalRealloc, to
which i don't have the souce code, do I?
Why I should not write drivers? When i was learning about virtual vs.
hardware memory the top of the lince CPU was 386...soo..
Besides, I know how realloc works on x86 CPUs, what i don't know is the
windows kernel API to get the same thing.

Post by David J. Craig
comes with Visual Studio and adapt it to kernel mode.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+) ?

[Maxim S. Shatskih]

Post by Bogdan Bejan
As i sed, I do know what is behind GlobalRealloc/realloc/etc.. from the x86
point of view (do you?)

There is nothing under realloc() "from x86 point of view" (except the virtual
memory, but this is too low a level - for instance, it only knows pages and not
small memory blocks).

realloc() is a function of the heap manager package, and, if the heap manager
has no such function - then sorry.

ExxxxPool heap manager has no such function.

[Bogdan Bejan]

Post by Maxim S. Shatskih

Post by Bogdan Bejan
As i sed, I do know what is behind GlobalRealloc/realloc/etc.. from the x86
point of view (do you?)

There is nothing under realloc() "from x86 point of view" (except the virtual

In realloc there isn't a single bit of x86 dependent code (ok , meybe one or
two :)
and it can never be.Why? Because it is a C/C++ runtime library that is ment
to be portable and has to realy on the OS to acctually do the reallocation.

"from x86 point of view" = read, I know how the x86 memory model works,I
know it at the lowest possible level, I don't need to buy some books for
that. Don't we have the Architecture/OS manual from Intel? Which is a free
download?

I don't want the want the damn C/C++ runtime in a driver, I only need to
resize a memory buffer. Simple as that

Post by Maxim S. Shatskih
memory, but this is too low a level - for instance, it only knows pages and not
small memory blocks).
realloc() is a function of the heap manager package, and, if the heap manager
has no such function - then sorry.
ExxxxPool heap manager has no such function.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
http://www.storagecraft.com

[Maxim S. Shatskih]

Post by Bogdan Bejan
I don't want the want the damn C/C++ runtime in a driver, I only need to
resize a memory buffer. Simple as that

The ExxxxPool memory allocator package does not support this operation. Maybe
this is because MS just forgotten to do this. It is rarely needed anyway.

[Maxim S. Shatskih]

Post by Bogdan Bejan
Let me guess, realloc is written using system calls like GlobalRealloc, to

This is not a syscall. This is hooked to user-mode ntdll!RtlxxxHeap, which do
all heap management.

VirtualAlloc and other Virtualxxx are syscalls, hooked to
ZwAllocateVirtualMemort.

Post by Bogdan Bejan
Besides, I know how realloc works on x86 CPUs, what i don't know is the
windows kernel API to get the same thing.

Just - no such API. Allocate the second block and copy.

[Bogdan Bejan]

Post by Maxim S. Shatskih

Post by Bogdan Bejan
Let me guess, realloc is written using system calls like GlobalRealloc, to

This is not a syscall. This is hooked to user-mode ntdll!RtlxxxHeap, which do
all heap management.

I didn't mean syscall (as in a call to kernel) i meant a system call or API,
imlemented in a user mode dll, like you sed :)

All the runtime function from C/C++ have to go trough windows user mode API
to do the real-work, so if I look at the realloc code I won't learn much, do
I?

So the real newbe here was David J.
:P

When i sed malloc/realloc some people understood it literaly when I only
ment them as the end result I was looking for, that is being able to resize
a kernel mode memory buffer.

Post by Maxim S. Shatskih
VirtualAlloc and other Virtualxxx are syscalls, hooked to
ZwAllocateVirtualMemort.

Post by Bogdan Bejan
Besides, I know how realloc works on x86 CPUs, what i don't know is the
windows kernel API to get the same thing.

Just - no such API. Allocate the second block and copy.

For startes I will do just that, but, you should be able to realocate the
number of fizical pages behind a buffer and map them in a diffrent location
(yes, i still have to finish looking at the zilions of memory functions from
the DDK) where there is enough virtual space, right?
That way you don't do the costly copy operation.

Post by Maxim S. Shatskih
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
http://www.storagecraft.com

[Doron Holan [MS]]

no such API exists. you must be able to update the pointer value atomically
if you need to have a bigger allocation

d

[Bogdan Bejan]

I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value atomically
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc based on
GlobalRealloc?Which i suppose works by allocating more hardware pages and
mapping them in a new vitual location, where you have enough virtual space
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode (W2k+) ?

[Doron Holan [MS]]

one of the semantics of realloc is that the same pointer as before can be
returned (i haven't used realloc in such a long time, i can't remember if
this true 100% of the time or not). Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have stopped
using it
2) update the field which contains the pointer value so that no one picks
up the old value and starts using the new one

d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc based on
GlobalRealloc?Which i suppose works by allocating more hardware pages and
mapping them in a new vitual location, where you have enough virtual space
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+)

Post by Bogdan Bejan
?

[Bogdan Bejan]

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before can be
returned (i haven't used realloc in such a long time, i can't remember if
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to return
the same adress, what if you don't have enought linear virtual memory there
to fulfill the request?

So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have stopped
using it
2) update the field which contains the pointer value so that no one picks
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel mode.

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc

based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware pages and
mapping them in a new vitual location, where you have enough virtual space
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+)

Post by Bogdan Bejan
?

[Alexander Grigoriev]

In most designs, realloc is so rarely used, that any implementation is left
up to you. You won't have an optimized realloc (which can resize a memory
block), though.

Post by Bogdan Bejan

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before can be
returned (i haven't used realloc in such a long time, i can't remember if
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to return
the same adress, what if you don't have enought linear virtual memory there
to fulfill the request?
So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have stopped
using it
2) update the field which contains the pointer value so that no one picks
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel mode.

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc

based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware pages

and

Post by Doron Holan [MS]

Post by Bogdan Bejan
mapping them in a new vitual location, where you have enough virtual

space

Post by Doron Holan [MS]

Post by Bogdan Bejan
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+)

Post by Bogdan Bejan
?

[Doron Holan [MS]]

maloc/ExAllocatePoolWithTag has a much higher likelyhood of failure in the
kernel then in user mode. so, yes, as such you need to guard against NULL
being returned as well w/out a doubt. But there is a difference between a
NULL check and making sure all current users of valid pool are not touching
memory when you decide to grow the allocation.

d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by Bogdan Bejan

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before can be
returned (i haven't used realloc in such a long time, i can't remember if
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to return
the same adress, what if you don't have enought linear virtual memory there
to fulfill the request?
So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have stopped
using it
2) update the field which contains the pointer value so that no one picks
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel mode.

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc

based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware pages

and

Post by Doron Holan [MS]

Post by Bogdan Bejan
mapping them in a new vitual location, where you have enough virtual

space

Post by Doron Holan [MS]

Post by Bogdan Bejan
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+)

Post by Bogdan Bejan
?

[G.]

Still, how do you perform the acctual resize?
Any reasons as to why there isn't a direct function in kernel to do this?

Post by Doron Holan [MS]
maloc/ExAllocatePoolWithTag has a much higher likelyhood of failure in the
kernel then in user mode. so, yes, as such you need to guard against NULL
being returned as well w/out a doubt. But there is a difference between a
NULL check and making sure all current users of valid pool are not touching
memory when you decide to grow the allocation.
d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by Bogdan Bejan

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before can

be

Post by Bogdan Bejan

Post by Doron Holan [MS]
returned (i haven't used realloc in such a long time, i can't remember

if

Post by Bogdan Bejan

Post by Doron Holan [MS]
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to

return

Post by Bogdan Bejan
the same adress, what if you don't have enought linear virtual memory

there

Post by Bogdan Bejan
to fulfill the request?
So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have

stopped

Post by Bogdan Bejan

Post by Doron Holan [MS]
using it
2) update the field which contains the pointer value so that no one

picks

Post by Bogdan Bejan

Post by Doron Holan [MS]
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel mode.

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc

based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware pages

and

Post by Doron Holan [MS]

Post by Bogdan Bejan
mapping them in a new vitual location, where you have enough virtual

space

Post by Doron Holan [MS]

Post by Bogdan Bejan
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel mode

(W2k+)

Post by Bogdan Bejan
?

[Doron Holan [MS]]

depends on the application for how to do the resize, but i would either keep
a reference count of all the threads who want access or perform all access
under a lock (like a spinlock) which guarantees only one user of the buffer
at once. one reason realloc does not exist is that VA space is limited and
more often then not, the same pointer cannot be returned so this is
functionally the same as free/alloc anyways.

d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by G.
Still, how do you perform the acctual resize?
Any reasons as to why there isn't a direct function in kernel to do this?

Post by Doron Holan [MS]
maloc/ExAllocatePoolWithTag has a much higher likelyhood of failure in the
kernel then in user mode. so, yes, as such you need to guard against NULL
being returned as well w/out a doubt. But there is a difference between a
NULL check and making sure all current users of valid pool are not

touching

Post by Doron Holan [MS]
memory when you decide to grow the allocation.
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before can

be

Post by Bogdan Bejan

Post by Doron Holan [MS]
returned (i haven't used realloc in such a long time, i can't remember

if

Post by Bogdan Bejan

Post by Doron Holan [MS]
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to

return

Post by Bogdan Bejan
the same adress, what if you don't have enought linear virtual memory

there

Post by Bogdan Bejan
to fulfill the request?
So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have

stopped

Post by Bogdan Bejan

Post by Doron Holan [MS]
using it
2) update the field which contains the pointer value so that no one

picks

Post by Bogdan Bejan

Post by Doron Holan [MS]
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel mode.

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing it.

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer value

atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't realloc

based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware

pages

Post by Doron Holan [MS]

Post by Bogdan Bejan
and

Post by Doron Holan [MS]

Post by Bogdan Bejan
mapping them in a new vitual location, where you have enough virtual

space

Post by Doron Holan [MS]

Post by Bogdan Bejan
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers

no

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]

Post by Bogdan Bejan
rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel

mode

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]
(W2k+)

Post by Bogdan Bejan
?

[G.]

Let's try to be as technical as possible.
In kernel mode (driver) how you do this:

1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.

Post by Doron Holan [MS]
depends on the application for how to do the resize, but i would either keep
a reference count of all the threads who want access or perform all access
under a lock (like a spinlock) which guarantees only one user of the buffer
at once. one reason realloc does not exist is that VA space is limited and
more often then not, the same pointer cannot be returned so this is
functionally the same as free/alloc anyways.
d
--
This posting is provided "AS IS" with no warranties, and confers no rights.
Please reply to newsgroups only.

Post by G.
Still, how do you perform the acctual resize?
Any reasons as to why there isn't a direct function in kernel to do this?

Post by Doron Holan [MS]
maloc/ExAllocatePoolWithTag has a much higher likelyhood of failure in

the

Post by G.

Post by Doron Holan [MS]
kernel then in user mode. so, yes, as such you need to guard against

NULL

Post by G.

Post by Doron Holan [MS]
being returned as well w/out a doubt. But there is a difference

between

Post by Doron Holan [MS]
a

Post by G.

Post by Doron Holan [MS]
NULL check and making sure all current users of valid pool are not

touching

Post by Doron Holan [MS]
memory when you decide to grow the allocation.
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan

Post by Doron Holan [MS]
one of the semantics of realloc is that the same pointer as before

can

Post by G.

Post by Doron Holan [MS]
be

Post by Bogdan Bejan

Post by Doron Holan [MS]
returned (i haven't used realloc in such a long time, i can't

remember

Post by G.

Post by Doron Holan [MS]
if

Post by Bogdan Bejan

Post by Doron Holan [MS]
this true 100% of the time or not).

Any for of realocation of a memory buffer can't possible guarantee to

return

Post by Bogdan Bejan
the same adress, what if you don't have enought linear virtual memory

there

Post by Bogdan Bejan
to fulfill the request?
So this applayes to maloc or any other function, i think.

Post by Doron Holan [MS]
Based on this assumption if you were to
resize a buffer, you would need to
1) make sure any threads that are using the previous buffer have

stopped

Post by Bogdan Bejan

Post by Doron Holan [MS]
using it
2) update the field which contains the pointer value so that no one

picks

Post by Bogdan Bejan

Post by Doron Holan [MS]
up the old value and starts using the new one

I really know this.
I just don't know how to acctualy resize a memory buffer in kernel

mode.

Post by G.

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers no

rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
I wasn't sure is shuch API exists in kernel, thanx for clearing

it.

Post by G.

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]
no such API exists. you must be able to update the pointer

value

Post by G.

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]

Post by Bogdan Bejan
atomically

Post by Doron Holan [MS]
if you need to have a bigger allocation

Don't you have to update the pointer with realloc to? Isn't

realloc

Post by G.

Post by Doron Holan [MS]

Post by Bogdan Bejan
based

Post by Doron Holan [MS]
on

Post by Bogdan Bejan
GlobalRealloc?Which i suppose works by allocating more hardware

pages

Post by Doron Holan [MS]

Post by Bogdan Bejan
and

Post by Doron Holan [MS]

Post by Bogdan Bejan
mapping them in a new vitual location, where you have enough

virtual

Post by G.

Post by Doron Holan [MS]

Post by Bogdan Bejan
space

Post by Doron Holan [MS]

Post by Bogdan Bejan
for the new buffer?

Post by Doron Holan [MS]
d
--
This posting is provided "AS IS" with no warranties, and confers

no

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]

Post by Bogdan Bejan
rights.

Post by Doron Holan [MS]
Please reply to newsgroups only.

Post by Bogdan Bejan
How can you alocate then realocate a memory buffer in kernel

mode

Post by Doron Holan [MS]

Post by Bogdan Bejan

Post by Doron Holan [MS]
(W2k+)

Post by Bogdan Bejan
?

[G.]

In user space, I can only think of only one way. don't use realloc, malloc
some other buffer then copy.

I kernel mode you should be able to do this without the copy operation, but
i don't know how..

What about the case where you would actually WANT a different pointer

returned from realloc()?

Is there any way to force it to move the block to another user-space VA

location?

[Tim Roberts]

Post by G.
Still, how do you perform the acctual resize?

char * newBuffer = ExAllocatePool( newBufferSize, NONPAGED_POOL );
memcpy( newBuffer, oldBuffer, oldBufferSize );
ExFreePool( oldBuffer );

Post by G.
Any reasons as to why there isn't a direct function in kernel to do this?

Yes, because it isn't that common an operation in kernel mode, and it
SHOULDN'T be common.

Memory is a precious resource in kernel mode; non-paged pool is fixed and
cannot be increased. In many cases a memory allocation failure is a
blue-screen-worthy event. As a result, well-designed drivers tend to
allocate memory at device initialization and keep it around forever.

Further, many kernel buffers are shared with user-mode apps. If a buffer
is going to change its address, the driver writer needs to make a conscious
decision to do that.

[G.]

That is verry smart, I never thought of that.
But, how you do that **without the copy** ?
You do know you can do that, don't you ?

Besides the real question is this, so answer this if you can (malloc/realloc

Post by G.
1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.

Post by G.
Still, how do you perform the acctual resize?

char * newBuffer = ExAllocatePool( newBufferSize, NONPAGED_POOL );
memcpy( newBuffer, oldBuffer, oldBufferSize );
ExFreePool( oldBuffer );

Post by G.
Any reasons as to why there isn't a direct function in kernel to do this?

Yes, because it isn't that common an operation in kernel mode, and it
SHOULDN'T be common.
Memory is a precious resource in kernel mode; non-paged pool is fixed and
cannot be increased. In many cases a memory allocation failure is a
blue-screen-worthy event. As a result, well-designed drivers tend to
allocate memory at device initialization and keep it around forever.
Further, many kernel buffers are shared with user-mode apps. If a buffer
is going to change its address, the driver writer needs to make a conscious
decision to do that.
--
Providenza & Boekelheide, Inc

[Don Burn]

Sure you can do this, of course you are taking a limited system resource
(kernel PTE's) and squandering them for your driver. I guess that is ok, if
your driver is in an embedded or limited environment, but if this is a
driver for a general commecial environment, this is crap.

I am friends with several folks who perform the acceptance of drivers, for
large firms who are buying parts OEM, they tell a number of amusing tales of
their firms telling a supplier to completely rewrite their driver (because
of bad code like this, or using a toolkit that large firm won't bless) and
giving them a short time to rewrite. Unless you are just playing at this,
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
That is verry smart, I never thought of that.
But, how you do that **without the copy** ?
You do know you can do that, don't you ?
Besides the real question is this, so answer this if you can

(malloc/realloc

Post by G.

Post by G.
1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.

Post by G.
Still, how do you perform the acctual resize?

char * newBuffer = ExAllocatePool( newBufferSize, NONPAGED_POOL );
memcpy( newBuffer, oldBuffer, oldBufferSize );
ExFreePool( oldBuffer );

Post by G.
Any reasons as to why there isn't a direct function in kernel to do this?

Yes, because it isn't that common an operation in kernel mode, and it
SHOULDN'T be common.
Memory is a precious resource in kernel mode; non-paged pool is fixed and
cannot be increased. In many cases a memory allocation failure is a
blue-screen-worthy event. As a result, well-designed drivers tend to
allocate memory at device initialization and keep it around forever.
Further, many kernel buffers are shared with user-mode apps. If a buffer
is going to change its address, the driver writer needs to make a

conscious

Post by G.
decision to do that.
--
Providenza & Boekelheide, Inc

[G.]

To spam...not to spam....oh well, to spam

So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system resource
(kernel PTE's) and squandering them for your driver. I guess that is ok, if
your driver is in an embedded or limited environment, but if this is a
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of drivers, for
large firms who are buying parts OEM, they tell a number of amusing tales of
their firms telling a supplier to completely rewrite their driver (because
of bad code like this, or using a toolkit that large firm won't bless) and
giving them a short time to rewrite. Unless you are just playing at this,
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
That is verry smart, I never thought of that.
But, how you do that **without the copy** ?
You do know you can do that, don't you ?
Besides the real question is this, so answer this if you can

(malloc/realloc

Post by G.

Post by G.
1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.

Post by G.
Still, how do you perform the acctual resize?

char * newBuffer = ExAllocatePool( newBufferSize, NONPAGED_POOL );
memcpy( newBuffer, oldBuffer, oldBufferSize );
ExFreePool( oldBuffer );

Post by G.
Any reasons as to why there isn't a direct function in kernel to do

this?

Post by G.

Post by G.
Yes, because it isn't that common an operation in kernel mode, and it
SHOULDN'T be common.
Memory is a precious resource in kernel mode; non-paged pool is fixed

and

Post by G.

Post by G.
cannot be increased. In many cases a memory allocation failure is a
blue-screen-worthy event. As a result, well-designed drivers tend to
allocate memory at device initialization and keep it around forever.
Further, many kernel buffers are shared with user-mode apps. If a

buffer

Post by G.

Post by G.
is going to change its address, the driver writer needs to make a

conscious

Post by G.
decision to do that.
--
Providenza & Boekelheide, Inc

[Don Burn]

You keep asking for a realloc, and people have given you approaches without
knowing the constraints you have we will just keep getting worthless barbs
back on this. The bottom line is it depends:

1. You allocate a big chunk, and manage your own heap. This is
wasteful in a general driver.

2. You can use the allocate, copy, free. But you don't seem to like
this.

3. You can look at your overall driver and determine if you really
need to use realloc. I haven't seen a need for realloc in the 10 years I've
been in the Windows kernel, nor have the driver writers I know, so perhaps
your upper level design is flawed.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
To spam...not to spam....oh well, to spam
So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system resource
(kernel PTE's) and squandering them for your driver. I guess that is

ok,

Post by G.
if

Post by Don Burn
your driver is in an embedded or limited environment, but if this is a
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of drivers, for
large firms who are buying parts OEM, they tell a number of amusing

tales

Post by G.
of

Post by Don Burn
their firms telling a supplier to completely rewrite their driver (because
of bad code like this, or using a toolkit that large firm won't bless) and
giving them a short time to rewrite. Unless you are just playing at this,
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

[David J. Craig]

I think he has decided that realloc without copying is his solution and
nothing else matters. Obstinacy doesn't make writing drivers easy and
just wastes everyone's time when they ask a question to which they don't
want a valid answer. Of course, the question could be about how do I
grab an executing image, enlarge it, and add some virus code.

Post by Don Burn
You keep asking for a realloc, and people have given you approaches without
knowing the constraints you have we will just keep getting worthless barbs
1. You allocate a big chunk, and manage your own heap. This is
wasteful in a general driver.
2. You can use the allocate, copy, free. But you don't seem to like
this.
3. You can look at your overall driver and determine if you really
need to use realloc. I haven't seen a need for realloc in the 10 years I've
been in the Windows kernel, nor have the driver writers I know, so perhaps
your upper level design is flawed.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
To spam...not to spam....oh well, to spam
So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system resource
(kernel PTE's) and squandering them for your driver. I guess that is

ok,

Post by G.
if

Post by Don Burn
your driver is in an embedded or limited environment, but if this is a
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of

drivers,

Post by Don Burn
for

Post by G.

Post by Don Burn
large firms who are buying parts OEM, they tell a number of amusing

tales

Post by G.
of

Post by Don Burn
their firms telling a supplier to completely rewrite their driver

(because

Post by G.

Post by Don Burn
of bad code like this, or using a toolkit that large firm won't

bless)

Post by Don Burn
and

Post by G.

Post by Don Burn
giving them a short time to rewrite. Unless you are just playing

at

Post by Don Burn
this,

Post by G.

Post by Don Burn
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

[G.]

Try this instead

How do I implement some streams in a driver.

And no, I don't care is the driver is not the place for streams.

Post by David J. Craig
I think he has decided that realloc without copying is his solution and
nothing else matters. Obstinacy doesn't make writing drivers easy and
just wastes everyone's time when they ask a question to which they don't
want a valid answer. Of course, the question could be about how do I
grab an executing image, enlarge it, and add some virus code.

Post by Don Burn
You keep asking for a realloc, and people have given you approaches

without

Post by Don Burn
knowing the constraints you have we will just keep getting worthless

barbs

Post by Don Burn
1. You allocate a big chunk, and manage your own heap. This is
wasteful in a general driver.
2. You can use the allocate, copy, free. But you don't seem to

like

Post by Don Burn
this.
3. You can look at your overall driver and determine if you

really

Post by Don Burn
need to use realloc. I haven't seen a need for realloc in the 10

years I've

Post by Don Burn
been in the Windows kernel, nor have the driver writers I know, so

perhaps

Post by Don Burn
your upper level design is flawed.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
To spam...not to spam....oh well, to spam
So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system

resource

Post by Don Burn

Post by G.

Post by Don Burn
(kernel PTE's) and squandering them for your driver. I guess that

is

Post by Don Burn
ok,

Post by G.
if

Post by Don Burn
your driver is in an embedded or limited environment, but if this

is a

Post by Don Burn

Post by G.

Post by Don Burn
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of

drivers,

Post by Don Burn
for

Post by G.

Post by Don Burn
large firms who are buying parts OEM, they tell a number of

amusing

Post by Don Burn
tales

Post by G.
of

Post by Don Burn
their firms telling a supplier to completely rewrite their driver

(because

Post by G.

Post by Don Burn
of bad code like this, or using a toolkit that large firm won't

bless)

Post by Don Burn
and

Post by G.

Post by Don Burn
giving them a short time to rewrite. Unless you are just playing

at

Post by Don Burn
this,

Post by G.

Post by Don Burn
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

[G.]

If i have a "virus" driver running in a system, do I reallly care about
infecting a file IN memory?

Do infecting a IN memory image really needs resizing a section? can you
really resize a section without unmapping it first(etc...etc..)?

"Should you be wrinting drivers?" :P

And sice after you the misteries of kernel memory management hide in the
realloc source code from your Visual C++ (how stupid is that?), i suppose
if I look in the fopen source code I will learn how to make file system
driver?

The fact is you don't know how to *really* answer the original question so
you spam,flame,etc..

Why do you even bother to post if your not going to add any meaningfull data
to the discussion?

Post by David J. Craig
I think he has decided that realloc without copying is his solution and
nothing else matters. Obstinacy doesn't make writing drivers easy and
just wastes everyone's time when they ask a question to which they don't
want a valid answer. Of course, the question could be about how do I
grab an executing image, enlarge it, and add some virus code.

Post by Don Burn
You keep asking for a realloc, and people have given you approaches

without

Post by Don Burn
knowing the constraints you have we will just keep getting worthless

barbs

Post by Don Burn
1. You allocate a big chunk, and manage your own heap. This is
wasteful in a general driver.
2. You can use the allocate, copy, free. But you don't seem to

like

Post by Don Burn
this.
3. You can look at your overall driver and determine if you

really

Post by Don Burn
need to use realloc. I haven't seen a need for realloc in the 10

years I've

Post by Don Burn
been in the Windows kernel, nor have the driver writers I know, so

perhaps

Post by Don Burn
your upper level design is flawed.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
To spam...not to spam....oh well, to spam
So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system

resource

Post by Don Burn

Post by G.

Post by Don Burn
(kernel PTE's) and squandering them for your driver. I guess that

is

Post by Don Burn
ok,

Post by G.
if

Post by Don Burn
your driver is in an embedded or limited environment, but if this

is a

Post by Don Burn

Post by G.

Post by Don Burn
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of

drivers,

Post by Don Burn
for

Post by G.

Post by Don Burn
large firms who are buying parts OEM, they tell a number of

amusing

Post by Don Burn
tales

Post by G.
of

Post by Don Burn
their firms telling a supplier to completely rewrite their driver

(because

Post by G.

Post by Don Burn
of bad code like this, or using a toolkit that large firm won't

bless)

Post by Don Burn
and

Post by G.

Post by Don Burn
giving them a short time to rewrite. Unless you are just playing

at

Post by Don Burn
this,

Post by G.

Post by Don Burn
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

[Maxim S. Shatskih]

Post by G.
Why do you even bother to post if your not going to add any meaningfull data
to the discussion?

There is no discussion.

NT kernel has NO routine which will reallocate the memory block with pointer
value preservation and without data copy. Period.

All of the rest - "why", "is it not bad?" - are unnecessary phylosophy, which
has nothing to do with professional development.

[G.]

Post by Maxim S. Shatskih

Post by G.
Why do you even bother to post if your not going to add any meaningfull data
to the discussion?

There is no discussion.
NT kernel has NO routine which will reallocate the memory block with pointer
value preservation and without data copy. Period.

First thing, I **did not asked that**, al least read the thread then post to
it? That is a reall challenge?
Second, You can never, ever, guarantee to grow a buffer and keep the old VA.
Third ****I DON'T CARE ABOUT THE MALLOC/REALLO/ETC... or.... PRESERVING
THE OLD VA****

The acctual question is how you do this:

In kernel mode, a dirver, how can do this steps?
1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.

I'm curious how many more dirver gurus will spam this thread without giving
a decent answer
Or if any of you acctualy knows the answer

Post by Maxim S. Shatskih
All of the rest - "why", "is it not bad?" - are unnecessary phylosophy, which
has nothing to do with professional development.

I guess spaming with pointless posts has something to do with the
professional development?

Post by Maxim S. Shatskih
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
http://www.storagecraft.com

[Maxim S. Shatskih]

Post by G.
1. Allocate a bunch of pf phisical pages

MmAllocatePagesForMdl

Post by G.
2. Map them to some VA, continguosly

MmGetSystemAddressForMdlSafe

Post by G.
3. Clear the VA mapping, but not free the phisical pages

MmUnmapLockedPages

Post by G.
7. Free the phisical pages.

MmFreePagesFromMdl/ExFreePool for MDL itself.

[G.]

I really thank you

It's a bit frustrating to post a simple and clear(sorry if i was not clear,
that is my fault, i though i was) question and to get aything but an answer

Post by Maxim S. Shatskih

Post by G.
1. Allocate a bunch of pf phisical pages

MmAllocatePagesForMdl

Post by G.
2. Map them to some VA, continguosly

MmGetSystemAddressForMdlSafe

Post by G.
3. Clear the VA mapping, but not free the phisical pages

MmUnmapLockedPages

Post by G.
7. Free the phisical pages.

MmFreePagesFromMdl/ExFreePool for MDL itself.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
http://www.storagecraft.com

[Tim Roberts]

Post by G.

Post by Maxim S. Shatskih
NT kernel has NO routine which will reallocate the memory block with
pointer value preservation and without data copy. Period.

First thing, I **did not asked that**, al least read the thread then post to
it? That is a reall challenge?
Second, You can never, ever, guarantee to grow a buffer and keep the old VA.
Third ****I DON'T CARE ABOUT THE MALLOC/REALLO/ETC... or.... PRESERVING
THE OLD VA****

Look, you jumped in on the middle of a thread entitled "Kernel
malloc/realloc" that absolutely DID start out with the exact question Maxim
referred to. Further, your first contribution to this thread, according to

Post by G.
Still, how do you perform the acctual resize?
Any reasons as to why there isn't a direct function in kernel to do this?

You did describe your algorithm a few days later, without changing the
subject line, but I think you are being a bit hard on those of us who might
have assumed the question was still about realloc.

Post by G.
In kernel mode, a dirver, how can do this steps?
1. Allocate a bunch of pf phisical pages
2. Map them to some VA, continguosly
3. Clear the VA mapping, but not free the phisical pages
4. Alocate more phisical pages
5. Continguosly in VA map the pages allocated from step 1 & 4
6. Destroy the VM mapping.
7. Free the phisical pages.
I'm curious how many more dirver gurus will spam this thread without giving
a decent answer
Or if any of you acctualy knows the answer

Maxim did give you the answer, but it is a pyhrric victory. No one would
ever really do this. Much of non-paged pool comes out of the 1:1
physical/virtual mapping that lives at 80000000. You can't change the
underlying page mapping of those pages.

Further, I will wager that this would not be faster than a new alloc and a
copy. Dinking with the page tables requires flushing the TLB, which turns
your performance to mush.

Post by G.
I guess spaming with pointless posts has something to do with the
professional development?

Ask a tacky question, get a tacky answer.

[Alex]

Post by Tim Roberts
Maxim did give you the answer, but it is a pyhrric victory. No one would
ever really do this. Much of non-paged pool comes out of the 1:1

Knolidge is power :)
even if it sems ussles at first....

Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

Post by Tim Roberts
physical/virtual mapping that lives at 80 000 000. You can't change the
underlying page mapping of those pages.

How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with
kernel memory?

Post by Tim Roberts
Further, I will wager that this would not be faster than a new alloc and a
copy. Dinking with the page tables requires flushing the TLB, which turns
your performance to mush.

Post by G.
I guess spaming with pointless posts has something to do with the
professional development?

Ask a tacky question, get a tacky answer.
--
Providenza & Boekelheide, Inc

[Ray Trent]

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

As far as I know, all the realloc functions (in user mode) that I've
ever seen have just done an alloc/copy/free unless they happened to have
extra space coincidentally in the heap right after the original allocation.

Post by Alex
How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with
kernel memory?

I think the only thing that one can say about this entire discussion is
that it proves the old maxim that 10 people that know nothing about a
topic know 10 times less than 1 person that knows nothing about the topic...

[Don Burn]

Actually, the reason I jumped in was I have at multiple times in the past
owned the realloc code for a compiler or an OS. The user space algorithm
was:

if (new_size < old_size) diddle the heap to reflect the new size plus
free space
if (new_size = old_size) do nothing
if (new_size > old_size) see if space after block on heap is free and
big enough, if not do the malloc, copy, free.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by Ray Trent

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

As far as I know, all the realloc functions (in user mode) that I've
ever seen have just done an alloc/copy/free unless they happened to have
extra space coincidentally in the heap right after the original allocation.

Post by Alex
How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with
kernel memory?

I think the only thing that one can say about this entire discussion is
that it proves the old maxim that 10 people that know nothing about a
topic know 10 times less than 1 person that knows nothing about the topic...
--
../ray\..

[Alex]

Post by Don Burn
Actually, the reason I jumped in was I have at multiple times in the past
owned the realloc code for a compiler or an OS. The user space algorithm
if (new_size < old_size) diddle the heap to reflect the new size plus
free space
if (new_size = old_size) do nothing
if (new_size > old_size) see if space after block on heap is free and
big enough, if not do the malloc, copy, free.

That is teribly suboptimal, the only reason a C/C++ runtime would do that is
because it needs to be OS portable...
...but, the OS specific win32 function GlobalRealloc dosn't need to be
portable, i really don't think it works that way (with copy alghoritm).

Meybe someone from MS can acctualy shed some light on this issue.

Post by Don Burn
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by Ray Trent

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Ray Trent

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual

realloc

Post by Ray Trent

Post by Alex
function :P) which we all use in user mode apps?

As far as I know, all the realloc functions (in user mode) that I've
ever seen have just done an alloc/copy/free unless they happened to have
extra space coincidentally in the heap right after the original

allocation.

Post by Ray Trent

Post by Alex
How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of

ram

Post by Don Burn
in

Post by Ray Trent

Post by Alex
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with
kernel memory?

I think the only thing that one can say about this entire discussion is
that it proves the old maxim that 10 people that know nothing about a
topic know 10 times less than 1 person that knows nothing about the

topic...

Post by Ray Trent
--
../ray\..

[Ray Trent]

Post by Alex
That is teribly suboptimal, the only reason a C/C++ runtime would do that is
because it needs to be OS portable...
...but, the OS specific win32 function GlobalRealloc dosn't need to be
portable, i really don't think it works that way (with copy alghoritm).
Meybe someone from MS can acctualy shed some light on this issue.

Any allocator (like GlobalAlloc) that normally allocates sub-page blocks
of memory doesn't have any other choice. What makes you think that the
address immediately after the end of your allocation, but still in the
same page, belongs to you necessarily? Among other things, remember: the
kernel memory physical/virtual mapping is the same in all processes.

Allocating 4k of memory for a "new char" wouldn't exactly be efficient
either...

Post by Alex

Post by Don Burn
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by Ray Trent

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Ray Trent

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual

realloc

Post by Ray Trent

Post by Alex
function :P) which we all use in user mode apps?

As far as I know, all the realloc functions (in user mode) that I've
ever seen have just done an alloc/copy/free unless they happened to have
extra space coincidentally in the heap right after the original

allocation.

Post by Ray Trent

Post by Alex
How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of

ram

Post by Don Burn
in

Post by Ray Trent

Post by Alex
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with
kernel memory?

I think the only thing that one can say about this entire discussion is
that it proves the old maxim that 10 people that know nothing about a
topic know 10 times less than 1 person that knows nothing about the

topic...

Post by Ray Trent
--
../ray\..

[Tim Roberts]

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and LocalRealloc/GlobalRealloc
use the new/copy/free model.

Post by Alex

Post by Tim Roberts
physical/virtual mapping that lives at 80 000 000. You can't change the
underlying page mapping of those pages.

How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with kernel memory?

What I mean is that there is a direct mapping of the first gigabyte of
physical memory at virtual address 80000000 hex. It's done using 4MB pages
in order to save page space. So, 80000000 has the DOS interrupt vectors
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

[Alex]

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and LocalRealloc/GlobalRealloc
use the new/copy/free model.

Is that a fact? Or a beleif?

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
physical/virtual mapping that lives at 80 000 000. You can't change the
underlying page mapping of those pages.

How can you map 1:1 the VA 80M, let's sey, if you don't have 80M of ram in
the system?
Or, isn't VA 80M in the first 2G of VA, which are mapped per process
context, and have mothing to do with kernel memory?

What I mean is that there is a direct mapping of the first gigabyte of
physical memory at virtual address 80000000 hex. It's done using 4MB pages

A little 'h' changes a lot of things :)

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt vectors
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

Post by Tim Roberts
--
Providenza & Boekelheide, Inc

[Tim Roberts]

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and LocalRealloc/GlobalRealloc
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with Visual
C++. You can look it up. If there's empty space following the block, it
just expands the block. Otherwise, it's new/copy/free.

Think about it for a bit. Your page table magic will ONLY work if all heap
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing up
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt vectors
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.

[Alex]

Do you even know the diffrence between a RTL function and a System API
function?

As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.

Sorry, if it's ofensive, but this is what it is

MSDN Library April 2003:
GlobalAlloc:
GMEM_MOVEABLE:
"Allocates movable memory. Memory blocks are never moved in physical memory,
but they can be moved within the default heap."

**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with Visual
C++. You can look it up. If there's empty space following the block, it
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if all heap
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing up
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt vectors
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.
--
Providenza & Boekelheide, Inc

[David J. Craig]

No, stupid is calling functions belonging to Win32, 'SYSTEM'. Windows
NT has a kernel that has several possible environments above it and one
of them is Win32. There is or was a POSIX environment. They also had a
OS/2 environment at one time because the OS/2 Brief would run under NT4.
The use of the GlobalAlloc function is not even recommended since the
overhead is high. So how does a function in kernel32.dll even belong in
the 'Development Device Drivers' newsgroup? The first question was
about allocating and reallocating memory under Windows 2000+.

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical memory,
but they can be moved within the default heap."
**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the

acctual

Post by Alex
realloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with Visual
C++. You can look it up. If there's empty space following the block, it
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if

all

Post by Alex
heap

Post by Tim Roberts
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing up
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt

vectors

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.
--
Providenza & Boekelheide, Inc

[Alex]

I'm getting really bored, really fast, Whatever, I know that...
And since you seem to support the look into RTL sources, learn kernel
programming, please share with the rest of the forum what you learn after
you look at those surces...

Post by David J. Craig
No, stupid is calling functions belonging to Win32, 'SYSTEM'. Windows
NT has a kernel that has several possible environments above it and one
of them is Win32. There is or was a POSIX environment. They also had a
OS/2 environment at one time because the OS/2 Brief would run under NT4.
The use of the GlobalAlloc function is not even recommended since the
overhead is high. So how does a function in kernel32.dll even belong in
the 'Development Device Drivers' newsgroup? The first question was
about allocating and reallocating memory under Windows 2000+.

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical

memory,

Post by Alex
but they can be moved within the default heap."
**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the

acctual

Post by Alex
realloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with

Visual

Post by Alex

Post by Tim Roberts
C++. You can look it up. If there's empty space following the

block, it

Post by Alex

Post by Tim Roberts
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if

all

Post by Alex
heap

Post by Tim Roberts
allocations are done in units of whole pages. As soon as you have

two

Post by Alex

Post by Tim Roberts
objects in the same page, you can't alter the mapping without

screwing up

Post by Alex

Post by Tim Roberts
the other objects in the page. The C run-time malloc and the Win32

heap

Post by Alex

Post by Tim Roberts
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt

vectors

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
from physical address 0, 800C0000 has the VGA BIOS at physical

address

Post by Alex

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and

operating

Post by Alex

Post by Tim Roberts
system routines rely on it. I know it's true for the NT-based

systems

Post by Alex

Post by Tim Roberts
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember

about

Post by Alex

Post by Tim Roberts
95/98.
--
Providenza & Boekelheide, Inc

[Alexander Grigoriev]

This comment is a remnant of Windows 3.1 standard mode. And even then the
words about physical memory were wrong (these were related to GMEM_FIXED).

Win32 API makes no guarantees about memory physical address.

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical memory,
but they can be moved within the default heap."
**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual

realloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with Visual
C++. You can look it up. If there's empty space following the block, it
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if all

heap

Post by Tim Roberts
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing up
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt

vectors

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.
--
Providenza & Boekelheide, Inc

[Alex]

No, it can't guarantees that the memory is not moved in physical, because as
somone pointed early, if you allocate 1 byte it is possible that the rest of
the 4k page be ussed by some one else, so if you grow the 1 byte to 1K then
it has to move the first 1 byte in a new free physical page...
...but...
If you alloc 4K then you grow the buffer, no coppy will be made, this is my
belief, this is how it should be done

Also, optimal, when allocating fractions of pages, when you grow a
allocation unit inside a 4K page, you have to look at the sizes, you can
ether copy the buffer being resized, or you can copy the rest of the 4k in
another physical page. If it is implemented like this it will be quite fast.

Besides, growable buffers are ussualy needed for streams, and low data rate
streams like HTTP over TCP/IP put a verry low strain on the whole resize
concept

Even better, on application side, you can allways allocate multiples of 4K
( if you know you don't waste space like this)

Post by Alexander Grigoriev
This comment is a remnant of Windows 3.1 standard mode. And even then the
words about physical memory were wrong (these were related to GMEM_FIXED).
Win32 API makes no guarantees about memory physical address.

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical

memory,

Post by Alex
but they can be moved within the default heap."
**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual

realloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with

Visual

Post by Alex

Post by Tim Roberts
C++. You can look it up. If there's empty space following the block,

it

Post by Alex

Post by Tim Roberts
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if all

heap

Post by Tim Roberts
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing

up

Post by Alex

Post by Tim Roberts
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt

vectors

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
from physical address 0, 800C0000 has the VGA BIOS at physical

address

Post by Alex

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.
--
Providenza & Boekelheide, Inc

[Maxim S. Shatskih]

In Win32, both GlobalAlloc and LocalAlloc are mapped to HeapAlloc.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
***@storagecraft.com
http://www.storagecraft.com

Post by Alexander Grigoriev
This comment is a remnant of Windows 3.1 standard mode. And even then the
words about physical memory were wrong (these were related to GMEM_FIXED).
Win32 API makes no guarantees about memory physical address.

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical

memory,

Post by Alex
but they can be moved within the default heap."
**"never moved in physical memory"**

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not

necesarly

Post by Tim Roberts

Post by Alex
realloc, i'm shure this thread has nothing to do with the acctual

realloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts

Post by Alex
function :P) which we all use in user mode apps?

No. Both relloc in the C run-time library and

LocalRealloc/GlobalRealloc

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
use the new/copy/free model.

Is that a fact? Or a beleif?

The source code for Microsoft's C run-time library is included with

Visual

Post by Alex

Post by Tim Roberts
C++. You can look it up. If there's empty space following the block,

it

Post by Alex

Post by Tim Roberts
just expands the block. Otherwise, it's new/copy/free.
Think about it for a bit. Your page table magic will ONLY work if all

heap

Post by Tim Roberts
allocations are done in units of whole pages. As soon as you have two
objects in the same page, you can't alter the mapping without screwing

up

Post by Alex

Post by Tim Roberts
the other objects in the page. The C run-time malloc and the Win32 heap
alloc work in units of 32 bytes.

Post by Alex

Post by Tim Roberts
in order to save page space. So, 80000000 has the DOS interrupt

vectors

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
from physical address 0, 800C0000 has the VGA BIOS at physical

address

Post by Alex

Post by Tim Roberts

Post by Alex

Post by Tim Roberts
C0000, and so on.

This mapping is documented behaviour? For what OS-es is it used?

I've never seen it documented by Microsoft, but it's fact, and operating
system routines rely on it. I know it's true for the NT-based systems
(NT/2K/XP). It used to be true on Windows 3.1. I do not remember about
95/98.
--
Providenza & Boekelheide, Inc

[Tim Roberts]

Post by Alex
Do you even know the diffrence between a RTL function and a System API
function?

You don't want to get into a war of credentials here.

Post by Alex
Isn't the same allgorithm behing the user mode reallocation (not necesarly
realloc, i'm shure this thread has nothing to do with the acctual realloc
function :P) which we all use in user mode apps?

YOU related this thread to user-mode reallocation. I answered your
question with a concrete, verifiable reference. If you have somehow
developed the idea that insulting the people who are trying to help you is
the right way to get them to continue that help, then you are sadly
mistaken.

Post by Alex
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.

Wrong. The key algorithms for most system-based functions were established
and refined decades ago. It is extremely unlikely that the API
implementation of a memory-related function will vary significantly from
the RTL implementation of that same function.

Post by Alex
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical memory,
but they can be moved within the default heap."
**"never moved in physical memory"**

As David pointed out, this comment is a leftover from the 16-bit days and
no longer applies in Win32 code.

Did you perform any actual experiments to see if what you describe is even
possible? Consider this program:

C:\tmp>type x.c

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

int main()
{
HANDLE heap = GetProcessHeap();
void* pv1[4];
int i;

for( i = 0; i < 4; i++ )
{
pv1[i] = HeapAlloc( heap, 0, 16 );
printf( "%08x\n", pv1[i] );
}
for( i = 0; i < 4; i++ )
HeapFree( heap, 0, pv1[i] );

return 0;
}


C:\tmp>cl x.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.

x.c
Microsoft (R) Incremental Linker Version 6.00.8168
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

/out:x.exe
x.obj

C:\tmp>x
00132f48
00132f60
00132f78
00132f90

C:\tmp>

The system API HeapAlloc has allocated those addresses within the same
page. You cannot alter the paging of one of those objects without screwing
up the others.

[Alex]

Post by Tim Roberts
YOU related this thread to user-mode reallocation. I answered your

The weord "kernel" in the thread title should make it quite clear what this
is about..
If i point to functions like GloballAlloc, is because some people took the
idea of acctualy reallocating a memory buffer without copy close to
blasfemy, while it is quite possible in "user mode" but was not verry clear
how to do it in "kernel mode"

Post by Tim Roberts
question with a concrete, verifiable reference. If you have somehow

You answer nothing, you pointed to the C++ Run Tim Library source code, and
that is...well...stupid
How can i learn how to realloc a memory buffer in "kernel" mode by looking
at that?

Post by Tim Roberts
developed the idea that insulting the people who are trying to help you is
the right way to get them to continue that help, then you are sadly
mistaken.

Fine, don't post anymore...whatver

Post by Tim Roberts

Post by Alex
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.

Wrong. The key algorithms for most system-based functions were established
and refined decades ago. It is extremely unlikely that the API

There is no algorithm in the RTL, the RTL is just a wrapper over some
windows API

Post by Tim Roberts
implementation of a memory-related function will vary significantly from
the RTL implementation of that same function.

Did you acctually look at this misterious RTL code? Well, i did (for reasons
unrelated to this thread) and it maps to the user mode HeapXXX windows API,
Same as GloballAlloc, as Max pointed out...

Post by Tim Roberts

Post by Alex
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical memory,
but they can be moved within the default heap."
**"never moved in physical memory"**

As David pointed out, this comment is a leftover from the 16-bit days and
no longer applies in Win32 code.

Really? this whole memory management is more of a x86 thing then a OS
dependent, the OS part is only the specific API to work with the memory, If
you ask me

Post by Tim Roberts
Did you perform any actual experiments to see if what you describe is even
C:\tmp>type x.c
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
int main()
{
HANDLE heap = GetProcessHeap();
void* pv1[4];
int i;
for( i = 0; i < 4; i++ )
{
pv1[i] = HeapAlloc( heap, 0, 16 );
printf( "%08x\n", pv1[i] );
}
for( i = 0; i < 4; i++ )
HeapFree( heap, 0, pv1[i] );
return 0;
}
C:\tmp>cl x.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for 80x86
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.
x.c
Microsoft (R) Incremental Linker Version 6.00.8168
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
/out:x.exe
x.obj
C:\tmp>x
00132f48
00132f60
00132f78
00132f90
C:\tmp>
The system API HeapAlloc has allocated those addresses within the same
page. You cannot alter the paging of one of those objects without screwing

Ok, fine... but another experiment is in order

In user mode, allocate a 1 page memory block, then keep resizing it in
multiples of 1 page
After each reallocation get the VA returned
Pass the VA to a kernel mode driver, find out the physical adress for the VA
Print the physical adress

Post by Tim Roberts
up the others.
--
Providenza & Boekelheide, Inc

[Gary G. Little]

Well folks, in a given heads-up as to whom to trust when answering
questions, I'd vote for Tim. Alex is proving himself to be nothing but an
ass with a broken keyboard and a busted spell checker. Oh ... and of course
he always has to have the last say ... so I'm going to let him have it, and
let him throw his little tantrum totally ignored.

Gary

Post by Alex

Post by Tim Roberts
YOU related this thread to user-mode reallocation. I answered your

The weord "kernel" in the thread title should make it quite clear what this
is about..
If i point to functions like GloballAlloc, is because some people took the
idea of acctualy reallocating a memory buffer without copy close to
blasfemy, while it is quite possible in "user mode" but was not verry clear
how to do it in "kernel mode"

Post by Tim Roberts
question with a concrete, verifiable reference. If you have somehow

You answer nothing, you pointed to the C++ Run Tim Library source code, and
that is...well...stupid
How can i learn how to realloc a memory buffer in "kernel" mode by looking
at that?

Post by Tim Roberts
developed the idea that insulting the people who are trying to help you is
the right way to get them to continue that help, then you are sadly
mistaken.

Fine, don't post anymore...whatver

Post by Tim Roberts

Post by Alex
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.

Wrong. The key algorithms for most system-based functions were

established

Post by Tim Roberts
and refined decades ago. It is extremely unlikely that the API

There is no algorithm in the RTL, the RTL is just a wrapper over some
windows API

Post by Tim Roberts
implementation of a memory-related function will vary significantly from
the RTL implementation of that same function.

Did you acctually look at this misterious RTL code? Well, i did (for reasons
unrelated to this thread) and it maps to the user mode HeapXXX windows API,
Same as GloballAlloc, as Max pointed out...

Post by Tim Roberts

Post by Alex
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical

memory,

Post by Tim Roberts

Post by Alex
but they can be moved within the default heap."
**"never moved in physical memory"**

As David pointed out, this comment is a leftover from the 16-bit days and
no longer applies in Win32 code.

Really? this whole memory management is more of a x86 thing then a OS
dependent, the OS part is only the specific API to work with the memory, If
you ask me

Post by Tim Roberts
Did you perform any actual experiments to see if what you describe is even
C:\tmp>type x.c
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
int main()
{
HANDLE heap = GetProcessHeap();
void* pv1[4];
int i;
for( i = 0; i < 4; i++ )
{
pv1[i] = HeapAlloc( heap, 0, 16 );
printf( "%08x\n", pv1[i] );
}
for( i = 0; i < 4; i++ )
HeapFree( heap, 0, pv1[i] );
return 0;
}
C:\tmp>cl x.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for

80x86

Post by Tim Roberts
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.
x.c
Microsoft (R) Incremental Linker Version 6.00.8168
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
/out:x.exe
x.obj
C:\tmp>x
00132f48
00132f60
00132f78
00132f90
C:\tmp>
The system API HeapAlloc has allocated those addresses within the same
page. You cannot alter the paging of one of those objects without

screwing
Ok, fine... but another experiment is in order
In user mode, allocate a 1 page memory block, then keep resizing it in
multiples of 1 page
After each reallocation get the VA returned
Pass the VA to a kernel mode driver, find out the physical adress for the VA
Print the physical adress

Post by Tim Roberts
up the others.
--
Providenza & Boekelheide, Inc

[BobF]

But what will we do for entertainment?
:-)

Post by Gary G. Little
Well folks, in a given heads-up as to whom to trust when answering
questions, I'd vote for Tim. Alex is proving himself to be nothing but an
ass with a broken keyboard and a busted spell checker. Oh ... and of course
he always has to have the last say ... so I'm going to let him have it, and
let him throw his little tantrum totally ignored.
Gary

Post by Alex

Post by Tim Roberts
YOU related this thread to user-mode reallocation. I answered your

The weord "kernel" in the thread title should make it quite clear what

this

Post by Alex
is about..
If i point to functions like GloballAlloc, is because some people took the
idea of acctualy reallocating a memory buffer without copy close to
blasfemy, while it is quite possible in "user mode" but was not verry

clear

Post by Alex
how to do it in "kernel mode"

Post by Tim Roberts
question with a concrete, verifiable reference. If you have somehow

You answer nothing, you pointed to the C++ Run Tim Library source code,

and

Post by Alex
that is...well...stupid
How can i learn how to realloc a memory buffer in "kernel" mode by looking
at that?

Post by Tim Roberts
developed the idea that insulting the people who are trying to help

you

Post by Gary G. Little
is

Post by Alex

Post by Tim Roberts
the right way to get them to continue that help, then you are sadly
mistaken.

Fine, don't post anymore...whatver

Post by Tim Roberts

Post by Alex
As it's been sed before, pointing to RTL source code to understand how
**SYSTEM** APIs work is stupid.

Wrong. The key algorithms for most system-based functions were

established

Post by Tim Roberts
and refined decades ago. It is extremely unlikely that the API

There is no algorithm in the RTL, the RTL is just a wrapper over some
windows API

Post by Tim Roberts
implementation of a memory-related function will vary significantly from
the RTL implementation of that same function.

Did you acctually look at this misterious RTL code? Well, i did (for

reasons

Post by Alex
unrelated to this thread) and it maps to the user mode HeapXXX windows

API,

Post by Alex
Same as GloballAlloc, as Max pointed out...

Post by Tim Roberts

Post by Alex
Sorry, if it's ofensive, but this is what it is
"Allocates movable memory. Memory blocks are never moved in physical

memory,

Post by Tim Roberts

Post by Alex
but they can be moved within the default heap."
**"never moved in physical memory"**

As David pointed out, this comment is a leftover from the 16-bit days

and

Post by Alex

Post by Tim Roberts
no longer applies in Win32 code.

Really? this whole memory management is more of a x86 thing then a OS
dependent, the OS part is only the specific API to work with the memory,

If

Post by Alex
you ask me

Post by Tim Roberts
Did you perform any actual experiments to see if what you describe is

even

Post by Alex

Post by Tim Roberts
C:\tmp>type x.c
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
int main()
{
HANDLE heap = GetProcessHeap();
void* pv1[4];
int i;
for( i = 0; i < 4; i++ )
{
pv1[i] = HeapAlloc( heap, 0, 16 );
printf( "%08x\n", pv1[i] );
}
for( i = 0; i < 4; i++ )
HeapFree( heap, 0, pv1[i] );
return 0;
}
C:\tmp>cl x.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 12.00.8168 for

80x86

Post by Tim Roberts
Copyright (C) Microsoft Corp 1984-1998. All rights reserved.
x.c
Microsoft (R) Incremental Linker Version 6.00.8168
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
/out:x.exe
x.obj
C:\tmp>x
00132f48
00132f60
00132f78
00132f90
C:\tmp>
The system API HeapAlloc has allocated those addresses within the same
page. You cannot alter the paging of one of those objects without

screwing
Ok, fine... but another experiment is in order
In user mode, allocate a 1 page memory block, then keep resizing it in
multiples of 1 page
After each reallocation get the VA returned
Pass the VA to a kernel mode driver, find out the physical adress for

the

Post by Gary G. Little
VA

Post by Alex
Print the physical adress

Post by Tim Roberts
up the others.
--
Providenza & Boekelheide, Inc

[Alexander Grigoriev]

I don't think it's true. A debugger doesn't show accessible memory there.

This used to be true for Linux, but I don't know if it still is.

Post by Tim Roberts
What I mean is that there is a direct mapping of the first gigabyte of
physical memory at virtual address 80000000 hex. It's done using 4MB pages
in order to save page space. So, 80000000 has the DOS interrupt vectors
from physical address 0, 800C0000 has the VGA BIOS at physical address
C0000, and so on.
--
Providenza & Boekelheide, Inc

[Tim Roberts]

Post by Alexander Grigoriev
I don't think it's true. A debugger doesn't show accessible memory there.

Which debugger? The addresses above 80000000 are in kernel space. A
user-mode debugger won't find them.

I KNOW it's true in NT/2K/XP. I made sure by using a kernel debugger
before I posted the comment.

Post by Alexander Grigoriev
This used to be true for Linux, but I don't know if it still is.

It was as of 2.4, at C0000000. I don't know about 2.6, but I'd be
surprised if things had changed.

[Alexander Grigoriev]

I'm afraid it's WinIce artifact (or caused by DisablePagingExecutive flag
set by WinIce).

I see it on a Win2K system with WinIce.

WinDbg connected to a kernel debugger on another system doesn't show such
mapping.

Post by Tim Roberts

Post by Alexander Grigoriev
I don't think it's true. A debugger doesn't show accessible memory there.

Which debugger? The addresses above 80000000 are in kernel space. A
user-mode debugger won't find them.
I KNOW it's true in NT/2K/XP. I made sure by using a kernel debugger
before I posted the comment.

Post by Alexander Grigoriev
This used to be true for Linux, but I don't know if it still is.

It was as of 2.4, at C0000000. I don't know about 2.6, but I'd be
surprised if things had changed.
--
Providenza & Boekelheide, Inc

[David J. Craig]

Congratulations you have become the only person in my blocked senders
list. Guess I will have to add what may be your other name, Bogdan
Bejan, since that was the name used on the first post. Both had the
same email address of ***@na.na.

I will have no tutorials on writing virus code, so no comment there.
Security software requires activities very similar, well OK, identical,
to virus code.

Post by G.
How do I implement some streams in a driver.
And no, I don't care is the driver is not the place for streams.

I care and won't provide an answer to a question that should have not
been asked.

Has anyone provided an answer other than some subtle and not so subtle
hints that your design is stupid or deficient in some way?

Post by G.
If i have a "virus" driver running in a system, do I reallly care about
infecting a file IN memory?
Do infecting a IN memory image really needs resizing a section? can you
really resize a section without unmapping it first(etc...etc..)?
"Should you be wrinting drivers?" :P
And sice after you the misteries of kernel memory management hide in the
realloc source code from your Visual C++ (how stupid is that?), i suppose
if I look in the fopen source code I will learn how to make file system
driver?
The fact is you don't know how to *really* answer the original

question so

Post by G.
you spam,flame,etc..
Why do you even bother to post if your not going to add any

meaningfull data

Post by G.
to the discussion?

Post by David J. Craig
I think he has decided that realloc without copying is his solution and
nothing else matters. Obstinacy doesn't make writing drivers easy and
just wastes everyone's time when they ask a question to which they don't
want a valid answer. Of course, the question could be about how do I
grab an executing image, enlarge it, and add some virus code.

Post by Don Burn
You keep asking for a realloc, and people have given you

approaches

Post by G.

Post by David J. Craig
without

Post by Don Burn
knowing the constraints you have we will just keep getting

worthless

Post by G.

Post by David J. Craig
barbs

Post by Don Burn
1. You allocate a big chunk, and manage your own heap. This is
wasteful in a general driver.
2. You can use the allocate, copy, free. But you don't seem to

like

Post by Don Burn
this.
3. You can look at your overall driver and determine if you

really

Post by Don Burn
need to use realloc. I haven't seen a need for realloc in the 10

years I've

Post by Don Burn
been in the Windows kernel, nor have the driver writers I know, so

perhaps

Post by Don Burn
your upper level design is flawed.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

Post by G.
To spam...not to spam....oh well, to spam
So, can you acctualy answer the question?

Post by Don Burn
Sure you can do this, of course you are taking a limited system

resource

Post by Don Burn

Post by G.

Post by Don Burn
(kernel PTE's) and squandering them for your driver. I guess that

is

Post by Don Burn
ok,

Post by G.
if

Post by Don Burn
your driver is in an embedded or limited environment, but if this

is a

Post by Don Burn

Post by G.

Post by Don Burn
driver for a general commecial environment, this is crap.
I am friends with several folks who perform the acceptance of

drivers,

Post by Don Burn
for

Post by G.

Post by Don Burn
large firms who are buying parts OEM, they tell a number of

amusing

Post by Don Burn
tales

Post by G.
of

Post by Don Burn
their firms telling a supplier to completely rewrite their driver

(because

Post by G.

Post by Don Burn
of bad code like this, or using a toolkit that large firm won't

bless)

Post by Don Burn
and

Post by G.

Post by Don Burn
giving them a short time to rewrite. Unless you are just playing

at

Post by Don Burn
this,

Post by G.

Post by Don Burn
or know your driver is a one of a kind, write it based on standard
techniques.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply