Posted on by Jason Kemp

P/Invoke, COPYDATASTRUCT, and something weird in unmanaged land.

Dare recently wrote on the limits of the .NET framework and why he can’t wait for Longhorn. I tend to agree. It’s like going to a movie theatre to watch a movie, only to find out that you have to learn how setup the speaker orientation to get the movie to play properly.

I have been doing a lot of unmanaged stuff recently, first with COM Interop, do deal with the AxWebBrowser class, and now I’m going full tilt with a full blown ActiveX Control and P/Invoke. I have to pass messages to the ActiveX Control from managed code. Having very liltle experience with unmanaged code ( I come from Java ) , I am a little intimidated, especially when things don’t work as expected. In order to pass messages to this ActiveX control, I created an invisible top level window, so that I can find it via FindWindow,(while we’re here, P/Invoke is really easy and simple, IF you know the library function to call AND how to marshal the types); that took some time, mostly on the unmanaged side.

In order to send the invisible window a message, I call SendMessage with WM_COPYDATA message. Should be pretty straightfoward except I ran into the same kind of problem that Dare did. I had to implement COPYDATASTRUCT. My first try was like so:

public struct COPYDATASTRUCT {
  public int dwData,
  public int cpData,
  public IntPtr lpData
}

My google-fu did me a service though because I found this article on CodeProject that fixed my first problem but revealed a new one. Before I get to that though, let me summarize the article: What one has to do to transfer a string from a managed app to an unmanaged app in the manner above is implement COPYDATASTRUCT like so: 

public struct COPYDATASTRUCT {
  public int dwData,
  public int cpData,
  public int lpData
}

Note the third parameter’s type. Now we send the string over to the dark side: 

string s = "hello unmanaged window";
//Allocate memory
IntPtr ptr = Marshal.AllocHGlobal(s.Length);
//Convert to array of bytes
byte[] b = Encoding.Default.GetBytes(s);
//copy contents to unmanaged memory
Marshal.Copy(b, 0, ptr, b.Length);

//Allocate COPYDATASTRUCT
COPYDATASTRUCT cds = new COPYDATASTRUCT();
cds.dwData = 0;
cds.cpData = b.Length;
cds.lpData = ptr.ToInt32();

//Call SendMessage....

Now this works, I call SendMessage and my unmanaged code gets the string, but a whole lot more. Appended to the string is a bunch of gobbledygook. I assume it’s looking for the null-terminator, but I figure .NET would be smart enough to handle this. You know what they say about assuming…

But that’s not the weird part: if we make the string to pass less than eight characters, we get the exact string in unmanaged code as we had in managed!!

<Update>
Ok, I’ve found the problem and fixed it. I’ll show the code, and then explain a few things about what was going wrong

string s = "hello unmanaged window";

//Allocate memory and copy string
IntPtr ptr = Marshal.AllocToHGlobalAnsi(s);

//Allocate COPYDATASTRUCT
COPYDATASTRUCT cds = new COPYDATASTRUCT();
cds.dwData = 0;
//important! add one for the null terminator
cds.cpData = s.Length + 1;
cds.lpData = ptr.ToInt32();

//Call SendMessage....

//Free the memory
Marshal.FreeHGlobal( ptr );

First, you should notice that I use a different method on the Marshal class. I was doing this earlier, but I didn’t think it mattered so I decided to go with what the CodeProject article had done. But if you read the documentation, Marshal.AllocToHGlobalAnsi() allocates and copies, so that kills a few lines (always a good thing). So in my code, I was calling AllocToHGlobalAnsi() and copying the array with Marshal.Copy(). So that is one problem fixed. Next, I had to add one to the length of the data being copied, for the null terminator that is needed for C++ strings, something that the call to AllocToHGlobalAnsi() also does for you (at least the way I did it (CStrings were something I was looking at, before I solved it this way)). Now it works for any string.

AllocHGlobal() or AllocToHGlobalAnsi() is a bit of misnomer if you don’t know what Global means in this case. Global means global to the process, in which every Windows program executes. I was under the (naive) impression that it was global to the machine, a giant shared memory space in which every process can play. What actually happens when one sends a WM_COPYDATA message is the following: the COPYDATASTRUCT is marshalled across the two processes. This means that it will de-reference the lpData pointer; grab the data up to cbData bytes; copy it and send it to the other process. The receiving process rebuilds the COPYDATASTRUCT by allocating cbData bytes, dumping the data there, and setting the lpData pointer to the data. So if you don’t add one to the length, the unmanaged program will keep going along in memory until it finds a null terminator. This is nothing like the bug that Dare found with unmanaged code; it’s actually a novice programmer’s first few experiences with the dark side. If it continues like this, I don’t think I’ll be tempted by it.

</Update>

I haven’t had an opportunity to get my hands on a Longhorn preview (alas, I am but a poor student), but I’d be curious how all this would work in completely managed code

CategoriesUncategorizedTags