This is part of some code that assembles and runs fine on XP Pro SP2.
It was assembled using Masm 6.14.

It failed when run on a Vista 64-bit system.

When I tried to assemble it using Masm 8.0.50727.104, I got
"invalid instruction operands movd MM0," on the asterisk marked lines.

Can someone help me understand why it no longer works?

Thanks.

    movq MM2,MM1
    movq MM1,MM0
*  movd MM0,        ; loop moves backward
    punpcklbw MM0,MM7
    paddw MM2,MM0
*  movd MM4,
Posted on 2009-07-16 14:29:27 by skywalker
32-bit pointers on a 64-bit system? It can work only in 32-bit applications running under WOW64.
Posted on 2009-07-16 15:02:32 by ti_mo_n

32-bit pointers on a 64-bit system? It can work only in 32-bit applications running under WOW64.


The bigger question is, did he assemble/link it for 32-bit?

I would imagine that he did, especially considering the use of MASM 6.14.

On 64-bit Windows, 32-bit PE programs are (automatically ?) run on WOW64, in which is only a compatibility layer on top the fact the x86-64 architecture supports the running of unmodified 32-bit applications, during 64-bit Long Mode, in what is called Compatibility Mode.

In short, the symptom is probably more due to a bug in MASM 6.14 than the fact that he is running 64-bit Vista.

Try using DWORD/QWORD PTR with the instruction instead.
Posted on 2009-07-16 15:18:00 by SpooK
Posted on 2009-07-16 15:21:46 by SpooK
Thanks, I will try your suggestion.

Posted on 2009-07-16 15:31:11 by skywalker



In short, the symptom is probably more due to a bug in MASM 6.14 than the fact that he is running 64-bit Vista.

Try using DWORD/QWORD PTR with the instruction instead.



I have attached the source and .exe.

I just changed the instructions that Masm 8.0 complained about.

There may be other instructions that need changing so that it will run on 64-bit machines.

It will be a while until I can try it out on a 64 bit machine using Vista.

Take care,

Andy

P.S. I would be curious as the the CPU usage of this program on other process
     On a .386 with a 733 Mhz, it close to 100%.

Attachments:
Posted on 2009-07-16 16:23:00 by skywalker

P.S. I would be curious as the the CPU usage of this program on other process
      On a .386 with a 733 Mhz, it close to 100%.
It's probably going to run 100% CPU usage on anything single-core... on my quadcore Q6600@3GHz, Win7 64bit, it runs at ~25% CPU usage (meaning a single core maxxed out) - if you want less CPU usage, set a fps limit :)
Posted on 2009-07-18 08:34:38 by f0dder
Except for these 3 lines, I have corrected the "Fireworks" code so that Masm 8.0
doesn't complain about "invalid instruction operands." The original code assembled fine under Masm 6.14.

I am trying to fix these 3 lines and have the program still run in the original video mode
which I think is 24 bit.

I am also trying to figure out what I need to look at in Ollydbg
to see the differences in what is in the registers right after movd ,MM0 with what is
in them when movq , MM0 is substituted.

Thanks.

    movd ,MM0        ; If this is changed to movq , MM0 , then
                                        ; Masm 8.0 no longer complains, but the video mode is changed
                                        ; to a lower resolution ??
    movd ,MM1    ; && 
    movd ,MM2  ; && line 284 Last line that Masm 8.0 is saying
                                        ; "invalid instruction operands"
Posted on 2009-07-21 18:50:23 by skywalker