I have been going through the AsmTools code and I came across a few things that didn't make sense in these two files.

First, is push ecx and pop ecx necessary in str_compare? I've tested it, and I don't see a need for it.

    global str_compare
str_compare:
    push ecx
sc_lp:
    cmp byte , 0
    je sc_done1
    cmpsb
    je sc_lp
    jmp short sc_exit
sc_done1:
    cmp byte , 0
sc_exit:
    pop ecx
    ret


And str_len with comments of what I had changed it to.

    global str_len    ;    global _strlen
str_len:                ;    strlen:
    push eax          ;    push eax
    push esi          ;
    mov ecx, esi      ;    mov ecx, esi
str_lp:                ;    strlen_l:
    lodsb              ;    lodsb
    or al, al          ;    or al, al
    jnz str_lp        ;    jnz _strlen_l
    dec esi            ;    sub esi, 1
    sub esi, ecx      ;    sub esi, ecx
    pop ecx            ;
    xchg esi, ecx      ;    xchg esi, ecx
    pop eax            ;    pop eax
    ret                ;    ret

I had tested this, too and it seemed to work.

Anyway, I'm just curious to know if there is/was some reason for the extra code?
Posted on 2010-02-07 01:21:00 by Mol_Bolom
Your improvements to Jeff's code look sound to me (untested!). In the str_cmp, the push/pop of ecx is probably a "leftover" from an earlier version that used "rep cmpsb"(?).

Jeff's str_len is kinda weird, in that it returns the length in ecx and preserves eax. (what was he thinking?). Your improvements there look sound, too. We've "preserved" esi in ecx, so no need to push/pop it.

In Linux, we often want the length in edx. With a standard "C-style" strlen, we need to mov edx, eax. With Jeff's str_len, we need to mov edx, ecx... and probably mov ecx, esi, too.

S'pose we had an "astrlen", which would optionally allow a "destination" (must be a register) where the length is returned. If only one parameter is given, the length is in eax.

astrlen mystring ; like C's strlen()

mov esi, mystr
astrlen ecx, esi ; like Jeff's str_len

mov ecx, mystring
astrlen edx, ecx  ; length in edx - ready for a sys_write

Being a macro, not a subroutine, this pastes the "whole thing" in, every time you use it. But if you look closely, there isn't a heck of a lot of code involved in the actual "business" of finding the length

I'm not much of a macro user, much less a macro writer, but here's my shot at it.


%imacro astrlen 0-*

; single operand - eax is implied dest
    %if %0 = 1
%ifidni %1, eax
    %error "single operand form - eax is implied destination!!!"
        %endif    

or eax, byte -1
%%getlen:
cmp [%1 + eax + 1], byte 1
inc eax
jnc %%getlen

; two operand form
    %elif %0 = 2
%ifidni %1, %2
    %error "src and dest must not be the same!!!"
%endif

; fake an "%ifnreg"

%assign %%isreg 0

%ifidni %1, eax
    %assign %%isreg 1
%elifidni %1, ebx
    %assign %%isreg 1
%elifidni %1, ecx
    %assign %%isreg 1
%elifidni %1, edx
    %assign %%isreg 1
%elifidni %1, esi
    %assign %%isreg 1
%elifidni %1, edi
    %assign %%isreg 1
%elifidni %1, ebp
    %assign %%isreg 1
%endif

%if %%isreg = 0
    %error "destination must be a GP register!!!"
%endif
   
   
or %1, byte -1
%%getlen2:
cmp [%1 + %2 + 1], byte 1
inc %1
jnc %%getlen2
    %else
%error "usage: strlen src (reg/label) or strlen dest (reg) src (reg/label)."
    %endif
%endm



I'm sure this can be improved, but I don't think you'll find any extraneous pushes in it. Of course, zero-terminated strings are a dumb data structure anyway. Generally, we know the length - either it's static, in which case Nasm can count it for us, or we read it from someplace, in which case we know what we read. If you're doing strlen a lot, retink your code.

Best,
Frank

Posted on 2010-02-07 16:06:18 by fbkotler

Jeff's str_len is kinda weird, in that it returns the length in ecx and preserves eax. (what was he thinking?). Your improvements there look sound, too. We've "preserved" esi in ecx, so no need to push/pop it.

In Linux, we often want the length in edx. With a standard "C-style" strlen, we need to mov edx, eax. With Jeff's str_len, we need to mov edx, ecx... and probably mov ecx, esi, too.

S'pose we had an "astrlen", which would optionally allow a "destination" (must be a register) where the length is returned. If only one parameter is given, the length is in eax.

astrlen mystring ; like C's strlen()

mov esi, mystr
astrlen ecx, esi ; like Jeff's str_len

mov ecx, mystring
astrlen edx, ecx  ; length in edx - ready for a sys_write



That's one thing I've been working on, I've yet to standardize which register to use for what.  ecx seemed the best bet for any kind of count because of the loopcc instructions, however, now that you've mentioned it, would be more prudent to use edx for string lengths.

So far I've organized the registers this way.
EAX: Return value of a call
ECX: Loop, count, etc
ESI:  Main string, or the 'source' string.
EDI:  Secondary, or the 'destination' string.
Stack: I've only started using it more often lately.  I try not to use it if I can, though.

And I'll start using EDX for string lengths.  That will simplify a few things.

Well, I'm still a beginner, so maybe I'll learn a better way.  I've been reading through several sites on the internet, Intel manuals, as well as trying to read through the pcasm book again, still got a ways to go.


Being a macro, not a subroutine, this pastes the "whole thing" in, every time you use it. But if you look closely, there isn't a heck of a lot of code involved in the actual "business" of finding the length

I'm not much of a macro user, much less a macro writer, but here's my shot at it.


%imacro astrlen 0-*

; single operand - eax is implied dest
    %if %0 = 1
%ifidni %1, eax
    %error "single operand form - eax is implied destination!!!"
        %endif    

or eax, byte -1
%%getlen:
cmp [%1 + eax + 1], byte 1
inc eax
jnc %%getlen

; two operand form
    %elif %0 = 2
%ifidni %1, %2
    %error "src and dest must not be the same!!!"
%endif

; fake an "%ifnreg"

%assign %%isreg 0

%ifidni %1, eax
    %assign %%isreg 1
%elifidni %1, ebx
    %assign %%isreg 1
%elifidni %1, ecx
    %assign %%isreg 1
%elifidni %1, edx
    %assign %%isreg 1
%elifidni %1, esi
    %assign %%isreg 1
%elifidni %1, edi
    %assign %%isreg 1
%elifidni %1, ebp
    %assign %%isreg 1
%endif

%if %%isreg = 0
    %error "destination must be a GP register!!!"
%endif
   
   
or %1, byte -1
%%getlen2:
cmp [%1 + %2 + 1], byte 1
inc %1
jnc %%getlen2
    %else
%error "usage: strlen src (reg/label) or strlen dest (reg) src (reg/label)."
    %endif
%endm



I'm sure this can be improved, but I don't think you'll find any extraneous pushes in it. Of course, zero-terminated strings are a dumb data structure anyway. Generally, we know the length - either it's static, in which case Nasm can count it for us, or we read it from someplace, in which case we know what we read. If you're doing strlen a lot, retink your code.

Best,
Frank




Someday soon I'll have to sit down and read through the Nasm manual and learn how to read and write macros.  I've done it once before, and just put it off until after I memorize the instructions a little better.  One thing that really bugs me about macros, since I'm learning, and if I use a macro, while debugging a program I'll come across some code and not have a clue why, what, or how it got there  :shock:.  So I try to stay away from them for the time being.


zero-terminated strings are a dumb data structure anyway. Generally, we know the length - either it's static, in which case Nasm can count it for us, or we read it from someplace, in which case we know what we read. If you're doing strlen a lot, retink your code.


While trying different methods to figure strings out, I've found that to be too true. Unfortunately I don't know an easier way to get the string lengths of command line arguments, too bad there isn't a empty string passed, then I could just use a pop and get all the string lengths from pop'ing the next arguments addr and subtracting it from the preceding arguments addr.

Hmm, just happen to think that the computer, OS, or whatever should know at least the length of the command line arguments.  It has the program directory and name, plus the arguments in memory, so there should be a way to find those out, and from that I could retrieve the arguments and their lengths easier. Perhaps that's what you meant by static, I am not sure (The hardest part to learn is the terminology)...

Well, thanks for the info.  Gives me something to work on.
Posted on 2010-02-08 20:06:36 by Mol_Bolom
Well I'm not sure "static" is the correct terminology. Maybe I should have said "initialized". I just meant that we can do:

msg db "Hi There!"
msg_len equ $ - msg

But command line arguments and environment variables come to us as zero-terminated strings - Linux being written in C, and all. I don't know if you can get the length by subtracting one address from another or not. It would work if the strings are located contiguously in memory... which they may be, but I don't know if we can count on it.

If you can spot those unneccessary instructions in Jeff's code, I'd say you were off to a good start!

Best,
Frank

Posted on 2010-02-09 21:02:06 by fbkotler
After I had written my last post I had searched static up, and you were right.

I've found myself using Jeff's style a little more often...
msg: db 'Howdy!'
end_msg:

mov edx, end_msg - msg

It's not perfect, but it does seem useful, say I want to create an array.


help:  db  'This is program what not                        ', 10
help_01:  db  '    -h              print help                      ', 10
help_last:db  '    -v              print version                  ', 10
end_help:


That way I wouldn't have to remember too much.  Thus for a write...

mov ecx, help
mov edx, end_help - help

; Or for only a single line...
mov ecx, help * linenumber
mov edx, end_help - help_last



I don't know if you can get the length by subtracting one address from another or not. It would work if the strings are located contiguously in memory... which they may be, but I don't know if we can count on it.

That's the million dollar question, can it be counted on?

So far everything I've tested it does work, however if there is a chance that it wont be like that always, then the code will not work.

Such that when I wrote this code an hour ago, it would be useless if the arguments weren't stored contiguously.


_get_arg:
pop ebp ; Since only testing, don't need to worry about this.
pop esi ; Get arg...
cmp ecx, 1
jz _get_arglen ; If ecx = 1 then we have to get the arg length by scanning the string.

mov edx, ; Get next arg...
sub edx, esi ; Get arg length+NULL.
dec edx ; Now string length...
jmp _end_get_arg
_get_arglen:
mov edx, esi
_get_arglen_l:
lodsb ; Get byte from esi
or al, al
jnz _get_arglen_l
xchg edx, esi
sub edx, esi
dec edx
_end_get_arg:
push ebp
ret


More to think about.

Thanks
Posted on 2010-02-09 21:53:57 by Mol_Bolom
I'm a little confused by your code. Where does the value in ecx come from? Passed in as a parameter? Your use of ebp is... unusual. Why not "mov esi, ", and leave the return address on the stack? Or maybe I don't understand how this is intended to be used.

In any case, keep thinkin' and keep codin' - you can't go wrong!

Best,
Frank

Posted on 2010-02-10 01:25:53 by fbkotler
Whoops. Well, I don't get to talk with others about this kind of thing, so I often forget to explain things.

Anyway, ecx was used with LOOP and stored the argument count, argc in c. For the most part, ebp &ecx weren't important in this example, it was the rest of the code that was, just showing that as long as the arguments are contiguous in memory then that's one way that they could be retrieved, possibly faster than parsing each string.


Your use of ebp is... unusual. Why not "mov esi, ", and leave the return address on the stack?


I thought I had read that it is a good idea to remove everything from the stack, so nothing existed after program execution. So I didn't think it would be a good idea to leave the arguments on the stack, and so one of my thoughts was to use something such as this to pop each arg off the stack for testing.  I guess that leaves me with another question, is it ok for the stack to remain after a program exits?
Posted on 2010-02-10 10:27:36 by Mol_Bolom
Mol_Bolom"]…is it ok for the stack to remain after a program exits?


Whole program or function? Modern OS releases resources of terminated program (including stack). As for function, it's the matter of calling conventions.

For example, cdecl function leaves it's parameters on stack, stdcall function discards them (using ret # instruction, usually).
Posted on 2010-02-10 11:57:49 by baldr
When your program exits, your stack, along with all your other memory, just vanishes - poof! The magic of virtual memory.

A program that begins at the "_start:" label has no return address on the stack, so we don't have to worry about that. We couldn't "ret" if we wanted to! A "call" instruction puts a return address on the stack, and the "ret" instruction expects to find it there, so the stack is critical in a called function. "main" is called. (and a Windows entrypoint is called, AFAIK) We can exit our program with "ret", in this case - and the stack has to be correct. If we exit with sys_exit, or with exit(), the stack can be trash.

A more "usual" subroutine would look like:


somefunction:
push ebp ; save caller's ebp, they're probably using it
mov ebp, esp
; body of our function
mov esp, ebp
pop ebp
ret


Sometimes the "enter" and "leave" instructions are used instead of explicit code like this - look 'em up. Now, in the body of our function, we can abuse esp any way we want, and it'll be restored to a sane state from ebp. Typically, "sub esp, ?" is used to reserve space for local variables, for example. It means we can't trash ebp, of course!

Your code, where I said your usage of ebp was "unusual", pops the return address into ebp, and pushes it back before the "ret". You indicate that's "not important" - but it is! That allows you to pop the next arg from the "original" stack. Works, but it isn't the "usual" way, and confused me at first.

Here's an example I happened to have on hand, that shows a different approach. This scans through environment variables instead of command line arguments, but it's the same idea...


; nasm -f elf myfile.asm
; ld -o myfile myfile.o

; finds the USER environment variable, and says hi,
; using sys_writev - write several buffers at a time.

global _start

section .text
_start:
    nop
commence:
    mov eax, ; "argc"

; "lea eax, [8 + esp + eax * 4]" is start of environment variables
; 4 bytes for each arg in argc, plus argc itself, plus the terminating 0
; since we're going to be adding 4, calculate 4 short of that.
    lea eax, [4 + esp + eax * 4]

; fancy search algorithm :)
finduser:
    add eax, byte 4
    mov ecx,
    test ecx, ecx ; this just sets flags so we can...
    jz end_of_env ; terminating zero at end of env array - not found
    cmp dword , 'USER'
    jne finduser
    cmp byte , '='
    jne finduser

; got it, advance to "value"
    add ecx, byte 5

; find the length
    or edx, byte -1
getlen:
    inc edx
    cmp byte , 0
    jne getlen

    mov , ecx
    mov , edx
    jmp short write_it

end_of_env:
    mov , dword whodat
    mov , dword whodat_len

write_it:
    mov eax, 146 ; __NR_writev
    mov ebx, 1 ; stdout
    mov ecx, my_vector ; ptr, len, ptr, len...
    mov edx, 3 ; three items to write
    int 80h

exit:
    mov eax, 1 ; __NR_exit
    int 80h

;-----------------------------------
section .data

    greet db "Hello, "
    greet_len equ $ - greet
   
    coda db "! Welcome to Linux Assembly!", 10
    coda_len equ $ - coda
   
    whodat db "Unknown User"
    whodat_len equ $ - whodat
   
; sys_writev takes a "vector", ptr, len, ptr, len... N times.
    my_vector:
dd greet
dd greet_len
username:
        dd 0 ; fill in at runtime
name_len:
        dd 0 ; fill in at runtime
        dd coda
        dd coda_len

;------------------------------


I guess that doesn't really address your question about the stack, since I pretty much leave it alone. I'm not really comfortable with popping argc, etc. off the stack. What if I've forgotten something, and want to go back and look again? :) If I were to "mov , esp" first thing (which I don't do in this example), I'd be able to find args and env from anywhere - even in deeply nested subroutines. Dunno, just a different approach...

If you were to keep popping, past all the args and environment variables, when you finally got to 0xC0000000, you'd segfault, I think. Try it...

Best,
Frank

Posted on 2010-02-15 06:18:19 by fbkotler
A program that begins at the "_start:" label has no return address on the stack, so we don't have to worry about that. We couldn't "ret" if we wanted to!


Actually, that depends on the OS.
Obviously the '_start:'-label isn't the beginning of all code. This code is called by the executable loader of the OS. In many cases (eg DOS/Windows), this loader stores a valid return address on the stack, so a ret will indeed exit your program cleanly. But yes, you have to be cautious with such an approach.
Posted on 2010-02-15 06:34:34 by Scali
Right. I should have said, "a Linux program that starts with the label _start:" (or non-default entrypoint supplied to ld). In this specific case, it's "jmp"ed to, not "call"ed. Good catch.

Best,
Frank

Posted on 2010-02-15 07:15:09 by fbkotler
Thanks a lot fbkotler.  Running that little example I was finally able to finally understand LEA.  I've been reading on that for quite some time, but couldn't grasp it.  Though I still need to tinker around with it to figure out how your example compiled and mine didn't, wasn't too different, I think, unfortunately I don't remember where I put that test program...

Anyway...


When your program exits, your stack, along with all your other memory, just vanishes - poof! The magic of virtual memory.

Strange how it took me so long to figure that out...Eh, at least I know now...



somefunction:
push ebp ; save caller's ebp, they're probably using it
mov ebp, esp
; body of our function
mov esp, ebp
pop ebp
ret


Sometimes the "enter" and "leave" instructions are used instead of explicit code like this - look 'em up. Now, in the body of our function, we can abuse esp any way we want, and it'll be restored to a sane state from ebp. Typically, "sub esp, ?" is used to reserve space for local variables, for example. It means we can't trash ebp, of course!

Yeah, I learned that early on when I first read about macros.  The only macros that I bother using, actually, are contending with local and passed variables and entering and exiting a call.


Your code, where I said your usage of ebp was "unusual", pops the return address into ebp, and pushes it back before the "ret". You indicate that's "not important" - but it is! That allows you to pop the next arg from the "original" stack. Works, but it isn't the "usual" way, and confused me at first.

Well, that's my fault, I should have mentioned that it was merely an example and those two registers didn't matter in that particular example. At least I know what your talking about... ;)


I guess that doesn't really address your question about the stack, since I pretty much leave it alone. I'm not really comfortable with popping argc, etc. off the stack. What if I've forgotten something, and want to go back and look again? :) If I were to "mov , esp" first thing (which I don't do in this example), I'd be able to find args and env from anywhere - even in deeply nested subroutines. Dunno, just a different approach...

If you were to keep popping, past all the args and environment variables, when you finally got to 0xC0000000, you'd segfault, I think. Try it...


The last paragraph, is exactly what was causing me the confusion.  I didn't have time to try any test programs for finding different ways to traverse the stack without actually pop'ing the stack, been rather busy, but I had thought about it.

The part that was the hardest to figure out was argc. So far, with what little time I've had to think on it, this is what I have so far come up with.
    Pop argc, need to have another register to compare it to.

    Leave argc on the stack, and read the arguments backwards.  Thus multiplying it by four, and subtracting it by four after every loop then using it to add to the stack to read the arguments. Therefore would not need any registers for the counter, but argc would be lost.

    Push a variable onto the stack, one that would be used for the counter.  Thus would have something to test with argc. Would only need one more 4.


Though, after reading through your example and all, I realized I maybe just plain nuts.  :D . Anyway, I should have this Wednesday to examine your code more closely and think about it and how I might go about it more clearly then.

Thanks.

Charles
Posted on 2010-02-16 00:07:06 by Mol_Bolom
Ok I finally got it, so it seems. Though now I know I really need to study up on the debuggers I use (Oh what fun was that debugging this code... :shock: )...

Anyway, this is what I got so far...


_getarg:
lea ecx,
mov ecx,
call _strlen
ret

_getenv:
or edx, edx
jz _getenv_len
lea ecx,
mov ecx,
call _strlen
jmp _end_getenv

_getenv_len:
inc edx
add ecx, byte 4
mov eax,
test eax, eax
jnz _getenv_len
_end_getenv:
ret

_strlen:
cmp byte , 0
je _end_strlen
xor edx, edx
_strlen_l:
inc edx
cmp byte , 0
jne _strlen_l
_end_strlen:
ret


I still want to work on the _genenv part, but eh, reality bytes sometimes.

So far I modified the code so I'd only need to remember two registers, ecx and edx.  Which now I try to use edx for array indexing, strings only. Of course this may change, but eh, I remember it and that's the important thing.

So this is the main program code that I've tested it with, it's not pretty, but it was just a spur of the moment kind of thing only needed to test the calls.


segment .data
argc: dd 0
argv: dd 0
env_begin: dd 0
env_count: dd 0
newline: db 10

segment .text
global _start
_start:
mov eax,
mov , eax      ; Save argc
mov ebx, esp
add ebx, 4
mov , ebx      ; Save stack address of arguments.

lea ecx, [4+esp+eax*4] ; Should be the beginning of the environ
mov , ecx        ; Save stack address of environ
xor edx, edx
call _getenv
mov , edx      ; Save # of environ variables.
mov edx, 1
push edx
_envloop:
mov ecx,
call _getenv
call _printout
call _printnewline
inc dword
mov edx,
cmp edx,
jnz _envloop
mov , dword 1
mov edx, 1
_argloop:
mov ecx,
call _getarg
call _printout
call _printnewline
inc dword
mov edx,
cmp edx,
jnz _argloop
pop edx              ; Clear the pop, though I know now I don't need to do so.
_exit:
mov eax, 1
mov ebx, 0
int 80h

_printnewline:
mov ecx, newline
mov edx, 1
_printout:
mov eax, 4
mov ebx, 1
int 80h
ret


And lastly, I've created two others, but haven't tested them yet.  These two calls were something I created that could be utilized in other programs as well.
What these "should" do is, _lcodestr "something=else" should be ecx="something" and edx = strlen("something"), and _rcodestr should be ecx = "else" and edx = strlen("else").


_lcodestr:
inc edx
cmp byte , '='
jnz _lcodestr
ret

_rcodestr:
inc ecx
cmp byte , '='
jnz _rcodestr
inc ecx
call _strlen
ret


Anyway, it's not pretty, but for only having a few minutes a day to work on it for the past several days, it's not too bad, IMO...

Again, thanks for everyone's help.
Posted on 2010-02-19 13:04:46 by Mol_Bolom
Mol_Bolom,

lea ecx, / mov ecx, is somewhat artificial way to write mov ecx, .

test edx, edx is better way to test.

_getenv(0) assumes that environment is not empty.

_strlen explicitly checks for zero-length string, yet returns undefined value in edx.

_lcodestr again assumes that edx will be zero upon call.

Both _lcodestr and _rcodestr don't check for terminating NUL.
Posted on 2010-02-21 12:06:50 by baldr