At first: Privalov, this thread is not directly relate FASM, so I don't know is here is the right place for it, but because of "in development" state of FASM, it's possible for you (or someone else) to implement this ideas.

So, yesterday I think about macroses and macro languages. It's strange that all macro assemblers have builded some macro engine more similar to HLL than ASM. Why? The difference between macro and code is only in the time of execution. Macro code is running during compilation but program code is running during execution of program. But this diference is not principle. On the other hand the power of macro engine (I mean capabilities for programming) always is poor in comparison with the native language.

The idea: Why not to use ASM language to write macroses. This code will be executed in the compilation time and will not be inserted in output file. The goal of this code is to generate some text in the source of the program, just like every macro. For example: in the input of macro: edi will points to the text of program, where macro is called and esi will point to some structure with macro arguments, so the macro will insert some text in , and then the assembler will compile the generated source.

The advantages of this idea are:
1. Removing of macro engine - the assembler and the macro engine are the same program.
2. Very powerful macro capabilities. Everything what you can imagine is possible.
3. Simple recursive structure of assembler.

The disadvantage:
1. Maybe a little more complex writing of macroses. But this is relative. For assembler programmers, assembler is simple and beautiful more than some HLL.
Posted on 2002-12-11 00:55:43 by JohnFound
Macros have to communicate with the assembler: the macros reserved word give orders to the assembler...
Talking with macro only in assembly would have a very limited effect...

In fact, it would be possible for commonly used operations (add, sub, xor, etc) to use pure assembly, but one advantage of macros is to be maintenable and easy to read, and quick to write: It is advantages of HLL, so i
t seems appropriated here, and as you said, this code will not be included in the executable: no impact on the compiled program performance...

I don't use macros very much but when I write them, I don't want to bother myself with register preservations issues or to clear some register before a division...
I don't know how the "pseudo-assembly" (macro) and the "real assembly" code are going to react to each other... (the macro told "mov ebx, value", but the program already uses ebx... who trust now ?).

Note also that macros are also used to create "pseudo" inlined functions with real assembly code inside of them: distingish all of this would be a mess, both for the assembler and the programmer...

I am not as experienced as some in writing compilers and assemblers, so it is just my 2 ?...
Posted on 2002-12-11 01:09:00 by JCP
As far as the asm macro will be started by assembler itself, the assembler will wrap the macro code with some other code:

1. preserving registers
2. set initial values of some registers ( esi, edi in my example )
3. exception handling

The macro code will be called via call, so it must ends with ret, and this ret will be inserted by assembler.

Here is small example, ilustrating just idea:

[size=12]

; edi points to source code of the program, where macro is called.
macro MyNOP {
code
mov esi,.instext
.loop:
lodsb
test al,al
jz .exit
stosb
jmp .loop
.exit:

data
.instext db 'xchg ax,ax',$0d,$0a,0
}
[/size]


BTW: At least there is no conflict between clasic macroses and asm macroses. It's possible to introduce new directive: "asmmacro" for new sintax.
Posted on 2002-12-11 01:49:25 by JohnFound

Either nobody noticed it, or nobody was interested to it, but some days ago I proposed something similar in the last post of this thread:

http://www.asmcommunity.net/board/index.php?topic=9297
Posted on 2002-12-11 02:33:40 by Maverick
JohnFound, I have a question.

Are you talking about "fictitious" registers during the macro execution?

If the macro is to be executed during assembly time, the registers are used by the assembler itself.
In order to process the macros, the assembler will be loaded with an interpreter more similar to a x86 emmulator.

What do you think?
Posted on 2002-12-11 02:36:54 by pelaillo
To pelaillo:

Why we need emulator, while we have the processor and assembler? The asm macroses will be assembled by the same assembler as the whole other code, only in the some temporary memory, and will be called by the assembler. The assembler will preserve the registers before calling macros, will load the registers and/or stack with macro arguments and destination source buffer adn take care about exceptions during execution of macro, so there must be no problems at all.

So, the whole idea is still a little raw, but i think it may be clarified.
Posted on 2002-12-11 02:55:13 by JohnFound
JohnFound:
It is clear now. It will be very powerful, but needs to be assisted by some library of string manip. functions (dynamically linked?) to avoid writing very large asm-macros.

Maverick:
I never see that thread. As the board increases in size, the information becomes unaffordable. :.(
Posted on 2002-12-11 03:31:17 by pelaillo
Hola Pelaillo :)
Maybe I'm saying the obvious, but I use much the board function:

http://www.asmcommunity.net/board/search.php?s=&action=getnew

Otherwise it would be really too messy to check each forum manually. :P

---

To return in topic, as I wrote in my post on that thread, and anyway even apart that, I fully agree that a custom code generator can be much more flexible than any macro system.. because a macro system to be that much flexible should be native code anyway, so no point anymore to keep them as two separed languages.
JohnFound: maybe we could all join forces and work on something like what I described in that thread? There are a lot of coders on this board that would produce tons of useful plugins.. be it DIV optimizations, or OOP frameworks, etc.. etc.. etc..
Posted on 2002-12-11 04:16:22 by Maverick
Hi Maverick.

I read your post carefully, but you have a slightly diferent idea.

I must say, that I don't like command line tools and modular systems (like UNIX/LINUX) where we have a hundreds of tools and must execute them one after other. (or write tons of bat file or use some other tools like "make" to get the result) This, I think, is the past of the computers.

Why not to build one entirely complete development system and make the computer work for us. And more, as I understand your idea, the plug-ins will be compiled modules, but I think that every macro must be accessible via his source code.
Posted on 2002-12-11 04:43:57 by JohnFound
Maverick, unfortunately the getnew are renewed too fast by tons of notices about worlds, polls and planets...I could connect only few times each month...

A common framework for plugins is an excellent idea, let's do it. Personally, I prefer the asm-macro working after some preprocessor activities (such as constant and text substitutions, normal fasm macros ...) and not before.
Posted on 2002-12-11 04:46:41 by pelaillo

Hi JohnFound/Pelaillo:
Yup, ideally having this functionality built in FASM would be better.. let's see if Privalov likes the idea.
In any case, whatever is the final solution, I would advice it to be powerful enough to produce code as in the "magicdiv" example.. i.e. to be rather flexible, more than actual MACROs are (the specific magicdiv example could be handled by a MACRO, but it would probably have to be big and hardcoded.. while instead some code to generate it would be better, rather than a switch-style MACRO block with all the possible solutions hardcoded and ready to be inlined).
Whatever, when MACROs start to become that flexible, they become a programming language by themselves.. hence it would be smart to just make macros in asm or in any powerful HLL language.
Posted on 2002-12-11 05:49:11 by Maverick

Either nobody noticed it, or nobody was interested to it, but some days ago I proposed something similar in the last post of this thread:

http://www.asmcommunity.net/board/index.php?topic=9297
:) I stated long ago my belief that the macro language should be the assembly language (ie no macro language to learn). My assembler will be designed around this idea.

http://www.asmcommunity.net/board/index.php?topic=1392&highlight=macro+langauge
http://www.asmcommunity.net/board/showthread.php?s=&postid=485&highlight=macro+language.msg485
Posted on 2002-12-11 08:37:07 by bitRAKE
Again you have reinvented the idea I was already considering in the fasm development, and I wanted something even more powerful - some kind of plugin engine for various plugin routines compiled at assembly time. But in the simplest variant it would just compile additional instruction handlers, and so follow the same rules - JohnFound's macro would look like:


opmacro MyNOP
{
mov ax,0C031h
stosw
}

The main reason why I've abandoned it was that it would completely break the compiler stability - because source would be allowed to do everything (when it's written in assembly, the only limitation are these from protected mode OS, and with real mode fasm even those don't exist), and all programmer's mistakes (or even intended "bad" behaviour of some other programmer giving you his source). Well, you are anyway risking things like that when you run compiled program after the assembly, but I wanted to separate it from the "safe" and OS independent compilation process.
Posted on 2002-12-12 06:29:08 by Tomasz Grysztar
Hi Privalov.

At this time, i think you don't understand completely my idea. Your "opmacro", as I can see, writes to the binary output of the assembler, but my "asmmacro" writes not binary instructions, but source text. So "opmacro" is executed on the "assembler" pass and "asmmacro" is executed on the "preprocessor" pass of FASM. This is the diferrence.

About the stabiliti and safeness of the code: I think it's possible for assembler to wrap the macro procedures with some kind of wraper code which will protect stability of the assembler, handling all exceptions. And if the macro writes only in the source code, which will be parsed and assembled at the next steps of the assembling, IMHO the stability will be acceptable.
Posted on 2002-12-12 08:11:18 by JohnFound
It was just the example how would it be done in the simplest way, macros generating source code would be allowed in the general plugin interface I thought about, and also formatting plugins, etc.
And the exception handling is impossible just because the basic fasm version runs in real mode (actually in FRM, but it's possible I'll release also unREAL version soon), and there exception handling is an utopia. And such plugin interface couldn't be guaranteed to be OS-independent as the rest of fasm, FRM version can execute 16-bit code only, on the other hand it's hard to execute 16-bit code from the Win32/Linux program. Forgive me, I'm not touching it now. But if someone wants to write his own - for example Win32-only - assembler based on fasm with such capabilities, I can help him, providing the neccessary information about fasm architecture (well, maybe I should write once again fasmGuide, this time complete and actual for fasm 1.42).
Posted on 2002-12-12 08:33:48 by Tomasz Grysztar
Everyone in this thread,
A MACRO works best when it just generates inline code based upon a set of given parameters. When you try to make a defined MACRO do fancy things like HLL, complex function calls, or exemption handling, you are competing with the programmer and the assembler itself. The assembler is far more capable of doing those things due to its dynamic processing capability and global knowledge of the destination addresses and data. If your MACRO is THAT smart, you will have written a part of the assembler in MACRO language. The area for more MACRO endeavor is feature enhancement. Examples: The ability to read its parameters in reverse easily, without defining another MACRO. The ability to output user messages in the listing (not the code itself) on a conditional basis. The ability to control and format its listing. The ability to output inline functions. Ratch
Posted on 2002-12-31 22:01:22 by Ratch
Ratch, what about features falling between the assembler and the programmer?

1. Not including duplicate constant data in readonly segments.

2. Encrypting a proceedure during assembly-time with a custom encryption algorithm. (The encryption algorithm changes based on the length of the proceedure). {Compression of code as well.}

3. Substituting other instructions for a constant divide/multiply.
Posted on 2003-01-01 12:07:58 by bitRAKE
bitRAKE,
I consider all your suggestions, and upon further consideration, a couple of mine I listed previously, to be something the main assembler and not the macro facility should implement. There is no reason to limit all or part of our wish list to just macros. Ratch
Posted on 2003-01-01 12:31:00 by Ratch
Ratch, each time I change the encrytion/compression algo I should create a new assembler? Please, help me understand how these features can be best handled by the assembler. I'm not saying putting these features in the assembler is not possible - just that it is not practical. I do understand the need to keep functionality separate, but believe this creates more work or eliminates possible features.

Analogously, this can be seen as a scriptable plug-in interface. Why doesn't Photoshop have all the features built in - why use plug-ins? Why does office have VBA? Why are there batch files for DOS/WIN? Maybe it is the word MACRO that causes the mis-understanding? I just want a way to extend the assembler using assembler at assemble-time.
Posted on 2003-01-01 16:25:28 by bitRAKE
bitRAKE,
I don't know how encryption relates to assembly, but if it does, one should be able to at least enter the keys as EQUs. Probably encryption/compresson is best done outside of assembly. Perhaps as a pre-processor. Ratch
Posted on 2003-01-01 18:18:25 by Ratch