ok I want suggestions on howto best design a master that processes data and sends part of its job to slaves on my homenetw(small renderfarm)
and slaves sends back result when finished
Posted on 2006-06-01 16:47:50 by daydreamer
Let's do some brainstorming, but first:

- do  you want/need security?
- will this always be lan-only, or could it be useful across the internet as well?
- how large do you reckon commands/data and send-back data will be?

...
Posted on 2006-06-02 04:21:46 by f0dder
Hi
This is a topic I'm very interested too.

- I'm not sure, but I think that if we restrict it to a LAN, security is not an issue
- For the first step, LAN should be our target
- Difficult to say at this moment.

Biterider
Posted on 2006-06-02 04:27:52 by Biterider
For communications, it's tempting to use UDP if you're on a LAN and don't need security, because "it's faster than TCP". But if extremely low latency and maxed-out bandwidth isn't necessary, I'd advise to use TCP instead. It's safer, a bit easier to handle since you've got state, and easier to expand (security, internet) in the future.

Two-way communication might be a good idea. Processing nodes ("clients") report to the server and say "hi, I'm available". Might be a good idea if the server can query some estimated performance value from the client.

Also a good idea if server can query current client state. If a client doesn't respond in a timely manner, the server can query what it's up to. If the client says "I'm not rendering, you fool", server should ask another client to handle that particular request. This also implies the server must maintain a list of currently running tasks and which clients are handling them.

...
Posted on 2006-06-02 04:45:17 by f0dder
Well the topic does sound interesting...

Since it's over LAN, we have all agreed that security is not an issue. f0dder is right - I think it is better to use TCP (due to the reliability factor) as I dont think that much data is gonna be transferred around

So lets say we have a server A. A's main duty would be to assign instructions to the other computer that is connected to A. Once the computers connected to A is done with the task, they will report back to A that they are done and A will reassign new jobs.

BUT what kind of asm-distributed comp on LAN? Are we cracking the RSA numbers?  ;)
Posted on 2006-06-03 01:28:25 by roticv

(small renderfarm)


I think he wants to compete with DreamWorks and Pixar :P
Posted on 2006-06-03 01:31:22 by SpooK
Why bother? I rather be earning money from cracking RSA numbers  ;)
Posted on 2006-06-03 01:43:18 by roticv
I already have 3d apps that can use netw render and hardware to use it
and now my interest in asm, isnt there a better way todo it more optimized to render animations, after learned RTRT techniques ala 4d rendering all rays that hit the same object/texture for all frames
and instead of huge send copy of all data at beginning to all clients and split work ala tile image evenly among clients, do it different, send only enough data to each client for one object to work on for all frames in animation at a time?

forgive my ramblings, first gonna start easy as client/server which sends some data back and forth and hey you calculate sin and sendback result or you calculate cos, input from this array I transfered to you


Posted on 2006-06-03 06:51:07 by daydreamer
I have thought this over and every renderer be in form of custome Virtual machine and instructions sent are bytes that feed a jump table
for example
TRACE 0 180 0 90 LIGHTMAP
opcode for VM's trace tell to trace rays between those angles with a lightmap, because its textured light
ligthmap follows 
LOAD TEXTURE
etc

philosophy:work while receiving, so send data in custom structures for best access in order used in renderer
so no dependency on data is slowing it down, for example sending first a chunk of meshes and after that wait for hires ,texture before be able to render but recieve meshes and data between UV coordinates for that section and start to work on first pixel that comes in
in CG art, often a big texture is UVmapped in several areas and whitespace between those arent nesserary to transfer

so a package could be a header section with instructions followed by a bigger data sector


Posted on 2006-06-17 03:53:57 by daydreamer
Most renderfarms I've seen work one of two ways:

A) each frame is Tiled, and the rendering of JUST that tile is allocated to some LAN slave.. the server receives complete Tiles and reconstructs each rendered Frame, compositing them to some kind of animation.. often the complete Frames are processed by a video codec on the Server and composited into an AVI or other video format.

B) each frame is allocated to some LAN slave.. the server compiles some kind of animation from the complete Frames as above.

The resources required to render any two consecutive Frames are likely to be the same, and as such, an opportunity for Caching of resources has presented itself..


Posted on 2006-06-17 09:53:38 by Homer