Welcome, Guest. Please Login or Register.
OpenDX User Forum
09/08/10 at 04:36:18
News: Quality paid support available from Visualization and Imagery Solutions


Pages: 1
Threaded OpenDX w/ better memory management (Read 3557 times)
gda
Forum Newbie
*


Got Viz?

Posts: 4
Threaded OpenDX w/ better memory management
10/22/09 at 13:02:17
 
Anybody interested in a threaded version of OpenDX?  By replacing the multi-process model with threads, I've been able to get rid of the complex shared-memory management and removed the limitation to 2GB max memory.  Although any individual data object still has to be <2GB, the cache can grow to any arbitrary cap.  Also alleviates the memory-fragmentation issue.
 
In the process of doing the above, I've fixed what seems to be an egregious bug in cache management, too.
Back to top
 
 
Email   IP Logged
dmoroian
Full Member
***


If you like CFD,
then you probably
like OpenDX!

Posts: 150
Re: Threaded OpenDX w/ better memory management
Reply #1 - 10/28/09 at 01:48:51
 
Wow, somebody still intrested in developing OpenDX! Would you mind making the sources available (full or just patches)?  
 
Dragos
Back to top
 
 
WWW   IP Logged
gda
Forum Newbie
*


Got Viz?

Posts: 4
Re: Threaded OpenDX w/ better memory management
Reply #2 - 11/24/09 at 11:45:41
 
Sorry about the delay.  I've created a SourceForge repository (sourceforge.net/projects/opendx) containing my updated source files.  There are no binaries yet; right now, please only use svn access:
 
svn co https://opendx.svn.sourceforge.net/svnroot/opendx opendx
 
There are quite a lot of changes here.  The most significant are:
 
       - Changed multi-process model for multi-threading.  This enabled me to remove shared-memory management.  Defaults to 1 thread; add '-processors n' to get more.
 
       - 64-bit addressing.  If built on a 64-bit machine, you can tell it to use a lot more memory.  You still specify a memory limit (default 2GB) so that caching happens without filling all available swap space -  I've been using 16GB (-memory 16000).  This entailed a lot of changes, particularly in the EDF code (thanks, Marylin).   Though I haven't seen any, I think there'll be problems if any individual allocation exceeds 2GB. Probably ought to throw an error.
 
There are a bunch of bug fixes, too.  The biggest was that it was never automatically flushing the cache.  When it hit the memory limit, it would recover a small bit of memory by flushing the dictionary (don't ask) and would then limp along a bit longer.  Now it will flush the cache and continue.  There's a lot of room to be more intelligent about cache flushing.
 
These are substantial changes, and I've not been able to really test them thoroughly.  I've been running on a 4-core Fedora R11 system, and have also tested somewhat on cygwin.  I'd appreciate whatever feedback I can get.
 
One outstanding problem has arisen since I've been able to run it a whole lot longer.  With many more objects in the cache, there have been cache collisions - two objects, created by different modules, can get the same cache tag.  This causes downstream modules to raise errors when they receive the wrong object as input, and it can look pretty odd.   I've only seen it when running a somewhat complicated network in a 30K sequencer loop.  I think I can fix it by implementing open hashing in the cache, but that'll take some work.
 
Please try it out.  To build from SVN, you need to enter the root directory (trunk) and run:
 
aclocal
autoheader
libtoolize
automake -a
autoconf
 
And then in a separate object directory, run {path to trunk}/configure.  I'll look in here (the opendx.org mailing lists and user forum) from time to time, but for the time being, I've creates a mailing list at SourceForge for discussion of the SF version.
 
Thanks,
Greg
Back to top
 
 
Email   IP Logged
joelr
OpenDX Forum Administrator
*****


Got Viz?

Posts: 313
Re: Threaded OpenDX w/ better memory management
Reply #3 - 11/25/09 at 04:32:33
 
Hey Greg,
 
This is great. I'm sure this is going to be a huge hit (potential bugs,
notwithstanding). On behalf of everyone, thanks.
 
Joel
Back to top
 
 
Email   IP Logged
dmoroian
Full Member
***


If you like CFD,
then you probably
like OpenDX!

Posts: 150
Re: Threaded OpenDX w/ better memory management
Reply #4 - 11/25/09 at 07:37:55
 
...that everyone includes me, too!
 
Dragos
Back to top
 
 
WWW   IP Logged
dmoroian
Full Member
***


If you like CFD,
then you probably
like OpenDX!

Posts: 150
Re: Threaded OpenDX w/ better memory management
Reply #5 - 11/30/09 at 02:39:16
 
Hello Greg,
Unfortunately I could not complete the "configure" part Sad. I get the following error:
Quote:

configure: error: cannot find macro directory `m4'

 
Any clues?
 
Dragos
Back to top
 
 
WWW   IP Logged
gda
Forum Newbie
*


Got Viz?

Posts: 4
Re: Threaded OpenDX w/ better memory management
Reply #6 - 11/30/09 at 08:39:37
 
Dragos,  I bet this is an automake/autoconf version issue.   Googling around, it looks like some versions expect there to be a 'm4' directory in the project root directory - or maybe the object root directory.  Could you make try making those and seeing if it fixes it?  BTW - I'm using automake 1.11 and autoconf 2.63.
 
Greg
Back to top
 
 
Email   IP Logged
dmoroian
Full Member
***


If you like CFD,
then you probably
like OpenDX!

Posts: 150
Re: Threaded OpenDX w/ better memory management
Reply #7 - 12/21/09 at 11:06:33
 
Hi Greg,
Yes it worked. I had to create an empty "m4" directory in the source directory.
Sorry for the late response.
Now I have to find some time to test it.
 
Regards,
Dragos
Back to top
 
 
WWW   IP Logged
cloudvis
Forum Newbie
*


Got Viz?

Posts: 14
Re: Threaded OpenDX w/ better memory management
Reply #8 - 02/18/10 at 11:28:51
 
I am definitely interested in using this, although I'll have to try to find someone to install it for me as I am a complete dummy on that end. Any news about further testing ?  
 
I found interesting the comments about the cache collision problem that seemed to arise in this new version, because I think I have seen a similar thing happen in the old version. A couple times I have gotten a fatal run error indicating that object types don't match or that a macro cannot operate on the given object type. Upon further investigation, I find that instead of the field object that is expected, the string "red" or something similarly bizarre is being feed down the pipe. This occurred within a network that is stable, being run hundreds or thousands of times with no problems of this sort.  
 
And about the cache not being flushed properly, I want to point out my experience that when DX does run out of memory, not even "Reset Server" works to restore it. Does anyone else concur with that observation ? I always have to take the more drastic action of Disconnect from Server to restore a peace conditon. Also I seem to recall that in (much) older versions, a proper warning was given when memory was running or had run out. Now the protocol seems to be that DX just starts acting totally weird and you have to waste a bunch of time trying to figure out what the problem is.  
 
Lastly, could the caching problem you refer to be the cause of another Most Annoying DX Behaviors ? Certain macros (Compute likely the biggest contender) get corrupted sometimes and have to be replaced with a fresh icon. Again, often there is an misguided error about object types not being compatible, etc. The problem is not limited to this condition, but it seems to occur mostly after a syntax (human) error is typed in interactively as the operational statement, and then corrected upon discovery. Again, Reset Server does nothing to flush or fix the corruption. I have to say I am pretty sure Disconnect from Server doesn't work either. Replacing the corrupted icon in the VPE is the only thing that does the trick. This has happened so many times that I have learned to be quick to replace icons before wasting too much time trying to figure out whatever else could be wrong. Another offender has been the Image macro which has on occasion not reset the camera upon updated images (yes, Reset Camera was set), although not so much recently. Maybe that has been fixed (I would suspect indirectly since I've never heard mention of it) somewhere along the way.
Back to top
 
 
Email   IP Logged
dmoroian
Full Member
***


If you like CFD,
then you probably
like OpenDX!

Posts: 150
Re: Threaded OpenDX w/ better memory management
Reply #9 - 03/09/10 at 00:35:47
 
Hello again,
The first superficial tests didn't bring up any abnormal behaviour, so far so good.
I don't recollect ever to encounter a cache collision problem (object types don't match error). However, I have experienced the memory limitation (due to cache or some other reason), then I was forced to restart the server.
Also I have never had the problem that couldvis mentions with the necessity of replacing a module in the VPE.
 
I will continue the tests with bigger meshes, and let you know about the results.
 
Dragos
Back to top
 
 
WWW   IP Logged
cloudvis
Forum Newbie
*


Got Viz?

Posts: 14
Re: Threaded OpenDX w/ better memory management
Reply #10 - 04/05/10 at 09:06:24
 
Not able to get this due to our firewall. Is there possibly another way to obtain it ?
 
Pretty please ?
 
Sharon
Back to top
 
 
Email   IP Logged
cafmike
Forum Newbie
*


Got Viz?

Posts: 1
Re: Threaded OpenDX w/ better memory management
Reply #11 - 04/13/10 at 20:21:52
 
could you use a proxy?
Back to top
 
 
Email WWW   IP Logged
Pages: 1