Home → Problems →

Long saving time

• Edit

beshoy.agayby

22 Oct, 2019 03:05 PM

Hi Chris,

If you remember, a while ago, i had an issue with data not being saved. While this issue is resolved; I have a different one now. Mworks takes ages closing the data file (more than 1.5 hours at this moment and going). The file size so far is about 3.6GB. I am using v 0.9 as well. Any idea what causes this? if it's unavoidable, is it possible to have another instance of Mworks to start my next experiment?

Cheers,
Beshoy

1. Support Staff 1 Posted by Christopher Sta... on 22 Oct, 2019 08:05 PM

   

   Hi Beshoy,

   It sounds like the same root problem as before: At some point in the experiment, the rate at which events are written falls drastically below the rate at which events are generated, producing a huge backlog, and the data file can't close until the backlog is cleared. Please see the linked comment for my suggestions on how to proceed.

   Cheers,
   Chris

   • Edit

2. 2 Posted by beshoy.agayby on 23 Oct, 2019 10:57 AM

   

   Hi Chris,

   I checked the data read and write rates. During the experiment. THe data read/sec is around 40KB and the written is about 13-25MB. Are those normal numbers?

   Cheers,
   Beshoy

   • Edit

3. Support Staff 3 Posted by Christopher Sta... on 23 Oct, 2019 07:25 PM

   

   Hi Beshoy,

   During the experiment. THe data read/sec is around 40KB and the written is about 13-25MB. Are those normal numbers?

   Yes, those are totally reasonable I/O rates.

   Can you try running the special MWorks build I provided previously? The log file it generates should provide some insight into what's happening.

   Chris

   • Edit

4. 4 Posted by beshoy.agayby on 27 Nov, 2019 11:26 AM

   

   Hi Chris,

   I tried this special build, but i am not sure how and where to find the log file to send you.

   Cheers,
   Beshoy

   • Edit

5. Support Staff 5 Posted by Christopher Sta... on 27 Nov, 2019 03:20 PM

   

   Hi Beshoy,

   It should be in the same place as before: /tmp/mwserver_event_file_log.txt.

   Thanks,
   Chris

   • Edit

6. 6 Posted by beshoy.agayby on 28 Nov, 2019 01:19 PM

   

   Hi Chris,

   I attached the log file. Again, it took hours to finish saving the file.
   Thanks in advance for the help.

   Cheers,
   Beshoy

   • mwserver_event_file_log.txt 25.4 MB
   • Edit

7. Support Staff 7 Posted by Christopher Sta... on 02 Dec, 2019 08:41 PM

   

   Hi Beshoy,

   Thanks for the log file. I'll take a look and see if it provides any new insight into the problem.

   Chris

   • Edit

8. Support Staff 8 Posted by Christopher Sta... on 05 Dec, 2019 09:02 PM

   

   Hi Beshoy,

   The log file confirms that writing events to disk is taking an unreasonably long time (e.g. 100ms or more per 1000 events, where a reasonable duration would be <10ms). I can induce this problem on my Mac Pro by using a stress-testing tool that loads the system with disk writes.

   I've also discovered that I can eliminate the problem, even with the stress-testing tool running, by setting SQLite's "synchronous" flag to "OFF". Disregarding for a moment whether this is really a good idea, it'd be interesting to know if this change resolves the issue for you, too. If you're willing to try it, I've created a modified build of MWorks that you can get at

   https://www.dropbox.com/s/chojes7fa5eepl9/MWorks-sqlite_synchronous...

   It's identical to the current nightly build, except for the change to the synchronous flag.

   I should note that, when I start the stress-testing tool, Activity Monitor's data written/sec figure jumps from around 60MB to 700MB or more. Since you aren't seeing a similarly high write volume, you may be experiencing a different issue. Still, I think this is a worthwhile test to run.

   Thanks,
   Chris

   • Edit

9. 9 Posted by beshoy.agayby on 07 Dec, 2019 05:29 PM

   

   Hi Chris,

   Thanks for the modified version. I will test it on Monday and let you know how it goes!

   Cheers,
   Beshoy

   • Edit

10. 10 Posted by beshoy.agayby on 09 Dec, 2019 06:27 PM

    

    Hi Chris,
    This worked! File size looks appropriate and I took a quick look at the events and things seem in order.

    Cheers,
    Beshoy

    • Edit

11. Support Staff 11 Posted by Christopher Sta... on 10 Dec, 2019 09:22 PM

    

    Hi Beshoy,

    That's great news! I have to think a bit more about whether this should be the default configuration going forward, although my current feeling is that it should.

    Cheers,
    Chris

    • Edit

12. Support Staff 12 Posted by Christopher Sta... on 07 Jan, 2020 03:43 PM

    

    Hi Beshoy,

    I have to think a bit more about whether this should be the default configuration going forward

    I thought about it and decided that this is the best default. The change is now in the nightly build and will be included in the next MWorks release.

    Cheers,
    Chris

    • Edit

13. Christopher Stawarz closed this discussion on 10 Jan, 2020 04:44 PM.