SD 2.1: error processing extended attributes: Cannot allocate memory

[dnanian]

OK. Attached is a little "prune_log" script that should work.

To use, unzip the attachment to an appropriate folder. (I suggest your personal Applications folder, but it's up to you.) Ensure that it looks like an "Unix executable file" when viewed in the finder.

Now, open SuperDuper! In the Advanced tab of Options, there's a selection under "After copy" to run a shell script when the copy completes. Check the box, and select the "prune_log" executable in the open panel.

That'll automatically remove any warnings from your system log on completion, assuming there were no errors during the run. All of these entries in the log are warnings, not errors, generated by the call I've spoken of earlier in the thread. Logging to the regular SuperDuper!.log will continue as usual, and no pruning will be done if an error occurs.

(For those who are interested, I tried to limit the pruning to just the SDCopy entries, but while the regular expression matching worked fine to restrict viewing to those entries, it wouldn't work to prune them, and I couldn't hardwire the location of SDCopy, which did work, because the user could have SuperDuper! installed anywhere.

What I attempted was:

syslog -p -k Sender Req .\*SDCopy

Without the -p, that only shows SDCopy entries. With the -p, it should prune those entries, but doesn't, and gives no errors. You'll need to put "sudo " in front of that if you want to try in Terminal, at least with -p.)

I've tested this here and it does properly prune the asl.log of these entries; let me know if it helps you out.

[Cuneyt]

Same problem! And my asl.log is 67 MB curently.

[Cuneyt]

After the script, my asl.log is 359 Kb. But, I noticed that SuperDuper! 2.1 was using 430 Mb of memory while running.

[richardl]

wow .. Just looked at my asl.log and it's 92mb .. Ran the script (through Terminal) and pruned it to 700kb .. Thanks

[dnanian]

430MB of memory doesn't mean much, actually, because it includes shared components and stuff... we're generally pretty frugal.

[sdsl]

Thanks for the quick response with the script! That was really fast.

By the way, I don't think the memory usage noted above is serious nor is it connected to the writing of messages to the log file. The messages are probably written a line at a time, which should have no effect on memory.

[priller]

Originally Posted by DaleMeyn

Are any of you having these problems running Macaroni?

Yes, Macaroni 2.0.7

[priller]

Originally Posted by dnanian

OK. Attached is a little "prune_log" script that should work.

FWIW ..... The 56,000 lines per Smart Update are in my /var/log/system.log. Since there is no "Level", the script has no impact.

/Applications/SuperDuper!.app/Contents/MacOS/SDCopy: error processing extended attributes: Cannot allocate memory

Thanks for continuing to look into this.

[dnanian]

Yes, I understand. This is to deal with the asl.log, not the system.log. As I indicated, we're looking into working around the logging.

The message is exactly the same, though, and is a warning (which you can see with syslog).

[Cuneyt]

Of course I am not sure but I feel like that the increasing memory problem seems like related with this excessive logging.

First, it started with SD! 2.1, and it was not the case with 2.0.1. If we think that SD copies files from one drive to another drive why the memory usage would increase continuously? Also, I am not sure whether the system writes the log messages one by one. There may be a buffer for those, again I am not sure.

May be this problem should be opened in another topic. But, as I noticed the memory usage is around 180 Mb after SD launched. And when SD starts copying it also starts and continue to increase. 180 ... 200 ... 220 ... 300 and upto 480 Mb. I wonder what will it be if the copy would continue. And it returns to 180-200 after SD finished with copying. 480-180= 300 Mb. If the author of SD would say that 300 Mb memory usage is normal for a copy operation, I would say nothing.

[george]

priller writes, "FWIW ..... The 56,000 lines per Smart Update are in my /var/log/system.log....."

Thou not necessarily the final solution, does not running an app like Macaroni (as you mention) flush/keep the size of your system.log file under control?

[priller]

Originally Posted by george

Thou not necessarily the final solution, does not running an app like Macaroni (as you mention) flush/keep the size of your system.log file under control?

Marcaroni insures that the system.log file is rotated every 24 hours, it does not reduce the size of the daily's.

[sdsl]

There were two issues discussed in this thread:

* asl.log growing in size due to many messages being written (harmless messages generated by Mac OS X Tiger)

* system.log growing due to similar messages

The asl.log issue is resolved by the neat little script (prune log) that was provided in an earlier post by the forum administrator. It works nicely.

I think the system.log file size issue is resolved by the DAILY maintenance script. I ran it manually and before doing so my system.log had been ~ 10 Meg having grown to that size just after running SD, but after the maintenance scripts the largest system.log (including the archived ones) was under 100 kbytes, which is nothing.

So SD has provided an automated cure to the asl.log growth (just add the script to your SD run), and it looks like to me that the automated Tiger maintenance scripts (which can be activated manually via terminal or via a free program like MacJanitor), which run every night automatically, somehow shrink the size of the system.log.

SD is not the only program that affects these log files this way. I think it's a defect in the way Mac OS X handles these log files. Until Apple cleans this up, I think it's really a non issue because Dave's prune-log script and the maintenance system scripts together remove the excess file sizes.

[priller]

Log file rotation is not a resolution, it just masks the problem.

The size, in bytes, of the file is not the issue. It's the number of log entries. If you inject 56,000 lines of crap into the syslog, it makes it that much more difficult to look for real issues, like a hacker attack.

[dnanian]

We're not satisfied with the prune_log workaround, and continue to try to figure out a way to stop it from logging entirely. Sorry for the time it's taking...