nonstop HD activity-freezing

UBUNTU 10.4.1 AMD 64 on Acer 5003WLMi Turion64 ML32 1.8Gh, 1Gb 400Mh RAM

I’m convinced I’ve got some Ubuntu virus or bug (yeah, I know they don’t exist) cause I was using unsecured neighbor wifi for 7 months (now got secure Internet): the HD just goes on rants of activity- 5 minutes and system semi-freezes with limited mouse movement and delay of any change, opening or closing of any program. It’s so bad and annoying I sometimes REISUB it. May be connected w FF, but has continued quite a while even after closing.

Anyway I scanned it with a new AVIRA Boot AV CD and got these results (didn’t complete cause it started doing all the other drives in Media- how to prevent that?). Is this history thing important (50 warnings), what is it, endless recursions bad, and can I delete it without consequences (I LIKE history)??? How about changelog? Full scan file attached.

WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/clamav-testfiles/clam_cache_emax.tgz → clam_cache_emax.tar → clam02.tgz → clam02.tar → clam03.tgz → clam03.tar → clam04.tgz → clam04.tar → clam05.tgz → clam05.tar → clam06.tgz

WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/doc/libxvidcore4/changelog.gz → changelog → object → object → object → object → object → object → object → object → object

WARNING: [Archive not completly scanned. Reason: maximum recursion level (10) reached] /media/Devices/hda9/usr/share/doc/w3m/ja/HISTORY.gz → HISTORY → object → object → object → object → object → object → object → object → object

All those warnings mean is that Avira can only scan 10 directories deep (maximum recursion level (10) reached) … it doesn’t mean you have a virus.

If you think it’s connected with Firefox … try clearing the Firefox cache.

But at a guess I’d say you have either a failing hard drive (bad sectors that need to be re-read several times before the data is read properly), or a screwed up file system (try running fsck).

Does “SMART Data” in “Disk Utility” suggest any errors ?

Well the results are strange. This Samsung 160GB disk was never too good, and initially had such weird numbers here without
being flagged that I didn’t believe it + ignored it. So they’re right + my disk is crap?- well I bought it in 2008 Ukraine where they dump all kinds of defective crap. Oh, man. Better back up files.

3 bad sectors; only warning in red is: CURRENT PENDING SECTOR COUNT: 3 waiting to be remapped;

UNCORRECTABLE SECTOR COUNT: 3

It says “GOOD” for read-write errors, but with a value of 1271733 Normalized 102, Worst 72, Threshhold 6,

SEEK ERROR RATE: “Good” , Value 8776831664, Norm: 79, Worst 60, Thresh 30

HARDWARE ECC Recovered: Value: 199191785, Norm 73, WOrs: 57, Thresh 0

LOAD/UNLOAD CYCLE COUNT: 96880, NORM 52, Wor: 52, Thresh: 0

POWER OFF RETRACT COUNT: 781, Norm 100, Worst 100, Thresh: 0

All GOOD: Power On 198 days (of almost 2 years); Power Cycle Count 1945: Start-STOP count: 1796

How do I just force it to ignore the bad sectors (how big is that)?

IS instruction: sudo fsck ? Is that analysis or FIXING IT?

whats the proper FSCK instruction to analyze and fix disk or file system: SDA9. Is the “check file system” on Disk Utility any good? Can I figure out which partition these errors are on?

DO NOT run fsck on a mounted disk/partition

First try this …

run:

sudo touch /forcefsck

then reboot.

That will run fsck at bootup

To run fsck on particular partitions (and ones not listed in fstab), you’ll want to boot to a LiveCD/LiveUSB (so the partitions aren’t mounted) … then let’s say you want to run fsck on /dev/sda1

sudo fsck /dev/sda1

As I said … DO NOT run fsck on a mounted partition.


Just a thought, but whilst booted to the hard drive … what’s the output from:

free -m

free -m
total used free shared buffers cached
Mem: 870 842 28 0 15 130
-/+ buffers/cache: 696 174
Swap: 1048 457 591
mako4@mak4-laptop:~$

Would it even permit fsck on a mounted disk?

Would it even permit fsck on a mounted disk?

From memory, I think it will … but only after warning you not to … but I could be wrong.

Or maybe it will but only if you use the “force” (-f ?) option … I can’t remember ???


It looks like you’re VERY low on memory … when you ran free -m was that with a load of applications open ?

Your system was already accessing the swap partition, which will cause “disk thrashing” and system slowdown as stuff is constantly swapped out of RAM to the hard drives swap partition (which is MUCH slower than RAM).

With only 1GB RAM, you’d probably be better off with a “light” distro such as Peppermint/Lubuntu/Crunchbang/etc.

Or just fit some more RAM :wink:

1 gb should be fine for Ubuntu. Even Windows XP runs OK, but Ubuntu was always lighter. Yeah it’s been open for a while and I’ve opened and closed disc burning software (think that eats up all memory). Wonder if have some bad RAM? Other times I ran that free program, it didn’t show any swap use. Thought Ubuntu doesn’t leak or retain memory loss if you close programs

But what about those horrible disc stats- think maybe these endless writes are trying to access bad sectors. What are risks of fsck- can it destroy big chunks of data, or trash OS? Seem to remember, I had these lousy stats before but other disk analysis programs in Windows said it was fine. What is difference between fsck (cleaner operation?) and using “check (+fix) file system” in disk utility?

What are risks of fsck- can it destroy big chunks of data, or trash OS?

If you are running fsck as Mark suggested then the risk is minimal (no worst than a failing drive), run:

sudo touch /forcefsck

then reboot. This will run the check on the boot up.
Ubuntu should automatically run the check on every 20 or so boot up if you got the flag set in your /etc/fstab:

/dev/sda1	/	ext4	rw,errors=remount-ro	0	1

Notice the 1 at the far right. If yours is 0 then change it.

You can also reduce the swappiness:
First see the current setting (default is 60):

cat /proc/sys/vm/swappiness

To change the system swappiness value, open /etc/sysctl.conf as root. Then, change or add this line to the file:

vm.swappiness = 10

Reboot for the change to take effect

1 gb should be fine for Ubuntu.

ROFL.

Looks like you have some corruption on your hard drive - it needs fscking. That said that, looking at your SMART output I’d say that drive is fsck’d and needs replacing asap. (Samsung drives in general are great on performance, cheap, but have a terrible average lifespan … I have ~ 50 drives, failure rate in the first 3 years is probably ~ 30-40%)

Another way to force a fsck on reboot is;

tune2fs -c199 -C200 /dev/sda1

Sample output for a Decent (expensive) drive; [WD 600G VelociRaptor]

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       1
  3 Spin_Up_Time            0x0027   232   231   021    Pre-fail  Always       -       3375
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       38
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   092   092   000    Old_age   Always       -       6220
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       36
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       28
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       9
194 Temperature_Celsius     0x0022   128   110   000    Old_age   Always       -       22
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

And a Samsung Spinpoint that’s starting to show a little wear, note the Multi-Zone error rate.
Note, the more faults you get, the more retries your drive is doing, the slower it will get …

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   051    Pre-fail  Always       -       1
  2 Throughput_Performance  0x0026   252   252   000    Old_age   Always       -       0
  3 Spin_Up_Time            0x0023   070   068   025    Pre-fail  Always       -       9224
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       45
  7 Seek_Error_Rate         0x002e   252   252   051    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0024   252   252   015    Old_age   Offline      -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       11383
 10 Spin_Retry_Count        0x0032   252   252   051    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   252   252   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       51
191 G-Sense_Error_Rate      0x0022   252   252   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0022   252   252   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0002   064   062   000    Old_age   Always       -       25 (Min/Max 13/39)
195 Hardware_ECC_Recovered  0x003a   100   100   000    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   252   252   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x002a   001   001   000    Old_age   Always       -       309314
225 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       51

Im not going to argue whether 1Gb of RAM is enough … at this point (as I don’t know what you were running at the time, and what you have set to autostart) all I can do is point out that when you ran free -m you had 28MB or RAM left, and your system was using the swapfile.

If that is the “normal” state of your PC … you need more RAM … simple as.


Here’s an idea … install iotop:

sudo apt-get install iotop

run it with:

sudo iotop

now leave that terminal open and watch which process(es) are making all the disk accesses.

iotop displays the top I/O using processes.

You might also want to run:

top

in another termiinal, and see what is using all your memory

top displays the top memory/cpu using processes.

@ Mad Penguin

That’s handy to know as (IIRC) Peppermint doesn’t take any notice of a /forcefsck file.

But using tune2fs to set the mount count higher than the maximum mount count (before checking) worked a treat :slight_smile:

[EDIT]

Turns out Peppermint 3 does work with /forcefsck … maybe it was Peppermint 2 or Linpus Lite :-[

:). … I’ve not seen the forcefsck option before , tune2fs has been my stock method since Noah emptied out the Arc … :wink:

1m or less is often fine on a server, but typically my Chrome instance eats more than that by itself … :slight_smile:

Sent from my Nexus S using Tapatalk 2

Thanks madmen. I tried the forcefsck (legitimate rape?) as per instructions but it didn’t seem to work- maybe 3 seconds longer boot + no output, same number of errors, but honestly I don’t trust that standard Disc Manager Ubuntu program- I tested that partition quite a while ago and think other disk check programs weren’t showing problems. What’s a good one in Ubuntu? Now I’m using Windows since it is working better and have secure Internet.

Yes Mark, seem to have many processes open eating all the RAM, even when I don’t open anything- I’ve studied the process manager quite a bit before- Clam eats alot, sometimes that “sleeping” xxxx (neutral process can’t remember but usually several instances open), evolution (would love to just prevent this from opening but can’t find start-up control for processes) has 3-5 instances and bizarrely auto-opens in workspace 2 (the phantom instances that I never could close). I’ll copy the output of it + send it to you when I mess w Ubuntu again. Which is most crucial- % use of processor, memory, cache or swap?

Thanks for diagnostic things- love to see what the hell is happening before I fix wrong problem. Ever replace a generator for $200 and a day’s work because of loose fan belt or cable? Oww Oww. There really is a word: “swappiness” -sounds like something I’d invent. IOTOP is kinda cute too, like OOOTOP of Quake. Think reducing swapiness might help, but better to reduce load by eliminating unnecissary (sp) auto-opening processes. Pretty scrupulous about that in Windows- 15 processes max.

Sorry it is a SEAGATE drive, legendarily crummy, and the Ubun partition is near the end- at 130 gig of 160 gig drive. Initially had hellish problems with it masking 60 gigs of drive by that bizarre spy program that makes all disparates drive an identical length (100gig in my case). Luckily the powerful +_ invaluable HDAT2 analyzed it and made it all writable. What is difference with that “tune 2fs” instruction. Must read the manual for fsck (how does one exit from a manual in terminal- always have to close window) to see if I need any options.

If you mean you don’t trust the SMART data from Disk Utility … trust it … it’s getting that information directly from the hard drive firmware itself.
(look up what S.M.A.R.T. (Self Monitoring Analysis and Reporting Technology) is)
http://en.wikipedia.org/wiki/S.M.A.R.T.

Effectively, if you say you don’t trust SMART, you’re saying you don’t trust the hard drive to monitor itself … if you don’t trust the hard drive to monitor itself, get a new drive.

Swappiness … changing swappiness will only change WHEN the swap file/partition begins to be used … it will make no difference if you are running out of available RAM … it will still start paging to swap when NECESSARY (which is when you run out of RAM).

Exit from a man page with the “Q” key … or exit from anything else in a terminal with “Ctrl+C” (be careful with Ctrl+C)

I’m sure there are tutorials about removing Evolution in 10.04 … but be careful, just uninstalling it in Synaptic removes half your desktop too IIRC.

ClamAV … dunno why you’re running that at all ???

Seriously, the easiest solution would be to FIT MORE RAM, or move to a “light” distro :wink: … and that HDD will need replacing SOON.

Anything less, and you can expect issues … Windows may currently be running perfectly because it’s not spanning the failing sectors of the HDD, but when drives start to fail it most often spreads … think about this … if the read head at some point touched the platter and scraped off some of the magnetic coating, there are now little bits of metal flying around inside your HDD accelerating it demise … same thing with the spindle bearings, etc.

Which is most crucial- % use of processor, memory, cache or swap?

If your CPU is at 100% usage, instruction will start to be queued … at some point the PC may freeze till the queues are cleared.

Memory and swap are intertwined … data is loaded to memory from the hard drive when it’s going to be needed to be read or changed quickly, but when you run low on memory, rather than your system just crash, it starts using the swap partition of the hard drive as additional memory (which is why its sometimes called virtual memory), but the hard drive is 1000’s of times slower than actual RAM … so again your system will eventually have to pause whilst it waits for data from swap.

Neither are really “more important” … but there’s less you can do about CPU usage, short of replacing the CPU and maybe motherboard, or just not using CPU intensive applications.

No, I don’t trust the SMART on this disc cause it says OK or GOOD even with these lousy numbers, and believe it did from the beginning. The memory shows 871 of 1024- that normal, and with only Opera open, uses 824. After disconnect Internet, and closing Opera, took 5 minutes for memory usage to drop to 26% (almost identical to SWAP, which was about 56%) on the SYSTEM MONITOR “Resources” tab. That ain’t right, is it? Now with FF open and Internet connected, only reading 50%, but lets see what happens. IOTOP is saying

1024 be/4 root 0.00 B 0.00 B ?unavailable? console-kit~–no-daemon
1 be/4 root 0.00 B 0.00 B ?unavailable? init
2 be/4 root 0.00 B 0.00 B ?unavailable? [kthreadd]
3 rt/4 root 0.00 B 0.00 B ?unavailable? [migration/0]
4 be/4 root 0.00 B 0.00 B ?unavailable? [ksoftirqd/0]
5 rt/4 root 0.00 B 0.00 B ?unavailable? [watchdog/0]
6 be/4 root 0.00 B 0.00 B ?unavailable? [events/0]
7 be/4 root 0.00 B 0.00 B ?unavailable? [cpuset]
8 be/4 root 0.00 B 0.00 B ?unavailable? [khelper]
9 be/4 root 0.00 B 0.00 B ?unavailable? [netns]
10 be/4 root 0.00 B 0.00 B ?unavailable? [async/mgr]
11 be/4 root 0.00 B 0.00 B ?unavailable? [pm]
12 be/4 root 0.00 B 0.00 B ?unavailable? [sync_supers]
13 be/4 root 0.00 B 0.00 B ?unavailable? [bdi-default]
14 be/4 root 0.00 B 0.00 B ?unavailable? [kintegrityd/0]
15 be/4 root 0.00 B 0.00 B ?unavailable? [kblockd/0]
16 be/4 root 0.00 B 0.00 B ?unavailable? [kacpid]
17 be/4 root 0.00 B 0.00 B ?unavailable? [kacpi_notify]
18 be/4 root 0.00 B 0.00 B ?unavailable? [kacpi_hotplug]
19 be/4 root 0.00 B 0.00 B ?unavailable? [ata/0]
20 be/4 root 0.00 B 0.00 B ?unavailable? [ata_aux]
21 be/4 root 0.00 B 0.00 B ?unavailable? [ksuspend_usbd]
22 be/4 root 0.00 B 0.00 B ?unavailable? [khubd]
CONFIG_TASK_DELAY_ACCT not enabled in kernel, cannot determine SWAPIN and IO %

when I open 12 FF windows though, w Gedit, terminal, + sysmonitor, memory use goes up to 86%, though it seems SWAP is inaccessible

TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
1024 be/4 root 0.00 B/s 0.00 B/s ?unavailable? console-kit~–no-daemon
1 be/4 root 0.00 B/s 0.00 B/s ?unavailable? init
2 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kthreadd]
3 rt/4 root 0.00 B/s 0.00 B/s ?unavailable? [migration/0]
4 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [ksoftirqd/0]
5 rt/4 root 0.00 B/s 0.00 B/s ?unavailable? [watchdog/0]
6 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [events/0]
7 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [cpuset]
8 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [khelper]
9 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [netns]
10 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [async/mgr]
11 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [pm]
12 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [sync_supers]
13 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [bdi-default]
14 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kintegrityd/0]
15 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kblockd/0]
16 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kacpid]
17 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kacpi_notify]
18 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [kacpi_hotplug]
19 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [ata/0]
20 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [ata_aux]
21 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [ksuspend_usbd]
22 be/4 root 0.00 B/s 0.00 B/s ?unavailable? [khubd]

AGAIN, how do I prevent unnecessary processes from starting up? IS there any startup control of processes? I think it is the memory use, not bad disk, cause NOW it is doing no disk writes cause it hasn’t maxed out. Here’s TOP. Plug in container (video, audio) uses alot and sometimes causes freezes by itself, and Xorg is big user too, but 70% of problem is browser. 400Mz DDR (1) 1gig is really expensive now: 16x modern RAM (need 2 cause have 2x 512 and 1 gig is max), and I feel like a fool to feed this old beast more, rather than find a nice Black Friday $300 powerhouse. Tough to wait in line from 5000 miles away though. You have those ultra-sales in Britannia yet? Guy Faulks or Hastings Day?

2469 mako4 20 0 552m 129m 10m R 19.9 14.9 17:40.61 plugin-containe
1212 root 20 0 315m 74m 9124 R 17.9 8.6 25:58.73 Xorg
2216 mako4 20 0 362m 12m 5580 S 11.9 1.4 17:40.03 gnome-system-mo
2300 mako4 20 0 1189m 386m 21m S 6.6 44.4 8:52.88 firefox
2345 mako4 20 0 318m 9.9m 6972 S 2.7 1.1 0:07.05 gnome-terminal
52 root 20 0 0 0 0 S 1.0 0.0 3:50.47 kondemand/0
1783 mako4 20 0 260m 7724 5228 S 0.3 0.9 0:05.77 metacity
1802 mako4 20 0 29116 364 288 S 0.3 0.0 0:05.09 syndaemon
1841 mako4 20 0 272m 4556 3088 S 0.3 0.5 0:30.10 sensors-applet
2663 mako4 20 0 19236 1404 1040 R 0.3 0.2 0:00.11 top
1 root 20 0 23800 1036 540 S 0.0 0.1 0:00.79 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.17 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
6 root 20 0 0 0 0 S 0.0 0.0 0:00.59 events/0

I reckon it’s both a lack of RAM, your choice of the full blown Ubuntu, and some bad sectors on the hard drive.

but primarily a lack of RAM … I’ve said it before, and I’ll say it again … fit more RAM or switch to a lighter desktop/distro … throwing more figures at me isn’t going to change that … 1GB is stretching it with Ubuntu

SMART data comes DIRECTLY from the HDD’s own firmware … so if you think the data is incorrect, then you think the HDD’s firmware is broken … in which case replace replace the drive.

Startup Applications is where to look for apps that start at bootup … which “processes” do you not want to start at bootup ?


I think it is the memory use, not bad disk, cause NOW it is doing no disk writes cause it hasn't maxed out.

if it hasn’t used your available RAM and isn’t using swap, I’m going to guess it wasn’t freezing at the time either … so how did you draw any conclusions ? … if on the other hand you still had spare RAM and no I/O and it WAS freezing, then the bottleneck must be the CPU.

But you always ask for numbers and stats- look, my problem is it worked fine: efficient, fast, no lockups or freezes for a year and half- these problems have only been in the last 2-3 months and the RAM and I think the disk stats haven’t changed. The start-up list programs, not processes, which there are maybe 60 of running (too much, I think). Is there some process startup control, maybe manually in terminal, and which can be safely shutdown- first thing people do with Windows is limit all this start-up crap. Hows about Xorg (25-30megs)? I shut down bluetooth + Ubuntu One synch service. Can Bluetooth be manually started OK? IDE drives aren’t easy to find now either, thought I might rip one out of cheap new externals, but they are already all SATA. Not really any room in a lappy for adapter. Sometimes- you are right- its the CPU that’s maxing out at 100%, which usually drops radically when you disconnect Internet, so think it’s some script or plug-in container loop; but most problems are from these infinite disc writes.

OK, I have the same processor with 2GiB memory and running ArchBang with memory usage after startup around 120MiB

You cannot just disable applications (services) like Xorg
Go to System > Preferences > Startup Applications.
This should bring up a dialog box similar to this:

http://cloud.addictivetips.com/wp-content/uploads/2009/06/startup-applicatoin-preferences.jpg

Where you can disable/enable applications. But be careful what you disable.
The following might not work on 10.10
To show hidden applications run:

cd /etc/xdg/autostart/
sudo sed --in-place 's/NoDisplay=true/NoDisplay=false/g' *.desktop

After changing anything you want, you can go back to the previous state:

cd /etc/xdg/autostart/
sudo sed --in-place 's/NoDisplay=false/NoDisplay=true/g' *.desktop