Dotaz: KVM vysoký load po ugradu na squeeze

[JimiK]

Zdravím, už si nevím rady co by mohlo způsobovat lenost virtuálních mašin. Posílám nějaké info, jestli vás něco napadne.

server-lojza:~# uname -a
Linux server-lojza 2.6.33 #6 SMP Tue Jun 22 04:16:12 CEST 2010 i686 GNU/Linux

server-lojza:~# lsmod | grep kvm
kvm_intel              32970  6 
kvm                   197153  1 kvm_intel

server-lojza:~# w
 20:59:46 up 57 min,  3 users,  load average: 10.36, 10.71, 12.08



server-lojza:~# vmstat 
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  1      0 1343920  20800 1572040    0    0   172   180  121  296  2  0 75 23


PRC | sys  43.32s | user   3m58s | #proc    152 | #zombie    0 | #exit      0 |
CPU | sys      1% | user      6% | irq       0% | idle    298% | wait     94% |
CPL | avg1  10.27 | avg5   12.40 | avg15  12.53 | csw  4555064 | intr 1835934 |
MEM | tot    3.9G | free    1.3G | cache   1.5G | buff   20.9M | slab   32.1M |
SWP | tot    7.5G | free    7.5G |              | vmcom   2.5G | vmlim   9.4G |
DSK |         sdb | busy     79% | read   52119 | write 183279 | avio   12 ms |
DSK |         sda | busy     57% | read     288 | write 158203 | avio   13 ms |
DSK |         sdd | busy      0% | read     376 | write      0 | avio    1 ms |
DSK |         sdc | busy      0% | read      85 | write      0 | avio    1 ms |
NET | transport   | tcpi   71404 | tcpo   71474 | udpi    2956 | udpo    3033 |
NET | network     | ipi   261023 | ipo   262142 | ipfrw 179245 | deliv  79788 |
NET | vnet0    0% | pcki   22268 | pcko   29679 | si   55 Kbps | so   11 Kbps |
NET | vnet1    0% | pcki   47684 | pcko   45462 | si   15 Kbps | so   15 Kbps |
NET | eth0     0% | pcki  185816 | pcko  158048 | si  176 Kbps | so  138 Kbps |
                *** system and process activity since boot ***
  PID  SYSCPU  USRCPU  VGROW  RGROW  RDDSK  WRDSK  ST EXC S  CPU CMD     1/4   
 6800  10.17s   1m46s   1.1G 518.1M 528.1M   1.0G  N-   - S   3% kvm
 6787   6.63s  89.98s   1.2G 511.2M 568.1M 299.2M  N-   - S   3% kvm
 6554   5.86s  37.39s 104.3M  7012K   272K   468K  N-   - S   1% libvirtd
  889   7.76s   0.00s     0K     0K     0K     0K  N-   - S   0% md3_raid1
 2227   3.75s   1.61s  6116K  1268K     0K     0K  N-   - S   0% zabbix_agentd
 7719   3.40s   1.40s  7860K  3712K   1.1G   1.2G  N-   - S   0% mc
 7892   0.86s   0.86s  8260K  2868K     0K     4K  N-   - S   0% sshd
    1   1.08s   0.00s  2032K   724K 133.5M  3536K  N-   - S   0% init
  874   0.92s   0.00s     0K     0K     0K     0K  N-   - S   0% md1_raid1
 7290   0.66s   0.00s     0K     0K     0K     0K  N-   - D   0% md1_resync
 1306   0.45s   0.00s     0K     0K     0K     0K  N-   - S   0% flush-9:1
 7897   0.40s   0.04s  2820K   480K     0K     0K  N-   - S   0% nc
  373   0.34s   0.00s     0K     0K     0K     0K  N-   - D   0% xfsdatad/1
 1890   0.11s   0.21s 88780K 30508K  1728K     8K  N-   - S   0% named
  111   0.25s   0.00s     0K     0K     0K 15248K  N-   - S   0% sync_supers
  115   0.17s   0.00s     0K     0K     0K     0K  N-   - S   0% kblockd/1
  369   0.12s   0.03s     0K     0K     0K     0K  N-   - S   0% xfslogd/1
 1937   0.06s   0.06s  2708K  2704K   164K   160K  N-   - S   0% atop
 1854   0.02s   0.06s 28844K  1756K   396K   744K  N-   - S   0% rsyslogd
 6769   0.03s   0.04s  8260K  2864K     0K     0K  N-   - S   0% sshd
 6119   0.00s   0.06s  4560K  1812K    16K     4K  N-   - S   0% ntpd
   18   0.02s   0.04s     0K     0K     0K     0K  N-   - S   0% events/3
 7258   0.02s   0.03s  8396K  2892K     0K     0K  N-   - S   0% sshd
   15   0.05s   0.00s     0K     0K     0K     0K  N-   - S   0% events/0
   17   0.05s   0.00s     0K     0K     0K     0K  N-   - S   0% events/2
 1915   0.02s   0.02s  4524K   920K     0K   528K  N-   - S   0% openvpn
 4561   0.02s   0.00s  8396K  2976K   440K     0K  N-   - S   0% sshd
  957   0.02s   0.00s  2520K   996K  1454K     8K  N-   - S   0% udevd
 2233   0.00s   0.02s  6116K   756K     0K     0K  N-   - S   0% zabbix_agentd
   16   0.00s   0.02s     0K     0K     0K     0K  N-   - S   0% events/1
  114   0.00s   0.02s     0K     0K     0K     0K  N-   - S   0% kblockd/0
  117   0.02s   0.00s     0K     0K     0K     0K  N-   - S   0% kblockd/3
 8304   0.00s   0.01s  8396K  2892K     0K     4K  N-   - S   0% sshd
 8027   0.00s   0.01s  8260K  2868K     0K     4K  N-   - S   0% sshd
 7999   0.00s   0.01s  8260K  2864K     0K     4K  N-   - S   0% sshd
 8561   0.01s   0.00s  2484K  2480K     0K     0K  N-   - R   0% atop
 4935   0.01s   0.00s  4552K  1860K   200K  1256K  N-   - S   0% bash
 7260   0.00s   0.01s  4544K  1836K    40K    24K  N-   - S   0% bash
 6774   0.01s   0.00s  2820K   476K     0K     0K  N-   - S   0% nc
 2210   0.01s   0.00s  1680K   456K    16K   312K  N-   - S   0% uml_switch
  116   0.01s   0.00s     0K     0K     0K     0K  N-   - S   0% kblockd/2
  867   0.01s   0.00s     0K     0K     0K     0K  N-   - S   0% md0_raid1

server-lojza:~# hdparm -tT /dev/md3

/dev/md3:
 Timing cached reads:   4080 MB in  2.00 seconds = 2040.26 MB/sec
 Timing buffered disk reads:  22 MB in  3.27 seconds =   6.72 MB/sec


server-lojza:~# kvm --version
QEMU emulator version 1.1.2 (qemu-kvm-1.1.2+dfsg-5~bpo60+1, Debian), Copyright (c) 2003-2008 Fabrice Bellard

Je mi jasné, že je to rychlostí disků, ale proč a jak z toho ven? Virtuály jsou na SW raidu 1,XFS, neswapují, používají virtio ovladače.

Odpovědi

[Milan Beneš]

Ahoj, zkus u disků nastavit režim cachování na "none".

[JimiK]

bohužel nepomohlo. Když dám ve virtuálu kopírovat 1G soubor, tak virtuální mašina lehne úplně :(

[JimiK]

přikládám lshw - sekci disky

        *-disk:0
                description: ATA Disk
                product: WDC WD5000AADS-0
                vendor: Western Digital
                physical id: 0
                bus info: scsi@0:0.0.0
                logical name: /dev/sda
                version: 01.0
                serial: WD-WCAV97469917
                size: 465GiB (500GB)
                capabilities: partitioned partitioned:dos
                configuration: ansiversion=5 signature=000ddefe
              *-volume:0
                   description: EXT3 volume
                   vendor: Linux
                   physical id: 1
                   bus info: scsi@0:0.0.0,1
                   logical name: /dev/sda1
                   version: 1.0
                   serial: cdd6959a-f71c-4093-9b5b-993e915d47cf
                   size: 188MiB
                   capacity: 188MiB
                   capabilities: primary bootable multi journaled extended_attributes recover ext3 ext2 initialized
                   configuration: created=2010-09-18 10:30:52 filesystem=ext3 modified=2013-02-05 20:02:12 mounted=2013-02-05 20:02:12 state=clean
              *-volume:1
                   description: Linux raid autodetect partition
                   physical id: 2
                   bus info: scsi@0:0.0.0,2
                   logical name: /dev/sda2
                   version: 3.6
                   serial: f0a5ea68-84e1-411c-9d62-594157a8a7cc
                   size: 18GiB
                   capacity: 18GiB
                   capabilities: primary multi journaled reiserfs initialized
                   configuration: filesystem=reiserfs hash=r5 state=unclean
              *-volume:2
                   description: Linux swap volume
                   physical id: 3
                   bus info: scsi@0:0.0.0,3
                   logical name: /dev/sda3
                   version: 1
                   size: 7632MiB
                   capacity: 7632MiB
                   capabilities: primary multi swap initialized
                   configuration: filesystem=swap pagesize=4096
              *-volume:3
                   description: Linux raid autodetect partition
                   physical id: 4
                   bus info: scsi@0:0.0.0,4
                   logical name: /dev/sda4
                   capacity: 439GiB
                   capabilities: primary multi
           *-disk:1
                description: ATA Disk
                product: WDC WD5000AADS-0
                vendor: Western Digital
                physical id: 1
                bus info: scsi@1:0.0.0
                logical name: /dev/sdb
                version: 01.0
                serial: WD-WCAV97468850
                size: 465GiB (500GB)
                capabilities: partitioned partitioned:dos
                configuration: ansiversion=5 signature=000ddefe
              *-volume:0
                   description: EXT3 volume
                   vendor: Linux
                   physical id: 1
                   bus info: scsi@1:0.0.0,1
                   logical name: /dev/sdb1
                   version: 1.0
                   serial: cdd6959a-f71c-4093-9b5b-993e915d47cf
                   size: 188MiB
                   capacity: 188MiB
                   capabilities: primary bootable multi journaled extended_attributes recover ext3 ext2 initialized
                   configuration: created=2010-09-18 10:30:52 filesystem=ext3 modified=2013-02-05 20:02:12 mounted=2013-02-05 20:02:12 state=clean
              *-volume:1
                   description: Linux raid autodetect partition
                   physical id: 2
                   bus info: scsi@1:0.0.0,2
                   logical name: /dev/sdb2
                   version: 3.6
                   serial: f0a5ea68-84e1-411c-9d62-594157a8a7cc
                   size: 18GiB
                   capacity: 18GiB
                   capabilities: primary multi journaled reiserfs initialized
                   configuration: filesystem=reiserfs hash=r5 state=unclean
              *-volume:2
                   description: Linux swap volume
                   physical id: 3
                   bus info: scsi@1:0.0.0,3
                   logical name: /dev/sdb3
                   version: 1
                   size: 7632MiB
                   capacity: 7632MiB
                   capabilities: primary multi swap initialized
                   configuration: filesystem=swap pagesize=4096
              *-volume:3
                   description: Linux raid autodetect partition
                   physical id: 4
                   bus info: scsi@1:0.0.0,4
                   logical name: /dev/sdb4
                   capacity: 439GiB
                   capabilities: primary multi
        *-serial
             description: SMBus
             product: 5 Series/3400 Series Chipset SMBus Controller
             vendor: Intel Corporation
             physical id: 1f.3
             bus info: pci@0000:00:1f.3
             version: 06
             width: 64 bits
             clock: 33MHz
             configuration: driver=i801_smbus latency=0
             resources: irq:18 memory:fbcf4000-fbcf40ff ioport:400(size=32)


        *-ide:1
             description: IDE interface
             product: 5 Series/3400 Series Chipset 2 port SATA IDE Controller
             vendor: Intel Corporation
             physical id: 1f.5
             bus info: pci@0000:00:1f.5
             logical name: scsi2
             logical name: scsi3
             version: 06
             width: 32 bits
             clock: 66MHz
             capabilities: ide pm bus_master cap_list emulated
             configuration: driver=ata_piix latency=0
             resources: irq:19 ioport:ac00(size=8) ioport:a880(size=4) ioport:a800(size=8) ioport:a480(size=4) ioport:a400(size=16) ioport:a080(size=16)
           *-disk:0
                description: ATA Disk
                product: WDC WD20EARX-00P
                vendor: Western Digital
                physical id: 0
                bus info: scsi@2:0.0.0
                logical name: /dev/sdc
                version: 51.0
                serial: WD-WCAZAJ577302
                size: 1863GiB (2TB)
                configuration: ansiversion=5
           *-disk:1
                description: ATA Disk
                product: WDC WD20EARX-00P
                vendor: Western Digital
                physical id: 1
                bus info: scsi@3:0.0.0
                logical name: /dev/sdd
                version: 51.0
                serial: WD-WCAZAJ579992
                size: 1863GiB (2TB)
                capabilities: partitioned partitioned:dos
                configuration: ansiversion=5 signature=000ddefe
              *-volume:0
                   description: Linux raid autodetect partition
                   physical id: 1
                   bus info: scsi@3:0.0.0,1
                   logical name: /dev/sdd1
                   capacity: 188MiB
                   capabilities: primary bootable multi
              *-volume:1
                   description: Linux raid autodetect partition
                   physical id: 2
                   bus info: scsi@3:0.0.0,2
                   logical name: /dev/sdd2
                   capacity: 18GiB
                   capabilities: primary multi
              *-volume:2
                   description: Linux raid autodetect partition
                   physical id: 3
                   bus info: scsi@3:0.0.0,3
                   logical name: /dev/sdd3
                   capacity: 7632MiB
                   capabilities: primary multi
              *-volume:3
                   description: Linux raid autodetect partition
                   physical id: 4
                   bus info: scsi@3:0.0.0,4
                   logical name: /dev/sdd4
                   capacity: 439GiB
                   capabilities: primary multi
  *-power UNCLAIMED

[JimiK]

Přílohy:

• dmesg.host (45553 bytů)
• dmesg.guest (17303 bytů)

nyní na hostovi běží ještě replikace pole, po nějakých mých experimentech, toho si nevšímej

[Nikola Ciprich]

poslete prosim vystup z dmesg (na hostovi i guestovi) a cely prikazovy radek kterym se spousti qemu (i s pripadnym vystupem)..

[JimiK]

/usr/bin/kvm -S -M pc -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name webhosting-lojza -uuid 3946955d-3b48-35a1-8bbf-81e9cc2b5374 -nodefconfig -nodefaults -chardev socket,id=charmonitor,pat h=/var/lib/libvirt/qemu/webhosting-lojza.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive fil e=/virtual_machines/webhosting-lojza.img,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vir tual_machines/webhosting-lojza_2.img,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -drive file=/virtual_machines/we bhosting-lojza_3.img,if=none,id=drive-virtio-disk2,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk2,id=virtio-disk2 -drive file=/virtual_machines/webhosting-lojza_4 .img,if=none,id=drive-virtio-disk3,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk3,id=virtio-disk3 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=host net0,id=net0,mac=52:54:00:27:83:4e,bus=pci.0,addr=0x3 -vnc 127.0.0.1:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8

[JimiK]

nevím jestli to může mít souvislost, ale tuším že před ugradem běželi guesti pod rootem