Transition Technology: Ticket Query

#641: Enable dynamic Munin graphs

chris — Mon, 02 Dec 2013 15:25:45 GMT

If we only generate munin graphs on the fly we will also get the zoom function working, this example config looks good: http://uname.pingveno.net/blog/index.php/post/2013/08/25/Configure-Munin-graphs-with-Nginx-and-Debian-7

#666: Parrot lockups

chris — Wed, 08 Jan 2014 11:31:16 GMT

We have this on the console, I'm going to reboot it:

[28800.164426]  [<ffffffffa001c1ba>] ? do_get_write_access+0x22c/0x452 [jbd2]
[28800.164435]  [<ffffffff81066360>] ? wake_bit_function+0x0/0x23
[28800.164443]  [<ffffffff8104b51c>] ? try_to_wake_up+0x289/0x29b
[28800.164453]  [<ffffffffa001c402>] ? jbd2_journal_get_write_access+0x22/0x33 [jbd2]
[28800.164475]  [<ffffffffa006289e>] ? __ext4_journal_get_write_access+0x4e/0x56 [ext4]
[28800.164492]  [<ffffffffa0042b8e>] ? ext4_reserve_inode_write+0x37/0x73 [ext4]
[28800.164508]  [<ffffffffa0042c05>] ? ext4_mark_inode_dirty+0x3b/0x1c4 [ext4]
[28800.164528]  [<ffffffffa005bdc7>] ? ext4_journal_start_sb+0xd4/0x10e [ext4]
[28800.164543]  [<ffffffffa0042eb0>] ? ext4_dirty_inode+0x30/0x46 [ext4]
[28800.164553]  [<ffffffff81109ead>] ? __mark_inode_dirty+0x25/0x14a
[28800.164560]  [<ffffffff8110138b>] ? file_update_time+0x101/0x130
[28800.164569]  [<ffffffff810b6835>] ? __generic_file_aio_write+0x16e/0x293
[28800.164578]  [<ffffffff810b69b3>] ? generic_file_aio_write+0x59/0x9f
[28800.164588]  [<ffffffff810f0316>] ? do_sync_write+0xce/0x113
[28800.164596]  [<ffffffff810fcd0c>] ? filldir+0x0/0xb7
[28800.164605]  [<ffffffff810549b1>] ? _local_bh_enable_ip+0x22/0x8f
[28800.164613]  [<ffffffff81066332>] ? autoremove_wake_function+0x0/0x2e
[28800.164626]  [<ffffffff8130f1a1>] ? _spin_lock_bh+0x9/0x25
[28800.164626]  [<ffffffff810f0c68>] ? vfs_write+0xa9/0x102
[28800.164632]  [<ffffffff810f0d18>] ? sys_pwrite64+0x57/0x77
[28800.164639]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b
[28800.164657] INFO: task apache2:31559 blocked for more than 120 seconds.
[28800.164665] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[28800.164675] apache2       D 0000000000000000     0 31559  28011 0x00000000
[28800.164691]  ffffffff8149f1f0 0000000000000286 0000000000000000 ffffffff81274f4f
[28800.164711]  ffff880002dc99d8 ffff8800bd1ca000 000000000000f9e0 ffff880002dc9fd8
[28800.164730]  00000000000157c0 00000000000157c0 ffff8800bd9746a0 ffff8800bd974998
[28800.164752] Call Trace:
[28800.164762]  [<ffffffff81274f4f>] ? sch_direct_xmit+0x7f/0x14c
[28800.164773]  [<ffffffff81066253>] ? bit_waitqueue+0x10/0xa0
[28800.164787]  [<ffffffffa001c1ba>] ? do_get_write_access+0x22c/0x452 [jbd2]
[28800.164798]  [<ffffffff81066360>] ? wake_bit_function+0x0/0x23
[28800.164812]  [<ffffffffa001c402>] ? jbd2_journal_get_write_access+0x22/0x33 [jbd2]
[28800.164833]  [<ffffffffa006289e>] ? __ext4_journal_get_write_access+0x4e/0x56 [ext4]
[28800.164852]  [<ffffffffa0042b8e>] ? ext4_reserve_inode_write+0x37/0x73 [ext4]
[28800.164871]  [<ffffffffa0042c05>] ? ext4_mark_inode_dirty+0x3b/0x1c4 [ext4]
[28800.164890]  [<ffffffffa005bdc7>] ? ext4_journal_start_sb+0xd4/0x10e [ext4]
[28800.164908]  [<ffffffffa0042eb0>] ? ext4_dirty_inode+0x30/0x46 [ext4]
[28800.164921]  [<ffffffff81109ead>] ? __mark_inode_dirty+0x25/0x14a
[28800.164932]  [<ffffffff8110138b>] ? file_update_time+0x101/0x130
[28800.164943]  [<ffffffff810b6835>] ? __generic_file_aio_write+0x16e/0x293
[28800.164958]  [<ffffffff8125227b>] ? sock_aio_write+0x0/0xbc
[28800.164969]  [<ffffffff8100cc43>] ? xen_make_pte+0x7b/0x83
[28800.164980]  [<ffffffff810b69b3>] ? generic_file_aio_write+0x59/0x9f
[28800.164992]  [<ffffffff810f0316>] ? do_sync_write+0xce/0x113
[28800.165003]  [<ffffffff81066332>] ? autoremove_wake_function+0x0/0x2e
[28800.165015]  [<ffffffff810ce24c>] ? handle_mm_fault+0x3b8/0x80f
[28800.165027]  [<ffffffff810f0c68>] ? vfs_write+0xa9/0x102
[28800.165038]  [<ffffffff810f0d7d>] ? sys_write+0x45/0x6e
[28800.165049]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b
[125024.867759] hrtimer: interrupt took 38561246 ns
[1412520.196163] INFO: task mysqld:7928 blocked for more than 120 seconds.
[1412520.196183] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[1412520.196191] mysqld        D 0000000000000000     0  7928  18454 0x00000000
[1412520.196203]  ffff8800bfa69530 0000000000000286 0000000000000000 0000000000000000
[1412520.196215]  0007ffffffffffff 0000000000000001 000000000000f9e0 ffff880074cb5fd8
[1412520.196226]  00000000000157c0 00000000000157c0 ffff880002ce1530 ffff880002ce1828
[1412520.196236] Call Trace:
[1412520.196259]  [<ffffffffa00232bf>] ? jbd2_log_wait_commit+0xbf/0x112 [jbd2]
[1412520.196273]  [<ffffffff81066332>] ? autoremove_wake_function+0x0/0x2e
[1412520.196293]  [<ffffffffa003fb41>] ? ext4_sync_file+0x199/0x25c [ext4]
[1412520.196304]  [<ffffffff8110d6e0>] ? vfs_fsync_range+0x73/0x9e
[1412520.196319]  [<ffffffff8110d78a>] ? do_fsync+0x28/0x39
[1412520.196325]  [<ffffffff8110d7b9>] ? sys_fsync+0xb/0x10
[1412520.196333]  [<ffffffff81011b63>] ? sysret_check+0x17/0x5a
[1412520.196341]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b

#632: Add accounts for Sam

chris — Mon, 25 Nov 2013 14:41:25 GMT

Following ticket:534

#739: Set up DEV and other drupal accounts for Annesley

ed — Wed, 11 Jun 2014 14:38:42 GMT

is this you chris? or a sam job? or paul?

#771: cannot access puffin over sftp again

annesley — Tue, 05 Aug 2014 09:24:10 GMT

not sure what happened this time. my connection to other servers is fine. i am using:

sftp://tn.ftp@puffin.transitionnetwork.org/static/transition-network-d6-32-s002-booker/

the password was accepted first time and i successfully navigated to /static

which worked until i tried to get to the transition-network-d6-32-s002-booker directory. now ssh fails also.

when i ssh in i get plenty of warnings and potential lock outs when i try various things. like trying to access /static.

#780: parrot

annesley — Wed, 27 Aug 2014 14:35:51 GMT

can't access parrot again also: it re-requests my password very quickly. what is the session timeout?

also: i cannot include my files. seems that the website is located under the /home/annesley directory. is this some sort of strange setup because the PHP cannot see that area. and i cannot cd to the root through SFTP...

#613: Parrot mysql console errors

chris — Mon, 04 Nov 2013 09:48:35 GMT

I connected to the wiki:ParrotServer wiki:XenShell to check the bandwidth usage and there was the following output, we should try to track down the cause of this:

parrot login: [41569.058444] hrtimer: interrupt took 59422559 ns
[442920.164115] INFO: task mysqld:3522 blocked for more than 120 seconds.
[442920.164133] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[442920.164142] mysqld        D ffff880003755c40     0  3522    711 0x00000000
[442920.164153]  ffff8800bfa68e20 0000000000000282 ffff8800565fb9e8 ffff8800565fb9e4
[442920.164165]  00000000000157c8 0000000000000000 000000000000f9e0 ffff8800565fbfd8
[442920.164175]  00000000000157c0 00000000000157c0 ffff88000180c6a0 ffff88000180c998
[442920.164185] Call Trace:
[442920.164200]  [<ffffffff8110fb96>] ? sync_buffer+0x0/0x40
[442920.164211]  [<ffffffff8130de82>] ? io_schedule+0x73/0xb7
[442920.164219]  [<ffffffff8110fbd1>] ? sync_buffer+0x3b/0x40
[442920.164228]  [<ffffffff8130f142>] ? _spin_unlock_irqrestore+0xd/0xe
[442920.164237]  [<ffffffff8130e292>] ? __wait_on_bit_lock+0x3f/0x84
[442920.164246]  [<ffffffff8110fb96>] ? sync_buffer+0x0/0x40
[442920.164253]  [<ffffffff8130e342>] ? out_of_line_wait_on_bit_lock+0x6b/0x77
[442920.164262]  [<ffffffff81066360>] ? wake_bit_function+0x0/0x23
[442920.164271]  [<ffffffff8110ffc3>] ? sync_dirty_buffer+0x29/0x93
[442920.164286]  [<ffffffffa001be04>] ? journal_dirty_data+0xd1/0x1b0 [jbd]
[442920.164301]  [<ffffffffa0032f4c>] ? ext3_journal_dirty_data+0xf/0x34 [ext3]
[442920.164312]  [<ffffffffa00313f5>] ? walk_page_buffers+0x65/0x8b [ext3]
[442920.164322]  [<ffffffffa0032f71>] ? journal_dirty_data_fn+0x0/0x13 [ext3]
[442920.164333]  [<ffffffffa0034a93>] ? ext3_ordered_write_end+0x73/0x10f [ext3]
[442920.164343]  [<ffffffff810b648a>] ? generic_file_buffered_write+0x18d/0x278
[442920.164352]  [<ffffffff810b6926>] ? __generic_file_aio_write+0x25f/0x293
[442920.164361]  [<ffffffff810b69b3>] ? generic_file_aio_write+0x59/0x9f
[442920.164369]  [<ffffffff810f0316>] ? do_sync_write+0xce/0x113
[442920.164377]  [<ffffffff81066332>] ? autoremove_wake_function+0x0/0x2e
[442920.164385]  [<ffffffff8130f6fa>] ? error_exit+0x2a/0x60
[442920.164394]  [<ffffffff8101251d>] ? retint_restore_args+0x5/0x6
[442920.164403]  [<ffffffff8102de30>] ? pvclock_clocksource_read+0x3a/0x8b
[442920.164411]  [<ffffffff810f0c68>] ? vfs_write+0xa9/0x102
[442920.164419]  [<ffffffff810f0d7d>] ? sys_write+0x45/0x6e
[442920.164427]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b
[442920.164435] INFO: task mysqld:21061 blocked for more than 120 seconds.
[442920.164441] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[442920.164448] mysqld        D ffff8800018b0e20     0 21061    711 0x00000000
[442920.164457]  ffff8800018b0e20 0000000000000282 ffffffff8100c426 ffffffff8100e233
[442920.164469]  ffff880003755d10 0000000000000002 000000000000f9e0 ffff8800bdde3fd8
[442920.164480]  00000000000157c0 00000000000157c0 ffff8800bd9462e0 ffff8800bd9465d8
[442920.164492] Call Trace:
[442920.164498]  [<ffffffff8100c426>] ? pfn_to_mfn+0xe/0x22
[442920.164506]  [<ffffffff8100e233>] ? arbitrary_virt_to_machine+0x28/0x73
[442920.164515]  [<ffffffff8110fb96>] ? sync_buffer+0x0/0x40
[442920.164523]  [<ffffffff8106ce4a>] ? timekeeping_get_ns+0xe/0x2e
[442920.164531]  [<ffffffff8110fb96>] ? sync_buffer+0x0/0x40
[442920.164538]  [<ffffffff8130de82>] ? io_schedule+0x73/0xb7
[442920.164545]  [<ffffffff8110fbd1>] ? sync_buffer+0x3b/0x40
[442920.164554]  [<ffffffff8130f142>] ? _spin_unlock_irqrestore+0xd/0xe
[442920.164561]  [<ffffffff8130e292>] ? __wait_on_bit_lock+0x3f/0x84
[442920.164569]  [<ffffffff8110fb96>] ? sync_buffer+0x0/0x40
[442920.164576]  [<ffffffff8130e342>] ? out_of_line_wait_on_bit_lock+0x6b/0x77
[442920.164584]  [<ffffffff81066360>] ? wake_bit_function+0x0/0x23
[442920.164592]  [<ffffffff8110ffc3>] ? sync_dirty_buffer+0x29/0x93
[442920.164600]  [<ffffffffa001be04>] ? journal_dirty_data+0xd1/0x1b0 [jbd]
[442920.164611]  [<ffffffffa0032f4c>] ? ext3_journal_dirty_data+0xf/0x34 [ext3]
[442920.164622]  [<ffffffffa00313f5>] ? walk_page_buffers+0x65/0x8b [ext3]
[442920.164632]  [<ffffffffa0032f71>] ? journal_dirty_data_fn+0x0/0x13 [ext3]
[442920.164643]  [<ffffffffa0034a93>] ? ext3_ordered_write_end+0x73/0x10f [ext3]
[442920.164652]  [<ffffffff810b648a>] ? generic_file_buffered_write+0x18d/0x278
[442920.164661]  [<ffffffff810b6926>] ? __generic_file_aio_write+0x25f/0x293
[442920.164670]  [<ffffffff810b69b3>] ? generic_file_aio_write+0x59/0x9f
[442920.164677]  [<ffffffff810f0316>] ? do_sync_write+0xce/0x113
[442920.164685]  [<ffffffff81066332>] ? autoremove_wake_function+0x0/0x2e
[442920.164693]  [<ffffffff810f0c68>] ? vfs_write+0xa9/0x102
[442920.164700]  [<ffffffff810f0d18>] ? sys_pwrite64+0x57/0x77
[442920.164706]  [<ffffffff81011b42>] ? system_call_fastpath+0x16/0x1b

#618: Migrate Penguin and Parrot to the ZFS fileserver

chris — Fri, 15 Nov 2013 09:37:36 GMT

Since wiki:PuffinServer has been running from the ZFS fileserver, see ticket:593, it has been performing better -- we should also migrate wiki:PenguinServer and wiki:ParrotServer to the ZFS server prior to upgrading them to Debian Wheezy on ticket:535.