[ale] sometimes you just need to reboot NOW!

[Jim Kinney]

On Wed, Jul 30, 2014 at 10:28 AM, Michael H. Warfield <mhw at wittsend.com>
wrote:

> On Wed, 2014-07-30 at 09:28 -0400, Jim Kinney wrote:
> > Stupid power strip can only monitor power usage. Addressable pdu was seen
> > as a waste of money. Sort of like cooling that shuts off and requires a
> > manual reset when there's a power hiccup.
>
> Ah, it's not the monitoring.  It's the ability to shoot the node in the
> head and force him to reboot.  When all else fails, it's hard for a
> server to resist the removal of primary power to the power supplies.


I am crafting a stonith script now to include in /root for emergencies. I
have many machines with no remote power control.


>  In
> the past, I've always used X10 controllers as a poor man's PDU for
> controlling power to my colo units.  Now, some relatively cheap web
> controlled power strips have become available that, when combined with a
> Raspberry Pi for access control and security, makes for a very powerful
> management solution.
>

Adding a RasPi with a thermistor as a manage point for the next cooling
system failure. Not much more and I can add stonith for the entire stack.
Will need to figure out the vmware CLI method of suspending VMs.

>
> Corollary to that:  Have a maintenance / forensice CD or USB drive in
> the system with serial consoles so you can intercept the boot process to
> an assured safe media to repair possible damage.
>
> Also...  Before doing a forced reboot, if you have ANY control at all,
> sync and unmount your disks.  SysRQ - S and U
>

Will add that ability plus a "just shoot me" version with no safety as a
last ditch fallback.

>
> > On Jul 30, 2014 9:18 AM, "Chris Fowler" <cfowler at outpostsentinel.com>
> wrote:
>
> > > Intelligent power strip.
> > >
> > > Power off.  Power on.
> > >
> > > _______________________________________________
> > > Ale mailing list
> > > Ale at ale.org
> > > http://mail.ale.org/mailman/listinfo/ale
> > > See JOBS, ANNOUNCE and SCHOOLS lists at
> > > http://mail.ale.org/mailman/listinfo
> > >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> http://mail.ale.org/pipermail/ale/attachments/20140730/bd900026/attachment.html
> >
> > _______________________________________________
> > Ale mailing list
> > Ale at ale.org
> > http://mail.ale.org/mailman/listinfo/ale
> > See JOBS, ANNOUNCE and SCHOOLS lists at
> > http://mail.ale.org/mailman/listinfo
> >
>
> --
> Michael H. Warfield (AI4NB) | (770) 978-7061 |  mhw at WittsEnd.com
>    /\/\|=mhw=|\/\/          | (678) 463-0932 |
> http://www.wittsend.com/mhw/
>    NIC whois: MHW9          | An optimist believes we live in the best of
> all
>  PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 465 bytes
> Desc: This is a digitally signed message part
> URL: <
> http://mail.ale.org/pipermail/ale/attachments/20140730/3bcba4f5/attachment.sig
> >
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>



-- 
-- 
James P. Kinney III

Every time you stop a school, you will have to build a jail. What you gain
at one end you lose at the other. It's like feeding a dog on his own tail.
It won't fatten the dog.
- Speech 11/23/1900 Mark Twain


*http://heretothereideas.blogspot.com/
<http://heretothereideas.blogspot.com/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20140730/6f9bf7f4/attachment.html>