MDM Trouble Report for 2014 Jan 01

Telescope: 1.3m
Observer(s): Jules Halpern (Columbia)
Instrument: Templeton

Problem(s) Encountered:
While on the last target of the night, I realized that the
TCS control panel could not send commands to the TCS.  The
dome stopped following the telescope.  None of the computers
could communicate with hill. Got the following messages:

ssh: connect to host hill port 22: No route to host

The TCS control panel gave this:

ERROR: Could not send command.
Socket is not connected
Is the TCS server running?
Cannot connect to TCS server: Socket operation timed out -
is server running?

I could see that the TCS was still running because the
telescope was tracking and the coordinates were updating
correctly.  I don't really understand the socket, or which
computer was at fault.  But there was no way to communicate
with hill, even using its monitor in the control room, which
remained blank.  So I concluded it was probably hill's fault,
and that I would have to physically power it down.

I used the ESTOP to stop the telescope from tracking into
the horizon, just in case this didn't work.

I powered down hill. mcgraw, and mdmarc2, and when they came back
up I could start everything fine: TCS, XMIS, OWL.
Remarkably, the telescope didn't lose pointing by more than
a couple of arcminutes.  I spent the early twilight time
zeroing the pointing, so I wouldn't have to think about it 
for the next night.  By the way, the telescope points very well,
even over long slews.


------------------------------

Submitted on 2014 Jan 2 [6:54:08]