.. _pumping: Pumping and cooling =================== This section describes how to pump and cool HiPERCAM's CCDs. It is assumed that the instrument has already been :ref:`powered up `. Click on the photos below for larger images. 1. You will need to access the Elevation (Folded Cassegrain) Platform in order to pump and cool HiPERCAM. Only HiPERCAM team members who have completed the relevant safety courses and GTC paperwork, and have GTC permission to do so, are allowed to access this area. Before going up to the platform, put on your personal protection equipment, including: helmet with chin strap, safety shoes/boots, safety harness with lanyard and scaffold hooks (located in the HiPERCAM crates), and safety gloves (if necessary). Make sure that you have a radio, set to channel 8, and that you have told the GTC Jefe de Turno (during the day) or Telescope Operator (during the night) if it is ok for you to go up to the platform. He or she will have to slew the telescope to vertical, insert the elevation axis locking pins and turn on the dome lights. On your way up to the platform, do not carry anything in your hands that would prevent you from safely using the stair rails on your way up - any such equipment should be transported to the platform using the cradle and crane. When descending the platform, always place one hand on the stair rail behind you and one on the rail in front of you, as shown in the photo below. .. Note: The policy on wearing safety harnesses when accessing the .. elevation platform at the GTC is in a state of flux - it is .. possible that it may be relaxed in the future due to the existence .. of the railing around the elevation platform. For up to date .. information on this, please ask the Jefe de Turno or Telescope .. Operator prior to accessing the platform. .. image:: photos/safety_kit2.jpg :width: 25% .. image:: photos/staircase2.jpg :width: 25% 2. Once on the platform, carefully pivot the access stairs if they are not already down. It has been known for the telescope elevation to be incorrectly positioned, forcing the access stairs to rest on a flimsy piece of metal conduit rather than the red railing. In this case, ask the Jefe de Turno or Telescope Operator to adjust the telescope elevation accordingly. .. image:: photos/access_stairs2.jpg :width: 25% 3. The next step is to vacuum pump the CCD heads. The principle here is that cooling the CCDs when the heads do not contain a good vacuum risks condensation forming on the detectors, which are the coldest part of the CCD head. So you must only ever cool down the CCDs whilst pumping. Prior to June 2019, we used the OSIRIS vacuum pump for the HiPERCAM CCD heads. For instructions on how to use this pump, refer to :ref:`OSIRIS pump `. The HiPERCAM pump is mounted on the collar, which is mounted on the rotator. The pump is permanently connected to the vacuum manifold on HiPERCAM via a stainless steel pipe. Power to the pump is provided by a custom mains cable that is permanently plugged into the patch panel on the rotator. The vacuum pump is mounted such that the rotation axis of the turbo is aligned with the rotation axis of the rotator, so it should not be a problem if the rotator is moved whilst the HiPERCAM CCD heads are being pumped. Moving the telescope elevation axis whilst pumping, however, could damage the turbo, so should be avoided if possible. .. image:: photos/vacuum_pump.jpg :width: 25% 4. When pumping, it is recommended that you position HiPERCAM in the orientation shown in the pictures below, with CCD2 pointing upwards. This ensures that there are no CCD heads pointing downwards, which reduces the risk of a person damaging one of the CCD heads as they pass underneath the instrument. .. image:: photos/pump_orientation1.jpg :width: 25% .. image:: photos/pump_orientation2.jpg :width: 25% .. image:: photos/pump_orientation3.jpg :width: 25% 5. If the instrument is not at the correct orientation, you will have to move the rotator. To do this, take the key hanging from the shelf at the rear of the HiPERCAM electronics cabinet, and open the rotator brake box. (Be careful not to lose the little loop of black velcro used to attach the key to the cabinet shelf.) If the switch inside the box is pointing to *AUTO*, the rotator brake is on and you will need to switch it to *MANUAL* to move the rotator. When you switch it to manual, you should hear air escaping continuously from the pneumatics near the switch. If you don't hear this noise, which is quite loud, this means that the brake is interlocked by the telescope control system and you will have to ask the Jefe de Turno or Telescope Operator to release it. With the brake released, rotate HiPERCAM to the correct orientation - you may need two people to do this. .. image:: photos/brake_box1.jpg :width: 25% .. image:: photos/brake_box2.jpg :width: 25% .. image:: photos/brake_box3.jpg :width: 25% .. image:: photos/brake_box4.jpg :width: 25% 6. This pumping procedure assumes that the vacuum manifold is already under a reasonable vacuum. If it is not, refer to item 7 below. To start the pump, simply press the red power button. The button will light up red and a blue LED will turn on indicating that the pump is operational. You will also hear the backing pump turn on immediately, and the pressure displayed on the front panel (in mbars) will start falling. Within a minute you will hear the turbo whining as it automatically turns on and speeds up, with the whining increasing in frequency as the pressure drops to the low e-5 mbar level. At this pressure, you should hear the frequency of the turbo whining stabilise, which indicates that it is up to speed. .. image:: photos/pump_pon.jpg :width: 25% 7. When the turbo is up to speed and the pressure on the pump gauge falls to the low e-5 mbar range, open the valve to the HiPERCAM vacuum manifold. You should see the pressure on the gauge rise slightly and then quickly drop again. .. image:: photos/vac_manifold2.jpg :width: 25% 8. If the HiPERCAM vacuum manifold is at atmospheric pressure, follow the above procedure, but open the valve to the vacuum manifold *before* you power on the vacuum pump. If you don't do this, and you open the manifold valve when the turbo is up to speed, you will damage the pump with the sudden rush of air. 9. Before you can open the vacuum valves on the CCD heads, you need to inspect the head pressures. To do this, open the front of the HiPERCAM electronics cabinet and view the vacuum gauge controllers at the top of the cabinet. The pressures they display are in mbars. .. image:: photos/vacuum_gauges.jpg :width: 25% 10. You are now ready to open the vacuum valves to the CCD heads. For the safety of the CCDs, this should be a two-person job, with one person opening the valves and the other watching the vacuum gauges. The person opening the valves must ensure they are hooked onto the railings for safety. Choosing the CCD head with the highest pressure, and ensuring that the pump pressure is below the CCD head pressure, slowly open the valve on the head, but do not open it fully yet. If the person watching the gauge sees the pressure rise, shout out to the other person to close the valve immediately and then check the pump and vacuum pipe connections for problems. If, on the other hand, the pressure drops, then shout OK and continue to open the valve until you feel it go loose, at which point you should start closing it until you feel resistance and then stop. Then move onto the CCD head with the next highest pressure. Only open the valve to this head if the pressure in the first CCD head is approximately equal to or lower than the head to be opened. In this way, one can avoid reverse pumping from one CCD head to the other. Repeat the procedure until all five CCD heads are open - you can identify which CCD head is which using the sticker on each head or the colour-coded anodised PCB boxes on the rear of each head. === ==== ====== *u* CCD1 purple *g* CCD2 blue *r* CCD3 orange *i* CCD4 red *z* CCD5 dark red === ==== ====== .. image:: photos/hook_railings.jpg :width: 25% .. image:: photos/ccd1_head2.jpg :width: 25% .. image:: photos/ccd2_head2.jpg :width: 25% .. image:: photos/ccd3_head2.jpg :width: 25% .. image:: photos/ccd4_head2.jpg :width: 25% .. image:: photos/ccd5_head2.jpg :width: 25% 11. If the CCD heads have not been pumped for weeks/months, their pressures are likely to be in the e-2 mbar range. When the CCD head valves are opened, the pressures should drop steadily, getting into the e-3 mbar range within a matter of minutes and then into the e-4 mbar range after a few hours. When the CCDs reach the low e-3 mbar range, it is safe to begin cooling them. 12. The HiPERCAM CCDs are cooled by pelter coolers (or Thermo-Electric Coolers, TECs) that are themselves cooled by the GTC +5degC water/glycol chiller system. The CCD heads are cooled in parallel, fed by two 6-way manifolds mounted on the instrument. The cooling pipes are colour coded, with blue markings indicating the cold water flow and red markings the hot water return. Each of the 6 parallel arms in the cooling circuit is equipped with a flow sensor that cuts power to the associated peltier if the flow falls below 0.3 litres/minute. As a backup, the power is also cut to the peltiers if their hot side goes above a temperature of +50degC. On first connecting HiPERCAM to the GTC cooling circuit, it is essential to ensure that no water bubbles pass through the HiPERCAM flow sensors, as this could damage them. So, disconnect all 6 of the flow sensors from the manifold and attach a single hose to the manifold to make a circuit. (You may actually find it easier to remove the hoses from the flow sensors rather than disconnect the flow sensors.) Then turn on the coolant flow by turning the yellow lever marked *HiPERCAM CCDs* underneath the rotator by 90 degrees so that it is pointing down to the ground, ensuring that the red valve next to the yellow lever is fully opened (at which point it reads 4.9 or 5.0). You can tell that the coolant is flowing by inspecting the red propeller mounted above it. .. image:: photos/flow_disconnect2.jpg :width: 25% .. image:: photos/gtc_valve2.jpg :width: 25% 13. Watch the red propeller for a few minutes, waiting for all of the air bubbles to pass through the system. Then close the yellow valve lever, reconnect the 6 flow sensors to the manifold and then re-open the yellow valve lever. .. image:: photos/propeller2.jpg :width: 25% .. image:: photos/gtc_valveclose2.jpg :width: 25% .. image:: photos/flow_connect2.jpg :width: 25% 14. You can check the flow rate through each CCD head of HiPERCAM, as well as the NGC, on the Honeywell data recorder at the front of the electronics cabinet. The values should all be approximately 1 litre/min. If no flow is indicated in any of the CCD heads, and you have verified that the red propeller is spinning, check that the lap-top style power supply at the rear of the electronics cabinet is plugged in. If there is no flow in just one of the CCD heads, it could be that the flow sensor has failed. Try swapping the offending flow sensor with the spare stored in the HiPERCAM crates. .. image:: photos/honeywell_zoom2.jpg :width: 25% 15. Once you have flow through all 5 CCD heads, you are ready to cool the detectors using the Meerstetter peltier controllers. To do this, open the front door of the cabinet and press the blue round OK button on the right-hand side of the front panel to display the CCD temperatures. If the coolant is not flowing (or is below a flow rate of 0.3 litres/min), the relays will be tripped in the Meerstetters and a circular power symbol will be indicated on both sides of the CCD temperatures. This means that cooling will not be possible. On the other hand, if the flow is above a rate of 0.3 litres/min, a tick appears on the left-hand side of each CCD temperature, and either an hour-glass symbol or a tick appears on the right-hand side of each CCD temperature, depending on whether or not the CCD has reached its target temperature. .. image:: photos/meerstetter_power2.jpg :width: 25% .. image:: photos/meerstetter_hourglass2.jpg :width: 25% .. image:: photos/meerstetter_tick2.jpg :width: 25% 16. Now set the CCD target temperatures, which are -90degC for CCDs 1, 4 and 5, and -88degC for CCDs 2 and 3. You can set the temperatures locally using the Meerstetters, or remotely using ``hdriver``. To set them locally: * Press the left arrow on the blue control panel as many times as necessary to get to the top-level *LDD/TEC* screen. * Press the middle blue *OK* button to display all 3 CCD temperatures. * Press the right arrow to get to the *LTR settings* screen. * Press the down arrow to get to the *Terminal 1* page, i.e. CCD1. * Press the right arrow to get to the temperatures menu. * Press the down arrow 5 times to get to the *Target object temperature* setting. * Press the middle button to enable the temperature to be changed. * Press the up and down arrows to change the value of the digit, the left and right buttons to change the digit, and the middle button to set the value. * Press the left arrow to return to the *Terminal 1* page and then the down arrow to enter the *Terminal 2* page, i.e. CCD2. Repeat the above to change its *Target object temperature*, and then repeat again for *Terminal 3*, i.e. CCD3. To set them remotely: * On ``hdriver``, click on the *Settings* menu at the top and *select Expert Level 1*. An additional button will appear towards the top-left hand side of the GUI labelled *CCD TECs*. * Click on the *CCD TECs* button to enter the temperatures page. Enter the target temperature setting for CCD1, for example, remembering to press carriage return after you have entered the value. Repeat this operation for the other CCDs. The Meerstetter peltier controller has a ramp function that has been set to cool down (or warm up) the CCDs at a rate of 0.1deg/s. Therefore, it takes approximately 15 minutes to cool the CCDs down. The power output of the Meerstetters has been limited to 85% of the maximum power that can be safely applied to the peltiers. This is the reason why the set temperatures of CCDs 2 and 3 are only -88degC: if set to -90degC, these CCDs would struggle to maintain this temperature and permanently draw the maximum power available, potentially shortening the lifetime of the system. 17. To prevent condensation forming on the exterior surfaces of the CCD head windows, HiPERCAM employs a nitrogen-gas flushing system, fed by the observatory supply via a 5-way gas manifold mounted on the large central plate on the instrument. To check that the gas is flowing, remove one of the black gas pipes going into one of the CCD heads by simultaneously pulling the pipe and pushing the shoulder of the self-sealing connector. Place the pipe near your lips - you should be able to feel a very gentle breeze. If you can't, check with the GTC staff to see if the gas supply has been turned on. If it hasn't, make sure you disconnect the main gas pipe feeding the manifold prior to turning the supply on, as otherwise you may send high pressure gas into the manifold, popping the pipes out of their connectors. 18. Once you have finished on the elevation platform, switch the rotator brake back to *AUTO*, lock the brake box with the key, and attach the key to the cabinet shelf using the velcro loop. It is essential you clear all loose material from the elevation platform to prevent damage to the mirror if the telescope is moved. Hence you must remove any equipment, tools and notebooks, ensure that both doors of the electronics cabinet are closed, and swing the access stairs back out of the way. Then inform the Jefe de Turno or Telescope Operator that you have finished working on the platform and that the telescope can be moved. 19. To log the pressures and temperatures during the pumping and cooling procedures, go to the control room and use the DRPC to open a terminal on the rack PC and, in the ``/home/insuser`` directory, type ``hw_monitor &``. This will open a ``matplotlib`` window displaying graphs of the pressures and temperatures of the CCDs as a function of time, starting from when the ``hw_monitor`` script is run. The values are also stored in an ASCII file called ``hw_log.txt``, also in the ``/home/insuser`` directory. To view an automatically-updating list of the pressure and temperature values, type ``tail -f hw_log.txt``. Note that ``hw_monitor`` script always appends to this file, never overwrites it, but only plots the pressures and temperature values obtained since the script was run. At the end of a run at the telescope, move the ``hw_log.txt`` file to ``/home/insuser/hw_log_oct18.txt``, for example, to prevent the file becoming overly large and disjoint. A new file will then automatically be created in the home directory when the ``hw_monitor`` script is next run. To plot old log files, use the python script ``/home/insuser/old_hw_logs/plot_hw_log.py``. The DRPC desktop should look like the screenshot below. It is recommended that you open the above windows in the middle-left virtual desktop of the DRPC. .. image:: photos/hwmonitor_screenshot.png :width: 75%