Transcription

White PaperCisco Unified Computing SystemSite Planning Guide: Data CenterPower and CoolingThis document provides a technical overview of power, space, and cooling considerations requiredfor successful deployment of IT equipment in the data center. Topics are introduced with ahigh-level conceptual discussion and then discussed in the context of Cisco products.The Cisco Unified Computing System (Cisco UCS ) product line works with industry-standardrack and power solutions that are generally available for the data center. Cisco also offers racksand power distribution units (PDUs) that have been tested with Cisco UCS and selected CiscoNexus products.This document is intended to inform those tasked with physical deployment of IT equipment in thedata center. It does not discuss equipment configuration or deployment from the viewpoint of asystem administrator. 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 1 of 22

ContentsData Center Thermal Considerations . 3Data Center Temperature and Humidity Guidelines . 3Best Practices . 4Hot-Aisle and Cold-Aisle Layout . 5Populating the Rack . 6Containment Solutions . 6Cable Management . 7Relationship between Heat and Power . 8Energy Savings in Cisco’s Facilities . 8Cisco Rack Solutions . 8Cisco Rack Options and Descriptions . 9Multi-Rack Deployment Solutions. 9Data Center Power Considerations . 9Overview . 10Power Planning . 10Gather the IT Equipment Power Requirements . 10Gather the Facility Power and Cooling Parameters. 14Design the PDU Solution . 15Cisco RP Series Power Distribution Unit (PDU) . 15Cisco RP Series Basic PDUs . 15Cisco RP Series Metered Input PDUs . 16Cisco RP Series PDU Input Plug Types . 16For More Information . 17Appendix: Sample Designs . 18Example 1: Medium Deployment (Rack and Blade Server) . 18Example 2: Large Deployment (Blade Server) . 20 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 2 of 22

Data Center Thermal ConsiderationsCooling is a major cost factor in data centers. If cooling is implemented poorly, the power required to cool a datacenter can match or exceed the power used to run the IT equipment itself. Cooling also is often the limiting factor indata center capacity (heat removal can be a bigger problem than getting power to the equipment).Data Center Temperature and Humidity GuidelinesThe American Society of Heating, Refrigeration, and Air Conditioning Engineers (ASHRAE) Technical Committee9.9 has created a widely accepted set of guidelines for optimal temperature and humidity set points in the datacenter. These guidelines specify both a required and an allowable range of temperature and humidity. ASHRAE2015 thermal guidelines are presented in the 2016 ASHRAE Data Center Power Equipment Thermal Guidelinesand Best Practices. Figure 1 illustrates these guidelines.Figure 1.ASHRAE and NEBS Temperature and Humidity LimitsAlthough the ASHRAE guidelines define multiple classes with different operating ranges, the recommendedoperating range is the same for each class. The recommended temperature and humidy are shown in Table 1.Table 1.ASHRAE Class A1 to A4 Recommended Temperature and Relative Humidity RangePropertyRecommended ValueLower limit temperature64.4 F [18 C]Upper limit temperature80.6 F [27 C]Lower limit humidity40% relative humidity and 41.9 F (5.5 C) dew pointUpper limit humidity60% relative humidity and 59 F (15 C) dew point 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 3 of 22

These temperatures describe the IT equipment inlet air temperature. However, there are several locations in thedata center where the environment can be measured and controlled, as shown in Figure 2. These points include: Server inlet (point 1) Server exhaust (point 2) Floor tile supply temperature (point 3) Heating, Ventilation, and Air Conditioning (HVAC) unit return air temperature (point 4) Computer room air conditioning unit supply temperature (point 5)Figure 2.Example of a Data Center Air Flow DiagramTypically, data center HVAC units are controlled based on return air temperature. Setting the HVAC unit return airtemperatures to match the ASHRAE requirements will result in very low server inlet temperatures, because HVACreturn temperatures are closer to server exhaust temperatures than inlet temperatures.The lower the air supply temperature in the data center, the greater the cooling costs. In essence, the airconditioning system in the data center is a refrigeration system. The cooling system moves heat generated in thecool data center into the outside ambient environment. The power requirements for cooling a data center dependon the amount of heat being removed (the amount of IT equipment in the data center) and the temperature deltabetween the data center and the outside air.The rack arrangement on the data center raised floor can also have a significant impact on cooling-related energycosts and capacity, as summarized in the next section.Best PracticesAlthough this document is not intended to be a complete guide for data center design, it presents some basicprinciples and best practices for data center airflow management. 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 4 of 22

Hot-Aisle and Cold-Aisle LayoutThe hot-aisle and cold-aisle layout in the data center has become standard (Figure 3). Arranging the racks intorows of hot and cold aisles minimizes the mixing of air in the data center. If warm air is allowed to mix with theserver inlet air, the air supplied by the air conditioning system must be at an even colder temperature tocompensate. As described earlier, lower supply-air temperatures cause increased energy use by the chiller andlimit the cooling efficiency of the data center by creating hot spots.Figure 3.Hot-Aisle and Cold-Aisle LayoutIn contrast, not using segregated hot and cold aisles results in server inlet air mixing. Air must be supplied from thefloor tile at a lower temperature to meet the server inlet requirements, as shown in Figure 4.Figure 4.Server Inlet Air Mixing 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 5 of 22

Populating the RackRacks should be populated with the heaviest and most power-dense equipment at the bottom. Placing heavyequipment at the bottom helps lower the rack’s center of mass and helps reduce the risk of tipping. Power-denseequipment also tends to draw more air. In the typical data center, in which air is supplied through perforated floortiles, placing power-dense equipment near the bottom of the rack gives that equipment the best access to thecoldest air.Unoccupied space in the rack can also cause hot air to penetrate back into the cold aisle. Blanking panels are asimple measure that can be used to prevent this problem, as shown in Figure 5.Figure 5.Using Blanking Panels to Prevent Airflow Short-Circuiting and BypassIn summary, populate racks from the bottom up and fill any gaps between hardware or at the top of the rack withblanking panels.Containment SolutionsAn effective extension of the hot- and cold-aisle concept is airflow containment. Figure 6 depicts hot-aislecontainment. Containment provides complete segregation of the hot and cold air streams, which has the benefit ofreducing energy use in the HVAC system by allowing the temperature of the cold air output to be raised. Becausethere is no mixing of air, there is no need to set the air temperature lower to compensate. This approach increasesthe temperature of the air returning to the HVAC system, which improves the efficiency of the HVAC system.For hot-aisle containment, care should be taken to not create pressure in the hot aisle. IT systems are designed sothat they have a near zero pressure difference between their air intake and exhaust. Backpressure in the hot aislecan cause fans to work harder in the system. 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 6 of 22

Figure 6.Hot-Aisle Airflow Containment ExampleCable ManagementTo the greatest extent possible, airflow obstructions should be removed from the intake and exhaust openings ofthe equipment mounted in the chassis. Lack of sufficient airflow may result in increased equipment fan powerconsumption to compensate for increased airflow impedance. If a rack door is installed, it should be perforated andshould be at least 65 percent open. Solid doors, made of glass or any other material, inevitably result in airflowproblems and should be avoided. Please consult the hardware installation guide for specific equipmentrequirements.Proper cable management is critical to reducing airflow blockage. Cisco UCS significantly reduces the number ofcables required. However, it is still important to properly dress the cables to provide the best airflow (Figure 7).Figure 7.Cisco UCS Power and Network Cabling 2017 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information.Page 7 of 22

Relationship between Heat and PowerAll power that is consumed by IT equipment is converted to heat. Though power is typically reported in watts (W)and heat is typically reported in British Thermal Units (BTUs) per hour (BTU/hr), these units are in factinterchangeable. Although power is almost always reported in watts, heat load is commonly reported in watts orBTU/hr. The conversion from watts to BTU/hr is 1W 3.412 BTU/hr. So, for example, a server that consumes100W produces approximately 341.2 BTU/hr of heat energy.Energy Savings in Cisco’s FacilitiesTo carefully study the effects of best practices to promote energy efficiency, Cisco underwent a data centerefficiency study in the Cisco research and development laboratories. As part of this study, the following bestpractices were applied: Redundant power was disabled where possible Power savings programs were used Computational fluid dynamics (CFD) modeling was used Virtualization was applied Blanking panels were used The floor grilles were rearranged The chilled water temperature was raised from 44 F to 48 F (7 C to 9 C)This study demonstrated major improvements in data center power and cooling efficiency. Even though anincrease in hardware installations caused the IT load to increase slightly (from 1719 to 1761 kilowatts [kW]), theoverhead for cooling the data center dropped (from 801 to 697 kW). The overall power usage effectiveness (PUE)dropped from 1.48 to 1.36. The payback from the proof of concept was 6 to 12 months. The ideas from this pilotproject are being applied to all Cisco facilities and are projected to save US 2 million per year.Cisco Rack SolutionsThe Cisco R42612 Rack is an industry standard EIA-310-D rack that is optimized for the Cisco UCS. It suppo