Take data center efficiency to the max on three-phase power

Single-phase power

Incoming power from the utility is 240-V, delivered by two hot wires and one neutral.

For decades, it was standard practice to power cabinets at 120 V -- the same voltage and wiring scheme used for utility outlets in homes and offices.

Every receptacle is serviced by two wires -- hot and neutral -- plus a non-voltage-carrying ground wire (see figure 1). A full cycle of utility power goes through a 360 degree phase change, so the two hot legs are 180 degrees apart, which is known as single-phase power.

Big appliances, ones that do not get unplugged frequently, run on 240 Volts of single-phase power, using both hot wires to deliver twice the 120-V hot-to-neutral voltage, as shown in figure 1. The higher voltage delivers more power (wattage) with the same-sized wires and circuit breaker ratings.

Power outages

If you connect too large a load to a receptacle, the breaker will trip and interrupt the circuit. If several small loads connect to each receptacle, but the total exceeds the circuit breaker capacity, the breaker will trip, failing each receptacle on that circuit. It is possible for loads to be within limits on every branch outlet and lighting circuit, yet still overwhelm the power supply.

Each hot leg of 240-V utility power must have a main circuit breaker, but electrical code requires that the handles on both service legs be physically tied together. Therefore, if the total load on either hot leg exceeds the main breaker rating, it will trip both circuits, causing a total power failure. Both wires are connected by 240-V circuits, so draw the same amount of power from each leg. But 120-V service comes from only one of the hot wires and the neutral. Multiple 120-V receptacles are wired to each hot wire, so if you connect too many things to receptacles on the same leg of your incoming service, the main circuit breaker will trip both legs, not just the overloaded one. Running reasonably equal loads on both phases avoids this problem.

Is three-phase power to the cabinets the answer?

Commercial buildings mostly use three-phase power. More actual power is delivered through the building with the same-sized wires and circuit breakers, leading to significant electrical cost savings. The principles are the same as described above, but phase balancing is trickier.

Three-phase power service is delivered by three hot wires and one neutral. The voltage between any phase or hot wire and neutral is still 120 V, just like at home or older data centers. But three-phase power moves the legs 120 degrees apart. The voltage between any two phases or hot legs is 208 V.

Figure 2 shows several different possibilities derived from a three-phase system. The 208 V-power delivered by any two legs of a three-phase system is still called single-phase power. It is unnecessary to bring the neutral to data center cabinets with a three-phase system. With just the three-phase power wires, you have multiple 208 V-circuits. If the neutral is extended, then both 120 V- and 208 V-receptacles can be provided on the same power strip.

We get considerably more power from three-phase circuits than from single-phase designs. Without delving into electrical theory, the reason is that three-phase wattage is volts times amps times √3, rather than just volts times amps. Power factor is excluded; it makes little difference in modern computing hardware.

To translate design into real power load, see the final column of chart 1: 80% capacity. This is the maximum power that equipment is allowed by code to draw through a circuit breaker on a continuous basis. This is the real power that will go to data center equipment.

Computing hardware running at 208 V operates at higher energy efficiency than equipment running at 120 V.

Three-phase systems deliver nearly twice the power capacity of single-phase systems, but via fewer wires. Since most cabinets today are dual circuited, the difference in wiring demand doubles, escalating cost savings with three-phase power.

The wiring differences become extreme when powering ultra-dense cabinets (25 to 30 kW), especially with higher amperage circuits where conductor sizes are also bigger.

Phase balance is highly important with three-phase circuits. The phase with the highest load determines the maximum output of the data centers' UPS or power distribution units, so phases must be balanced to use the total power capacity. All three handles of a three-phase breaker are tied together, so if you overload one phase, it will trip the breaker and cut power.

Examples of power loading in data center cabinets

The goal for good data center power distribution is to have no more than a 5% load differential among phases. If you have a combination of 120- and 208-V devices in the data center cabinets, the 120-V circuits are easy to move around on a three-phase system. While 120-V circuits can make phase balancing easier, it's a poor reason alone to use them, because of the efficiency losses.

Calculating amperage per phase on unbalanced three-phase circuits is not straightforward. Metering is the best way to get actual per-phase amps in the data center.

Assume you have a cabinet full of the same type of servers, and one-third of them are plugged into every circuit group. The phases are automatically balanced as illustrated in Example 1. This cabinet will run on a 30-Amp, three-phase circuit since each phase is less than 24 Amps, or 80% of the 30 Amp breaker rating.

It's harder to phase balance if devices draw different power levels or if too many devices are connected to the same phases. Example 2 shows that when devices with different power requirements are randomly connected to a three-phase power strip, the loads go way more than 5% out of balance, and two of the phases are over the limit for a 30-Amp breaker. A larger circuit would solve that limit problem, but is actually unnecessary for the total load presented.

In Example 3, the data center operator moves two servers (highlighted in the chart) in an attempt to balance the phase loads. But when 208-V loads move, the load changes on only one phase. The loads are better matched on Phases A and C, but the load on Phase B is the same as before. Further, the differences among phase currents still significantly exceed 5% and Phase B is still above the 30-Amp circuit's maximum. Over time, the breaker could trip and shut down each circuit in the cabinet.

In Example 4, operators move two more servers (highlighted in the chart), with different power requirements, to achieve better phase balance. The result is only 1% differential and all three phases are now within the limits of a 30-Amp breaker, without changing the power equipment.

These examples show that you may need to move several devices around to achieve phase balance on a three-phase circuit.