Warning: While interesting, this build won't be very practical for most folks. The build will also be - and this pains me greatly - made up mostly of parts for which I paid full price. Oh the shame!
Build Name: UberDB
Operating System/ Storage Platform: Windows and Solaris 11.1 - we'll see which one wins the database benchmarking war
CPU: Quad Xeon E5-4640 engineering sample revision C1
Motherboard: Supermicro X9QR7-TF-JBOD
Chassis: Supermicro SuperServer 4047R-7JRFT. I'll also use one of my Supermicro SC216 chassis that has been converted to JBOD.
Drives: 48 Samsung 120GB SSD drives, 16 1TB spinning rust drives, 1 120GB SSD boot drive. Will expand further, up to 72 drives, if the server can make good use of the extra disk IO.
RAM: 256GB DDR3-1600 ECC. This isn't much RAM, but with really fast disk IO I have not seen any benefits from going bigger in my usage scenarios.
Add-in Cards: 4x LSI 9300-8i, 2x LSI SAS2308 on the motherboard, 2x Intel 10GbE ports on the motherboard, 1x Mellanox dual-port QDR Infiniband. I will add up to 4 LSI 9207-8e cards if necessary.
Power Supply: 3x Supermicro 1,620 watt
Other Bits: Oracle ASM, Oracle 12c
Usage Profile: An all-SSD Oracle analytical database. A single "Data Warehouse" query can easily use all available IO, and quite a bit of CPU, so almost no investment in speed will go un-utilized on a daily basis.
More Information: My favorite server ever is the HP DL585 G7. It's absurdly fast, and I built it, thanks to eBay, for remarkably little money. True, my 16 CPU Dell c6145 cluster was faster for many (but not all) database queries, but the HP wins for overall productivity. Now, however, it's time to see if I can do better.
The only real bottleneck with the HP DL is IO. That may sound funny given my signature, but with a data warehouse there is no such thing as too much IO. The DL585 G7 has an insane 11 PCIe slots, all of which are x16 or x8, and sports a full four IO chips, but those PCIe slots are PCIe2. I'm hoping that PCIe3, directly wired to the CPUs as in the Xeon E5, can do better. If it can't then I'm out some serious coin.
![]()
The Supermicro SuperServer 4047R-7JRFT sounds expensive at $3,100 new. It's not as bad as it seems. You get a quad Xeon E5 motherboard, two embedded LSI SAS2308 disk controllers, a pair of Intel 10GbE ports, some massive power supplies, and 48 2.5" drive bays with sleds. Buy two 24-bay disk JBOD disk chassis and you'll spend $1,600 even if you buy used. Add another $500 for two LSI cards and $200 more for 10GbE and you are up to $2,500 already. That means you get a chassis, motherboard, and power supplies for the equivalent of $600. Not so bad.
The CPUs were $400 each. Not much of a deal compared to AMD 6128s and 6172s, or Xeon L5520s and L5639s, but a lot less than retail. This, the Mellanox card, and RAM are the items for which I was able to rely on eBay. The SSD drives were bought new, but long ago.
The four new LSI 9300-8i cards were $1,148 total. I could have saved a bit of money without compromising performance by using 9207-8i cards, but the 12G SAS might be useful in the future. The LSI 9207-8e cards are recycled from another server, as is the Mellanox card and the RAM.
Build Name: UberDB
Operating System/ Storage Platform: Windows and Solaris 11.1 - we'll see which one wins the database benchmarking war
CPU: Quad Xeon E5-4640 engineering sample revision C1
Motherboard: Supermicro X9QR7-TF-JBOD
Chassis: Supermicro SuperServer 4047R-7JRFT. I'll also use one of my Supermicro SC216 chassis that has been converted to JBOD.
Drives: 48 Samsung 120GB SSD drives, 16 1TB spinning rust drives, 1 120GB SSD boot drive. Will expand further, up to 72 drives, if the server can make good use of the extra disk IO.
RAM: 256GB DDR3-1600 ECC. This isn't much RAM, but with really fast disk IO I have not seen any benefits from going bigger in my usage scenarios.
Add-in Cards: 4x LSI 9300-8i, 2x LSI SAS2308 on the motherboard, 2x Intel 10GbE ports on the motherboard, 1x Mellanox dual-port QDR Infiniband. I will add up to 4 LSI 9207-8e cards if necessary.
Power Supply: 3x Supermicro 1,620 watt
Other Bits: Oracle ASM, Oracle 12c
Usage Profile: An all-SSD Oracle analytical database. A single "Data Warehouse" query can easily use all available IO, and quite a bit of CPU, so almost no investment in speed will go un-utilized on a daily basis.
More Information: My favorite server ever is the HP DL585 G7. It's absurdly fast, and I built it, thanks to eBay, for remarkably little money. True, my 16 CPU Dell c6145 cluster was faster for many (but not all) database queries, but the HP wins for overall productivity. Now, however, it's time to see if I can do better.
The only real bottleneck with the HP DL is IO. That may sound funny given my signature, but with a data warehouse there is no such thing as too much IO. The DL585 G7 has an insane 11 PCIe slots, all of which are x16 or x8, and sports a full four IO chips, but those PCIe slots are PCIe2. I'm hoping that PCIe3, directly wired to the CPUs as in the Xeon E5, can do better. If it can't then I'm out some serious coin.
The Supermicro SuperServer 4047R-7JRFT sounds expensive at $3,100 new. It's not as bad as it seems. You get a quad Xeon E5 motherboard, two embedded LSI SAS2308 disk controllers, a pair of Intel 10GbE ports, some massive power supplies, and 48 2.5" drive bays with sleds. Buy two 24-bay disk JBOD disk chassis and you'll spend $1,600 even if you buy used. Add another $500 for two LSI cards and $200 more for 10GbE and you are up to $2,500 already. That means you get a chassis, motherboard, and power supplies for the equivalent of $600. Not so bad.
The CPUs were $400 each. Not much of a deal compared to AMD 6128s and 6172s, or Xeon L5520s and L5639s, but a lot less than retail. This, the Mellanox card, and RAM are the items for which I was able to rely on eBay. The SSD drives were bought new, but long ago.
The four new LSI 9300-8i cards were $1,148 total. I could have saved a bit of money without compromising performance by using 9207-8i cards, but the 12G SAS might be useful in the future. The LSI 9207-8e cards are recycled from another server, as is the Mellanox card and the RAM.