Valve Hardware Day 2006 - Multithreaded Edition
by Jarred Walton on November 7, 2006 6:00 AM EST- Posted in
- Trade Shows
Test Setup
Obviously valve is pretty excited about what can be done with additional processing power, and they have invested a lot of time and resources into building tools that will take advantage of the possibilities. However, Valve is a software developer as opposed to a hardware review site, and our impression is that most of their systems are typical of any business these days: they are purchased from Dell or some other large OEM, which means they are a bit more limited in terms of what kind of hardware is available. That's not to say that Valve hasn't tested AMD hardware, because they have, but as soon as they reached the conclusion that Core 2 Duo/Core 2 Quad would be faster, they probably didn't bother doing a lot of additional testing. We of course are more interested in seeing what these new multiprocessor benchmarks can tell us about AMD and Intel hardware -- past, present, and future -- and we plan on utilizing these tests in future articles. As a brief introduction to these benchmark utilities, however, we thought it would be useful to run them on a few of our current platforms to see how they fare.
In the interest of time, we did not try to keep all of the tested platforms identical in terms of components. Limited testing did show that the processor is definitely the major bottleneck in both benchmarks, with a variance between benchmark runs of less than 5% on all platforms. Besides the processor, the only other area that seems to have any significant impact on benchmark performance is memory bandwidth and timings. We tested both benchmarks three times on each platform, then we threw out the high and low scores and took the remaining median score. In many instances, the first run of the particle simulation benchmark was slightly slower than the next two runs, which were usually equal in performance. The variability between benchmark runs of the map compilation test was less than 1%, so the results were very consistent.
Here are the details of the tested systems.
We did test all of the systems with the same graphics card configuration, just to be consistent, but it really made little to no difference. On the Athlon 64 configuration, for example, we got the same results using the integrated graphics as we got with the X1900. We also tested at different resolutions, and found once again that on the graphics cards we used resolution seemed to have no impact on the final score. 640x480 generated the same results as 1920x1200, even when enabling all of the eye candy at the high resolution and disabling everything at the low resolution. To be consistent, all of the benchmarking was done at the default 1024x768 0xAA/8xAF. We tried to stay consistent on the memory that we used -- either for DDR or DDR2 - though the Pentium D test system had issues and would not run the particle simulation benchmark. Finally, to give a quick look at performance scaling, we overclocked all of the tested systems by 20%.
For now we are merely providing a short look at what Valve has been working on and some preliminary benchmarks. We intend to use these benchmarks on some future articles as well where we will provide a look at additional system configurations. Note that performance differences of one or two points should not be taken as significant in the particle simulation test, as the granularity of the reported scores is relatively coarse.
Obviously valve is pretty excited about what can be done with additional processing power, and they have invested a lot of time and resources into building tools that will take advantage of the possibilities. However, Valve is a software developer as opposed to a hardware review site, and our impression is that most of their systems are typical of any business these days: they are purchased from Dell or some other large OEM, which means they are a bit more limited in terms of what kind of hardware is available. That's not to say that Valve hasn't tested AMD hardware, because they have, but as soon as they reached the conclusion that Core 2 Duo/Core 2 Quad would be faster, they probably didn't bother doing a lot of additional testing. We of course are more interested in seeing what these new multiprocessor benchmarks can tell us about AMD and Intel hardware -- past, present, and future -- and we plan on utilizing these tests in future articles. As a brief introduction to these benchmark utilities, however, we thought it would be useful to run them on a few of our current platforms to see how they fare.
In the interest of time, we did not try to keep all of the tested platforms identical in terms of components. Limited testing did show that the processor is definitely the major bottleneck in both benchmarks, with a variance between benchmark runs of less than 5% on all platforms. Besides the processor, the only other area that seems to have any significant impact on benchmark performance is memory bandwidth and timings. We tested both benchmarks three times on each platform, then we threw out the high and low scores and took the remaining median score. In many instances, the first run of the particle simulation benchmark was slightly slower than the next two runs, which were usually equal in performance. The variability between benchmark runs of the map compilation test was less than 1%, so the results were very consistent.
Here are the details of the tested systems.
| Athlon 64 3200+ 939 | |
| CPU | Athlon 64 3200+ (939) - 2.0GHz 512K OC 3200+ @ 10x240 HTT = 2.40GHz |
| Motherboard | ASUS A8N-VM CSM - nForce 6150 |
| Memory | 2x1GB OCZ OCZ5001024EBPE - DDR-400 2-3-2-7 1T OC DDR-480 3-3-2-7 1T |
| GPU | X1900 XT |
| HDD | Seagate SATA3.0Gbps 7200.9 250GB 8MB cache 7200 RPM |
| Athlon X2 3800+ 939 | |
| CPU | Athlon X2 3800+ (939) - 2.0GHz 2x512K OC 3800+ @ 10x240 HTT = 2.40GHz |
| Motherboard | ASUS A8R32-MVP - ATI Xpress 3200 |
| Memory | 2x1GB OCZ OCZ5001024EBPE - DDR-400 2-3-2-7 1T OC DDR-480 3-3-2-7 1T |
| GPU | X1900 XT |
| HDD | Western Digital SATA3.0Gbps SE16 WD2500KS 250GB 16MB cache 7200 RPM |
| Athlon X2 3800+ AM2 | |
| CPU | Athlon X2 3800+ (AM2) - 2.0GHz 2x512K OC 3800+ @ 10x240 HTT = 2.40GHz |
| Motherboard | Foxconn C51XEM2AA - nForce 590 SLI |
| Memory | 2x1GB Corsair PC2-8500C5 - DDR2-800 4-4-4-12 OC DDR2-960 4-4-4-12 |
| GPU | X1900 XT |
| HDD | Western Digital SATA3.0Gbps SE16 WD2500KS 250GB 16MB cache 7200 RPM |
| Core 2 Duo E6700 NF570 | |
| CPU | Core 2 Duo E6700 - 2.67GHz 4096K OC E6700 @ 10x320 FSB = 3.20GHz |
| Motherboard | ASUS P5NSLI - nForce 570 SLI for Intel |
| Memory | 2x1GB Corsair PC2-8500C5 - DDR2-800 4-4-4-12 OC DDR2-960 4-4-4-12 |
| GPU | X1900 XT |
| HDD | Western Digital Raptor 150GB 16MB 10000 RPM |
| Core 2 Quad QX6700 975X | |
| CPU | Core 2 Quad QX6700 - 2.67GHz 2 x 4096K OC QX6700 @ 10x320 FSB = 3.20GHz |
| Motherboard | ASUS P5W DH Deluxe - 975X |
| Memory | 2x1GB Corsair PC2-8500C5 - DDR2-800 4-4-4-12 OC DDR2-960 4-4-4-12 |
| GPU | X1900 XT |
| HDD | 2 x Western Digital Raptor 150GB in RAID 0 |
| Pentium D 920 945P | |
| CPU | Pentium D 920 - 2.8GHz 2 x 2048K OC 920 @ 14x240 HTT = 3.36GHz |
| Motherboard | ASUS P5LD2 Deluxe - 945P |
| Memory | 2x1GB Corsair PC2-8500C5 - DDR2-667 4-4-4-12 OC DDR2-800 4-4-4-12 |
| GPU | X1900 XT |
| HDD | Western Digital SATA3.0Gbps SE16 WD2500KS 250GB 16MB cache 7200 RPM |
We did test all of the systems with the same graphics card configuration, just to be consistent, but it really made little to no difference. On the Athlon 64 configuration, for example, we got the same results using the integrated graphics as we got with the X1900. We also tested at different resolutions, and found once again that on the graphics cards we used resolution seemed to have no impact on the final score. 640x480 generated the same results as 1920x1200, even when enabling all of the eye candy at the high resolution and disabling everything at the low resolution. To be consistent, all of the benchmarking was done at the default 1024x768 0xAA/8xAF. We tried to stay consistent on the memory that we used -- either for DDR or DDR2 - though the Pentium D test system had issues and would not run the particle simulation benchmark. Finally, to give a quick look at performance scaling, we overclocked all of the tested systems by 20%.
For now we are merely providing a short look at what Valve has been working on and some preliminary benchmarks. We intend to use these benchmarks on some future articles as well where we will provide a look at additional system configurations. Note that performance differences of one or two points should not be taken as significant in the particle simulation test, as the granularity of the reported scores is relatively coarse.
Facebook
Twitter
RSS
55 Comments
View All Comments
primer - Tuesday, November 7, 2006 - link
it's a nice article. i'm anxious to see what develops out of Valve in the near future.how about using some Athlon FX, Opteron 100/1200 series or higher speed Athlon X2s for crying out loud. i know that here is a performance gap even with the higher AMD models currently, but please show us an attempt at not being as one-sided.
jm20 - Tuesday, November 7, 2006 - link
The tests at 2.0 and 2.4 Ghz for the AMD chips are there, you can plot how it 'should' improve. Here are my scaling projections based on the data given for the 939 platform.AMD Speed single dual
2.0 14.0 27.0
2.2 15.5 29.5
2.4 17.0 32.0
2.6 18.8 35.0
2.8 20.8 38.2
3.0 23.1 41.7
I just hope this will show up correctly :/
Very interesting article, I'm very excited to see an almost linear improvement from single to dual core.
yacoub - Tuesday, November 7, 2006 - link
Your Tested Systems details on page 8 - the last three show a CPU of a Venice 3200+ oc'd to 2.4GHz (hey that's what I run too!), but the headings suggest they should really show Intel chips. :)yacoub - Tuesday, November 7, 2006 - link
Yes please make it only visual and not actually a part of the game, because much like trying too hard to make graphics look realistic, it really just adds more frustration for the player than anything else. It's already annoying in HL2 getting caught up on the edge of prop_physics related objects. The last thing we need is also getting caught or bumped by another player ;)
JarredWalton - Tuesday, November 7, 2006 - link
If the world is more interactive, there might not actually be a need for "prop physics" objects. There's also the potential to make a game with different gameplay mechanics depending on how the character interacts with the world. It sounds like initially Valve will make it a visual enhancement, to make entities look more lifelike in their behavior, but down the road others could potentially do more. Like most of the enhancements, what we want to see is how they can actually change and improve gameplay beyond just being visual.yacoub - Tuesday, November 7, 2006 - link
"You're doing a disservice to the customer if you're not using all of the CPU power."okay so that alone shows the need for dual-core so the customer can offload background tasks to a second core so the game can truly get 100% of a core. ;)
timmiser - Tuesday, November 7, 2006 - link
Background tasks?? I agree with Gabe that I want all of my CPU working the game. I'm sitting here with an Athlon X2 4800+ and I can't run Flight Sim X at an acceptable frame rate all the while I watch the graph that shows CPU#2 at idle while CPU#1 tries to run the entire program by itself!On another note, how about Microsoft develope some type of new API (DirecThread?) that automatically takes care of utilizing multiple cores so that game companies like Valve don't have to employ entire multi-threaded R&D divisions just so their games use both cores.
It seems like we are going back to the caveman days before DirectInput, DirectSound, & DirectX etc. Remember when you had to choose your sound card, joystick, and video card from a list within the game?? Let's not go there again!!
-Tim
JarredWalton - Tuesday, November 7, 2006 - link
That was a direct quote from... Gabe I think. Others might disagree, but I thought it was an interesting take on things. As for background tasks, they usually only need a bit of CPU time (less than 5% in most cases), so unless you really want to encode videos and play games at the same time....yacoub - Tuesday, November 7, 2006 - link
What a worthless attempt at sarcasm:Why not just say something normal like:
It's not omg-witty, but at least that sentence doesn't end with a preposition. @___@
JarredWalton - Tuesday, November 7, 2006 - link
Which obviously ruined the whole article. Seriously yacoub, a little tact with posting would be nice. Rather than stating:You could have just said something normal like:
We are real people, and derogatory adjectives like "worthless" tend to irritate more than help. You may not intend it that way, but imagine for a second someone talking about what you worked on for a couple days last week and describing it as "absolute garbage". The key word in "constructive criticism" is to actually make it constructive.
Now, thanks for the suggestion, and I'm more than happy to change things a bit to appeas people. Like most people, I just appreciate a bit more consideration, even if I'm just doing my job here. :) As the old cliche goes, you can catch more flies with honey than with vinegar. (Well, my grandpa used to say that anyway.)