Bapco Benchmarks?

[Javabri]

Interesting read about Sysmark and Bapco. If true, this could really invalidate the accuracy of Sysmark scores.

http://www.amdzone.com/#11

Java

[Ioman]

Originally posted by Javabri
Interesting read about Sysmark and Bapco. If true, this could really invalidate the accuracy of Sysmark scores.

http://www.amdzone.com/#11

Java

I truely questions the validity of this. I heard a while ago about this incident and was told that Intel purchased the company that owns Sysmark, now whether or not they changed the benchmarks to favor the Intel processors or not, I do not know. I doubt it.

You are also reading this story on an AMD website so I am sure there is bias there as well. I cant tell you that Sysmark is a program run by people and manufacturers that know better. If Intel was indeed influencing this program then those manufacturers and people would know better, so why would they do that? It just does not add up....I would not believe everything you read in the Inquirer either. I remember reading an article on Nvidia trying to purchase AMD I beleive. That of course was not the case.

[Entropy]

Originally posted by Ioman

I remember reading an article on Nvidia trying to purchase AMD I beleive. That of course was not the case.

Man... Ioman... Ur memory if funny man :P

That article on nVIDIA buyin AMD. That was an April fools joke on www.tomshardware.com :D


Anyway. I've both heard and read about Sysmark being biased to Intel. I've also tested this theory as well.
I benched my Athlon XP 1800+ with Sysmark and with SiSoft SANDRA. and Sysmark was the only one that told me a P4 1.8ghz had any chance of competing with my 1800+. :P

So I'm inclined to belive what I've heard about Sysmark bein biased ;)

[Cloud]

Hey I remember that article. It did not say Nvidia bought them but that they were thinking about it. In anycase, I don't think Sysmark is biased either, It is not in their best interest to be biased.

I think you guys are all just zealots! :argue

[Unregistered]

Hey ioman,

While AMDzone is a site dedicated to reviews of AMD products and platforms, it is not a site run by AMD. Therefore, I would not necessarily jump to the conclusion that they are biased. In addition, I do not necessarily believe everything at the Inquirer, however there are multiple sites reporting similar things. Van's has tested the theory and their report seems to verify this info. From his previous articles and reviews, I have found him to be a person of integrity and I trust his results. I don't wear my emotions on my sleeve nor do I make judgements on limited information. I try to fully investigate a topic before coming to a conclusion. If additional data is added, I review my previous decisions.

I find it very disconcerting that Bapco shared the same address as Intel for quite some time. It was only recently they moved out of the Intel building. While this alone is circumstantial, it is certainly very interesting.

How do you account for the radical difference in scores between Sysmark 2001 and Sysmark 2002? AMD's documented evidence is very persuasive. However, they do have a vested interest. I also find it interesting that many other benchmarks often provide very different scores for similar tests. I personally think there is something fishy going on, but that is just my opinion.

Why would Intel do it? How many people actually know and understand how a benchmark works? I would venture to guess that many people think they know, but few really understand the mechanics of a benchmark. Any person can download the s/w and run it, but few know what and how the tests are performed. People only seem to care about their "score" with no idea how this score was derrived. Given this, it is completely viable that a benchmark or set of benchmarks could be modified to favor one platform or processor over another.

I ask again, what accounts for the change in scores from 2001 to 2002? I can only come to the conclusion that if the platform remains the same, but the scores radically change, then something changed within the application.

Regards,
Java

[Ioman]

I cannot argue with you if you have done some research. This would have to be one of the biggest mess ups courtesy of Sysmark and Intel.

Now will someone please show me a document saying Intel owns Sysmark? I would look for it if I knew where to look. Most states allow you to look up a business based on company name and the state they are in. For example if I went to the state of Oregon's website and did a business search for Designtechnica or Bizzbum, it would say "owned by Designtechnica Corporation".

Maybe there are two seperate issues. Is Sysmark "owned" by Intel or simply influenced by Intel? In either case I think the trust in the software, benchmarks and company name of Sysmark is gone. They have probably ruined themselves into bankruptcy.

But at the same time, I would not put it past other companies to be doing the same thing. I can tell you that we get more traffic than most sites on the net and we still cannot get an AMD processor to use in any of our projects, yet any side with "AMD" in it will get a top of the line processor. I would question ANY benchmarks from either Intel or AMD where a site has their name in their website name.

[Javabri]

Hi ioman,

I don't recall ever reading that Bapco was owned by Intel nor do I know of anyone making those claims. However, I do believe that Bapco was founded by Intel, Dell and a few other major players. AMD just recently joined Bapco in an effort to better understand how they are performing their benchmarks. What I do know is that Bapco used to be located on the bottom floor of Intel's headquarters in Santa Clara, CA. Again, this does not indicate ownership, but it does seem a little strange for a company that is supposed to be providing independent benchmark software.

I think this is really an issue about if the benchmarks were skewed and if so, how were they skewed. If this turns out to be true, then I think Bapco has some explaining to do. In addition, I would hope that reviewers would not use Sysmark until Bapco provides a solid and unbiased benchmark routine.

Regards,
Java

[Ioman]

Thanks for bringing this to our attention Java, we will not be using any Bapco products for benchmarking.

On another note, go the front page of the forums and log in down in the bottom right hand corner of the screen. If that does not work, e-mail me at ianbell@designtechnica.com and I will send you the correct e-mail password.

Thanks

[Ioman]

After doing research, I have found that I do not agree with the similar statements. Here is an article from Toms Hardware I found. He specifically says these claims were simply not true.

http://www6.tomshardware.com/blurb/02q3/020825/index.html

It is about the 3rd page in.

Sorry Java, difference in opinions :(

[Entropy]

Originally posted by Ioman
After doing research, I have found that I do not agree with the similar statements. Here is an article from Toms Hardware I found. He specifically says these claims were simply not true.

http://www6.tomshardware.com/blurb/02q3/020825/index.html

It is about the 3rd page in.

Sorry Java, difference in opinions :(


There in brings up a whole new facet to the convo Ioman. For a long while I've found Tom's Hardware to be a somewhat Intel-biased themselves. This is not to say that they dont make great articles, but the team of reviewer's at Tom'sHW have made it clear they prefer Intel. ;)

Tom'sHW does, however, give accurate results in their benchmarks (from what I've expierienced), they simply have a tendancy to provide more support for Intel in their comments within the article(s).


P.S. Ioman... When did you ever start using or paying any mind to Tom'sHW? :P

[Ioman]

Originally posted by Entropy



There in brings up a whole new facet to the convo Ioman. For a long while I've found Tom's Hardware to be a somewhat Intel-biased themselves. This is not to say that they dont make great articles, but the team of reviewer's at Tom'sHW have made it clear they prefer Intel. ;)

Tom'sHW does, however, give accurate results in their benchmarks (from what I've expierienced), they simply have a tendancy to provide more support for Intel in their comments within the article(s).


P.S. Ioman... When did you ever start using or paying any mind to Tom'sHW? :P

Well, I have always been a fan of his site. I usually go to his, Sharkyextreme.com and Anandtech.com

Someone pointed me to this article, I did not find it on my own. I think his article was OK and not Intel biased. I also gained some respect for him and his site. I am glad he stays out of the battles between websites and companies. I as a reader appreciate that,

[Unregistered]

Hi ioman,

While I understand your comments and we may have a difference of opinions and that's OK. I think there should be some investigation of this issue.

I too read Tom's comment on the subject. I am not convinced that Tom's has really evaluated the material provided. He mentions how "AMD came to me first...". I don't know if this is true or not, but I did see the AMD powerpoint while it was posted on the web. I think are reasonable questions raised that deserve answers. To me, it seems Tom's article was more about bashing Van Smith and Kyle Bennet and this is clearly stated.

"Today, however, I was informed about a fight that's going on between Kyle Bennett and Van Smith, the two people in this 'PC Hardware World' that I despise most. First I considered this battle quite amusing. It's always nice to watch two of your enemies fighting each other. "

Later he wrote

"It is somehow unbelievable that fate would make it so good for me. I never thought that I would come clean with this behind-the-scenes stuff that happened with Kyle Bennett and Van Smith, two of the loudest and most ruthless "journalists" on the Internet. However, fate threw me a bone and the two guys started to hack at each other." This is clearly not what I would expect from any journalist. These types of comments are best left out of any article.

Not exactly objective journalism. When an article is peppered with these types of comments, I have to question the credibility of the article in question. While Tom's often has good articles, he also posts a lot of Intel Biased articles and directly attacks AMD supporters. Read the Second Hand Smoke series, if you think I am mistaken.

Out of a several page posting, I only found 1 paragraph where he comments on the material provided by AMD. Yet, no where do these comments directly address the questions raised by AMD. The article on Van's hardware directly addresses the questions in addition to posting his own results of the investigation. Why did Tom not do the same thing?

Now, no where did I say in any of my posts that the claims made by AMD or anyone else were true, only that if true, this really raises suspicions about Bapco and Sysmark. I think this calls for independent investigation since AMD did post some very hard numbers and clearly documented their findings.

I just can't get over the test results. This is very strange. I have not seen anything that explains why the vast differences in scores from Sysmark 2001 to Sysmark 2002 even though the platform remains the same.

I think the first problem is that people rely too much on benchmarks which are inherrently flawed in the first place.

I am posting some links that are relevent to the topic at hand.

Benchmarking comments in general
http://www.aapltalk.com/shootouts/benchmark_notes.html

Dean Kent's posting over at Aces Hardware:
http://www.aceshardware.com/forum?read=80028558

Van's Review (you can also download the AMD powerpoint here)

http://www.vanshardware.com/reviews/2002/08/020822_AthlonXP2600/020822_AthlonXP2600.htm

Sorry for the lengthy post, I just think we are all entitled to the truth. If there is nothing wrong with Bapco's benchmarks then that is great, but right now there seems to be enough evidence to call this into question, even if Tom does not agree.

Please understand that I am not trying to be argumentative, only trying to clarify my position on the subject. I have a great deal of respect for this site as well as for you, ioman.

[Javabri]

Hey ioman,

I made the previous post. By the way, I did try logging in and for some reason it is still showing me as Unregistered. I also logged back in to make this post. I don't know if this is going to work right, but I am trying.

Not sure what is going on.

Regards,
Java

[Ioman]

Javabri,

Thank you for the kind words. Your opinions are always welcome here and will always be respected.

I read your links and at this point in time, I am unsure of which to believe. It is sad to see such well respected website journalists like Tom, Kyle, and Van disagreeing and bashing each other in unprofessional ways. It is even more upsetting to see companies use these very sites for press and for influence.

Thankfully we have not done any processor reviews on this site so we have no incorrect results posted. What is discerning is that there are such a large number of people that have been taking these benchmarks and the reviewer’s words as being truthful, with all parties being blind.

In all honesty, I have not taken any benchmarks too seriously when making a CPU purchase. I think that the general consumer thinks the same way. If the CPU is affordable and significantly faster than what I had before, then that is enough reason to upgrade.

A little debate is never bad as long as it is constructive, thank you for the challenge Javabri!