Submission Calibration

[Neil]

Hi Guys!

Some of our members have already made contribitions which is great.

I'd like to call attention to these to make sure the questions are understood:

http://www.mtbs3d.com/m3ga/index.php?sh ... _id&val=10" onclick="window.open(this.href);return false;

I don't have this game, but if are getting serious shadow errors, and you can turn them off, please do so. Otherwise the remaining error undermines the game's rating. We are after what remains after all options are explored (e.g. settings are reduced). The way this is written, it suggests that while shadows can be reduced, there are still shadow errors on the screen.

http://www.mtbs3d.com/m3ga/index.php?sh ... s_id&val=8" onclick="window.open(this.href);return false;

A depth-only situtation means reducing convergence to near minimum. I think that would have solved this problem. However, in general, should separated light sources be a critical error, or a secondary error?

http://www.mtbs3d.com/m3ga/index.php?sh ... _id&val=13" onclick="window.open(this.href);return false;

We will clarify the question a bit more. When we are talking about the HUD, we are talking about the in-game HUD, not the game menus system. It's also important to check for that minimum convergence option (depth only) to counteract this problem.

Please share your questions and ideas so we can fine tune M3GA's Q&A process.

Regards,
Neil

[BlackShark]

Bug found : it seems the game list in doesn't like the apostrophe character : '
The game Mirror's Edge shows in the list as "Mirror"
The game Assassin's Creed shows in the list as "Assassin"

A few suggestions :
Put a small checklist of the titles of the 4 steps at the top of the page to make the user know there are 4 steps and knows roughly where he is in the process.
Provide a way to go back to a previous page in the user made a mistake.
Add a final choice "validate report / cancel report" on the last page in case the user made a mistake.

Part 2 and 3 are not clear enough : make sure the user understands this part is only if there is no way to remove the bug by disabling some graphic option.
in order to make sure the user understand I suggest you change the order of the steps :
-step1 config
-step2 reduced settings used to correct bugs
-step3 major issues that cannot be corrected
-step4 minor issues that cannot be corrected


In part 2 : "Check here if ALL these flaws go away in an estimated ... blablabla .... AND if necessary, game settings are reduced"
-> remove the end : it introduces a lot of confusion about this part : the instructions say this is about stuff that can't be corrected by reducing settings and this line talks about having stuff fixed by reducing settings

Make two different items for ingame HUD and menus (2D menu only), it seems you are already aware about this issue.

Reduced graphic option (other) -> add a text box to describe the option (keep some room, in case this option needs to be described with an ini file)
Add a remark box if the user wants to add something important like :
3D mode needs to be activated manually in ini file located in the following folder c:\ blablabla
Use a modified profile <text>


I have trouble describing the clipped decals in the unreal engine 3 games and the clipped flashlight to the side in left4dead when using high separation/convergence with the iZ3D driver.
You should add something in the minor issues report (currently part3)

The croshair part is also confusing .
I suggest you make the crosshair a three-part item.
tick1- the in-game main crosshair is at wrong depth
tick2- some of the in-game secondary crosshair are at wrong depth (sniper rifle scopes, binoculars, etc...)
tick3- the driver backup crosshair features are not working
tick this item if none of the following works : driver lasersight feature, driver left/right eye shift mode
or if the ingame crosshair cannot be disabled and interferes with the driver backup crosshair system.

[Neil]

Excellent feedback, and thanks for your submissions!

Regards,
Neil

[BlackShark]

bug report :
it seems the DX9 and DX10 report crush one another, i noticed that with resident evil 5 and assassin's creed

[stepsbarto]

Hi Guys!

Some of our members have already made contribitions which is great.

I'd like to call attention to these to make sure the questions are understood:

I don't have this game, but if are getting serious shadow errors, and you can turn them off, please do so. Otherwise the remaining error undermines the game's rating. We are after what remains after all options are explored (e.g. settings are reduced). The way this is written, it suggests that while shadows can be reduced, there are still shadow errors on the screen.

..Hi Neil,

I think this was me

to Mirrors Edge: http://www.mtbs3d.com/m3ga/index.php?sh ... _id&val=10" onclick="window.open(this.href);return false;
... the problem is that you cannot turn shadows off inside the game ,you have to edit some .ini file ,which is a bit difficult to do.And this file needs write-protection for me otherwise its changed by the game .
so how to describe this ...?
and

A depth-only situtation means reducing convergence to near minimum. I think that would have solved this problem. However, in general, should separated light sources be a critical error, or a secondary error?

...to Trackmania: http://www.mtbs3d.com/m3ga/index.php?sh ... s_id&val=8" onclick="window.open(this.href);return false;

.. Trackmania is really great in 3D and has even a built-in stereoscopic engine (which works without flaws for me with Zalman->line interleave).With the nVidia driver some halos are wrong but there are not many lights in the game actually and decreasing depth is not necessary.
In fact it should be rated much higher the lights are the only (small) problem ...


....so maybe there should be some place for notes to add where you can write some free text ?
... and an additional option for games with stereoscopic engine ?

regards
steps

[BlackShark]

Another suggestion.
On the reduced graphics settings :
"Check here if total display setting was a general "medium" setting option or similar, and no advanced options settings are available. '

I suggest a rephrase :
Check here if you cannot isolate the specific graphic option that causes the bug. (you have to turn the overall graphic setting down to fix the bug)

[Neil]

Ahah! It sounds like you did things more or less correctly! This is excellent news. I was worried it would be too hard to understand.

If we move the light question from Tier 1 to Tier 2 errors, that would increase it to Silver level from Bronze. Keep in mind that secondary scoring (numeric) can be handled completely differently.

Is Trackmania strictly anaglyph, or does it support additional options? The Native Support list needs to be handled differently, I think.

Regards,
Neil

[BlackShark]

Trackmania supports anaglyph, line interleaved and side by side (squashed)
There are bugs though, and some of them vary according to the output used. (side by side has completely wrong hud, line interleaved has vertical separation issues when zooming with custom cameras)

[BlackShark]

I forgot one very important suggestion :
on the main issues :
-> the driver does not hook the game, 3D not working

[BlackShark]

I've just found this : http://www.mtbs3d.com/m3ga/index.php?sh ... s_id&val=7" onclick="window.open(this.href);return false;

Turn shadows off completely = gold ranking 90% ???
In my opinion having to turn shadows off completely should be at best silver ranking.

[BlackShark]

Again an other suggestion :
"Scenery
# The background terrain is rendered in 2D or at screen depth instead of S-3D. For example, while looking at the screen without 3D glasses, the background terrain or sky refuses to split apart at all (e.g. mountains, clouds, buildings, etc.) compared to the other parts of the image."

I suggest you split this item in different items :

Background scenery :
-tick1 : 2D skybox at wrong depth (some games use a 2D panoramic picture for the far background scenery, tick this box if this picture is displayed at the wrong depth , eg. in front of other 3D objects or at screen depth)
-tick2 : 3D skybox at wrong depth (some games use 3D objects for the background scenery, tick this box if these objects are displayed with a wrong scale, with eyes swapped or in 2D)

In the screen objects section :
"Gun or weapon separates too far apart (>80% doubled) when achieving out of screen effects. "
I know your tendency to use these words when fps weapons respect the world scale and YOU are the culprit who deliberately uses waaay too much pop-out (because you love pop-out effects), in order to avoid false statements I suggest you modify this item as follows :

*World and objects relative scaling issues (eg car interiors, guns or swords in the player's hands)
Tick1 -> Close objects are drawn at a smaller scale than they should be compared to the rest of the world making them immediately separate too much when using pop-out
Tick2 -> Close objects are drawn at a bigger scale than they should be compared to the rest of the world, making them seem to pierce through farther scenery

[Neil]

Fixing, fixing, fixing.

Keep the feedback coming.

The gun issue might be tougher to fix. We will try.

Regards,
Neil

[BlackShark]

scoring :
"Rare flicker or flashing patterns that can be easily ignored" -> straight down to 80% ???
In my opinion should be more like gold 90% or 95% (from the "can easily be ignored")

I've also found hard to describe the event when a minor special FX causes small glitches that can easily be ignored.
Examples : in the iZ3D driver in Portal, the portal opening FX on the rim of the opening hole looks different between the two eyes, it's easy to miss but it's there.
others tiny bugs can be found like that and I feel reluctant to put 100% platinum mark when I see even the tinyest glitch.

Maybe introduce a 99% platinum ranking with a tickbox, "I found a tiny glitch but it's hard to spot"

[Neil]

Don't worry about the numeric scoring. That is going to be heavily calibrated right down to the decimal.

The certification levels have hard and fast rules, while the secondary scoring can follow a very different path.

Regards,
Neil

[Neil]

We added a lot of BlackShark's recommendations and more. There is more to come, but it will take time.

See if it is better now.

Regards,
Neil

[BlackShark]

Thanks, neil.

It seems there is still a bug with Assassin's Creed and the two DX9/DX10 reports. The game name has been cut again.
There are two game entries now with this game : - Assassin's creed (only one DX9 report) and - Assassin (with two DX9+DX10 reports)

New suggestion :
The game titles are currently sorted by certification ranking : add an other sorting method by game name (ignoring ranking)

Another suggestion, the database seems to be missing entries for games with native support.
I suggest on the first page, add "native 3D" to the list of 3D drivers.

When native 3D is selected, add immediately after a second page that allows to tell what 3D outputs the game supports
add a checkbox for every known 3D display output :

--> API based outputs
-Nvidia 3Dvision API
-iZ3D API (when it will be available)
-VR920 API

--> Direct to display based outputs
-Anaglyph Red/Cyan
-Anaglyph Yellow/Blue
-Anaglyph Magenta/Green
-Anaglyph (other)
-Dual Head
-Dual Head with mirror
-Dual Head iZ3D
-Checkerboard
-Hdmi 1.4 Frame-Packing
-Frame Alternate
-Interleave (Row)
-Interleave (Column)
-RealD proprietary
-Sensio proprietary
-Side by side (Left-Right)
-Side by side (Top-Bottom)

-------------------------
Just noticed one thing :
Avatar(native) : ther real name of the game is "James Cameron's AVATAR : the game"
Do not put "native" in the game name, when a game has a native mode, it may be interesting to compare between the native mode and drivers in case some display is not supported.

[Guig2000]

I've just found this : http://www.mtbs3d.com/m3ga/index.php?sh ... s_id&val=7" onclick="window.open(this.href);return false;

Turn shadows off completely = gold ranking 90% ???
In my opinion having to turn shadows off completely should be at best silver ranking.

In supreme commander 2 this issue is not important I think.
Depends of games but I think lost 20% for only disbling shadows would be too much.


Questions about M3GA:
When I've tryed a game on the last iZ3D beta and the last iZ3D final release, should I report it two times even if they have the same result?

Is submiting a second time a game with the exact same config and driver than the first time I submited it is a way to edit my first submission or it will simply add a second submission on the database?
If it will add a second submission, I suggest to add an edit option so that people can correct their errors.

The games a member submitted could appear on the member "User Control Panel".

[Neil]

Add them both if it doesn't take too much time. No harm.

I will look into Assassin's Creed.

Regards,
Neil

[BlackShark]

scoring again :
the 3D driver does not hook the game -> 20%
I suggest the absolute minimum rating instead, or maybe a special rating "N/A"

[BlackShark]

BUMP
I repeat these feature requests :

Another suggestion, the database seems to be missing entries for games with native support.
I suggest on the first page, add "native 3D" to the list of 3D drivers.

When native 3D is selected, add immediately after a second page that allows to tell what 3D outputs the game supports
add a checkbox for every known 3D display output :

--> API based outputs
-Nvidia 3Dvision API
-iZ3D API (when it will be available)
-VR920 API

--> Direct to display based outputs
-Anaglyph Red/Cyan
-Anaglyph Yellow/Blue
-Anaglyph Magenta/Green
-Anaglyph (other)
-Dual Head
-Dual Head with mirror
-Dual Head iZ3D
-Checkerboard
-Hdmi 1.4 Frame-Packing
-Frame Alternate
-Interleave (Row)
-Interleave (Column)
-RealD proprietary
-Sensio proprietary
-Side by side (Left-Right)
-Side by side (Top-Bottom)

-------------------------
Just noticed one thing :
Avatar(native) : ther real name of the game is "James Cameron's AVATAR : the game"
Do not put "native" in the game name, when a game has a native mode, it may be interesting to compare between the native mode and drivers in case some display is not supported.

[Neil]

We can add this a bit later. It shouldn't be hard to do. Right now we are focusing on editing entries and scoring.

Regards,
Neil

[Guig2000]

Maybe instead of have a major problem page and a minor problem page, the two could be merged and the user could have to choice if the issue is major or minor by ticking a box or by using a slider.
Because for example, on dragon age: origin, the issue "# Player tags or markings associated with game characters are rendered only at screen depth, and not at the depth of the individual objects or characters." is very minor, while due to the massive numbers of units on screen, it could be considered as major or intermediate in forged alliance.

[BlackShark]

Maybe instead of have a major problem page and a minor problem page, the two could be merged and the user could have to choice if the issue is major or minor by ticking a box or by using a slider.
Because for example, on dragon age: origin, the issue "# Player tags or markings associated with game characters are rendered only at screen depth, and not at the depth of the individual objects or characters." is very minor, while due to the massive numbers of units on screen, it could be considered as major or intermediate in forged alliance.

Agree that's a good idea.
The same issue happens in UT3, the character names are huge and are a major annoyance whereas in Left4Dead they're only minor.
The same goes with most other items : sky at screen depth is minor issue in Assassin's Creed but a major issue in Shattered Horizon
Dynamic Shadows at screen depth are a minor issue in Tomb Raider Anniversary or in UT3 but are a major issue in Racedriver GRID, etc... etc...

[Neil]

Thanks for the idea. I think this would get very messy very fast. Keep in mind that the scoring and certification levels are handled separately, and there will be a comments section at the end that you can share subjective feedback, etc.

Regards,
Neil

[cybereality]

You should be able to decide how severe the bug is or isn't. A set scoring is never going to work.

[Neil]

First, the scores need to be refreshed. What we are seeing is out of date as far as how well a game is scoring , etc. in numeric form. You will see big changes once this is done.

It's important to remember that the results shown on M3GA aren't just for the benefit of gamers, it's for the benefit of game developers. Game developers can't have a case of "this error is critical in this game, but not critical in this game", etc. We can have a comments section for this. We can even encourage gamers to read the comments for a fuller story, etc. to draw attention to this.

At FMX 2010, we went through this with several game developers in person and presented M3GA too. I'm hesitant to tell you who we spoke with because it's not fair to put them on record, but so far, everyone liked what they saw - and it ties in with the logic they follow when troubleshooting internally. In fact, they weren't surprised with the results they got with some of their games, which is good because they recognized the problems (and were able to verbalize them).

Something to think of too. I saw a great presentation by Sony and how they keep consistency with all their studios and partners. There are rules that are followed 100% of the time so everyone is on the same page. I will talk about it more in a later editorial, but it completely tied in with the logic we follow with M3GA.

Regards,
Neil