Gamma correction

[Threepwood]

Dell U2410 IPS with wide gamut CCFL backlight calibrated with i1 Display Pro in srgb color space, ICC in windows, 3DLUT in MadVR.
2002 Sony Trinitron KV-29FX30, also calibrated.

Thank James. I had a quick look and while that Dell was a nice display when it hit the market 10 years ago, it does not feature a hardware LUT and as soon as you attach your MiSTer (or anything else than the OS you calibrated it with) your calibration is gone. Same for the Sony.

I assume you use the i1Profiler included with the i1Display Pro (I use that colorimeter, too)? If not, take a look at DisplayCAL (OpenSource), you can get even better results with that.

EDIT:

Again, these 1.2-1.4 number which result in 2.64 to 3.08 gamma is a ridiculously wrong thing to do,, especially if it is baked into the core.
I understand you advocate for this because for some reason it looks good on your display to your eyes (very subjective), but we don't want something like this permanent in a core especially if it is a purely cosmetic feature and done wrong (imo).

I guess with "we" you mean "I" and your impression is purely subjective, too, when you use the MiSTer. Your displays are not calibrated outside of Windows/Mac OS and more likely to display wrong colors, due to the white point calibration that requires you to setup RGB individually and then are corrected by a LUT. Besides, since hardly any user will use a real calibrated display it is good to have a few options to choose from individually. That is the point: It is an optional feature that you can as well leave off if you don't like it.

[JamesF]

Yep, using DisplayCAL and MadVR for years now, vastly more accurate than all the other options.
HCFR+MadVR for TV's, it's also freeware.

Anyway, I hope paulb-nl can make that polynomial gamma curve work, as it is the 'right' way to do it (imo).
From there, we can adjust the parameters to look even 'better', or closer to a CRT.
Hopefully Sorgelig can think of something that works outside of the core, like a filter.

EDIT:
WE, as 'all mister users',, but as I can only speak for myself, yes, I mean 'I'.
I would have though that someone that knows a thing or two about display calibration would understand that a gamma of 3.0 is the wrong thing to do.

@paulbnl
How do you set your Brightness level on your CRT, with a blackscreen?

[Threepwood]

I would have though that someone that knows a thing or two about display calibration would understand that a gamma of 3.0 is the wrong thing to do.

In an ideal world where everybody had a good, calibrated screen that displays colors accurately forever and out of the box, I would agree to you. Sadly there are a variety of displays, technologies and different assumptions with which the games for the individual MiSTer cores were made. Those assumptions are made with consumer devices in mind, not usually professional equipment (same in audio production where audio is mixed using studio monitors AND an cheap radio speaker, because it has to sound good on those, too). The correction is relative to what the display shows, not to a fixed reference point and as paulbnl pointed out, byuu even recommends a gamma correction of 1.5 and his example images look way better to me, too.

Therefore I would like to repeat my question @paulbnl: "Would it be possible to integrate a reference image, like many games use them to roughly calibrate gamma (e.g. The Mark of the Ninja and many horror games) that says "adjust gamma till [object] is barely visible"?

Anyway, I hope paulb-nl can make that polynomial gamma curve work, as it is the 'right' way to do it (imo).
From there, we can adjust the parameters to look even 'better', or closer to a CRT.
Hopefully Sorgelig can think of something that works outside of the core, like a filter.

Whatever proves to be the best and also convenient solution in the end, I am all for it and I trust paulbnl and the devs to make the best decision.

[kitrinx]

I'm not a pro with the fancy calibrations and color corrections and things, but gamma is IMHO the responsibility of the TV, not MiSTer. If your TV can't do it and you care a lot about gamma, then you should probably get a better display. Features aren't free in FPGA. They take up space and make it harder to build stable cores, so adding them "just because" isn't really a great choice. If you feel like it really must be part of mister, the correct place for this is as part of the framework, with a standard setting along with the filters. It doesn't belong as a core specific setting, and it *definitely* doesn't belong baked in.

[Threepwood]

I'm not a pro with the fancy calibrations and color corrections and things, but gamma is IMHO the responsibility of the TV, not MiSTer. If your TV can't do it and you care a lot about gamma, then you should probably get a better display

We covered these arguments already. No LCD display allows for the necessary gamma curve corrections and MiSTer is the device emulating old hardware where games were created with different gamma curves in mind than you will find today on the market. Therefore it is the responsibility of the emulating device or software to do the corrections that definetly no consumer display can do and I am not aware of any other LCD that can.

In the article paulbnl posted earlier, byuu describes what needs to be done to fix this in emulation. MiSTer is, in my humble opinion, the best emulation device there is and it would be great if it would exceed or at least keep up with software emulation in all areas to preserve the authenticity of the look and feel of the games.

It is a great idea paulbnl had and looks much closer to what it was supposed to look. You can also check byuu's article where screenshots show the severity of the fault in colors and gamma without correction.

It doesn't belong as a core specific setting, and it *definitely* doesn't belong baked in.

It needs to be an option that can be set per core as you can see with byuu's article. Different old display technologies (Game Boy vs CRT) need different corrections. Where it is best implemented and how, is for the devs to decide. It sounds like you already have a solid idea on how to do this and filter settings sound like a good option.

[paulbnl]

Again, these 1.2-1.4 number which result in 2.64 to 3.08 gamma is a ridiculously wrong thing to do,, especially if it is baked into the core.
I understand you advocate for this because for some reason it looks good on your display to your eyes (very subjective), but we don't want something like this permanent in a core especially if it is a purely cosmetic feature and done wrong (imo).

Can you elaborate on what does not look good specifically? Like some specific spots in games? It seems like you are calling it wrong just because the numbers are high. Maybe you are just used to everything looking greyish and shadow detail being very bright.

I have added a build with your polynomial curve in the same link as the previous builds. It seems like a minor difference to me compared to 1.3. The bright shades are slightly brighter.

Would it be possible to integrate a reference image, like many games use them to roughly calibrate gamma (e.g. The Mark of the Ninja and many horror games) that says "adjust gamma till [object] is barely visible"?

That is above my pay grade Maybe that is something for the 240p test suite.

[JamesF]

By 'looks wrong' I mean the middle shades are much too dark compared to pure white,, pure white is used for text and plenty other things (clouds, eyes, etc..). That's the false effect of too high gamma value.
To fix this abrupt transition we use polynomial gamma to gradualy return to 2.2 from around 70% input to 100% input.

The 'poly' gamma curve is not a minor difference, it looks much more natural and doesn't transition from from middle to bright tones so abruptly, much closer to a CRT.
It's the 0% to 20% input (0-50 from 255) we want to concentrate on, not the entire gamma curve, since a CRT gamma is overall closer to 2.2-2.4 as mentioned earlier but with darker blacks.
Remember, we are trying to slightly reverse the darker side of sRGB gamma curve of LCDs, not change the entire gamma curve.

Please try 2.9_3.8 poly values as that will be closer to a gamma of 2.6 at 0-20% input, and 2.2 at 75-100% input.
As 3.15_4.4 produces a gamma of 2.8 at 0-20% and 2.2 at 75%-100%,, and imo that's slightly too dark compared to my Trinitron, but already much better than the pure 1.2-1.3 values.
You can just replace the 1.2 and 1.3 values with the two poly equations, since as explained above, we don't want to touch the brighter shades.
Maybe: Off, Poly 1.2, Poly 1.3, and 1.3.



PS.
What I do is look at the first black step in the grey ramp in 240p test suite.
Then Yoshi's Island first level mountains in the background.
Also, my Dell U2410 has a gamma curve closer to the Ploy already, so it looks 'right' in the OFF setting and does not need correction,, that tells us not all LCD monitors calibrated equally from the factory.

[Threepwood]

What I do is look at the first black step in the grey ramp in 240p test suite.
Then Yoshi's Island first level mountains in the background.

I assume you take screenshots on the MiSTer that you then view on your PC?

EDIT: This test suit was written in 2014. Is there any information wether or not it was written with adjustments for CRT Gamma curves in mind? I would think this outputs unscewed test patterns that look normal on a LCD, too, while games had certain ranges boosted to compensate for a CRT's gamma curve. In that case using the 240p test suite would be useless for our use case here.

Also, my Dell U2410 has a gamma curve closer to the Ploy already, so it looks 'right' in the OFF setting and does not need correction,, that tells us not all LCD monitors calibrated equally from the factory.

How old is your monitor by now?

EDIT: Attached you find two comparisons of gamma levels using Yoshi's Island and Super Metroid (screenshots taken on MiSTer with the latest core modified by paulbnl)

[paulbnl]

I am going to make it like the scaling filters so we can just use txt files for the different curves.

It is better than having fixed curves in the core.

[Threepwood]

I am going to make it like the scaling filters so we can just use txt files for the different curves.

It is better than having fixed curves in the core.

Awesome, thanks for the hard work!

[JamesF]

I am going to make it like the scaling filters so we can just use txt files for the different curves.

Thank you.

[BitsNStuff]

That sounds fantastic, paulbnl. Thanks for putting the work in.

[Sorgelig]

I just want to remind: if it will use LUT for gamma correction it's very unlikely will be integrated into framework as it will produce a lot of timing violataions at 148MHz. For framework integration this LUT needs to be highly optimized with pipe-lining to cope with high HDMI data rate.

[paulbnl]

I just want to remind: if it will use LUT for gamma correction it's very unlikely will be integrated into framework as it will produce a lot of timing violataions at 148MHz. For framework integration this LUT needs to be highly optimized with pipe-lining to cope with high HDMI data rate.

Sure I can add pipe-lining. With the SNES I haven't seen any timing differences with using it in the framework or in the core.

As I understand it it just needs to work on CLK_VIDEO because the LUT will be used on the input to the scaler so for the SNES it will only be at 42Mhz.

[Sorgelig]

Because direct video requires high super-sampling clock, some cores may output video at up to 100MHz (or even a little higher).
But surely it can be guarded by clock enabler as it's on input side where clock enabler is still valid. Though i don't know how timings will be calculated

[yxkalle]

Doesn't it make more sense to apply the LUT before the scaler, or is that simply not possible?

[softtest1]

Maybe you can make it so that we have two filter files; one for gamma and one for scaling. Both would be selected in the menu the same way, and the gamma filter would be applied before the scaling filter.

[paulbnl]

I have updated the link with updated cores. You'll need to copy the Gamma folder to the sdcard and replace the MiSTer binary for the Gamma correction option to appear under Scaling Filter.

The txt files for the gamma curves are just 256 lines of values 0-255.

The Genesis is now also included because it has a 100Mhz video clock. I haven't seen negative changes to the timings in the Timing analyzer.

https://drive.google.com/file/d/12FrBLf ... sp=sharing

Doesn't it make more sense to apply the LUT before the scaler, or is that simply not possible?

Maybe you can make it so that we have two filter files; one for gamma and one for scaling. Both would be selected in the menu the same way, and the gamma filter would be applied before the scaling filter.

It already works that way

[JamesF]

This is absolutely amazing.
I hope to see this merged to main.

Matlab script for Poly Gamma:

Code: Select all

in = 0:1:255;
a = ((in/255) .^ 2.9);
b = ((in/255) .^ 3.8);
Out = 255 * ((2 * a - b) .^ (1/2.2));
Y = round(Out);
Y1=Y(:)

Script for normal Gamma:

Code: Select all

in = 0:1:255;
Out = 255 * ((in/255) .^ 1.1);
Y = round(Out);
Y1=Y(:)

Use here if you don't have matlab:
https://www.tutorialspoint.com/matlab/try_matlab.php
Just copy the result and paste in the gamma text file.
.

[softtest1]

It already works that way

Thank you. I tried it now and it works really well.

[JamesF]

@paulbnl
Are you going to PR it so Sorgelig can review it?

[Threepwood]

Indeed, I hope it makes it into the main release and all cores get updated over time. I tried the Genesis core with different games yesterday and it works amazing. The games really benefit from gamma adjustment.

[paulbnl]

Are you going to PR it so Sorgelig can review it?

Sure, I just had to figure out how to add scrolling to the system menu but now that's working properly I have sent the pull requests.

[bhamadicharef]

For those interested in research and implementation aspects
there is this old IEEE paper, PDF is online ...

A Flexible Architecture for Precise Gamma Correction
Dong-U Lee, Ray C. C. Cheung, and John D. Villasenor
IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Year: 2007 | Volume: 15, Issue: 4 | Journal Article | Publisher: IEEE
https://www.researchgate.net/publicatio ... Correction

Best regards

[Sorgelig]

Is it possible to re-use ADV chip output converter dsp for gamma correction? It's not LUT of course..