Gamma correction

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

JamesF wrote:Dell U2410 IPS with wide gamut CCFL backlight calibrated with i1 Display Pro in srgb color space, ICC in windows, 3DLUT in MadVR.
2002 Sony Trinitron KV-29FX30, also calibrated.
Thank James. I had a quick look and while that Dell was a nice display when it hit the market 10 years ago, it does not feature a hardware LUT and as soon as you attach your MiSTer (or anything else than the OS you calibrated it with) your calibration is gone. Same for the Sony.

I assume you use the i1Profiler included with the i1Display Pro (I use that colorimeter, too)? If not, take a look at DisplayCAL (OpenSource), you can get even better results with that.

EDIT:
JamesF wrote:Again, these 1.2-1.4 number which result in 2.64 to 3.08 gamma is a ridiculously wrong thing to do,, especially if it is baked into the core.
I understand you advocate for this because for some reason it looks good on your display to your eyes (very subjective), but we don't want something like this permanent in a core especially if it is a purely cosmetic feature and done wrong (imo).
I guess with "we" you mean "I" and your impression is purely subjective, too, when you use the MiSTer. Your displays are not calibrated outside of Windows/Mac OS and more likely to display wrong colors, due to the white point calibration that requires you to setup RGB individually and then are corrected by a LUT. Besides, since hardly any user will use a real calibrated display it is good to have a few options to choose from individually. That is the point: It is an optional feature that you can as well leave off if you don't like it.
Last edited by Threepwood on Tue Oct 01, 2019 11:58 am, edited 1 time in total.
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
JamesF
Captain Atari
Captain Atari
Posts: 159
Joined: Sat Dec 15, 2018 6:46 am

Re: Gamma correction

Post by JamesF »

Yep, using DisplayCAL and MadVR for years now, vastly more accurate than all the other options.
HCFR+MadVR for TV's, it's also freeware.

Anyway, I hope paulb-nl can make that polynomial gamma curve work, as it is the 'right' way to do it (imo).
From there, we can adjust the parameters to look even 'better', or closer to a CRT.
Hopefully Sorgelig can think of something that works outside of the core, like a filter.

EDIT:
WE, as 'all mister users',, but as I can only speak for myself, yes, I mean 'I'.
I would have though that someone that knows a thing or two about display calibration would understand that a gamma of 3.0 is the wrong thing to do.

@paulbnl
How do you set your Brightness level on your CRT, with a blackscreen?
Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

JamesF wrote:I would have though that someone that knows a thing or two about display calibration would understand that a gamma of 3.0 is the wrong thing to do.
In an ideal world where everybody had a good, calibrated screen that displays colors accurately forever and out of the box, I would agree to you. Sadly there are a variety of displays, technologies and different assumptions with which the games for the individual MiSTer cores were made. Those assumptions are made with consumer devices in mind, not usually professional equipment (same in audio production where audio is mixed using studio monitors AND an cheap radio speaker, because it has to sound good on those, too). The correction is relative to what the display shows, not to a fixed reference point and as paulbnl pointed out, byuu even recommends a gamma correction of 1.5 and his example images look way better to me, too.

Therefore I would like to repeat my question @paulbnl: "Would it be possible to integrate a reference image, like many games use them to roughly calibrate gamma (e.g. The Mark of the Ninja and many horror games) that says "adjust gamma till [object] is barely visible"?
JamesF wrote:Anyway, I hope paulb-nl can make that polynomial gamma curve work, as it is the 'right' way to do it (imo).
From there, we can adjust the parameters to look even 'better', or closer to a CRT.
Hopefully Sorgelig can think of something that works outside of the core, like a filter.
Whatever proves to be the best and also convenient solution in the end, I am all for it and I trust paulbnl and the devs to make the best decision.
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
User avatar
kitrinx
Captain Atari
Captain Atari
Posts: 192
Joined: Wed Sep 26, 2018 6:03 am

Re: Gamma correction

Post by kitrinx »

I'm not a pro with the fancy calibrations and color corrections and things, but gamma is IMHO the responsibility of the TV, not MiSTer. If your TV can't do it and you care a lot about gamma, then you should probably get a better display. Features aren't free in FPGA. They take up space and make it harder to build stable cores, so adding them "just because" isn't really a great choice. If you feel like it really must be part of mister, the correct place for this is as part of the framework, with a standard setting along with the filters. It doesn't belong as a core specific setting, and it *definitely* doesn't belong baked in.
Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

kitrinx wrote:I'm not a pro with the fancy calibrations and color corrections and things, but gamma is IMHO the responsibility of the TV, not MiSTer. If your TV can't do it and you care a lot about gamma, then you should probably get a better display
We covered these arguments already. No LCD display allows for the necessary gamma curve corrections and MiSTer is the device emulating old hardware where games were created with different gamma curves in mind than you will find today on the market. Therefore it is the responsibility of the emulating device or software to do the corrections that definetly no consumer display can do and I am not aware of any other LCD that can.

In the article paulbnl posted earlier, byuu describes what needs to be done to fix this in emulation. MiSTer is, in my humble opinion, the best emulation device there is and it would be great if it would exceed or at least keep up with software emulation in all areas to preserve the authenticity of the look and feel of the games.

It is a great idea paulbnl had and looks much closer to what it was supposed to look. You can also check byuu's article where screenshots show the severity of the fault in colors and gamma without correction.
kitrinx wrote:It doesn't belong as a core specific setting, and it *definitely* doesn't belong baked in.
It needs to be an option that can be set per core as you can see with byuu's article. Different old display technologies (Game Boy vs CRT) need different corrections. Where it is best implemented and how, is for the devs to decide. It sounds like you already have a solid idea on how to do this and filter settings sound like a good option.
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
paulbnl
Captain Atari
Captain Atari
Posts: 151
Joined: Wed Oct 24, 2018 9:43 am

Re: Gamma correction

Post by paulbnl »

JamesF wrote:Again, these 1.2-1.4 number which result in 2.64 to 3.08 gamma is a ridiculously wrong thing to do,, especially if it is baked into the core.
I understand you advocate for this because for some reason it looks good on your display to your eyes (very subjective), but we don't want something like this permanent in a core especially if it is a purely cosmetic feature and done wrong (imo).
Can you elaborate on what does not look good specifically? Like some specific spots in games? It seems like you are calling it wrong just because the numbers are high. Maybe you are just used to everything looking greyish and shadow detail being very bright.

I have added a build with your polynomial curve in the same link as the previous builds. It seems like a minor difference to me compared to 1.3. The bright shades are slightly brighter.
Threepwood wrote:Would it be possible to integrate a reference image, like many games use them to roughly calibrate gamma (e.g. The Mark of the Ninja and many horror games) that says "adjust gamma till [object] is barely visible"?
That is above my pay grade :) Maybe that is something for the 240p test suite.
JamesF
Captain Atari
Captain Atari
Posts: 159
Joined: Sat Dec 15, 2018 6:46 am

Re: Gamma correction

Post by JamesF »

By 'looks wrong' I mean the middle shades are much too dark compared to pure white,, pure white is used for text and plenty other things (clouds, eyes, etc..). That's the false effect of too high gamma value.
To fix this abrupt transition we use polynomial gamma to gradualy return to 2.2 from around 70% input to 100% input.

The 'poly' gamma curve is not a minor difference, it looks much more natural and doesn't transition from from middle to bright tones so abruptly, much closer to a CRT.
It's the 0% to 20% input (0-50 from 255) we want to concentrate on, not the entire gamma curve, since a CRT gamma is overall closer to 2.2-2.4 as mentioned earlier but with darker blacks.
Remember, we are trying to slightly reverse the darker side of sRGB gamma curve of LCDs, not change the entire gamma curve.

Please try 2.9_3.8 poly values as that will be closer to a gamma of 2.6 at 0-20% input, and 2.2 at 75-100% input.
As 3.15_4.4 produces a gamma of 2.8 at 0-20% and 2.2 at 75%-100%,, and imo that's slightly too dark compared to my Trinitron, but already much better than the pure 1.2-1.3 values.
You can just replace the 1.2 and 1.3 values with the two poly equations, since as explained above, we don't want to touch the brighter shades.
Maybe: Off, Poly 1.2, Poly 1.3, and 1.3.

Image

PS.
What I do is look at the first black step in the grey ramp in 240p test suite.
Then Yoshi's Island first level mountains in the background.
Also, my Dell U2410 has a gamma curve closer to the Ploy already, so it looks 'right' in the OFF setting and does not need correction,, that tells us not all LCD monitors calibrated equally from the factory.
Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

JamesF wrote: What I do is look at the first black step in the grey ramp in 240p test suite.
Then Yoshi's Island first level mountains in the background.
I assume you take screenshots on the MiSTer that you then view on your PC?

EDIT: This test suit was written in 2014. Is there any information wether or not it was written with adjustments for CRT Gamma curves in mind? I would think this outputs unscewed test patterns that look normal on a LCD, too, while games had certain ranges boosted to compensate for a CRT's gamma curve. In that case using the 240p test suite would be useless for our use case here.
JamesF wrote:Also, my Dell U2410 has a gamma curve closer to the Ploy already, so it looks 'right' in the OFF setting and does not need correction,, that tells us not all LCD monitors calibrated equally from the factory.
How old is your monitor by now?

EDIT: Attached you find two comparisons of gamma levels using Yoshi's Island and Super Metroid (screenshots taken on MiSTer with the latest core modified by paulbnl)
You do not have the required permissions to view the files attached to this post.
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
paulbnl
Captain Atari
Captain Atari
Posts: 151
Joined: Wed Oct 24, 2018 9:43 am

Re: Gamma correction

Post by paulbnl »

I am going to make it like the scaling filters so we can just use txt files for the different curves.

It is better than having fixed curves in the core.
Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

paulbnl wrote:I am going to make it like the scaling filters so we can just use txt files for the different curves.

It is better than having fixed curves in the core.
Awesome, thanks for the hard work!
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
JamesF
Captain Atari
Captain Atari
Posts: 159
Joined: Sat Dec 15, 2018 6:46 am

Re: Gamma correction

Post by JamesF »

paulbnl wrote:I am going to make it like the scaling filters so we can just use txt files for the different curves.
Thank you.
User avatar
BitsNStuff
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 122
Joined: Tue Oct 16, 2018 7:55 am
Contact:

Re: Gamma correction

Post by BitsNStuff »

That sounds fantastic, paulbnl. Thanks for putting the work in.
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Gamma correction

Post by Sorgelig »

I just want to remind: if it will use LUT for gamma correction it's very unlikely will be integrated into framework as it will produce a lot of timing violataions at 148MHz. For framework integration this LUT needs to be highly optimized with pipe-lining to cope with high HDMI data rate.
paulbnl
Captain Atari
Captain Atari
Posts: 151
Joined: Wed Oct 24, 2018 9:43 am

Re: Gamma correction

Post by paulbnl »

Sorgelig wrote:I just want to remind: if it will use LUT for gamma correction it's very unlikely will be integrated into framework as it will produce a lot of timing violataions at 148MHz. For framework integration this LUT needs to be highly optimized with pipe-lining to cope with high HDMI data rate.
Sure I can add pipe-lining. With the SNES I haven't seen any timing differences with using it in the framework or in the core.

As I understand it it just needs to work on CLK_VIDEO because the LUT will be used on the input to the scaler so for the SNES it will only be at 42Mhz.
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Gamma correction

Post by Sorgelig »

Because direct video requires high super-sampling clock, some cores may output video at up to 100MHz (or even a little higher).
But surely it can be guarded by clock enabler as it's on input side where clock enabler is still valid. Though i don't know how timings will be calculated
yxkalle
Atarian
Atarian
Posts: 6
Joined: Fri Oct 04, 2019 3:08 pm

Re: Gamma correction

Post by yxkalle »

Doesn't it make more sense to apply the LUT before the scaler, or is that simply not possible?
softtest1
Atari User
Atari User
Posts: 34
Joined: Tue Apr 30, 2019 6:37 pm

Re: Gamma correction

Post by softtest1 »

Maybe you can make it so that we have two filter files; one for gamma and one for scaling. Both would be selected in the menu the same way, and the gamma filter would be applied before the scaling filter.
paulbnl
Captain Atari
Captain Atari
Posts: 151
Joined: Wed Oct 24, 2018 9:43 am

Re: Gamma correction

Post by paulbnl »

I have updated the link with updated cores. You'll need to copy the Gamma folder to the sdcard and replace the MiSTer binary for the Gamma correction option to appear under Scaling Filter.

The txt files for the gamma curves are just 256 lines of values 0-255.

The Genesis is now also included because it has a 100Mhz video clock. I haven't seen negative changes to the timings in the Timing analyzer.

https://drive.google.com/file/d/12FrBLf ... sp=sharing

yxkalle wrote:Doesn't it make more sense to apply the LUT before the scaler, or is that simply not possible?
softtest1 wrote:Maybe you can make it so that we have two filter files; one for gamma and one for scaling. Both would be selected in the menu the same way, and the gamma filter would be applied before the scaling filter.
It already works that way :wink:
JamesF
Captain Atari
Captain Atari
Posts: 159
Joined: Sat Dec 15, 2018 6:46 am

Re: Gamma correction

Post by JamesF »

This is absolutely amazing.
I hope to see this merged to main.

Matlab script for Poly Gamma:

Code: Select all

in = 0:1:255;
a = ((in/255) .^ 2.9);
b = ((in/255) .^ 3.8);
Out = 255 * ((2 * a - b) .^ (1/2.2));
Y = round(Out);
Y1=Y(:)
Script for normal Gamma:

Code: Select all

in = 0:1:255;
Out = 255 * ((in/255) .^ 1.1);
Y = round(Out);
Y1=Y(:)
Use here if you don't have matlab:
https://www.tutorialspoint.com/matlab/try_matlab.php
Just copy the result and paste in the gamma text file.
.
You do not have the required permissions to view the files attached to this post.
softtest1
Atari User
Atari User
Posts: 34
Joined: Tue Apr 30, 2019 6:37 pm

Re: Gamma correction

Post by softtest1 »

paulbnl wrote:It already works that way :wink:
Thank you. I tried it now and it works really well.
JamesF
Captain Atari
Captain Atari
Posts: 159
Joined: Sat Dec 15, 2018 6:46 am

Re: Gamma correction

Post by JamesF »

@paulbnl
Are you going to PR it so Sorgelig can review it?
Threepwood
Captain Atari
Captain Atari
Posts: 154
Joined: Thu Jan 10, 2019 10:06 am

Re: Gamma correction

Post by Threepwood »

Indeed, I hope it makes it into the main release and all cores get updated over time. I tried the Genesis core with different games yesterday and it works amazing. The games really benefit from gamma adjustment.
2x MiSTer FPGA: [Official Stormtrooper Case, USB Hub 2.1 with Bridge Board, 128MB SDRAM, ADC, RTC 1.3, I/O v5.5] + [3D printed MiSTer XS Case v2, 128MB SDRAM, I/O v5.6 XL]
paulbnl
Captain Atari
Captain Atari
Posts: 151
Joined: Wed Oct 24, 2018 9:43 am

Re: Gamma correction

Post by paulbnl »

JamesF wrote: Are you going to PR it so Sorgelig can review it?
Sure, I just had to figure out how to add scrolling to the system menu but now that's working properly I have sent the pull requests.
bhamadicharef
Atariator
Atariator
Posts: 23
Joined: Tue Jul 18, 2017 8:31 am
Location: Singapore

Re: Gamma correction

Post by bhamadicharef »

For those interested in research and implementation aspects
there is this old IEEE paper, PDF is online ...

A Flexible Architecture for Precise Gamma Correction
Dong-U Lee, Ray C. C. Cheung, and John D. Villasenor
IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Year: 2007 | Volume: 15, Issue: 4 | Journal Article | Publisher: IEEE
https://www.researchgate.net/publicatio ... Correction

Best regards
Brahim HAMADI CHAREF:: Singapore
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Gamma correction

Post by Sorgelig »

Is it possible to re-use ADV chip output converter dsp for gamma correction? It's not LUT of course..
Locked

Return to “MiSTer”