Possibility of sharper HDMI output via integer scaling?

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 4:56 pm

cacophony wrote:I compared Sonic (JPN) version for the latest core vs the version from the 10th, and the smoothness of movement for the clouds look identical.


That's because your 4K display's native resolution is 3840 x 2160 pixels, which is an exact multiply of 320x240!! It's pure LUCK that you can't see them. But there are other perfectly fine HDMI displays that don't have such resolution, and hence artifacts appear. You are not the only person with an HDMI monitor in the world.

cacophony
Atari maniac
Atari maniac
Posts: 79
Joined: Sun Jul 22, 2018 11:14 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby cacophony » Wed Oct 17, 2018 5:00 pm

vanfanel wrote:
cacophony wrote:I compared Sonic (JPN) version for the latest core vs the version from the 10th, and the smoothness of movement for the clouds look identical.


That's because your 4K display's native resolution is 3840 x 2160 pixels, which is an exact multiply of 320x240!! It's pure LUCK that you can't see them. But there are other perfectly fine HDMI displays that don't have such resolution, and hence artifacts appear. You are not the only person with an HDMI monitor in the world.


Never said I was, but I do feel we should optimize for the most common preferences and use cases, and I imagine your particular monitor resolution is not too common at this point.

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 5:03 pm

@Sorgelig: some questions arise then:
-Would the new scanlines algorithm work with POLYPHASE or is it only intended to work with NN?
-Would it be enough to change

Code: Select all

<parameter name="ALGORITHM_NAME" value="NEAREST_NEIGHBOUR" />
into

Code: Select all

<parameter name="ALGORITHM_NAME" value="POLYPHASE" />
to revert the situation?
-Are there instructions to build the cores from sources?

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 5:04 pm

Sorgelig wrote:It's possible, but as i've said it's quite unstable. It uses DDR3 for buffer and due to instability it may corrupt the Linux memory as well.

It is my understanding that DDR3 is biparted and used both by Linux and cores, but there is no inner protection method, so if a core misbehaves it’s game over. If you have time and will I have some questions:
Why did you choose DDR3? For burst speed?
Is your scaling algorithm frame buffered or do you multiply line by line?
Anyway I hope I somehow piqued your curiosity with the integer scaling + bilinear proof of concept. :wink:

Thank you in advance.
Best regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 5:09 pm

vanfanel wrote:Are there instructions to build the cores from sources?

This is a good starting point https://github.com/MiSTer-devel/Main_Mi ... ting-notes
And here you can see the changes in Genesis vip.qsys https://github.com/MiSTer-devel/Genesis ... 76a93e58e9

Regards.
Locutus73

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 5:18 pm

@Locutus73: Thanks!

However, this warning is frightening me:

Note: Scaler uses "Video and Image Processing"(VIP) modules from Megacore IP bundled with Standard Quartus license. Free/Web version of Quartus can compile the scaler code, but it will work as a trial version - video will work around 1 hour and then will start to blink. Trial version won't have RBF and can be uploaded to FPGA only through USB Blaster.


So I won't be able to produce a core that works for more than 1 hour if I use the free Quartus version, right?

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 5:19 pm

I've just tested on my 1920x1200 24" monitor.
I've used 1920x1080 and 1280x720 resolutions. I've starred a lot into picture while watching Sonic demo and even in 10cm distance i couldn't notice any shimmering while scrolling on clouds and playfield. Original game resolution is 320x224. I've tried forced scandoubler which gives 320x448 resolution. And none of configs gave any shimmering while scrolling.

Locutus73 wrote:Why did you choose DDR3? For burst speed?

1) For speed. High resolutions require high speed.
2) many cores use SDRAM, so it's fully occupied by core itself.
3) Some cores don't require SDRAM, so adding requirement of SDRAM just because of scaler isn't good way.
My HDMI lite scaler renders the full frame to allow output frame rate be different from input. So it requires triple full frame buffer of max possible size (1920x1080).

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 5:25 pm

vanfanel wrote:So I won't be able to produce a core that works for more than 1 hour if I use the free Quartus version, right?

Exactly.

Regards.
Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 5:32 pm

Sorgelig wrote:My HDMI lite scaler renders the full frame to allow output frame rate be different from input. So it requires triple full frame buffer of max possible size (1920x1080).

Right, and that’s because it directly outputs to HDMI. But could it be theoretically reduced to a single line buffer for integer multiplying both horizontal and vertical axis and pass the result to Quartus VIP Bilinear scaler in order to implement the hypothetical "integer+bilinear scaler" maintaining low latency? And in this case, would it require DDR3 for a single line (multiplied) buffer?

Thank you in advance
Regards.

Locutus73

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 5:41 pm

@Sorgelig: Can you please test Snow Bros scrolling intro sequece? Can't you see horrible shimmering there with NN?

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 5:44 pm

For single line buffer it's possible to use BRAM 1920x24 bits. If use it to feed into scaler through standard serial input then it will require variable pixel clock (or at least maximum possible pixel clock). It sounds not very complicated until u start to implement it.
Still prefer someone will make it.
Btw, even 2 stages scaling will produce different quality, as sometimes resolution after first stage may be close to final resolution while other cases may have edge case with resolution almost twice smaller than final one, so smoothness will be stronger.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 5:53 pm

vanfanel wrote:@Sorgelig: Can you please test Snow Bros scrolling intro sequece? Can't you see horrible shimmering there with NN?

no shimmering.
May be you mean stuttering which which is more noticeable due to hard edges? For smooth scrolling you need to use vscroll_adjust=1 option in MiSTer.ini. And if your monitor supports arbitrary pixel clock then you will get perfect VSync between core and HDMI output.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 5:59 pm

And for smoothness you can enable HQ2x. It gives new, modern hi-res feeling for retro systems.

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 6:02 pm

Sorgelig wrote:
vanfanel wrote:@Sorgelig: Can you please test Snow Bros scrolling intro sequece? Can't you see horrible shimmering there with NN?

no shimmering.
May be you mean stuttering which which is more noticeable due to hard edges? For smooth scrolling you need to use vscroll_adjust=1 option in MiSTer.ini. And if your monitor supports arbitrary pixel clock then you will get perfect VSync between core and HDMI output.


I have vsync_adjust=1 on my MiSTer.ini file, of course, and my monitor works nicely with that.
I don't get any stuttering, but a BIG dose of shimmering. Yesterday I wouldn't get ANY image artifact, it was just perfect until this new scaling algorithm appeared.

HQ2X isn't exactly good-looking for me :(

Please, could you consider a 100 Euros donation to produce two versions of each core (NN and POLYPHASIC) when you do these periodic releases? I would also send you some FPGA boards for free, like two Altera DE1 and 3xZX-UNOs, appart from the money.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 6:21 pm

vanfanel wrote:I have vsync_adjust=1 on my MiSTer.ini file, of course, and my monitor works nicely with that.
I don't get any stuttering, but a BIG dose of shimmering. Yesterday I wouldn't get ANY image artifact, it was just perfect until this new scaling algorithm appeared.

HQ2X isn't exactly good-looking for me :(

Please, could you consider a 100 Euros donation to produce two versions of each core (NN and POLYPHASIC) when you do these periodic releases? I would also send you some FPGA boards for free, like two Altera DE1 and 3xZX-UNOs, appart from the money.


As i see you are the only panicking here and see what others don't see. Don't forget that i'm the user as well. If it's really so much worse than before, then why i need to do this for myself? I'm not a masochist :)

This is free project without obligations. If you want to donate and appreciate my work then you can donate by either Paypal or by hardware. This is pure voluntary actions. If you require some promise from me, then don't do such donation. There won't be any obligation from my side to do something.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 6:29 pm

i've created the poll: viewtopic.php?f=117&t=34526

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 7:34 pm

vanfanel wrote:Please, could you consider a 100 Euros donation to produce two versions of each core (NN and POLYPHASIC) when you do these periodic releases? I would also send you some FPGA boards for free, like two Altera DE1 and 3xZX-UNOs, appart from the money.

Oh, my... :roll:
Donations are for saying thank you and showing appreciation for the work that has been done, not for preordering future features... this is an open source project!

Regards.
Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Wed Oct 17, 2018 7:46 pm

Sorgelig wrote:
vanfanel wrote:@Sorgelig: Can you please test Snow Bros scrolling intro sequece? Can't you see horrible shimmering there with NN?

no shimmering.
May be you mean stuttering which which is more noticeable due to hard edges? For smooth scrolling you need to use vscroll_adjust=1 option in MiSTer.ini. And if your monitor supports arbitrary pixel clock then you will get perfect VSync between core and HDMI output.


I know that you know, but just in order to have a common and established frame of reference with everyone in this discussion: this is scrolling shimmering
Image
The upper MARIO has scrolling shimmering too
Image
These images are taken from here https://gamedev.stackexchange.com/quest ... inemachine

The best tool in order to judge a scaler quality (sharpness, artifacts, shimmering, stuttering, whatever) with different consoles is the “240p test suite” you can find here http://junkerhq.net/xrgb/index.php?titl ... test_suite

Regards.

Locutus73

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 8:46 pm

Locutus73 wrote:I know that you know, but just in order to have a common and established frame of reference with everyone in this discussion: this is scrolling shimmering

There is no noticeable shimmering as i've already told.
So, scrolling is like bottom Mario.

vanfanel
Atari User
Atari User
Posts: 35
Joined: Tue Oct 09, 2018 10:19 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby vanfanel » Wed Oct 17, 2018 9:00 pm

Sorgelig wrote:
Locutus73 wrote:I know that you know, but just in order to have a common and established frame of reference with everyone in this discussion: this is scrolling shimmering

There is no noticeable shimmering as i've already told.
So, scrolling is like bottom Mario.


That's setup-dependant, while we had no shimmering on any setup before.

cacophony
Atari maniac
Atari maniac
Posts: 79
Joined: Sun Jul 22, 2018 11:14 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby cacophony » Wed Oct 17, 2018 10:16 pm

vanfanel wrote:
Sorgelig wrote:
Locutus73 wrote:I know that you know, but just in order to have a common and established frame of reference with everyone in this discussion: this is scrolling shimmering

There is no noticeable shimmering as i've already told.
So, scrolling is like bottom Mario.


That's setup-dependant, while we had no shimmering on any setup before.


Yes, but everyone with a decent 1080p or 4k panel had to have a fuzzier image with noticeable ringing.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Wed Oct 17, 2018 10:33 pm

Locutus73 wrote:I know that you know, but just in order to have a common and established frame of reference with everyone in this discussion: this is scrolling shimmering

I cannot change algo at runtime, but i can change coefficients. Since you are familiar with scaling, may be you can explore the possibility of set specific coefficients for Lanczos to make it work as NN scaling.

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Thu Oct 18, 2018 7:12 am

Sorgelig wrote:For single line buffer it's possible to use BRAM 1920x24 bits. If use it to feed into scaler through standard serial input then it will require variable pixel clock (or at least maximum possible pixel clock). It sounds not very complicated until u start to implement it.

Amen bro’, I’m a software guy but I feel your pain; I think that’s an universal constant.

Sorgelig wrote:Still prefer someone will make it.

I can understand that and I’m not trying to force you to do that; I’m just trying to explore the idea with you and everyone else… but I must be honest, I hope that talking with you I’m piquing your curiosity, so maybe, in a boring Sunday afternoon… :wink:

Sorgelig wrote:Btw, even 2 stages scaling will produce different quality, as sometimes resolution after first stage may be close to final resolution while other cases may have edge case with resolution almost twice smaller than final one, so smoothness will be stronger.

I see your point, but the first stage has not to be limited to produce an image slightly smaller than the final one, it can be slightly bigger so, with your example, instead of being almost twice smaller than final one it can be doubled being slightly bigger than the final one. i.e. I tried my previous NES Super Mario Bros example with 5x+bilinear instead of 4x+bilinear (5x produces a bigger image than the final one) getting the same final visual results than 4x+bilinear.
Do you have any strange resolution example I could try to edit with GIMP to see if the integer+bilinear idea still works? Maybe the super high horizontal res Amiga?

Thank you in advance.
Best regards.

Locutus73

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Thu Oct 18, 2018 7:34 am

Locutus73 wrote:I see your point, but the first stage has not to be limited to produce an image slightly smaller than the final one, it can be slightly bigger so, with your example, instead of being almost twice smaller than final one it can be doubled being slightly bigger than the final one. i.e. I tried my previous NES Super Mario Bros example with 5x+bilinear instead of 4x+bilinear (5x produces a bigger image than the final one) getting the same final visual results than 4x+bilinear.
Do you have any strange resolution example I could try to edit with GIMP to see if the integer+bilinear idea still works? Maybe the super high horizontal res Amiga?

While working with pictures on computer it's basically size doesn't matter. In video processing you have to think how to process it in real time. If HDMI resolution is 1920x1080, it's already hard to process this resolution while you offer to magnify beyond this? You will climb to incredible clocks and hit the low-cost FPGA limits. Even 1920x1080 requires 2 pixel clock processing in parallel which means many VIP components double the resource requirements.
Currently i only see the one possible adjustment - coefficients. If it will be possible to neutralize Lanczos function and act it as Near Neighbor, then it will be possible to choose between sharper and filtered looks.

Locutus73
Captain Atari
Captain Atari
Posts: 170
Joined: Wed Feb 07, 2018 6:13 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Locutus73 » Thu Oct 18, 2018 7:34 am

Sorgelig wrote:I cannot change algo at runtime,

I know that.

Sorgelig wrote:but i can change coefficients.

Oh my, that's interesting...

Sorgelig wrote:Since you are familiar with scaling,

Don’t think I’m so expert… I know random stuff learned over the internet since I’m interested in gaming and I’m a software guy, but I’m not an expert in signal analysis at all. Anyway I can try to do my best and I can communicate with a wide range of other people.

Sorgelig wrote:may be you can explore the possibility of set specific coefficients for Lanczos to make it work as NN scaling.

And that’s brilliant, so you’re thinking instead of switching algorithm at runtime (impossible with Quartus VIP), using edge case parameters in order to make one algorithm to work like another one!!!
And that’s not limited to Lanczos3; i.e. a optiroc, a guy in the Classic Gaming Discord, proposed to changing bilinear biasing in order to alter its linearity… so it could work both as a Nearest Neighbor and as a “pixel edge interpolation”, basically the same result as integer scaling+bilinear approach, but in a single pass… in the end the interpolation needed in a non integer scaling scenario where we want something that works just like NN inside an enlarged pixel, but interpolates (possibly using a single final pixel) when it reaches it edges.
That’s an explanatory picture optiroc made.
IMG_2663.jpg

Basically 0% is linear (bilinear if used on both axis), 100% is Nearest Neighbor and 50% (or another complex curve) is what we want for interpolating just on pixel edges.

Thank you in advance.
Best regards.

Locutus73
You do not have the required permissions to view the files attached to this post.


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 9 guests