Scaler

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, Greenious, spiny, Sorgelig, Moderator Team

JamesF
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 146
Joined: Sat Dec 15, 2018 6:46 am

Re: Scaler

Postby JamesF » Mon Feb 11, 2019 2:29 pm

I've compiled the latest ascal (Feb 10th) and it seems it has problem with interlaced modes with vsync_adjust=0 or 1, I see double image.
Strangely it does not happen in low-lag more vsync_adjust=2.
The previous ascal builds did not have this issue in any mode.

I also should mentioned that low-lag mode with HDMI_PLL_ADJ_MULDIV.BHD (Feb 10th) with multiplier/divider does not work on my TV or monitor.
The standard HDMI_PLL_ADJ (Feb 10th) version works but if I'm not mistaken it is the same as the previous (Jan 27th) build, but with updated code.

EDIT:
Thank you Grabulosaure.
Fixed in updated (Feb 11th) ascal.vhd.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Thu Mar 07, 2019 8:04 am

Small feature request: color for border. Border is outside the core's picture but inside the video resolution.
Not urgent and not important, but would be good to have it.

c0d3h4x0r
Atarian
Atarian
Posts: 5
Joined: Mon Apr 22, 2019 8:39 am
Location: Sammamish, WA, USA
Contact:

Re: Scaler

Postby c0d3h4x0r » Wed Apr 24, 2019 4:51 am

Bug report: Over MiSTer's HDMI port, the scaler does not cope well with resolution mode changes by the core. It's especially noticeable with the Minimig core, especially when it switches between interlaced and non-interlaced resolution modes. There are full-screen graphical glitches across the resolution mode switch, as seen in the videos I attached to this closed Minimig bug report.

Is there any way the scaler can be updated to handle such resolution changes more gracefully, without these visual glitches?

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Mon May 13, 2019 7:04 am

Grabulosaure wrote:You like scalers, right ?

I see you test new version of scaler. Can you tell any ETA for new version? I plan to release updated Arcade cores. It's quite a hassle to update ~50 cores, so i would like to wait for new scaler first.
What is 3D in Ascal?

User avatar
kitrinx
Captain Atari
Captain Atari
Posts: 166
Joined: Wed Sep 26, 2018 6:03 am

Re: Scaler

Postby kitrinx » Mon May 13, 2019 12:47 pm

Sorgelig wrote:What is 3D in Ascal?


You miss a lot not being on Discord :)

He made a really cool version of ascal that can do NES 3d stuff as actual 3d on a 3d tv.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Mon May 13, 2019 1:10 pm

Would be good to keep in course here.

Grabulosaure
Atari maniac
Atari maniac
Posts: 89
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Mon May 13, 2019 9:21 pm

I was busy with other stuff these past months, halting ASCAL development.

Current changes
- Low lag mode fix. Tearing.
- Reduced minimal horizontal blanking time
- Fix image header for 64bits data bus.
- "3D" Variant
- Border colour

- Low lag mode.
Some tearing occurred in low lag mode, principally when VSCALE_MODE=1.
When the output image display time is shorter than the input (even when the vertical frequency is identical), then it must be aligned with the bottom, not the top.
Something like that :

Input < Output
ImageImage

Input > Output
ImageImage

(Lag can be different between top and bottom!)

- Minimal blanking time.
Changed a bit the input pipeline.

- Header for 64bit data
I have made a few tests with a second instance of ASCAL over the 64bits bus as a frame grabber to compare VIP and ASCAL (/compare folder). While results are different, there is no obvious quality difference. Changing a bit subpixel phase can sometimes reduce banding.

- 3D
The ASCAL3D version transform interlaced video into side-by-side pictures suitable for modern 3D TVs.
As commented by Kitrinx here above, it can be used for SMS and NES games which had a 3D mode with LC shutter glasses "SegaScope 3D" and "Famicom 3D system". Still some glitches at the top of screen.

- Border.
For border colour, there are several options :
1) Add a 24bits RGB input to the scaler.
2) Sample the colour of a chosen pixel (top-left, bottom-right, ...)
3) Copy input colour when I_DE=0

I have tried (3) and it works with C64, but it doesn't work with all cores.

What else? Any idea?

Some have suggested adding "hybrid scanlines", AFAIU it means reducing scanline constrast in high intensity areas, but I'm not sure what could be the correct way to implement that.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Mon May 13, 2019 9:53 pm

Thanks for updating!

1) Is version on your site OK to use, or you still testing?
2) Border feature is interesting. Should be good for debugging the video as well.
3) 3D should be interesting but i think it's not urgent to implement and can be postponed for future. Especially only couple cores have 3D.
4) So, tearing happens only in integer scaling?

BBond007
Captain Atari
Captain Atari
Posts: 382
Joined: Wed Feb 28, 2018 3:23 am

Re: Scaler

Postby BBond007 » Mon May 13, 2019 10:27 pm

Is the scaler compatible will all video modes the Minimig core can produce?

The reason I ask is because many of the sequences in the new De Profundis demo don't properly display on HDMI or VGA with vga_scaler=1. For example, the very first sequence with the clock/key thing does not display for me.

De Profundis --> http://www.pouet.net/prod.php?which=81081

Thanks :)

Grabulosaure
Atari maniac
Atari maniac
Posts: 89
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Mon May 13, 2019 11:09 pm

Sorgelig wrote:Thanks for updating!

1) Is version on your site OK to use, or you still testing?
2) Border feature is interesting. Should be good for debugging the video as well.
3) 3D should be interesting but i think it's not urgent to implement and can be postponed for future. Especially only couple cores have 3D.
4) So, tearing happens only in integer scaling?


1) I've been doing some non regression tests lately. It looks fine so far. I can finish this week if no new feature is added.
2) What do you prefer for selecting the border colour? Do we take the convention of having cores output the border colour during blanking? You want to be able to display "overscan" debug information?
3) Well it is implemented, just some details to fix. I'd rather keep it as a separate variant rather than complexifying ASCAL though.
4) Tearing is more likely to happen when the output picture has large top/bottom borders. For example Genesis, 1080p, VSYNC=2, VSCALE=1, any game with fast scrolling. But now it's fixed!

BBond007 wrote:Is the scaler compatible will all video modes the Minimig core can produce?
The reason I ask is because many of the sequences in the new De Profundis demo don't properly display on HDMI or VGA with vga_scaler=1. For example, the very first sequence with the clock/key thing does not display for me.
De Profundis --> http://www.pouet.net/prod.php?which=81081
Thanks :)

I have tested that demo, there is a regression in Minimig between version 2019_03_09 and 2019_05_01, with the same scaler. Ask Minimig maintainers...

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Mon May 13, 2019 11:30 pm

Grabulosaure wrote:What do you prefer for selecting the border colour? Do we take the convention of having cores output the border colour during blanking?

I was thought some time ago "It would be good to define the border color INI". So it can be pretty much static. Some cores may have border like Genesis - so it can be used in scaler. I just not sure if such tight integration to core would really look good. For example 4:3 Genesis screen on 16:9 resolution will have very large side borders. Will it look good?
Probably can be added as an OSD option for such cores.
Computer cores such as ZX or C64 could use their border feature but it will require serious core redesign. There are many ZX demos using the border graphics - it will be broken. C64 can display the sprites on border - so it won't work on scaler's border as well.
May be instead of INI the border color should be added as output for for emu entity, so each core will decide how to use the border.

Grabulosaure wrote:You want to be able to display "overscan" debug information?

nope. I mean to use the border to see the edges of video.

Grabulosaure wrote:I have tested that demo, there is a regression in Minimig between version 2019_03_09 and 2019_05_01, with the same scaler. Ask Minimig maintainers...

Interesting.. i will check it.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Mon May 13, 2019 11:33 pm

Grabulosaure wrote:3) Well it is implemented, just some details to fix. I'd rather keep it as a separate variant rather than complexifying ASCAL though.

Not urgent anyway. May be at some stage you will decide to integrate it.
I have no TV to display 3D :) I have PS4 VR - not sure if it can be re-used as it uses HDMI as well.

zomgugoff
Atari freak
Atari freak
Posts: 52
Joined: Tue Jan 29, 2019 10:12 am

Re: Scaler

Postby zomgugoff » Tue May 14, 2019 12:32 am

Sorgelig wrote:
Grabulosaure wrote:3) Well it is implemented, just some details to fix. I'd rather keep it as a separate variant rather than complexifying ASCAL though.

Not urgent anyway. May be at some stage you will decide to integrate it.
I have no TV to display 3D :) I have PS4 VR - not sure if it can be re-used as it uses HDMI as well.


The PSVR can be used as a normal HDMI display. Early on, people tried this as a cheaper alternative to the Oculus and Vive. There was work being done on software to use webcams for head tracking and reading the gyro/accelerometer. For this type of stationary 2d application, that software would be unnecessary. I'm not if it supports any other resolutions than 1920x1080, though.

From what I've read, the Oculus Rift doesn't work like this. It would probably take some serious reverse engineering to initialize a dedicated PC VR headset to even start the video feed to the display. I'll try one I have access to later tonight.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue May 14, 2019 12:42 am

Grabulosaure wrote:I can finish this week if no new feature is added.

i don't really want to slowdown the Scaler development but i think it would be really useful to have special mode for scaler to output the video from frame buffer without filling it. So, HPS side will be able to output whatever it needs with specified resolution.
What it needs:
1) A way to define a fixed input resolution - this should be already implemented i just never tried it.
2) Scaler need to accept the memory address which will be the current buffer on next frame. So HPS will manage the buffers itself.
Probably that's all.
May be if it's not hard, let on-the-fly switch between normal mode and new mode. So HPS will be able to switch the output between core and HPS. Probably it can be done through external control. Just need to be sure the scaler will be able to handle framebuffer/core switch back and forth.
This can lead to a whole new experience which i even didn't fully realise yet :)

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Tue May 14, 2019 12:44 am

zomgugoff wrote:The PSVR can be used as a normal HDMI display. Early on, people tried this as a cheaper alternative to the Oculus and Vive. There was work being done on software to use webcams for head tracking and reading the gyro/accelerometer. For this type of stationary 2d application, that software would be unnecessary. I'm not if it supports any other resolutions than 1920x1080, though.

head tracking is not required. What required is ability to output 3D to it. So it should accept some 3D format like side-by-side or frame alternating.

zomgugoff
Atari freak
Atari freak
Posts: 52
Joined: Tue Jan 29, 2019 10:12 am

Re: Scaler

Postby zomgugoff » Tue May 14, 2019 12:54 am

Sorgelig wrote:head tracking is not required. What required is ability to output 3D to it. So it should accept some 3D format like side-by-side or frame alternating.


It uses side-by-side video. I suspect the only thing that may make it look weird would be distortion from the lenses. Given that motion wasn't considered on the original headsets, my guess is they did not have the same shape of lenses that current VR headsets use.

User avatar
kitrinx
Captain Atari
Captain Atari
Posts: 166
Joined: Wed Sep 26, 2018 6:03 am

Re: Scaler

Postby kitrinx » Tue May 14, 2019 1:39 am

Grabulosaure wrote:What else? Any idea?[/b]
Some have suggested adding "hybrid scanlines", AFAIU it means reducing scanline constrast in high intensity areas, but I'm not sure what could be the correct way to implement that.


The things I get asked about the most that are scaler related:

Cropping so people can use 5x integer scaling.
Freely adjustable aspect ratio and canvas size. (this is more of a menu thing, I think it's already possible?)
Hybrid scanline/shadow mask/LCD color blending support, which Soltan can explain better than me. He has an idea what's needed.

jemismyname
Atari freak
Atari freak
Posts: 70
Joined: Wed Feb 27, 2019 8:25 pm

Re: Scaler

Postby jemismyname » Tue May 14, 2019 4:06 am

can you implement supereagle scalar with scanlines? because that would make me happy. that's what i use on mednafen and it makes everything look just perfect to me. I wish i understood enough to implement it myself. :P

Everyone else on the internet wants HD mode 7, in case you haven't heard :D

ghogan42
Atari maniac
Atari maniac
Posts: 89
Joined: Wed Oct 17, 2018 7:27 pm

Re: Scaler

Postby ghogan42 » Tue May 14, 2019 5:02 am

kitrinx wrote:Hybrid scanline/shadow mask/LCD color blending support, which Soltan can explain better than me. He has an idea what's needed.


There are two basic effects people talk about when they "hybrid scanlines"
(1) Fade out the dark scanlines for high brightness pixels
(2) "Bloom" for the bright part of the scanline so the scanlines vary in thickness depending on pixel brightness.

For the scaler, both would probably have the same implementation. There are two ways to go about it.

Code: Select all

Hybrid Scanline Method 1:
(step1) Double all of the polyphase filter hardware in the scaler.
(step2) Each scaler gets different coefficients. Like Thick Scanline + Thin scanlines. You would just select both in the OSD.
(step3) After interpolation you would lerp between the two output pixels base on pixel brighness.  This could be a simple 8 or 16 entry lut and use bit shifting for mixing if that would save resources.


Code: Select all

Hybrid Scanline Method 2:
(step1) Load two sets of coefficients like above. But we don't double the pixel pipeline.
(step2) When you get the row of coefficients based on phase you load two coeff sets: C0,C1,C2,C3 from the first coefficient set and the corresponding D0, D1, D2, D3 from the second set of coefficients
(step3) You lerp between the coefficients based on pixel brightness. So if brightness is B then
E0 = (1-B)*C0 +B*D0
E1 = (1-B)*C1 +B*D1
E2 = (1-B)*C2 +B*D2
E3 = (1-B)*C3 +B*D3
As above, if the lerp is too much you can use a small lut. Even 4 or 8 entries would be good enough!
(step4)  Use those coefficients for the polyphase filter.


Pixel brightness could be something like 0.5(Green) + 0.25(Red) + 0.25(Blue) so it is easy to use bit shifting for it.

It seems to me as though Method2 is more resource efficient but it is harder to code. And Method1 is easier to code but uses more hardware.
I don't know if it is worth the work to add this feature but it does look good for scanlines and lcd effects when done in pixel shaders.

Also there is a very simple Method3 that would not look as good.

Code: Select all

Hybrid Scanline Method 3:
(step1) Add nothing to the scaler until the very end of the pipeline.
(step2) Lerp between the polyphase filter output pixel and either the bicubic/bilinear/sharp bilinear output pixel base on pixel brightness.

In this case it may be good to clamp the brightness at lower than 1.0 to keep the scanlines from fading out completely.

Grabulosaure
Atari maniac
Atari maniac
Posts: 89
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Sat May 18, 2019 10:36 pm

I've tried a few things with the scaler.

- Border. 24 bits input, defaults to black but can be set to display patterns, anything.
- Added an pixel enable input which allows to select pixels that don't update the framebuffer.Could be used for overlays.
(See Genesis demo build). Using that extra byte enable bits don't use much extra resources, particularly as RAM blocks have extra "parity" bits. Enabled by ALPHA generic.
An alternative would be to change the framebuffer to 32bits/pixels and have the extra byte select between input pixels and pixels drawn by the ARM allowing any shape for overlays, but limited colours range.

- Scaler in framebuffer mode fixes. To use the scaler as a framebuffer :
Clear MODE[3] : Single buffering.
Set freeze : Disable writes
Set vimax/himax for picture size.
Clear iauto : set image dimensions.
While the input image is ignored in that mode, input clock must still be active.
(note that in this mode, the scaler cannot downscale)

- ASCAL3D. Option to force side-by-side mode even when the source is mono.

jemismyname wrote:can you implement supereagle scalar with scanlines? because that would make me happy. that's what i use on mednafen and it makes everything look just perfect to me. I wish i understood enough to implement it myself. :P

It is something to add in the part which does HQ2X, not in the scaler.

kitrinx wrote:- Cropping so people can use 5x integer scaling.
- Freely adjustable aspect ratio and canvas size. (this is more of a menu thing, I think it's already possible?)


- The scaler can crop images, but managing different configurations, resolutions changes, would be a bit difficult. Shaving the top and bottom lines can be easily implemented in cores though, for example the ZXSpectrum core has a "Zoom" mode. Maybe adding that feature to a couple of cores like SNES or Genesis would be enough to address most complaints.

ghogan42 wrote:[...] "hybrid scanlines" [...]

Method 2 doesnt' seem much more complex than Method 1. For Method3, as the bicubic/bilinear are not used on Mister, they are removed during synthesis (well, except for AscalTest). In any case, adding hardware means more pipelining [sigh...]
Besides adding hardware for interpolers, mixing should be applied separately for horizontal and vertical coefficients, to avoid using extra line buffers.
I can try to make a demo version with that extra stuff. For colour mask effects, the trivial and cheap option is to use separate RGB polyphase coefficients, but it wouldn't allow things like delta shadow mask effects.
Feature creep will ask one day for pixel shaders and geometry transformations which needs lots of internal RAM and hardware, and get something at least as large and complex as the previous VIP. And efforts.
Last edited by Grabulosaure on Sat May 18, 2019 10:39 pm, edited 1 time in total.

Grabulosaure
Atari maniac
Atari maniac
Posts: 89
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Sat May 18, 2019 10:37 pm

.(error).

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Sun May 19, 2019 4:48 am

Grabulosaure wrote:See Genesis demo build

can you upload modified parts source to see how it's coded?

Grabulosaure wrote:To use the scaler as a framebuffer :

can you add ability to supply the frame buffer address in this mode? When new address is supplied then it will be active from the next frame. In this case HPS can manage the buffer and implement its own dual and triple buffer if needed. It also will allow to keep the HPS frame buffer separate from core's buffer.

Also for speed wise scaler needs to implement either 16bit or 32bit mode. 24bit writing in HPS is too slow. Probably 16bit color (X1:B5:G5:R5) will be enough - and won't require big buffer.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Sun May 19, 2019 5:00 am

Grabulosaure wrote:While the input image is ignored in that mode, input clock must still be active.

Does this clock must match the selected size of framebuffer or any clock is fine?

Grabulosaure
Atari maniac
Atari maniac
Posts: 89
Joined: Tue Sep 05, 2017 9:35 pm
Contact:

Re: Scaler

Postby Grabulosaure » Sun May 19, 2019 10:06 am

Sorgelig wrote:Does this clock must match the selected size of framebuffer or any clock is fine?

Any clock. Just propagating signals (calculating the input image dimensions is done in the i_clk clock domain.). I can try to remove that constraint, if needed.
Sorgelig wrote:
Grabulosaure wrote:See Genesis demo build

can you upload modified parts source to see how it's coded?

I've posted the diff, but there is nothing fancy.
Do you think that the ability of disabling pixels update is of any use? (for example to display the OSD menu, but it's tied to the input resolution)

Sorgelig wrote:can you add ability to supply the frame buffer address in this mode? When new address is supplied then it will be active from the next frame. In this case HPS can manage the buffer and implement its own dual and triple buffer if needed. It also will allow to keep the HPS frame buffer separate from core's buffer.

Also for speed wise scaler needs to implement either 16bit or 32bit mode. 24bit writing in HPS is too slow. Probably 16bit color (X1:B5:G5:R5) will be enough - and won't require big buffer.

I've long wanted to support different colour depths (the unused FORMAT generic).
I can turn the base address and format from a generic to a signal.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5078
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Scaler

Postby Sorgelig » Sun May 19, 2019 12:44 pm

Grabulosaure wrote:Do you think that the ability of disabling pixels update is of any use? (for example to display the OSD menu, but it's tied to the input resolution)

i don't see much use of this. As you've told it's too tied to original resolution. It also will be used only on HDMI, so VGA output will miss all these effects. With full frame from HPS, it will be possible to temporary switch VGA to scaler output, so in this case i don't worry much about dual output.


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 6 guests