Possibility of sharper HDMI output via integer scaling?

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

User avatar
Newsdee
Atari God
Atari God
Posts: 1200
Joined: Fri Sep 19, 2014 8:40 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Newsdee » Sun Oct 21, 2018 7:30 am

I love the 1920x1200 resolution for my desktop, but it's not for classic gaming. And those monitors should be able to handle 1920x1080 fine as well.

Kev79
Atarian
Atarian
Posts: 5
Joined: Sat Oct 06, 2018 12:20 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Kev79 » Mon Oct 22, 2018 12:32 am

1080p is not great resolution for retro games because you can't use the entire screen with integer scaling. 1600x1200 is a perfect 5x scale so it is much more useful. So if you have a monitor that has this vertical resolution you should absolutely use it.

Kev79
Atarian
Atarian
Posts: 5
Joined: Sat Oct 06, 2018 12:20 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby Kev79 » Mon Oct 22, 2018 12:32 am

Removed, accidental double post.

Indcsion
Atariator
Atariator
Posts: 23
Joined: Fri Oct 12, 2018 11:48 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Indcsion » Mon Oct 29, 2018 11:37 am

How do you use that coeff_nn.txt file? I copied to the genesis folder but I didnt notice it was on

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2863
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Mon Oct 29, 2018 11:42 am

Indcsion wrote:How do you use that coeff_nn.txt file? I copied to the genesis folder but I didnt notice it was on

with name coeff.txt as i told?

Indcsion
Atariator
Atariator
Posts: 23
Joined: Fri Oct 12, 2018 11:48 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Indcsion » Mon Oct 29, 2018 11:56 am

I didnt rename it before but I just took off the nn and I dont notice any different.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2863
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Mon Oct 29, 2018 12:23 pm

may be you didn't update MiSTer binary

Indcsion
Atariator
Atariator
Posts: 23
Joined: Fri Oct 12, 2018 11:48 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Indcsion » Mon Oct 29, 2018 12:31 pm

Its 1019 right. I changed it just incase before I tested.
I did atleast figure out how to change my rtc to my timezone.
I guess I got to start over. Windows finally got what it wanted and formatted f.

Indcsion
Atariator
Atariator
Posts: 23
Joined: Fri Oct 12, 2018 11:48 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Indcsion » Mon Oct 29, 2018 5:38 pm

Sdcard back to being setup. I copied that coeff.txt file to every folder being accessed to run the genesis core and it still didnt work

BitsNStuff
Retro freak
Retro freak
Posts: 12
Joined: Tue Oct 16, 2018 7:55 am
Contact:

Re: Possibility of sharper HDMI output via integer scaling?

Postby BitsNStuff » Mon Oct 29, 2018 9:55 pm

Have you changed the names of the folders your ROMs are in? I'm not sure if it will effect that but it can effect some things. Are you wanting to try nearest neighbour? Do you also have an ini file installed with integer scaling set up?

atmn
Atarian
Atarian
Posts: 3
Joined: Tue Oct 30, 2018 9:05 am

Soft image with MiSTer

Postby atmn » Tue Oct 30, 2018 9:09 am

Hi
Just getting started with this fantastic project, get my dram board working and started to fire up some sweet cores.

Works great, however the image is quite soft, its very noticeable. Im running HDMI out, the res is 1280x720p.
I have using 720p with a raspberry pi and a HiDef-NES with great sharpness on this very monitor.

How come its like this, does anyone have a clue? Looks like composite blurness without the artifacts.

Indcsion
Atariator
Atariator
Posts: 23
Joined: Fri Oct 12, 2018 11:48 am

Re: Possibility of sharper HDMI output via integer scaling?

Postby Indcsion » Tue Oct 30, 2018 4:27 pm

The problem I had was that the genesis folder wasnt in the root. I had everything separated into folders arcade, consoles, and conputers. Its working now. Thanks sorgelig

cacophony
Atari maniac
Atari maniac
Posts: 82
Joined: Sun Jul 22, 2018 11:14 pm

Re: Soft image with MiSTer

Postby cacophony » Tue Oct 30, 2018 4:56 pm

atmn wrote:Hi
Just getting started with this fantastic project, get my dram board working and started to fire up some sweet cores.

Works great, however the image is quite soft, its very noticeable. Im running HDMI out, the res is 1280x720p.
I have using 720p with a raspberry pi and a HiDef-NES with great sharpness on this very monitor.

How come its like this, does anyone have a clue? Looks like composite blurness without the artifacts.


Steps for sharper picture across all current cores:

- Make sure you have a MiSTer.ini file in your config directory (https://github.com/MiSTer-devel/Main_Mi ... MiSTer.ini)
- Change video_mode to 8 (this will give you 1080p60
- Change forced_scandoubler to 1 (this will increase sharpness across *all* cores when using HDMI)

For tack sharp Genesis output:

- Make sure you have latest Genesis core and MiSTer (https://github.com/MiSTer-devel/Main_Mi ... r/releases)
- Have a "Genesis" folder at the same level as MiSTer binary, and in that folder place this file: https://github.com/MiSTer-devel/Main_Mi ... eff_nn.txt (but make sure to rename it to coeff.txt first)

In time, the other cores will be updated to utilize this new coeff.txt file (which in the case above just enables nearest neighbor scaling for maximum sharpness). If you really like the improvement of that and don't want to wait for other cores to be updated you can download one of these custom NN builds: https://github.com/Kitrinx/MiSTer_NN

ghogan42
Atariator
Atariator
Posts: 22
Joined: Wed Oct 17, 2018 7:27 pm

Re: Soft image with MiSTer

Postby ghogan42 » Tue Oct 30, 2018 6:24 pm

cacophony wrote:
atmn wrote:Hi
Just getting started with this fantastic project, get my dram board working and started to fire up some sweet cores.

Works great, however the image is quite soft, its very noticeable. Im running HDMI out, the res is 1280x720p.
I have using 720p with a raspberry pi and a HiDef-NES with great sharpness on this very monitor.

How come its like this, does anyone have a clue? Looks like composite blurness without the artifacts.


Steps for sharper picture across all current cores:

- Make sure you have a MiSTer.ini file in your config directory (https://github.com/MiSTer-devel/Main_Mi ... MiSTer.ini)
- Change video_mode to 8 (this will give you 1080p60
- Change forced_scandoubler to 1 (this will increase sharpness across *all* cores when using HDMI)

For tack sharp Genesis output:

- Make sure you have latest Genesis core and MiSTer (https://github.com/MiSTer-devel/Main_Mi ... r/releases)
- Have a "Genesis" folder at the same level as MiSTer binary, and in that folder place this file: https://github.com/MiSTer-devel/Main_Mi ... eff_nn.txt (but make sure to rename it to coeff.txt first)

In time, the other cores will be updated to utilize this new coeff.txt file (which in the case above just enables nearest neighbor scaling for maximum sharpness). If you really like the improvement of that and don't want to wait for other cores to be updated you can download one of these custom NN builds: https://github.com/Kitrinx/MiSTer_NN


Thanks to sorgelig and anyone else working on this. Having this available for all of the cores would be fantastic. I think we can exploit the scaler as sort of a "poor man's pixel shader" and do a little more with it than we're doing now.

I've written a Matlab/Octave script to generate the scaler coefficients for Bilinear, Sharper Bilinear Variants, Bicubic (a couple of variations), Lanczos, etc. I'm waiting on delivery of my de10 nano which should be here today. Once I get it set up and verify that I'm not generating garbage then I'll post the script here along with all of the coefficients that I've produced.

I just don't want a script out in the wild that produces unusable results, so I have to test it on actual miSTer hardware before I post. But if it goes well, then I'll get something posted here before bed tonight.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2863
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Tue Oct 30, 2018 6:54 pm

Since appearance of the source of decent scaler and possible addition of bicubic/polyphase algos in it, i hold my propagation of changes i've made in Genesis/SMS scaler to other cores.
I hope to get opensource scaling solution which will simplify the HDMI output code and won't require special license.

ghogan42
Atariator
Atariator
Posts: 22
Joined: Wed Oct 17, 2018 7:27 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby ghogan42 » Tue Oct 30, 2018 7:21 pm

Sorgelig wrote:Since appearance of the source of decent scaler and possible addition of bicubic/polyphase algos in it, i hold my propagation of changes i've made in Genesis/SMS scaler to other cores.
I hope to get opensource scaling solution which will simplify the HDMI output code and won't require special license.


Oh I see. That makes sense. The open source scaler doesn't have the same feature set as the big polyphase scaler, but I can see why you'd want the open source version since it popped up. If I can get setup to compile fpga stuffs, I might be able to contribute to it a little (but I've never done fpga work and I do math not computers). But it wouldn't take much to improve on the straight bilinear that the open source scaler has right now. We can probably get it to have adjustable sharpness without too much work.

I'll still post my script and stuff for the polyphase scalar in case it's useful.

Sorgelig
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2863
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: Possibility of sharper HDMI output via integer scaling?

Postby Sorgelig » Tue Oct 30, 2018 8:15 pm

ghogan42 wrote:I'll still post my script and stuff for the polyphase scalar in case it's useful.

of course it will be useful!
Even with opensource scaler coefficients will be required for bicubic and polyphase.

ghogan42
Atariator
Atariator
Posts: 22
Joined: Wed Oct 17, 2018 7:27 pm

Re: Possibility of sharper HDMI output via integer scaling?

Postby ghogan42 » Wed Oct 31, 2018 5:19 am

Remember that you can only use coeff.txt with recent versions of the genesis core. 10/19 for sure, 10/29 I think, and I don't know about 10/31

Hello All,
I needed to fix up a few mistakes in my code. But here is a badly working script for Octave (Matlab should work too) to generate Filter Coefficients for the polyphase scaler.

The script is called polyphase_filter_calculator.m and can be found on my google drive, in the attached ZIP file in this post and in the code block below: https://drive.google.com/drive/folders/1A_yR4Ii53iCKqRLdPPohRVtfQLi2xvVQ?usp=sharing

Code: Select all

%% This script was mostly stolen from:
%% http://www.zipcores.com/datasheets/app_note_zc003.pdf
%%
%% This script is supposed to generate the coefficients needed
%% for the polyphase video filter as sorgelig uses it in the
%% miSTer project as of 10/29/2018.
%%
%% The filter is a 16 phase 4-tap filter with T0,T1,T2,T3 taps
%% A phase of 0 is centered on the T1 tap (according to intel docs)
%% and coefficients are from [-128,128]
%%
%%
%% If I understand correctly, this means that:
%%
%% ROW1 of the coefficient list
%% corresponds to the weights for T0,T1,T2,T3 at exactly the position of T1
%%
%% and ROW2 of the coefficients list
%% is the weights for position of T0 + 1/16 * D
%% where D is the distance from T0 to T1
%%
%% and ROW3 of the coefficients list:
%% is the weights for position of T0 + 2/16 * D
%% where D is the distance from T0 to T1
%%
%% and so on.
%%
%% If the above is not true, then this is probbaly broken!

1.0; %This line just tells Octave/Matlab that this file isn't a function m-file

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%  Filter Coefficient Generator Functions   %%%
%%                                           %%%
%%    are ALL called "filterfunction"        %%%
%%                                           %%%
%%  Uncomment only one of these at a time    %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% LANCZOS N Coefficients
%function y = filterfunction(x)
%  N = 2.0; %sets the N of LanczosN
%  x = abs(x);
%  y = 0;
%  if (x>0.0 && x<N)
%    y = sin(pi*x)/(pi*x)*sin(pi*x/N)/(pi*x/N);
%  elseif (x==0);
%    y=1;
%  end
%end


% Gaussian Scaling for y-axis with scanline effect
%function y=filterfunction(x)
%  %y = 0.955*exp(-3.8*x^2); % 25% scanlines
%  y = 0.94*exp(-3.4*x^2); % 20% scanlines
%  %y = 0.925*exp(-3.1*x^2); % 15% scanlines

%end

%% Bilinear Coefficients
%function y = filterfunction(x)
%  y = 0;
%  if (x>=0.0 && x<=1.0)
%    y=1.0-x;
%  end
%  if (x>=-1.0 && x< 0);
%    y=1+x;
%  end
%end

 %Bicubic Coefficients. Parameter A controls...something
 %Normal values are A = -0.5 , A = -0.75, with A=-0.6 as
 %a compromize
function y = filterfunction(x)
  A = -0.75;
  y = 0;
  x = abs(x);
  if (x <= 1.0)
    y = (A+2)*x^3-(A+3)*x^2+1;
  end
  if (x > 1.0  && x <= 2.0);
   y = A*x^3-5*A*x^2+8*A*x-4*A;
  end
end

% %%Sharp bilinear (not actually linear)
% %%Combined with an OPTIONAL scanline effect for use on y-axis
% %%IMO this this the best tradeoff for sharpness
%function y = filterfunction(x)
%  y = 0.0;
%  if (x>=0.0 && x<=1.0)
%   y=1.0-x;
%  end
%  if (x>=-1.0 && x< 0);
%    y=1+x;
%  end
%  pos = y;
%  if (y >= 0 && y<=0.5)
%    y=4*y^3;
%  end
%  if (y > 0.5 && y<=1.0)
%    y=1+4*(y-1)^3;
%  end
%
%  %If you uncomment ONE of the three lines below you can add a scanline effect
%  %y = y*0.965*(exp(-3.4*pos^2)+exp(-3.4*(pos-1)^2) ); %% 20% scanlines
%  y = y*0.975*(exp(-3.8*pos^2)+exp(-3.8*(pos-1)^2) ); %% 20% scanlines
%  %y = 0.985*y*(exp(-4.25*pos^2)+exp(-4.25*(pos-1)^2)); %% 30% scanlines
%  %y = 0.995*y*(exp(-5.5*pos^2)+exp(-5.5*(pos-1)^2)); %% 50% scanlines
%  %y =y*(exp(-15*pos^4)+exp(-15*(pos-1)^4)); %% fake LCD Thick Border
%  %y =y*(exp(-60*pos^6)+exp(-60*(pos-1)^6)); %% fake LCD Thin Border
%end


%% Even Sharper bilinear (not actually linear)
%% Combined with a scanline effect for use on y-axis
%% Also, you can do a neat "LCD effect" with vertical and horizontal scanlines
%function y = filterfunction(x)
%  y = 0.0;
%  if (x>=0.0 && x<=1.0)
%   y=1.0-x;
%  end
%  if (x>=-1.0 && x< 0);
%    y=1+x;
%  end
%  pos = y;
%  if (y >= 0 && y<=0.5)
%    y=16*y^5;
%  end
%  if (y > 0.5 && y<=1.0)
%    y=1+16*(y-1)^5;
%  end
%
%  %If you uncomment ONE of the three lines below you can add a scanline effect
%  y = y*0.965*(exp(-3.4*pos^2)+exp(-3.4*(pos-1)^2) ); %% 20% scanlines
%  %y = 0.985*y*(exp(-4.25*pos^2)+exp(-4.25*(pos-1)^2)); %% 30% scanlines
%  %y = 0.995*y*(exp(-5.5*pos^2)+exp(-5.5*(pos-1)^2)); %% 50% scanlines
%  %y =y*(exp(-15*pos^4)+exp(-15*(pos-1)^4)); %% fake LCD Thick Border
%  %y =y*(exp(-60*pos^6)+exp(-60*(pos-1)^6)); %% fake LCD Thin Border
%end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%  Below this is the code that actually     %%%
%%                                           %%%
%%  generates the coefficients by calling    %%%
%%                                           %%%
%%  the filterfunction you defined above.    %%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%% Generates the x-values for each tap for all 16 phases
%% Then computes the weight in [0,1.0] and stores it
for p_index = 1:16
 for t_index = 1:4
  p = (p_index - 1)/16;
  t = (t_index - 1);
  x = t - 1 - p;
 coeff(p_index, t_index) = filterfunction(x);
 end
end


% Quantize to [-128,128].  This is easy to change...
coeff_quant = round(coeff * 128);
% Check they sum to 1
sum_coeff = sum(coeff,2);
sum_coeff_quant = sum(coeff_quant,2);


% Write coefficients to a file I=Phase, J=Tap
fid = fopen('coeffs_4tap.txt', 'w');
fprintf(fid, 'PHASE: TAP0 TAP1 TAP2 TAP3 SUM\n\n');
for I = 1:16
 fprintf(fid, 'PHASE%2d : ',I-1 );
 for J = 1:4
 fprintf(fid, '%4.0f,', coeff_quant(I,J));
 end

 if sum_coeff_quant(I) == 128
 fprintf(fid, '%5d\n', sum_coeff_quant(I));
 else
 fprintf(fid, '%5d *\n', sum_coeff_quant(I));
 end
end

fclose(fid);

%Throw the coefficients out to the screen as well as the file
coeff_quant


I threw a variety of basic image resizing filter kernels in the script. Including: three variants of bicubic (different levels of sharpness/ringing), lanczos2, bilinear, a sharper version of bilinear, and an even sharper version of bilinear.

In the script, they are all named "filterfunction". You have to uncomment the one you want and then run the script. It pops the coefficients up on the screen and and saves a file called "coeffs_4tap.txt". There is an extra column after the coefficients for the row sum to help you get them to add up to 128.

Additionally, it was pretty simple to bake scanlines into the filter coefficients because you can use separate coefficients for X and Y scaling. So a few of the functions have several lines implementing Gaussian profile scanlines of different strengths.

I saved out most of the filter coefficients that I computed in a file called "Filter_Coefficients.txt" available on my google drive. But I can copy/paste them into a code block here I suppose:

Code: Select all

Regular Bilinear

  0, 128,   0,  0
  0, 120,   8,  0
  0, 112,  16,  0
  0, 104,  24,  0
  0,  96,  32,  0
  0,  88,  40,  0
  0,  80,  48,  0
  0,  72,  56,  0
  0,  64,  64,  0
  0,  56,  72,  0
  0,  48,  80,  0
  0,  40,  88,  0
  0,  32,  96,  0
  0,  24, 104,  0
  0,  16, 112,  0
  0,   8, 120,  0


Regular Bicubic - Parameter A=-0.5
A few coefficients were adjust to sum to 128

  0, 128,   0,  0
 -4, 127,   5,  0
 -6, 123,  12, -1
 -8, 118,  20, -2
 -9, 111,  29, -3
 -9, 102,  39, -4
 -9,  93,  50, -6
 -9,  83,  61, -7
 -8,  72,  72, -8
 -7,  61,  83, -9
 -6,  50,  92, -9
 -4,  39, 103, -9
 -3,  29, 111, -9
 -2,  20, 118, -8
 -1,  12, 123, -6
  0,   5, 127, -4


Regular Bicubic - Parameter A=-0.6
A few coefficients were adjusted to sum to 128

   0, 128,   0,   0
  -4, 127,   5,   0
  -7, 124,  12,  -1
 -10, 119,  21,  -2
 -11, 112,  31,  -4
 -11, 103,  41,  -5
 -11,  94,  52,  -7
 -11,  84,  63,  -8
 -10,  74,  74, -10
  -8,  63,  84, -11
  -7,  52,  94, -11
  -5,  41, 103, -11
  -4,  31, 112, -11
  -2,  21, 119, -10
  -2,  13, 124,  -7
   0,   5, 127,  -4

Regular Bicubic - Parameter A=-0.75
A few coefficients were adjusted to sum to 128

   0, 128,   0,   0
  -5, 127,   6,  -0
  -9, 124,  14,  -1
 -12, 119,  24,  -3
 -14, 113,  34,  -5
 -14, 105,  43,  -6
 -14,  96,  54,  -8
 -13,  86,  65, -10
 -12,  76,  76, -12
 -10,  65,  86, -13
  -8,  54,  96, -14
  -6,  43, 105, -14
  -5,  34, 113, -14
  -3,  24, 119, -12
  -1,  14, 124,  -9
  -0,   6, 127,  -5


Lanczos 2 - Notice this is essentially
identical to Bicbuc with A=-0.6 above
A few coefficients were adjusted to sum to 128 

   0, 128,   0,   0
  -4, 127,   5,   0
  -8, 124,  13,  -1
  10, 119,  21,  -2
  11, 112,  29,  -2
  11, 104,  39,  -4
  10,  95,  50,  -5
  10,  84,  61,  -7
  -8,  72,  72,  -8
  -7,  61,  84, -10
  -5,  50,  94, -11
  -4,  40, 103, -11
  -2,  30, 111, -11
  -1,  21, 118, -10
  -1,  13, 124,  -8
  -0,   5, 127,  -4

Sharp Bilinear (not linear anymore...)

   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   1,   0
   0, 125,   3,   0
   0, 120,   8,   0
   0, 112,  16,   0
   0, 101,  27,   0
   0,  85,  43,   0
   0,  64,  64,   0
   0,  43,  85,   0
   0,  27, 101,   0
   0,  16, 112,   0
   0,   8, 120,   0
   0,   3, 125,   0
   0,   1, 127,   0
   0,   0, 128,   0

Even Sharper Bilinear - Possibly some shimmering when scrolling

   0, 128,   0,   0
   0, 128,   0,   0
   0, 128,   0,   0
   0, 128,   0,   0
   0, 126,   2,   0
   0, 122,   6,   0
   0, 113,  15,   0
   0,  95,  33,   0
   0,  64,  64,   0
   0,  33,  95,   0
   0,  15, 113,   0
   0,   6, 122,   0
   0,   2, 126,   0
   0,   0, 128,   0
   0,   0, 128,   0
   0,   0, 128,   0
   
%%%%%% Coefficients to Add Scanlines to the Y-Axis Below %%%%%%
   
Sharp Bilinear with 20% scanlines (for y-axis scaling)

   0, 128,   0,   0
   0, 128,   0,   0
   0, 125,   1,   0
   0, 119,   3,   0
   0, 111,   7,   0
   0, 100,  14,   0
   0,  86,  23,   0
   0,  71,  36,   0
   0,  53,  53,   0
   0,  36,  71,   0
   0,  23,  86,   0
   0,  14, 100,   0
   0,   7, 111,   0
   0,   3, 119,   0
   0,   1, 125,   0
   0,   0, 128,   0

Sharp Bilinear with 25% scanlines (for y-axis scaling)

   0, 128,   0,   0
   0, 127,   0,   0
   0, 123,   1,   0
   0, 116,   3,   0
   0, 106,   7,   0
   0,  94,  13,   0
   0,  80,  21,   0
   0,  65,  33,   0
   0,  48,  48,   0
   0,  33,  65,   0
   0,  21,  80,   0
   0,  13,  94,   0
   0,   7, 106,   0
   0,   3, 116,   0
   0,   1, 123,   0
   0,   0, 127,   0

Sharp Bilinear with 30% scanlines (for y-axis scaling)

   0, 128,   0,   0
   0, 127,   0,   0
   0, 122,   1,   0
   0, 113,   3,   0
   0, 101,   7,   0
   0,  88,  12,   0
   0,  74,  20,   0
   0,  59,  30,   0
   0,  44,  44,   0
   0,  30,  59,   0
   0,  20,  74,   0
   0,  12,  88,   0
   0,   7, 101,   0
   0,   3, 113,   0
   0,   1, 122,   0
   0,   0, 127,   0


Sharp Bilinear with 50% scanlines (for y-axis scaling)

   0, 128,   0,   0
   0, 126,   0,   0
   0, 118,   1,   0
   0, 105,   3,   0
   0,  90,   6,   0
   0,  74,  10,   0
   0,  58,  16,   0
   0,  44,  22,   0
   0,  32,  32,   0
   0,  22,  44,   0
   0,  16,  58,   0
   0,  10,  74,   0
   0,   6,  90,   0
   0,   3, 105,   0
   0,   1, 118,   0
   0,   0, 126,   0


This is a Gaussian Profile for 35% Scanlines.
Use on the y-axis for a scanline effect
This is blurrier than the Sharp Bilinear Scanlines

   0, 127,   1,   0
   1, 125,   2,   0
   0, 118,   4,   0
   0, 108,   6,   0
   0,  96,  10,   0
   0,  82,  15,   0
   0,  67,  22,   0
   0,  54,  31,   0
   0,  41,  41,   0
   0,  31,  54,   0
   0,  22,  67,   0
   0,  15,  82,   0
   0,  10,  96,   0
   0,   6, 108,   0
   0,   4, 118,   0
   0,   2, 125,   1
   
This is a Gaussian Profile for 25% Scanlines.
Use on the y-axis for a scanline effect

   3, 122,   3,   0
   2, 120,   4,   0
   1, 115,   7,   0
   1, 107,  10,   0
   0,  96,  14,   0
   0,  84,  20,   0
   0,  72,  28,   0
   0,  59,  37,   0
   0,  47,  47,   0
   0,  37,  59,   0
   0,  28,  72,   0
   0,  20,  84,   0
   0,  14,  96,   0
   0,  10, 107,   1
   0,   7, 115,   1
   0,   4, 120,   2

This is a Gaussian Profile for 20% Scanlines.
Use on the y-axis for a scanline effect
This is blurrier than the Sharp Bilinear Scanlines
   
   4, 120,   4,   0
   3, 119,   6,   0
   2, 114,   9,   0
   1, 107,  13,   0
   1,  97,  18,   0
   0,  86,  24,   0
   0,  75,  32,   0
   0,  63,  41,   0
   0,  51,  51,   0
   0,  41,  63,   0
   0,  32,  75,   0
   0,  24,  86,   0
   0,  18,  97,   1
   0,  13, 107,   1
   0,   9, 114,   2
   0,   6, 119,   3
   

This is a Gaussian Profile for 15% Scanlines.
Use on the y-axis for a scanline effect
This is blurrier than the Sharp Bilinear Scanlines

   5, 118,   5,   0
   4, 117,   7,   0
   2, 113,  11,   0
   1, 106,  15,   0
   1,  98,  21,   0
   1,  87,  27,   0
   0,  77,  35,   0
   0,  65,  44,   0
   0,  55,  55,   0
   0,  44,  65,   0
   0,  35,  77,   0
   0,  27,  87,   1
   0,  21,  98,   1
   0,  15, 106,   1
   0,  11, 113,   2
   0,   7, 117,   4

This is Even Sharper Bilinear with a thin
scanline effect. It's meant to be used as the X and Y axis
coefficients for an LCD effect for handheld systems

   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   0,   0
   0, 125,   0,   0
   0, 120,   2,   0
   0, 110,   6,   0
   0,  95,  13,   0
   0,  76,  26,   0
   0,  50,  50,   0
   0,  26,  76,   0
   0,  13,  95,   0
   0,   6, 110,   0
   0,   2, 120,   0
   0,   0, 125,   0
   0,   0, 127,   0
   0,   0, 128,   0

This is Even Sharper Bilinear with an extra thin
scanline effect. It's meant to be used as the X and Y axis
coefficients for an LCD effect for handheld systems

   0, 128,   0,   0
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   0,   0
   0, 124,   2,   0
   0, 115,   6,   0
   0,  99,  13,   0
   0,  77,  26,   0
   0,  50,  50,   0
   0,  26,  77,   0
   0,  13,  99,   0
   0,   6, 115,   0
   0,   2, 124,   0
   0,   0, 127,   0
   0,   0, 128,   0
   0,   0, 128,   0

This is Bicubic with two "notches" for trinitron style aperture grille
efffect.  Use for the x-axis only
   0, 128,   0,   0
  -4, 126,   5,  -0
  -6, 119,  11,  -1
  -8, 107,  18,  -2
  -9,  98,  26,  -3
  -9,  93,  35,  -4
  -9,  90,  48,  -6
  -9,  83,  61,  -7
  -8,  72,  72,  -8
  -7,  61,  83,  -9
  -6,  48,  90,  -9
  -4,  35,  93,  -9
  -3,  26,  98,  -9
  -2,  18, 107,  -8
  -1,  11, 119,  -6
  -0,   5, 126,  -4


In my opinion, if you want a sharp image with defined pixels without bad shimmering then you should use my "sharp_bilinear" coefficients. Here's the coeff.txt (Also on my google drive in the Working_Coeffs.zip).

Code: Select all

# range -128..128
# sum of line must not exceed the range!

# Sharp Bilinear on x-axis and y-axis

# horizontal coefficients
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   1,   0
   0, 125,   3,   0
   0, 120,   8,   0
   0, 112,  16,   0
   0, 101,  27,   0
   0,  85,  43,   0
   0,  64,  64,   0
   0,  43,  85,   0
   0,  27, 101,   0
   0,  16, 112,   0
   0,   8, 120,   0
   0,   3, 125,   0
   0,   1, 127,   0
   0,   0, 128,   0

# vertical coefficients
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   1,   0
   0, 125,   3,   0
   0, 120,   8,   0
   0, 112,  16,   0
   0, 101,  27,   0
   0,  85,  43,   0
   0,  64,  64,   0
   0,  43,  85,   0
   0,  27, 101,   0
   0,  16, 112,   0
   0,   8, 120,   0
   0,   3, 125,   0
   0,   1, 127,   0
   0,   0, 128,   0

It looks like this:
Sharp_Bilinear.jpg


If you like sharpness combined with scanlines you might try sharp bilinear with 25% scanlines:

Code: Select all

# range -128..128
# sum of line must not exceed the range!

# Sharp Bilinear on x-axis and y-axis
# 25% Scanlines on y-axis

# horizontal coefficients
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   1,   0
   0, 125,   3,   0
   0, 120,   8,   0
   0, 112,  16,   0
   0, 101,  27,   0
   0,  85,  43,   0
   0,  64,  64,   0
   0,  43,  85,   0
   0,  27, 101,   0
   0,  16, 112,   0
   0,   8, 120,   0
   0,   3, 125,   0
   0,   1, 127,   0
   0,   0, 128,   0

# vertical coefficients
   0, 128,   0,   0
   0, 127,   0,   0
   0, 123,   1,   0
   0, 116,   3,   0
   0, 106,   7,   0
   0,  94,  13,   0
   0,  80,  21,   0
   0,  65,  33,   0
   0,  48,  48,   0
   0,  33,  65,   0
   0,  21,  80,   0
   0,  13,  94,   0
   0,   7, 106,   0
   0,   3, 116,   0
   0,   1, 123,   0
   0,   0, 127,   0

It looks like this:
Sharp_Bilinear_Scanlines.jpg

If that's too sharp, try blurrier scanlines with bicubic scaling:

Code: Select all

# range -128..128
# sum of line must not exceed the range!

# This is bicubic for the x-axis and Gaussian on y-axis
# 25% Scanlines on y-axis

# horizontal coefficients
  0, 128,   0,  0
 -4, 127,   5,  0
 -6, 123,  12, -1
 -8, 118,  20, -2
 -9, 111,  29, -3
 -9, 102,  39, -4
 -9,  93,  50, -6
 -9,  83,  61, -7
 -8,  72,  72, -8
 -7,  61,  83, -9
 -6,  50,  92, -9
 -4,  39, 103, -9
 -3,  29, 111, -9
 -2,  20, 118, -8
 -1,  12, 123, -6
  0,   5, 127, -4

# vertical coefficients
   3, 122,   3,   0
   2, 120,   4,   0
   1, 115,   7,   0
   1, 107,  10,   0
   0,  96,  14,   0
   0,  84,  20,   0
   0,  72,  28,   0
   0,  59,  37,   0
   0,  47,  47,   0
   0,  37,  59,   0
   0,  28,  72,   0
   0,  20,  84,   0
   0,  14,  96,   0
   0,  10, 107,   1

Sorry, I forgot to make a picture of this one. :(


I also made a sort of "LCD Screen effect" for handheld systems like Gameboy. It doesn't look great for genesis, but it's based on my rpi3 shader code and I know at least I like it for low res systems. It looks like this:
LCD_Effect.jpg


And here's that coefficient set:

Code: Select all

# range -128..128
# sum of line must not exceed the range!

# Even Sharper Bilinear with pixel outline
# effect (for LCD simulation)

# horizontal coefficients
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   0,   0
   0, 125,   0,   0
   0, 120,   2,   0
   0, 110,   6,   0
   0,  95,  13,   0
   0,  76,  26,   0
   0,  50,  50,   0
   0,  26,  76,   0
   0,  13,  95,   0
   0,   6, 110,   0
   0,   2, 120,   0
   0,   0, 125,   0
   0,   0, 127,   0
   0,   0, 128,   0

# vertical coefficients
   0, 128,   0,   0
   0, 128,   0,   0
   0, 127,   0,   0
   0, 125,   0,   0
   0, 120,   2,   0
   0, 110,   6,   0
   0,  95,  13,   0
   0,  76,  26,   0
   0,  50,  50,   0
   0,  26,  76,   0
   0,  13,  95,   0
   0,   6, 110,   0
   0,   2, 120,   0
   0,   0, 125,   0
   0,   0, 127,   0
   0,   0, 128,   0


The attached ZIP file (Also at my google drive linked at the beginning of this post) contains sample "coeff.txt" files containing most of the different scaling/scanline methods. You should grab that if you to try out the scaling methods. Also, you can replace the coefficients for X or Y with the precalculated coefficients in Filter_Coefficients.txt to mix and match methods for X and Y directions.

If we move away from the polyphase filter to an open source one, I'll see what I can do about getting the same kind of thing going with the new scaler.
You do not have the required permissions to view the files attached to this post.

BitsNStuff
Retro freak
Retro freak
Posts: 12
Joined: Tue Oct 16, 2018 7:55 am
Contact:

Re: Possibility of sharper HDMI output via integer scaling?

Postby BitsNStuff » Wed Oct 31, 2018 9:54 am

Interesting work there ghogan42.

It's always good to have options like these so that people can set up the system how they want it to be, it'll be really to see how the project moves forwards with scaling. I'll hopefully have time to try a few of the coeff files that you've uploaded when I return home later.

atmn
Atarian
Atarian
Posts: 3
Joined: Tue Oct 30, 2018 9:05 am

Re: Soft image with MiSTer

Postby atmn » Wed Oct 31, 2018 12:43 pm

cacophony wrote:
Steps for sharper picture across all current cores:

- Make sure you have a MiSTer.ini file in your config directory (https://github.com/MiSTer-devel/Main_Mi ... MiSTer.ini)
- Change video_mode to 8 (this will give you 1080p60
- Change forced_scandoubler to 1 (this will increase sharpness across *all* cores when using HDMI)

For tack sharp Genesis output:

- Make sure you have latest Genesis core and MiSTer (https://github.com/MiSTer-devel/Main_Mi ... r/releases)
- Have a "Genesis" folder at the same level as MiSTer binary, and in that folder place this file: https://github.com/MiSTer-devel/Main_Mi ... eff_nn.txt (but make sure to rename it to coeff.txt first)

In time, the other cores will be updated to utilize this new coeff.txt file (which in the case above just enables nearest neighbor scaling for maximum sharpness). If you really like the improvement of that and don't want to wait for other cores to be updated you can download one of these custom NN builds: https://github.com/Kitrinx/MiSTer_NN



Thanks, those settings made a big difference!
MUCH sharper now!, thanks!


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 11 guests