MidiLink 2.0 looking for testers. (no MIDI devices required)

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

Locked
duhproject
Atari freak
Atari freak
Posts: 56
Joined: Fri Jan 15, 2016 6:57 pm

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by duhproject »

Paradroyd wrote:
BBond007 wrote:

Is there any way to change flow control of the emulated modem? (this would normally be done with at&k on a real or Wifi modem)
I think being able to set at&k0 is necessary to make "64 Door" (a popular Amiga PETSCII terminal program) work. Otherwise, it starts up but the "modem" is unresponsive.

I've been using Term on the Amiga, Procomm Plus and even the Windows 3.1 terminal on AO486 with this with great success. Looking forward to trying The Atari ST core with it later tonight.

Again, thanks for your work on this!
I tested it today with Amiga and ao486 and it works VERY well. I also verified that we'll need the AT&K0 command to work to turn off XON/XOFF flow control to allow some terminals to work, like 64Door and Color 64. I also verified it does not like BBSs that use the TELNET protocol.

Otherwise, it works very well and is very stable. Thanks again!
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

duhproject wrote:
Paradroyd wrote:
BBond007 wrote:

Is there any way to change flow control of the emulated modem? (this would normally be done with at&k on a real or Wifi modem)
I think being able to set at&k0 is necessary to make "64 Door" (a popular Amiga PETSCII terminal program) work. Otherwise, it starts up but the "modem" is unresponsive.

I've been using Term on the Amiga, Procomm Plus and even the Windows 3.1 terminal on AO486 with this with great success. Looking forward to trying The Atari ST core with it later tonight.

Again, thanks for your work on this!
I tested it today with Amiga and ao486 and it works VERY well. I also verified that we'll need the AT&K0 command to work to turn off XON/XOFF flow control to allow some terminals to work, like 64Door and Color 64. I also verified it does not like BBSs that use the TELNET protocol.

Otherwise, it works very well and is very stable. Thanks again!
OK, 0, 3 & 4 have been implemented. I have not tested this functionality other than it does not crash.

HAYES FLOW CONTROL
---------------------
at&k0 - disable local flow control
at&k1 - ?
at&k2 - ?
at&k3 - RTS/CTS bi-directional hardware flow control
at&k4 - XON/XOFF bidirectional software flow control
at&k5 - uni-directional XON/XOFF flow control

I've modified the installer script so that It won't re-download that huge SC-55 sound font or MT-32 ROMS if you already have them.

Thanks for testing :)

EDIT: installer script moved to first post.
Last edited by BBond007 on Wed Jan 09, 2019 1:46 am, edited 3 times in total.
User avatar
Paradroyd
Captain Atari
Captain Atari
Posts: 300
Joined: Tue Sep 10, 2013 10:50 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Paradroyd »

BBond007 wrote:
OK, 0, 3 & 4 have been implemented. I have not tested this functionality other than it does not crash.

HAYES FLOW CONTROL
---------------------
at&k0 - disable local flow control
at&k1 - ?
at&k2 - ?
at&k3 - RTS/CTS bi-directional hardware flow control
at&k4 - XON/XOFF bidirectional software flow control
at&k5 - uni-directional XON/XOFF flow control

I've modified the installer script so that It won't re-download that huge SC-55 sound front or MT-32 ROMS if you already have them.

Thanks for testing :)
Amazing..Thank you!

I'm in the midst of transferring the contents of my 32GB SD card to a 128GB card. As soon as that's done I'll do some testing with the flow control and report back.

Thanks again!
- Paradroyd
@paradroyd on Twitter, @paradroyd@mastodon.sdf.org on Mastodon
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

Sorgelig wrote:
BBond007 wrote:You lost me at "AXI bridge" :)
Hopefully we can figure something out. MUNT already has support for streaming to WAV file in the xwindows front end. I'm looking at that code to see how it works. I don't know if that will help.
In simple words i need solution which will output the PCM stream into memory by just writing into circular buffer. Format is pretty much simple. for example first word is sample rate (44/48KHz), second word is buffer size, third word is current position in the buffer and then buffer itself. So if you can make this part on HPS side, then i will be able to read it on FPGA side.
The latest kernel includes my first shot at the audio circular buffer...

# cat /dev/MrAudioBuffer
0xc0c08364|512000|69628|376256|512
MrBuffer Address --> 0xc0c08364 (address of the buffer in kernel space)
MrBuffer Size --> 512000
MrBuffer Index --> 69628 (current buffer index)
MrBuffer Write Count --> 376256 (how many times the buffer has been written to)
MrBuffer Max Write Length --> 512 (this is typically 512 for MUNT and 280 for FSYNTH - this is the largest packet that has been written)

#define MR_BUFFER_LEN 512 * 1000
typedef struct RingBuffer
{
unsigned int rate;
unsigned int len;
unsigned int index;
char buf [MR_BUFFER_LEN];
} RingBuffer_t;

static RingBuffer_t MrBuffer;

In order to get ALSA to write to the device, asound.conf needs to be put in /etc

Unless you are a Sorgelig there is probably no reason for you to install this file this at this point :)

This can be used with or without a USB sound card (snd-dummy is used absence) so it will be easy to compare the results. I am very optimistic this is going to work well, writing to this buffer seems totally negligible as far as system overhead. I see no additional load...

I have a new MiSTer menu which does not block MidiLink : MUNT/FSYNT options even if a PCM device is not present - provided you have /etc/asound.conf.

I will pull the latest changes from the official branch and post a new menu momentarily.

Thanks.
You do not have the required permissions to view the files attached to this post.
User avatar
Paradroyd
Captain Atari
Captain Atari
Posts: 300
Joined: Tue Sep 10, 2013 10:50 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Paradroyd »

BBond007 wrote:
OK, 0, 3 & 4 have been implemented. I have not tested this functionality other than it does not crash.

HAYES FLOW CONTROL
---------------------
at&k0 - disable local flow control
at&k1 - ?
at&k2 - ?
at&k3 - RTS/CTS bi-directional hardware flow control
at&k4 - XON/XOFF bidirectional software flow control
at&k5 - uni-directional XON/XOFF flow control

I've modified the installer script so that It won't re-download that huge SC-55 sound front or MT-32 ROMS if you already have them.

Thanks for testing :)
It works perfectly!

Thanks again!
You do not have the required permissions to view the files attached to this post.
- Paradroyd
@paradroyd on Twitter, @paradroyd@mastodon.sdf.org on Mastodon
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

bitfan2011 wrote: It would be pretty cool to actually implement the Roland MT-32 using FPGA, it looks like they are doing this?
Also, it would be great to be able to real USB-->MIDI adapters like the ones from Roland.
If Mister can achieve both, wow 8)
You have been able use MIDI adapters and modules in the mainstream MiSTer install for a while now --> https://github.com/MiSTer-devel/Main_Mi ... o486-Cores
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

BBond007 wrote:
Sorgelig wrote:
BBond007 wrote:You lost me at "AXI bridge" :)
Hopefully we can figure something out. MUNT already has support for streaming to WAV file in the xwindows front end. I'm looking at that code to see how it works. I don't know if that will help.
In simple words i need solution which will output the PCM stream into memory by just writing into circular buffer. Format is pretty much simple. for example first word is sample rate (44/48KHz), second word is buffer size, third word is current position in the buffer and then buffer itself. So if you can make this part on HPS side, then i will be able to read it on FPGA side.
The latest kernel includes my first shot at the audio circular buffer...

# cat /dev/MrAudioBuffer
0xc0c08364|512000|69628|376256|512
MrBuffer Address --> 0xc0c08364 (address of the buffer in kernel space)
MrBuffer Size --> 512000
MrBuffer Index --> 69628 (current buffer index)
MrBuffer Write Count --> 376256 (how many times the buffer has been written to)
MrBuffer Max Write Length --> 512 (this is typically 512 for MUNT and 280 for FSYNTH - this is the largest packet that has been written)

#define MR_BUFFER_LEN 512 * 1000
typedef struct RingBuffer
{
unsigned int rate;
unsigned int len;
unsigned int index;
char buf [MR_BUFFER_LEN];
} RingBuffer_t;

static RingBuffer_t MrBuffer;

In order to get ALSA to write to the device, asound.conf needs to be put in /etc

Unless you are a Sorgelig there is probably no reason for you to install this file this at this point :)

This can be used with or without a USB sound card (snd-dummy is used absence) so it will be easy to compare the results. I am very optimistic this is going to work well, writing to this buffer seems totally negligible as far as system overhead. I see no additional load...

I have a new MiSTer menu which does not block MidiLink : MUNT/FSYNT options even if a PCM device is not present - provided you have /etc/asound.conf.

I will pull the latest changes from the official branch and post a new menu momentarily.

Thanks.
Wonderful!
Actually buffer should be mmap from FPGA address space. But i didn't decide the address yet. Probably it will be after Scaler buffer. I will finish some feature in the MiSTer binary and will be back to Linux.
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

It would be better if you will provide pull request to Kernel sources. I will need to play with options and sources to find the working config.
duhproject
Atari freak
Atari freak
Posts: 56
Joined: Fri Jan 15, 2016 6:57 pm

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by duhproject »

Paradroyd wrote:
BBond007 wrote:
OK, 0, 3 & 4 have been implemented. I have not tested this functionality other than it does not crash.

HAYES FLOW CONTROL
---------------------
at&k0 - disable local flow control
at&k1 - ?
at&k2 - ?
at&k3 - RTS/CTS bi-directional hardware flow control
at&k4 - XON/XOFF bidirectional software flow control
at&k5 - uni-directional XON/XOFF flow control

I've modified the installer script so that It won't re-download that huge SC-55 sound front or MT-32 ROMS if you already have them.

Thanks for testing :)
It works perfectly!

Thanks again!
Works here, too! Excellent. Thank you very much!
ijor
Hardware Guru
Hardware Guru
Posts: 4012
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by ijor »

Sorgelig wrote:Actually buffer should be mmap from FPGA address space. But i didn't decide the address yet. Probably it will be after Scaler buffer. I will finish some feature in the MiSTer binary and will be back to Linux.
If you will use the AXI bridges for MIDI, please make it optional. I am using the bridges for transferring copy protected images.
Fx Cast: Atari St cycle accurate fpga core
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

ijor wrote:
Sorgelig wrote:Actually buffer should be mmap from FPGA address space. But i didn't decide the address yet. Probably it will be after Scaler buffer. I will finish some feature in the MiSTer binary and will be back to Linux.
If you will use the AXI bridges for MIDI, please make it optional. I am using the bridges for transferring copy protected images.
I don't use AXI bridges. They tend hang the whole system, especially if core is loaded through USB blaster.
ijor
Hardware Guru
Hardware Guru
Posts: 4012
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by ijor »

Sorgelig wrote:I don't use AXI bridges.
I see, I understood you were going to use them.
They tend hang the whole system, especially if core is loaded through USB blaster.
I don't have problems as long as I kill MiSTer before loading a core through USB blaster. There is a problem if MiSTer and the FPGA are not in sync and the HPS attempts to access the bridges without an FPGA core acknowledging the transaction. The transaction doesn't complete and never timeouts until the FPGA acks. This shouldn't happen unless a core is changed or disabled without MiSTer knowing. Hence the need of killing MiSTer (or somehow signal MiSTer to avoid the bridges) if the core is going to change through USB blaster.
Fx Cast: Atari St cycle accurate fpga core
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

ijor wrote: I don't have problems as long as I kill MiSTer before loading a core through USB blaster. There is a problem if MiSTer and the FPGA are not in sync and the HPS attempts to access the bridges without an FPGA core acknowledging the transaction. The transaction doesn't complete and never timeouts until the FPGA acks. This shouldn't happen unless a core is changed or disabled without MiSTer knowing. Hence the need of killing MiSTer (or somehow signal MiSTer to avoid the bridges) if the core is going to change through USB blaster.
If you explain it to me, then I know this pretty well since beginning of work on MiSTer when it even had no name. Good for you if only killing the MiSTer helps you. By design, it's not supposed to reload the core at run-time. It must be loaded from bootloader and never change til power off. So, AXI/MPFE bridges work with respect this rule. There is no safe way to reload the core without power cycle or reset.
Fortunately MPFE is not that fragile as AXI and usually survives, but time after time it also brakes and requires hard reset or power cycle.

Even official socfpga kernel lost ability to reload the core at runtime. In kernels 3.xx it was possible simply load the core through /dev/*
in 4.x they still allow it (through devicetree overlays) but hide very deep and don't mention it in docs clearly.

I plan to use AXI in the future but very careful with predictable access, like disk access or input event. Bridge must be accessed by user interaction, so if no interaction from user, then AXI is idling. Another option for AXI are special cores with hybrid emulation. Those cores probably will need to be loaded (and also load the next core after) only through reset sequence like it's done now for LXDE core.
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

Sorgelig wrote:It would be better if you will provide pull request to Kernel sources. I will need to play with options and sources to find the working config.
I have provided a pull request to Linux-Kernel_4.5.0_MiSTer with the driver..

I have also made a pull request to Main_MiSTer with the system menu changes in order to accommodate TCP, UDP, MUNT and FSYNTH.

Menuconfig:

Device Drivers --->
----Sound Card Support --->
--------Advanced Linux Sound Architecture --->
------------<*> MiSTer Audio Ring buffer
------------[*] Generic Sound devices --->
--------------------<M> Dummy (/dev/null) soundcard

Dummy (snd-dummy.ko) is left as a module so that MidiLink can load it in the absence of an ALSA compatible USB sound device. This will allows the use of the FPGA buffer sound and USB sound simultaneously for testing/comparison purposes.

Thanks for looking into this!
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

ijor wrote: If you will use the AXI bridges for MIDI, please make it optional. I am using the bridges for transferring copy protected images.
Ijor,

I think the topic of AXI bridge came up because Sorgelig why explaining the technical reason he did not want to use use one and my response was - You lost me at "AXI bridge" :)

Anyway, just to clarify, this implementation is not just about MIDI. This is about ANY application running on the HPS that would like to produce sound via the Advanced Linux Sound Architecture. You could do any number of things with this beyond MIDI. For example: MP3 player, speech synthesizer, etc.

In other news, I fixed the problem where FluidSynth occasionally had no audio output. What was happening is I was not delaying long enough after killing the MUNT process (residing on sequencer port 128). Sometimes FluidSynth would get loaded on port 129. My initial idea was to simply always put MUNT on 128 and FluidSynth on 129, but that assignment seems entirely up to ALSA and there is no easy way to control it.

Anyway, what happens now is that I scan through the sequencer client list looking for a match and then use whatever client port it happens show up on. If I don't find which port its on then I delay 2 seconds and scan again up to 3 attempts.

So far its a good fix.
Last edited by BBond007 on Wed Jan 09, 2019 9:42 pm, edited 1 time in total.
NegSol
Captain Atari
Captain Atari
Posts: 355
Joined: Sat Dec 05, 2015 9:22 pm

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by NegSol »

Great, thanks for fixing this so fast! :mrgreen:
ijor
Hardware Guru
Hardware Guru
Posts: 4012
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by ijor »

Sorgelig wrote:By design, it's not supposed to reload the core at run-time. It must be loaded from bootloader and never change til power off. So, AXI/MPFE bridges work with respect this rule. There is no safe way to reload the core without power cycle or reset.
Fortunately MPFE is not that fragile as AXI and usually survives, but time after time it also brakes and requires hard reset or power cycle.
The whole process is a bit delicate. I think both the MPFE interface and the AXI bridges are fragile, but in a slightly different way. From the documentation, notes and some statements by Intel/Altera engineers the guidelines seem to be:

The FPGA-HPS bridges must be idle while reloading a new core, and more general, the bridges should not be accessed from the HPS side without an active FPGA core acking the bridge transactions.

The MPFE doesn't affect the Linux side so much because it doesn't really interacts with the processor, it connects directly to the DRAM controller. But it has a different type of problem. A new core must use the same FPGA-MPFE interface configuration. Attempting to load a new core with a different configuration will not work. The FPGA-HPS SDRAM interface can only be configured at boot time. Or more precisely, the configuration must be activated from the HPS side with the DRAM controller completely idle. Which means that the code must be running from cache. And that can't be done, at least not reliably, except at boot time.

I realize you know this already. I am writing this just for the record because it is not clearly documented and I learnt it the hard way. Although it might be better placed in a different thread, perhaps.
Fx Cast: Atari St cycle accurate fpga core
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

That's why i've set the same MPFE config for all cores. I think i've set more or less ideal config giving one highspeed channel for video processing and 2 less speedy channels for everything else. Actually one channel can be shared among many users if qsys blocks are used - they have good interconnect adapters hiding the whole complexity of different widths, burst length and even different versions of AvalonMM giving great flexibility to choose whatever you need.

Still time after time MPFE gets screwed. And it seems doesn't depend on usage MPFE. I did many experiments in the beginning of MiSTer development in order to find the root of the problem, but coudn't track it down. Sometimes MPFE screwed after core loading even if previous core didn't use MPFE at all. So user may see either black screen with no sync, or if monitor tries to catch the sync you can see corrupted video. Since Linux never hangs in this condition I assume it's pure FPGA part of DDR3 access while the memory itself works fine.
BBond007
Captain Atari
Captain Atari
Posts: 466
Joined: Wed Feb 28, 2018 3:23 am

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by BBond007 »

Paradroyd wrote: The only gotcha I've run into is that it seems to choke with actual telnet servers (things using telnet protocol, as opposed to raw TCP), like Linux telnet servers or BBSs that expect it to answer telnet negotiations. It seems to just hang on connect in that case, probably because the server never gets the response it's waiting for.
duhproject wrote: I also verified it does not like BBSs that use the TELNET protocol.
I have added very basic telnet negotiations to the TCP option.

its on by default but can be disabled:
ATTEL0 <-- disable
ATTEL1 <-- enable

Hopefully its enough for BBSs.

I have been able to connect to telnet to my Raspberry Pi (running RetroPie) where it would hang before.

Running some apps like top complain about unknown terminal type.

'export TERM=vt100' seems to fix that.

Thanks for testing!
User avatar
Paradroyd
Captain Atari
Captain Atari
Posts: 300
Joined: Tue Sep 10, 2013 10:50 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Paradroyd »

BBond007 wrote: I have added very basic telnet negotiations to the TCP option.

its on by default but can be disabled:
ATTEL0 <-- disable
ATTEL1 <-- enable

Hopefully its enough for BBSs.

I have been able to connect to telnet to my Raspberry Pi (running RetroPie) where it would hang before.

Running some apps like top complain about unknown terminal type.

'export TERM=vt100' seems to fix that.

Thanks for testing!
I've done some testing tonight and this is working great..well done!

Thanks for working this out so quickly!
- Paradroyd
@paradroyd on Twitter, @paradroyd@mastodon.sdf.org on Mastodon
duhproject
Atari freak
Atari freak
Posts: 56
Joined: Fri Jan 15, 2016 6:57 pm

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by duhproject »

BBond007 wrote: I have added very basic telnet negotiations to the TCP option.

its on by default but can be disabled:
ATTEL0 <-- disable
ATTEL1 <-- enable

Hopefully its enough for BBSs.

I have been able to connect to telnet to my Raspberry Pi (running RetroPie) where it would hang before.

Running some apps like top complain about unknown terminal type.

'export TERM=vt100' seems to fix that.

Thanks for testing!

Works very well. Thank you VERY much! This is awesome.
ijor
Hardware Guru
Hardware Guru
Posts: 4012
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by ijor »

Sorgelig wrote:Actually one channel can be shared among many users if qsys blocks are used - they have good interconnect adapters hiding the whole complexity of different widths, burst length and even different versions of AvalonMM giving great flexibility to choose whatever you need.
The MPFE is extremely efficient. I admit much more than I expected.
Still time after time MPFE gets screwed.
I've never seen it, but I'm sure I don't run, not nearly, as many tests and with as many different cores, as you regularly do. But it means that is not very frequent and then it is reasonable and acceptable.
Fx Cast: Atari St cycle accurate fpga core
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

If you try to switch between cores fast (once per couple seconds) - then you will get this problem after 5-10 loads.
In normal usage it seldom happens.
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by Sorgelig »

New MiSTer binary includes changes from BBond007, but i didn't announce it because other parts outside the binary aren't integrated yet.
duhproject
Atari freak
Atari freak
Posts: 56
Joined: Fri Jan 15, 2016 6:57 pm

Re: MidiLink 2.0 looking for testers. (no MIDI devices required)

Post by duhproject »

Sorgelig wrote:New MiSTer binary includes changes from BBond007, but i didn't announce it because other parts outside the binary aren't integrated yet.
This is excellent news! Thank you very much!
Locked

Return to “MiSTer”