hatari and midi

[607]

Does anyone have copson's program? I would like to use MIDI with Hatari. (or with Atari emulation in general, if it doesn't work with Hatari, I suppose...)

[607]

Does anyone have copson's program? I would like to use MIDI with Hatari. (or with Atari emulation in general, if it doesn't work with Hatari, I suppose...)

I got it! You can register an account at copson's website to download it. I'll let you know later if I get it working. (possibly with an edit to this post, if I get it working/give up quickly)
Edit: It does work! I couldn't get output to work, I think, and using MaxYMiser yields error messages about the pipe. Maybe I can figure that out later. But input works, and I have no issues with TTRAK! Except some sharp edges and a lag spike here and there, but I think that's to be expected.

[607]

Edit: It does work! I couldn't get output to work, I think, and using MaxYMiser yields error messages about the pipe. Maybe I can figure that out later. But input works, and I have no issues with TTRAK! Except some sharp edges and a lag spike here and there, but I think that's to be expected.

Actually, I was mistaken. I was storing the MIDI in file on my external hard drive, and when I changed it to my (internal) SSD the lag spikes and sharp edges completely disappeared. Amazing!
I still couldn't get MaxYMiser to work, though. Here is the error message:

[Eero Tamminen]

A quick look in midi.c, I think I can see one issue: the "running status" is not supported as the runningStatus byte is commented out.
When byte is higher than 0x80, we have a status byte and it should be saved, as the commented code suggests.
However, it has also to be used when the first byte after a complete message is a non-status byte and it seems that this logic is lacking.

Something like this could maybe work:
...
Unfortunately not tested as I haven't figured out how to build with PortMIDI... Do you know where I declare that PortMIDI should be used for Linux/Windows cross compiling?

I tested it already earlier, but forgot to say that it did not help.

I've now looked into PortMidi sources and "pmBadData" error comes in following situations:

• Previous sysex message was not terminated (with EOX: 0xf7) when new one starts
• Sysex data that is not realtime (0xf8 bits are not set for it)
• Non-sysex data does not have status bit (0x80) set

More info for PortMidi's data processing is in its header: https://github.com/philandstuff/portmid ... portmidi.h
And in the Pm_Write() function in: https://github.com/philandstuff/portmid ... portmidi.c

[federik0]

I confirm there is an error in midi.c, like copson wrote. It produces a midi stream with wrong status byte (zero) and that's the error.
I've done some changes in Midi_BuildEvent function and recompiled it and now it works, Notator runs in Hatari and is able to play correctly the internal wavetable synth in Windows 10. Without external mappers.

[Eero Tamminen]

I confirm there is an error in midi.c, like copson wrote. It produces a midi stream with wrong status byte (zero) and that's the error.
I've done some changes in Midi_BuildEvent function and recompiled it and now it works, Notator runs in Hatari and is able to play correctly the internal wavetable synth in Windows 10. Without external mappers.

Sounds great! Can you provide a patch? Or the modified versions of midi.c, if that is easier?

[federik0]

Sure, attached midi.c.
I can attach also the compiled version of Hatari (2.3.1 with midi.c modified) for people interested in testing. If it's allowed and size don't exceed the limit for attachments.

[federik0]

In the meantime, i'm investigating why it works in ST/STE mode (Notator and Cubase seem to run fine), while in Falcon mode Cubase doesn't produce a note! It seems a IRQ issue rather then a PortMidi issue.

Edit. Solved alone. Flag "midi emulation" was missing in my configuration of Falcon machine. The code works also with Falcon

Short video in action:

https://youtu.be/MjnmARd01Dk

[Eero Tamminen]

Sure, attached midi.c.

Thanks! If it works fine also in my testing, what name / email address you'd like to get to contributors list in Hatari "authors.txt"?

[Eero Tamminen]

While your changes fix (almost all of) the PortMidi errors about MIDI events, and I can now play previously failing MIDI songs in "SMF-Player" (and "Fractal Music" seems to work on quick test), most of the failing applications still do not work properly:

• Cubase Lite, Sequencer One, Dr T's KCS: MIDI events come with huge delay (tens of secs or even minutes) [1]
• Notator v3.0: While complex imported MIDI files play fine, playing its own "DEMO8.SON" still gave me PortMidi write error -10000 (Host error') after a while

This was tested with QSynth under Debian Stable. Could you try "DEMO8.SON", and the failing MIDI programs, especially Sequencer One?

[1] I checked that dataLength[] array accesses in Midi_GetDataLength() do not go over bounds.

[federik0]

Yes i'll try. I remember that DEMO8.SON has a very long sysex dump as first sequence in the arrange, for a Korg synth (M1 or Wave, don't remember), i'll see if that's the problem.

[federik0]

• Cubase Lite, Sequencer One, Dr T's KCS: MIDI events come with huge delay (tens of secs or even minutes) [1]

Did you change midi-device from the Hatari GUI while doing this tests? Because i found another bug while testing.
That's what happens: if you do a device change, Hatari calls Midi_UnInit procedure then try to open the new device calling Midi_Init (in which attempts to connect to a new midihost). There is a problem in this: un-init procedure disable midi interrupt, while init forget to enable. Midi_Reset, called after a reset, does.
So, if you change device during a GEM session (while testing a sequencer for example), midi events are not handled anymore by the cyclic timer of the emulator and are not sent to the host. Everything stop working until, guess when, you move the mouse. I think this is the effect of the emulation of the common IRQ line to MFP between ACIA (ikbd/mouse) and MIDI.
This can explain the long delay of the events.
Solution: i have added a call to Midi_Reset in Midi_Init, so it will set everything in the correct way (like after a reset).
I keep on testing.

[Eero Tamminen]

I've been setting MIDI device at Hatari start, before starting Atari sequencer program. I thought that was fine because things worked with SMF-Player.

But cold-booting Hatari after device change got Sequencer One to work too!

=> I'll try the programs (having huge delays) tomorrow with your Midi_Reset() call workaround.

[Eero Tamminen]

Solution: i have added a call to Midi_Reset in Midi_Init, so it will set everything in the correct way (like after a reset).

Doing it in Midi_Init() causes it Hatari freeze at startup with the new (much optimized) Hatari IRQ handling code. Doing conditional Midi_Reset() call in change.c fixes all the issue with the other programs though!

The remaining Notator "DEMO8.SON" issue is fairly bad. I do not hear any notes, after a while there's multi-sec freeze, and then comes the PortMidi error. Could that song have e.g. longer SysEx events than 4 bytes?

(While testing above, I noticed also EmuTOS bug in regards to MIDI handling with Dr T's KCS, I'll report that to its devs.)

PS. Regarding your pull request: https://github.com/hatari/hatari/pull/18

You don't need to care about the comments there, I've cleaned & reformatted your changes and will push that version when I'm done with testing.

[federik0]

Doing it in Midi_Init() causes it Hatari freeze at startup with the new (much optimized) Hatari IRQ handling code. Doing conditional Midi_Reset() call in change.c fixes all the issue with the other programs though!

Ok, didn't know the optimized IRQ code.

The remaining Notator "DEMO8.SON" issue is fairly bad. I do not hear any notes, after a while there's multi-sec freeze, and then comes the PortMidi error. Could that song have e.g. longer SysEx events than 4 bytes?

There are several sysex event in pattern 20, the first is on track 16 @1:1:8, the second in track 15 @2:4:4:96 (14179 bytes!), then in track 14 @4:1:1:49 (16350 bytes), and other folllow (shorter). They are all global setting or patch dump, so are very long.
I've played the song in my real STE, with original-dongled Notator 3.21 and it (seems to) freeze (at the second and third points above) while it send this long dumps, but even if the signal "overflow" comes up, it doesn't loose time.
In any case, i don't get any Portmidi error during these freeze and emulation seems quite close to the real hardware. I've taken a look with an external tool (MidiOx) to these dump and they are all sent correctly from Hatari.
I'm using this version (re-compiled under windows):
https://github.com/mixxxdj/portmidi

[federik0]

PS. Regarding your pull request: https://github.com/hatari/hatari/pull/18

You don't need to care about the comments there, I've cleaned & reformatted your changes and will push that version when I'm done with testing.

Thank you, I'm not a stylist programmer and it was my very first time with Git.

[Eero Tamminen]

I've played the song in my real STE, with original-dongled Notator 3.21 and it (seems to) freeze (at the second and third points above) while it send this long dumps, but even if the signal "overflow" comes up, it doesn't loose time.

In any case, i don't get any Portmidi error during these freeze and emulation seems quite close to the real hardware. I've taken a look with an external tool (MidiOx) to these dump and they are all sent correctly from Hatari.

It possible that this issue happens on real machine, but is invisible because real machines may not care about few broken sysex data bytes, especially if they are for some other manufacturer, whereas PortMidi library throws a fit and refuses all further work.

After more testing with "DEMO8.SON" in Hatari, PortMidi error always happens in the same place (in "OVERALL" section of the song, after it pauses around 6:92), but whether the error actually happens is somewhat random.

It may also partially depend on machine type and TOS version. With TOS v1.62 on STE, I was able to get it working several times in a row, but then it failed several times in a row too (with Hatari restarts in-between). With TOS v2.06, it worked only once (on STE) despite trying multiple times. It never worked with EmuTOS (whether on ST or STE) or TOS v1.04 versions. [1]

Maybe MIDI handling in this case is on edge of what ST(e) can process, so what happens depends when interrupts happen to fire? No idea whether the IRQ changes in Hatari Git head have anything to do with it. Or because you did not see it with Notator 3.21, maybe it's something related to Notator v3.0 (MCA) I have?

I tried the other available sysex file "REQUEST2.SON", and that worked without errors also on ST with EmuTOS (there were no midi notes, but I assume it's just sysex data).

[1] I tested all the other programs on emulated ST with EmuTOS, so in general that combo works fine.

[Eero Tamminen]

I've pushed the current change to Hatari upstream: https://git.tuxfamily.org/hatari/hatari ... f51e90b201

(Please tell what name / mail you'd like to have for authors.txt update.)

If you can reproduce / track down / fix the remaining Notator 3.0 issue, that can be separate update.

PS. I'm discussing on the hatari-devel mailing list changes to fix the IRQ re-enabling issue, and adding separate trace option for the IRQ & reg read / write output (which you had commented out in your own midi.c version).

[Eero Tamminen]

The more verbose MIDI trace options are now enabled only with "--trace midi_raw", so there is no need to comment them out any more: https://git.tuxfamily.org/hatari/hatari ... a81731805a

PS. The EmuTOS issue with KCS I mentioned above, turned out to be nothing related to MIDI, but a small (and invisible) difference in AES desktop window size reporting, which in KCS however caused rather drastic difference in behaviour. EmuTOS devs have not yet debugged the issue I noticed in A.C.T. MIDI program, but that's EmuTOS AES related too.

It seems that MIDI programs in overall are more "demanding" AES users than 99% of the other single-TOS apps...

[Eero Tamminen]

Fix for re-enabling MIDI IRQ on MIDI device change that does not imply emulation reset, is also now in: https://git.tuxfamily.org/hatari/hatari ... 268f5408f5

[Eero Tamminen]

I've added PortMidi HostError output to Hatari trace. With that the "DEMO8.SON" error looks like this:

Code: Select all

...
MIDI: event 7 9 8 63
MIDI: event 0 0 5 7F
MIDI: event 0 7F 0 1
MIDI: SYX END event 9 F F7 0
WARN : MIDI: PortMidi write error -10000: 'Operation not permitted'
MIDI: write error -> stop MIDI

I also found out that just by stracing (syscall tracing) Hatari, the error went away. I.e. it does not seem emulation, but host related.

=> I think that error is actually ALSA / Qsynth on Linux actually rejecting the PortMidi data from "DEMO8.SON" sometimes and/or closing the connection, not a Hatari issue.

I.e. latest Hatari Git daily snapshot should work fine with PortMidi, if the receiving host end actually accepts MIDI data produced on Atari side.

[federik0]

Last weekend i've tested Qsynth and Hatari on Windows, had a look to PortMidi sources also. It seems that Windows version doesn't soffer of that issue, hosterror is an event that PortMidi gets from host during a write. Windows and Linux versions seems different in host interface, in Windows code there is a long comment related to buffering implementation that Linux doesn't seem to have. It could have a sense that the error is related to a buffer overflow in Linux host write, but i'm not sure of this as we are emulating a 8 Mhz machine with a slow serial bus on todays Gigahertz monsters ....

[Eero Tamminen]

If there are Mac users reading this thread, Troed's daily Hatari builds have now this change along with some other ones: https://troed.ddns.net/d/b90229b625/

Note: I've changed how MIDI devices are handled in the SDL GUI, in the hopes that some day PortMidi will append new devices to end of device list without needing first call PmTerminate + PmInitialize (which Hatari cannot do as it would break current MIDI connection). If it would, new Hatari code could show new devices in Hatari SDL GUI without user needing to restart Hatari.

However, this required changing Hatari MIDI code API towards GUI code. I've changed Mac GUI code enough to have it building, but as I cannot test it, I do not know whether the Mac GUI MIDI device selection still actually works as expected... => Please test and report.

(If Mac GUI device selection does not work, just use SDL GUI to check out how well Federico's PortMidi event handling fix works.)

[darwinmac]

Eero,

I was able to select the dummy MIDI device (IAC Driver Bus 1) that you can set up from Audio MIDI Setup. However, I do not have an actual or software MIDI device to actually test the changes. Hopefully, there will be a Mac Hatari user that uses Hatari for MIDI use that can do actual testing on the MIDI changes.

Bob C

[Eero Tamminen]

I was able to select the dummy MIDI device (IAC Driver Bus 1) that you can set up from Audio MIDI Setup. However, I do not have an actual or software MIDI device to actually test the changes. Hopefully, there will be a Mac Hatari user that uses Hatari for MIDI use that can do actual testing on the MIDI changes.

Thanks!

If PortMidi ever gets support for adding new devices on the fly (= without PmTerminate+PmInitialize), then the Mac GUI code may need additional changes (SDL GUI should not), but for now your testing should be enough. Federico's MIDI data processing fixes were in the generic code, only the device selection is (partly) Mac code.