Context:
I'm trying to play audio files over an AppleSequencer. The audio file playbacks should be triggered in accurate sync with a predefined sequence of AppleSequencer beats: file 1 playback starts at beat 4, file 2 starts at beat 8, file N starts at beat 4N (there's no overlap: each audio file lasts less than 4 beats).

Question:
Is there a solution to this problem which allows seeking to any offset of the sequencer? For instance, when we seek to beat 4.123, then file 1 playback should start at the offset which corresponds to "beat 0.123" (by this I mean that the playback should be in sync with the sequencer, the same sync it would have if the sequencer were started at beat 0).

Current attempt:
My current sync attempt is not compatible with seeking at any offset (when I seek at an offset in the middle of an audio file, that file isn't played at all, only next files are). This attempt tries to use this SO answer, which answers the case N = 1: start AppleSequencer instance, fetch its hostTime corresponding to beat 4, and then start AudioPlayer instance at that time.

Actually, this attempt does not lead to accurate synchronization when using AudioKit AppleSequencer, so to get perfect sync I picked AVAudioSequencer as sequencer instead.

For general N >= 1, I use a MIDICallbackInstrument which will run the above strategy a bit before each beat that is a multiple of 4 (NB: I play the same AudioPlayer instance in the snippet below for simplicity):

// Add MIDI events, which will be picked by a callback, just before multiples of 4
let track = appleSequencer.tracks[0]
for i in 0..<N {
    track.add(noteNumber: 66, velocity: 127, position: Duration(beats: 3.5 + 4.0 * Double(i)), duration: Duration(beats: 1))
}

// Set a callback to play the AudioPlayer instance when it encounters these events
midiCallbackInstrument.callback = { _, note, _ in
    if note == 66 {
        let nextAudioStartInBeats = ceil(self.appleSequencer.currentPosition.beats / 4.0) * 4.0
        do {
            let nextAudioStartInHostTime = try AVAudioTime(hostTime: self.appleSequencer.hostTime(forBeats: nextAudioStartInBeats))
            audioplayer.play(at: nextAudioStartInHostTime)
        } catch { /* ... */ }
    }
}