Real time audio in Qt

[In my previous post I went through the use of QAudioDevice, but didn't show real time generation of audio. This post will not offer a solution to this either. Discusses the topic and leaves it where I left it, in case I want to re-take it. It will be a bit of rambling, sorry, but still, if you are on this topic and have not found the answer anywhere else, may point you in the right direction. If you have found a good example of this, please add a comment... Thanks!

Note: ha, like if anybody was ever going to read this :) ]

We are trying to have an audio system that has almost no latency, i.e, that a user can't notice it. This is important, for instance, in the case of games, where a player would see an explosion and should listen it right at the same time. On our example before, we had an audio buffer of about 1.5s (131072 bytes --> 65k samples --> ~1.5s at 44100samples/s audio). That meant that whenever we add something to that buffer, we will hear it a while later... Not good.

A thought was to reduce the length of that buffer, but if we simply were doing that we were getting some weird noises, maybe due to buffer overrun (?). We found documentation where they did mention that the buffer could be 10ms, but again, it did not work when we tried straight forward.

Using simply the audio play/stop may be fast, but still, the buffer has whatever it has, so, the latency from whenever the audio data is generated to the time is played would not change.

We also have the question on how signals work... Say we have a signal to service the timer updating the audio and one to service the audio generation itself, like creating the audio for an explosion. How do we service both on the right order? Something tells me this is done automatically on some kind of even queue.

But actually I am thinking that there may be several events happening at the same time, like two explosions, one after another, but with the first being long enough that is not finished by the time the next one comes... How do we deal with it? Several audio channels? Or mix first the audio and then output? Probably the second.

As we are talking about games, here is a good summary of the base framework in Qt:
http://boards.openpandora.org/index.php?/topic/108-qt-4x-as-a-game-engine/
Wish I had more time to play. Look at the demos!

Some people talks about using SDL which their authors describe as:
"Simple DirectMedia Layer is a cross-platform multimedia library designed to provide low level access to audio, keyboard, mouse, joystick, 3D hardware via OpenGL, and 2D video framebuffer. It is used by MPEG playback software, emulators, and many popular games, including the award winning Linux port of "Civilization: Call To Power."
http://www.libsdl.org/
http://www.libsdl.org/intro.en/usingsound.htmlhttp://www.libsdl.org/projects/SDL_mixer/

On the same topic of real time audio, there is a good post here:
http://lists.trolltech.com/qt-interest/1997-10/thread00245-0.html

Scroll down all the way, to read it. Basically it is saying that
"The conclusion is that you need to decouple the audio task from Qts event processing at all. I`ve done this by creating two threads: one that reads audio data from a mp3 decoder and a second one that copies the data to /dev/dsp. Both threads simply block on read/write and communicate through a simple buffer queue.

Calls into Qt from concurrent threads seems to be a bad idea as Qt is not thread safe by itself. That is: no signal/slot communication from within the threads, no interactions with the framework at all ;-)"

====

On a separate topic, on our previous post, we left unanswered what push vs pull mode were:

http://stackoverflow.com/questions/3426868/play-audio-data-using-qiodevice-qt4-6-with-vc

Explains it pretty well, showing that we indeed were using push. Notice that we used the setNotifyInterval at 50ms (we checked this by reading it too). That, at 44100 samples/s would mean every 4410 bytes, but this number does not really show up anywhere. It is not, certainly the periodSize, which is larger than this. So, notify happens more often but we do not get to see it. We plot every write call, but emptyBytes is at least periodSize long, not 4410?!?

One important thing I left out is that we are assuming that all what we write is written, but that is not truth. If the buffer is full, it wouldn't. Nevertheless, I think it is working because we are checking a priori how many bytes are empty.

Cheers!

PS.: If I ever figure it out may want to go reply other folks with the same doubt, like http://qt-project.org/forums/viewthread/20391