Screen tap feedback latency

Questions and Discussions about G-Stomper
TomVal
Posts: 41
Joined: Fri Nov 21, 2014 9:47 pm

Screen tap feedback latency

Postby TomVal » Wed Sep 02, 2015 6:38 pm

I'm playing with G-Stomper Studio (current beta version) on my Sony XPeria Z4 tablet. I have switchet to OpenSL (or how it's exactly called) audio engine with lower latency, set internal latency on minimum, the rest of audio parameters are on their defaults. Demos play smoothly, pattern position LEDs are in sync with music without noticeable delay - but playing pads/keys on touchscreen feels like playing an organ with pneumatic traction - the feedback comes with quite a big latency, well over 100ms. Any tips how to make on-screen pads/keyboards actually playable for a person not used to such a latency? Thanks.
User avatar
planet-h
Posts: 1545
Joined: Wed Jun 19, 2013 4:46 pm

Re: Screen tap feedback latency

Postby planet-h » Thu Sep 03, 2015 7:40 am

TomVal wrote:I'm playing with G-Stomper Studio (current beta version) on my Sony XPeria Z4 tablet. I have switchet to OpenSL (or how it's exactly called) audio engine with lower latency, set internal latency on minimum, the rest of audio parameters are on their defaults. Demos play smoothly, pattern position LEDs are in sync with music without noticeable delay - but playing pads/keys on touchscreen feels like playing an organ with pneumatic traction - the feedback comes with quite a big latency, well over 100ms. Any tips how to make on-screen pads/keyboards actually playable for a person not used to such a latency? Thanks.


Unfortunately there's nothing you can do about that. The complete latency, from the moment you press a piano key on the touchscreen until the sound hits the speaker, is a combination of multiple parts.

- Display latency
- Application latency (in G-Stomper)
- System Latency

The application latency can be optimized very well by the app itself. Using OpenSL with the latest 4.4 beta release allows an internal latency of 10ms, which is very good for an android app.

The System latency can also be optimized slightly. Using OpenSL makes the audio pipeline (which comes after the sound leaves G-Stomper) much shorter and therefor reduces the latency. But there still is System latency, at least around 30-60ms, which cannot be avoided. The reason why the nexus is faster in that sector is because it comes with support for an extra fast low latency audio path.

The display latency is under full control of the Android OS, resp. under the derivated Android which has been created by Sony.
There's nothing you can do to improve it.

The only way to get a lower display reaction time is to use a Nexus 9 device. The Nexus 9 (with GSS 4.4 BETA) runs at 5ms internal latency + around 5ms Display reaction + around 25-30ms system latency. This means you get a complete latency of 35-40ms. There's no other android device which comes even close to that numbers.

For more information, please check the following article and doc chapter:
http://www.musicalandroid.com/articles/audio-output-latency-on-android-by-planet-hcom
http://www.planet-h.com/gstomper/docs/G-Stomper_UserManual_LiveRecording.htm#_Toc423417834

Also remember that Best Tablet thread. I mentioned many times, that the only Android Tablet which allows low latencies for real, is the N9, and that for good reasons (the display reaction time is only one of them).

Sorry for not having better news.
TomVal
Posts: 41
Joined: Fri Nov 21, 2014 9:47 pm

Re: Screen tap feedback latency

Postby TomVal » Thu Sep 03, 2015 7:55 am

Thanks for your reply. Well, I got used to fact realtime playing of virtual instrument on touchscreen is quite tedious for me. Fortunately, I prefer step entering of data, so I can live with this. I don't blame your app, it's great.
User avatar
planet-h
Posts: 1545
Joined: Wed Jun 19, 2013 4:46 pm

Re: Screen tap feedback latency

Postby planet-h » Thu Sep 03, 2015 8:45 am

TomVal wrote:Thanks for your reply. Well, I got used to fact realtime playing of virtual instrument on touchscreen is quite tedious for me. Fortunately, I prefer step entering of data, so I can live with this. I don't blame your app, it's great.


I fully agree, it can be tedious. Did you also try a MIDI keyboard for real-time playing? Just wondering if there is a significant difference on your device between the on screen and the MIDI KB.
Branis
Posts: 8
Joined: Tue Sep 15, 2015 8:23 am

Re: Screen tap feedback latency

Postby Branis » Wed Sep 16, 2015 7:43 pm

Would this superpowered audio sdk be useful to lower the latency?
User avatar
planet-h
Posts: 1545
Joined: Wed Jun 19, 2013 4:46 pm

Re: Screen tap feedback latency

Postby planet-h » Thu Sep 17, 2015 5:09 pm

Branis wrote:Would this superpowered audio sdk be useful to lower the latency?


Partially, yes, there might be a minor improvement.
But regarding the fact that most modern devices come down to 10ms now (with Android 5.x), it won't be a big difference.

Superpowered would definitely improve the part of the latency which happens in Android (after the audio stream left G-Stomper).

But keep in mind, that the part of the latency which happens before a key press reaches an app (the "display latency") will still be there.
Especially the display latency varies a lot between the manufacturers. There are displays with 50-100ms delay, which happens before an audio app gets notified to play a sound.

Bottom line:
If you want a low latency android tablet, get a Nexus 9.
It is the undoubted king of android tablets if you look for serious real time playing.
It comes with a lightning fast touchscreen (near zero latency), while G-Stomper runs smoothly on it at 5ms (internal latency).
And the best of all, it costs much less than most slower big branded tablets.

There are other devices since a while which come close, but these are mostly phones, not tablets.
dag
Posts: 26
Joined: Thu Jul 21, 2016 8:59 am

Re: Screen tap feedback latency

Postby dag » Thu Aug 11, 2016 10:45 pm

planet-h wrote:
Branis wrote:Would this superpowered audio sdk be useful to lower the latency?


Partially, yes, there might be a minor improvement.
But regarding the fact that most modern devices come down to 10ms now (with Android 5.x), it won't be a big difference.


Do they, though? In Superpowered's own tests the best scenario latency on Nexus 9 is still 15 ms, and that's only on Android 6 with the flag for pro audio enabled. On Android 5 it's more than twice that! Note that this is testing loopback latency from headphones jack to microphone, but that's the sort of latency they're talking about when they're talking about the "10 ms problem" on Android from my understanding. And almost no modern Android devices come close to the performance of the Nexus 9. The closest is the Nexus 6P, a phone and at a premium cost.

With their new media server, this is cut down to 8 ms: http://superpowered.com/superpowered-an ... dia-server

(Was going to post about that so I found this thread by searching the forum for "superpowered".)
Cire86
Posts: 55
Joined: Tue Nov 26, 2013 1:47 pm

Re: Screen tap feedback latency

Postby Cire86 » Fri Aug 12, 2016 9:36 am

As a nexus9 user i can confirm the low latency on it still on android 5.0.1 since performance problems with marshmallow and trust me playing live audio on this device is absolutely wonderfull i never had any problem in any audio creation app recording my beats life you realy dont feel this small latency so if you realy care that much about latency, get one you wont be dissapointed
User avatar
planet-h
Posts: 1545
Joined: Wed Jun 19, 2013 4:46 pm

Re: Screen tap feedback latency

Postby planet-h » Fri Aug 12, 2016 1:19 pm

dag wrote:
planet-h wrote:
Branis wrote:Would this superpowered audio sdk be useful to lower the latency?


Partially, yes, there might be a minor improvement.
But regarding the fact that most modern devices come down to 10ms now (with Android 5.x), it won't be a big difference.


Do they, though? In Superpowered's own tests the best scenario latency on Nexus 9 is still 15 ms, and that's only on Android 6 with the flag for pro audio enabled. On Android 5 it's more than twice that! Note that this is testing loopback latency from headphones jack to microphone, but that's the sort of latency they're talking about when they're talking about the "10 ms problem" on Android from my understanding. And almost no modern Android devices come close to the performance of the Nexus 9. The closest is the Nexus 6P, a phone and at a premium cost.

With their new media server, this is cut down to 8 ms: http://superpowered.com/superpowered-an ... dia-server

(Was going to post about that so I found this thread by searching the forum for "superpowered".)


Thanks for the info, dag.
As you surely expect, I'm informed about the Superpowered Audio API.
Superpowered is great if you want to create a cross-platform app (which GSS clearly isn't), since it comes with the same API for Android, IOS and more.
But they do not at all anything special or anything magic. They just abstract and encapsulate the complexity of the internal audio system setup (OpenSL for example).

The reason why they lately brought the latency down is that they now code against Android N (7.0). And keep in mind that the low latency you've mentioned can be achieved on 2 devices worldwide, the Nexus 9 and the Nexus 6p, since these are the only ones that support the Pro Audio flag in Android N.

Also, this reduction of the latency does not only apply to Superpowered, it applies to any app that has a low latency audio system integrated.
On a Nexus 9, G-Stomper runs smooth at an internal latency of 2.5-3ms (the Nexus 6p has similar results). Of course that's not the whole thing, there's touch screen latency, post processing (for the speakers) latency and more. But also Superpowered has that problem.

Bottom line:
Achieving low latency on Nexus devices is nothing new, independent if you use Superpowered or not, because these devices have been built for that.
No other device models will achieve these low latency at the moment (with or without Superpowered).

However, I fully respect the effort of Superpowered, and I'm sure many devs will be happy with it.
The only thing I really don't like about it, is their non-transparent pricing model. They give it for free for new apps, but once you have reached some threshold number of installs installs, they start to charge you. How much that will be, is not clearly specified, they can factually charge whatever they want (correct me if I'm wrong, maybe they changed that meanwhile). I'd much prefer to pay a one time fee per app, independent of the number of installs.

As mentioned already in some other topic, Google anyway works on an out-of-the-box solution, which hopefully will be finished with Android 8.0 in 2017.

Cire86 wrote:As a nexus9 user i can confirm the low latency on it still on android 5.0.1 since performance problems with marshmallow and trust me playing live audio on this device is absolutely wonderfull i never had any problem in any audio creation app recording my beats life you realy dont feel this small latency so if you realy care that much about latency, get one you wont be dissapointed


Absolutely agree, Cire86. And there are good news for the Future.
I'm using the Android N (7.0) preview on my Nexus 9, and it runs perfectly smooth again. No more lags like with Marshmallow;).
And the best of all, Android N is just around the corner, as far as I know it'll be released very soon.
dag
Posts: 26
Joined: Thu Jul 21, 2016 8:59 am

Re: Screen tap feedback latency

Postby dag » Sat Aug 13, 2016 4:17 am

planet-h wrote:Thanks for the info, dag.
As you surely expect, I'm informed about the Superpowered Audio API.
Superpowered is great if you want to create a cross-platform app (which GSS clearly isn't), since it comes with the same API for Android, IOS and more.
But they do not at all anything special or anything magic. They just abstract and encapsulate the complexity of the internal audio system setup (OpenSL for example).


As mentioned in my other thread on this, this is a new (Android specific) development and not about their cross-platform Audio API.

The reason why they lately brought the latency down is that they now code against Android N (7.0). And keep in mind that the low latency you've mentioned can be achieved on 2 devices worldwide, the Nexus 9 and the Nexus 6p, since these are the only ones that support the Pro Audio flag in Android N.


I haven't seen tests on Android N. The Pro flag is new in Android 6, which is where we see the first bump. The second bump is brought by their new media server, not available in any standard Android.

Also, this reduction of the latency does not only apply to Superpowered, it applies to any app that has a low latency audio system integrated.


True about the Pro flag, in deed. The further halving of the latency does depend on the app using the new Superpowered Android media server, though.

On a Nexus 9, G-Stomper runs smooth at an internal latency of 2.5-3ms (the Nexus 6p has similar results). Of course that's not the whole thing, there's touch screen latency, post processing (for the speakers) latency and more. But also Superpowered has that problem.


Yes, internal latency can be quite far from real measured round-trip latency, though. As I understand it, it's really a measurement of number of buffers and their size?

Bottom line:
Achieving low latency on Nexus devices is nothing new, independent if you use Superpowered or not, because these devices have been built for that.
No other device models will achieve these low latency at the moment (with or without Superpowered).


Again, the point here is that they've built a completely new audio path, side-stepping the normal libraries and services, using a polling method instead of push, and achieving much lower latency regardless of device. Yes, the final latency will still depend greatly on the device, but it may well still be halved compared to before.

However, I fully respect the effort of Superpowered, and I'm sure many devs will be happy with it.
The only thing I really don't like about it, is their non-transparent pricing model. They give it for free for new apps, but once you have reached some threshold number of installs installs, they start to charge you. How much that will be, is not clearly specified, they can factually charge whatever they want (correct me if I'm wrong, maybe they changed that meanwhile). I'd much prefer to pay a one time fee per app, independent of the number of installs.


Did not know that. I was suspicious about Superpowered but they seemed legit and respected enough at a glance. Perhaps still are, but I agree about a need for (and apparent lack of) cost transparency.

As mentioned already in some other topic, Google anyway works on an out-of-the-box solution, which hopefully will be finished with Android 8.0 in 2017.


That's great news! I watched an I/O talk from 2016 about pro audio and didn't hear any plans that specific? They seem to mostly have been, and continuing to, fixing many small issues here and there and generally improving the results but still sticking to the same broken-at-the-core technologies that date back from the earliest days of even pre-Google Android.

Cire86 wrote:As a nexus9 user i can confirm the low latency on it still on android 5.0.1 since performance problems with marshmallow and trust me playing live audio on this device is absolutely wonderfull i never had any problem in any audio creation app recording my beats life you realy dont feel this small latency so if you realy care that much about latency, get one you wont be dissapointed


I bought a used Nexus 9 recently due to all the praise it gets and I had some issues on Android 5, in part with the OS itself (automatic brightness and screen rotation) but also with audio apps like DRC (where MIDI just crashed the app) so I upgraded to Android 6 and had an awful time with OpenSL in G-Stomper (crackling audio and glitchy sound) although AudioTrack worked fine (but with noticeable latency). It's running very well on Android 7 now, though. The developer preview also runs very well itself, doesn't feel like a beta in my experience so far. I think it's also the last beta version before the public release anyway, so.

Return to “General Questions and Discussions”

Who is online

Users browsing this forum: No registered users and 54 guests