Bugzilla – Bug 900
Faster spectrogram, and slightly faster effects
Last modified: 2018-08-20 11:51:49 UTC
Drawing of the spectrogram causes at least one opening and closing of a block file for each column of pixels. Therefore zooming in and out of spectrogram views is much slower than it could be even without more clever mathematics. Loop-play of a very short selection (say 1 ms or less) causes a number of openings and closings inversely proportional to the length, and a noticeable delay between pressing start and playing of the sound. Effects may have a slight inefficiency because WaveTrack::GetBestBlockSize() does not do what it is supposed to do (report a size that aligns fetches of sample data to block boundaries). So each block file may be opened and closed twice, not once. These problems could be addressed separately but are related, and I present a patch that remedies them all.
Created attachment 586 [details] Performance enhancement for spectrogram and some other things
Notice also with this patch that if you play a long selection in a spectrogram view, the delay in updating the screen when it scrolls a "page" down may be much less. But depending on spectrogram window size and zoom level, it may still be significant because calculation time dominates. I think more progress with that performance problem might be solved with anticipatory calculation of the spectrogram of the next screenful before it appears, and that might be done with threads. Now we are talking about a bigger project.
(In reply to comment #2) The patch for short loop play in bug 860 seems more effective than this patch on Win 7 HDD 2.4 GHz 6 GB RAM. Paul wrote: > Notice also with this patch that if you play a long selection in a spectrogram > view, the delay in updating the screen when it scrolls a "page" down may be > much less. But depending on spectrogram window size and zoom level, it may > still be significant because calculation time dominates. On the above machine at 32768 window size, the patch only gives modest speedup when switching from wave track to spectrogram views and almost no improvement when scrolling a long track at high zoom levels. In particular in spectrogram view at high zoom levels and high window size I don't get to see the playback cursor move across the screen at all, and I wait almost as long for a zoom to redraw the waveform with the patch as without it.
Gale, do you agree that with a more modest window size of say 1024, this is an improvement.
(In reply to comment #4) > Gale, do you agree that with a more modest window size of say 1024, this > is an improvement. I see no improvement on that Win 7 machine until I get up to 16384 window size. At sizes lower than that, response is already near immediate (I have not tested longer than 30 minutes of audio). At 16384 and 32768 the bug 900 patch offers same or slightly faster response (still some delay).
That answer surprises me, but my question was imprecise. Did you try scrolling or zooming? I expect greater savings with zooming.
More precisely: This patch does nothing to help drawing of waveform views. If you thought you saw a slight difference, I would put that down to random variations. I expect no improvement if you switch from spectrogram to waveform and back. The switch back uses a cache of already computed spectrum values. This patch helps when populating that cache, but not when reusing it. I expect improvement if you draw spectrograms for the first time after loading a project. I expect improvement if you change zoom level in spectrograms, because that completely invalidates the cached results. I expect less improvement if you scroll a spectrogram view horizontally by less than a screen, as with shift-mousewheel or clicking the arrowheads of the scrollbar. In these cases only the portion that comes on screen requires calculation of spectra. The rest is cached already. I expect improvement like that of zooming, when you scroll by an entire screenful, as with page-up and -down keys. Cache does nothing to help in those cases. I expect that the absolute time savings for drawing a whole screen of spectrogram will vary little with zoom level or with spectrogram window size, and so should be proportionally greater when the window size is less. I expect that computers with slower disk drives should show greater improvement. I observe in a few trials of my own that with a 1024 sample window, the change in speed for zooming in and out with ctrl-mousewheel in spectrogram view is very noticeable.
(In reply to comment #7) > I expect improvement if you draw spectrograms for the first time after loading > a project. Yes, I guessed there was caching so I only tested "drawing the first time". There is repeatable improvement drawing the first time at the two largest window sizes, but it is only a modest improvement. There is still a few seconds delay in switching first time to spectrogram. > I expect improvement if you change zoom level in spectrograms, because that > completely invalidates the cached results. I observe in a few trials of my > own that with a 1024 sample window, the change in speed for zooming in and > out with ctrl-mousewheel in spectrogram view is very noticeable. I had only zoomed using shortcuts or buttons. I understand now that mousewheel zoom is a sterner test because you can zoom faster. Yes, I see some improvement even at 1024 with mousewheel zoom, but very minor because it is already responsive. The improvement I see only becomes noticeable at 4096 or larger, despite what you seem to say. I really see no lag when zooming in/out on spectrograms with shortcuts or buttons until the two largest window sizes, so I see no improvement until those two largest sizes. But I am using release build - are you using debug? I also had not tried Page UP / Page DOWN. Like mousewheel zoom, those shortcuts show some improvement at 1024, but only noticeably so in the largest four sizes. I expect improvement like that of zooming, when you scroll by an entire screenful, as with page-up and -down keys. Cache does nothing to help in those cases.
Gale, isn't one of your computers an the underpowered Linux notebook? That one might demonstrate greater savings.
(In reply to comment #9) Paul wrote: > isn't one of your computers an the underpowered Linux notebook? > That one might demonstrate greater savings. Yes, though it can run Windows 7 as well. I tried the patch on it on Linux and Windows. Obviously spectrograms are much slower on that machine, but the patch shows the same modest, not dramatic improvements on either OS. The modest improvements can be seen from 1024 size upwards rather than from greater sizes. Patch or not, there are significant delays at 16384 or 32768 - they average out at about five seconds with the patch and six seconds without it.
I think 16% or so improvement is significant.
Note that the performance of zooming of spectrograms may depend not only on the speed of the computer, but also the speed of the storage device. If you test with a project that stores only temporary files on faster media, you may notice less improvement.
Submitted here: https://github.com/audacity/audacity/commit/ad42f1642846175fc319e2c15b58fc17cf22dcb2 I decided not to use this to fix short loop play. Let Bug 860 track that. I will fix by other means.
I removed "short loop play" from the title, per comment 13. I still see the improvement as modest but welcome, and not more marked on the slow Linux netbook than on my somewhat faster machines.
