Evaluating LCEVC Decoding Performance on Android Platform
This article is part 2 of our evaluation of the LCEVC codec , the first part can found here.
We compared CPU-usage on an android device when playing video encoded with LCEVC enhanced HEVC to CPU-usage when playing video encoded with native HEVC.
We found that using LCEVC enhanced HEVC gave a moderate CPU-usage overhead of ~35 % for a 25fps video, and ~45% for a 50fps video. While the CPU-usage overhead measured is not big enough to make using LCEVC enhanced HEVC unfeasible on mobile devices, it is something that needs to be taken into account when considering deploying video encoded with LCEVC enhanced HEVC.
All results and conclusions presented here are based on tests on a limited selection of test videos and playback devices. As such, the results may not accurately reflect playback performance for different devices and/or test videos.
Introduction
At Eyevinn we are always looking to stay up to date with new developments in video and streaming technologies. LCEVC is a new interesting video codec standardized by MPEG, and an implementation is available from V-NOVA. This article is part 2 of our evaluation of LCEVC. In part 1 (Will LCEVC-enhanced x265 contribute to sustainability in streaming) we evaluated the encoding performance of the V-nova LCEVC codec. Of equal importance to encoding performance is decoding performance, especially for platforms like mobile phones where increased CPU-usage can have a significant impact on power consumption.
LCEVC is a new interesting video codec standardized by MPEG, and with an implementation available from V-NOVA. An introduction to the LCEVC codec can be found in our previous blog post Will LCEVC-enhanced x265 contribute to sustainability in streaming or at lcevc.org
In our previous blog post we had a look at the encoding performance of the V-nova LCEVC codec. Arguably decoding performance is of equal importance to encoding performance, especially for platforms like mobiles where increased CPU-usage can have a significant impact on power consumption. In this article we will investigating playback performance of LCEVC on android devices, by evaluating V-nova LCEVC enabled exoplayer with regards to
- Decoding performance
- Ease of integration
Test Method
Our aim was to get some measure of the decoding performance of the LCEVC decoder when decoding an LCEVC enhanced HEVC stream, with the base stream decoded in hardware. We decided that a suitable proxy measure for this was to measure how much more CPU was used when decoding a LCEVC enhanced HEVC stream compared to a native HEVC stream decoded in hardware. For reference we also did measurements of CPU usage when decoding a native HEVC stream with software decoding [1].
We used Eyevinn android player, a a simple android video player app based on exoplayer as our test player, together with LCEVC enabled exoplayer libraries supplied by V-Nova. Eyevinn android player allows loading a list of urls from a network location, which fits our purpose well. For measuring CPU-usage we used standard linux tool run over ADB (Android Debug Bridge), more details on this can be found in Appendix A. We measured cpu usage for relevant processes, i.e. our app process, and processes owned by the mediacodec user.
For each test run, we measured CPU usage during 60s of playbck. We measured the CPU usage of the processes as total CPU time used. This is of course a rather course-grained measure. It does not take into account the fact that the CPU frequency may be scaled up or down, or that the scheduler might schedule a process on a more power-efficient core resulting in a lower overall performance. Perhaps a more controlled environment could be achieved by disabling frequency scaling and setting the core affinity for the relevant processes, but this was left out of the scope of this investigation.
We do belive that the CPU time as measured is still a reasonably good indicator of the processing power used by the application.
Test Video
We encoded SVT natural complexity both as LCEVC enchanced HEVC and as native HEVC. In both cases x265 was used for HEVC encoding. For each of these two different encodes were produced:
- ~8000kbit/s, 1920x1080, 50fps, yuv420 8bit
- ~8000kbit/s, 1920x1080, 25fps, yuv420 8bit
The videos where packaged as HLS with shaka packager.
Test Devices
We used the below devices for testing. We did not have dedicated test devices available, the devices had other applications running in the background etc. It was assumed this would not have a major impact on the tests. A more varied set of devices with a more controlled enviroment would of course have been better.
- Device 1: Samsung galaxy S22 with Samsung exynos 2200 chipset.
- Device 2: Samsung galaxy S22 Ultra with Samsung exynos 2200 chipset
The samsung exynos chipset has 3 processors and a total of 8 cores:
- 2.8GHz ARM Cortex-X2 (Single-core)
- 2.5GHz ARM Cortex-A710 (Triple-core)
- 1.8GHz ARM Cortex-A510 (Quad-core)
Test Results
We did 10 measurements for each case on device 1, and 5 measurements of each case on device 2.
Table 1 shows total CPU time in miliseconds, while Table 2 shows relative CPU time compared to the base case (native x265).
Table 1: Total CPU-usage in ms
+----------+-------------------------------+-------------+---------------------+-------------------------------+
| Device | Test case | native x265 | LCEVC enhanced x265 | native x265 software decoding |
+----------+-------------------------------+-------------+---------------------+-------------------------------+
| Device 1 | SVT natural complexity 25 fps | 32560 | 47155 | 60355 |
| Device 1 | SVT natural complexity 50 fps | 43891 | 66584 | 103710 |
| Device 2 | SVT natural complexity 25 fps | 39214 | 51912 | 61886 |
| Device 2 | SVT natural complexity 50 fps | 57848 | 79086 | 91100 |
+----------+-------------------------------+-------------+---------------------+-------------------------------+Table 2: Relative CPU-usage compared to native x265
+----------+-------------------------------+-------------+---------------------+-------------------------------+
| Device | Test case | native x265 | LCEVC enhanced x265 | native x265 software decoding |
+----------+-------------------------------+-------------+---------------------+-------------------------------+
| Device 1 | SVT natural complexity 25 fps | 100% | 144% | 185% |
| Device 1 | SVT natural complexity 50 fps | 100% | 151% | 236% |
| Device 2 | SVT natural complexity 25 fps | 100% | 132% | 158% |
| Device 2 | SVT natural complexity 50 fps | 100% | 137% | 157% |
+----------+-------------------------------+-------------+---------------------+-------------------------------+The results differed somewhat between the devices, but were consistent with each device. Since the devices use the same chipset this is somewhat unexpected. We assume this is due to some differences in environment on the two devices. The general trend of the measurements were the same on both devices.
Our measurements indicate that when decoding a LCEVC enhanced HEVC stream, CPU time used was higher compared to when decoding a native HEVC stream with hardware decoding. This is not suprising since the decoding of the LCEVC enhancement layer is primarily done on the CPU, although with some hardware acceleration.
Conclusion
Our measurements showed a clear cpu-usage overhead when playing LCEVC enhanced HEVC compared to native HEVC. It is not trivial to estimate what impact this cpu-usage overhead has on the overall power consumption of the device when streaming video. However it is likely that the CPU-usage overhead is large enough to have some effect on power consumption.
While using CPU-decoding on mobile devices is generally not desirable because of the increased power consumption, both Netflix and Meta have deployed cpu-decoding for the AV1 codec to mobile devices, see [2] and [3]. The CPU overhead of using LCEVC enhanced HEVC according to our measurements is not as big as when using pure software decoding. Putting this together, the CPU-usage overhead does not neccesarily rule out using LCEVC on mobile devices, allthough until LCEVC is fully supported in harware it does put LCEVC enhanced HEVC somwhat at a disadvantage compared to native HEVC when it comes to power consumption of playback on mobile devices. From a sustainability perspective, the potential bandwidth savings from using LCEVC enhanced HEVC would have to be weighted against the increased power consumption from playback.
Experience from integrating V-Nova exoplayer libraries
Integrating the exoplayer libraries from v-nova in our application was straightforward, all that was needed was to import the supplied aar-libraries instead of the standard exoplayer-libraries. To enable playback of LCEVC-enhanced streams it was also necessary to set EXTENSION_RENDERER_MODE_PREFER on exoplayers RenderesFactory.
Appendix A Cpu usage measurement details
To measure cpu-usage, we used standard linux command run over ADB (Android Debug Bridge) Tthe ps command was used to find the PID of the relevant processes. There appeared to be two processes relevant for our measurements, our app and a video decoding process owned by the mediacodec user. For each of these processes, we logged CPU-usage from /proc/PID/stat every second.4 The cputime used taken as was the sum of utime and stime from the stats files, converted to milliseconds according to the CLK_TCK setting of the system.
For more information about this evaluation contact Gustav Grusell at Eyevinn Technology. Eyevinn Technology are vendor independent experts on sustainable video streaming. Contact us at sales@eyevinn.se if you want to know more on how we can help you reduce your service’s carbon footprint.
[1] This was done by providing ExoPlayer with a MediaCodecSelector that only allowed software codecs.
[2] https://netflixtechblog.com/bringing-av1-streaming-to-netflix-members-tvs-b7fc88e42320
[3] https://engineering.fb.com/2023/02/21/video-engineering/av1-codec-facebook-instagram-reels/
