March 8, 2022
When selecting an Application Framework for your next project it is important to understand what the performance benefits are. One of the main recurring questions is "How does Lightning compare to HTML5?" and in this page we'll answer the performance aspect of Lightning versus HTML5.
We'll use 2 different classes of embedded devices, one on the low-end and one of the higher-end spectrum of performance (at the time of writing this article).
Please note these devices are very bespoke and differ greatly from your typical target PC. Albeit Lightning can be perfectly used for PCs or high end devices, the performance gain is higher on low-end devices.
The devices used in this analysis:
CPU: Broadcom SOC 7428 (Dual-core MIPS CPU 3000 DMIPS)
Mem: 1 GB Shared (~200 MB available)
GPU: VideoCore 3 (~195 MB Gfx memory available)See software stack details below. For those unfamiliar with STB hardware this particular device has a more or less equivalent processing power of an iPhone 4G (4k DMIPS on Apple A8 ARM based) released in 2010.
CPU: Broadcom SOC 7268 (Quad-core ARM CPU 14k DMIPS)
Mem: 2 GB Shared (~250 MB available)
GPU: VideoCore 5 (~195 MB Gfx memory available)The equivalent processing power of the high-end STB is roughly of an iPhone SE released in 2016. Though the iPhones get a very powerful PowerVR GPU's compared to the embedded platforms making it harder to do a direct compare.
Both devices are running an RDK software stack which is built on a custom Yocto embedded Linux distro using the 4.1 Linux kernel.
The browser used is an embedded Safari/WebKit: WPEWebKit version 2.22.
To run the performance test we're running an IMDB application with a custom design. One application written in LightningJS and the other application in HTML5 using VueJS.
Both apps are loaded onto the STBs, wait until the splash screen ends and are ran using a pre-scripted set of key sequences.
The average/current Frame per Second (FPS) is captured using a requestAnimationFrame counter on screen.
At the end of the test sequence the average FPS is noted for comparison.
Each test sequence is recorded using a direct HDMI capture to avoid delays/processing of a TV using an elgato capture device.
These are the results of the two applications on the low-end STB:
As you can see the frame drops of the HTML5 version on the Low-end STB are very frequent. Resulting into a degraded or "stuttering" experience of the UI. Whereas Lightning stays relatively stable through all the animations.
So even though the average FPS is pretty close, the HTML5 version frequently drops below 10 FPS. Especially when there is a lot happening at the same time.
These are the results on the high-end STB:
The high end STB shows a different picture. Where the HTML5 drops are still present it does seem to run a but smoother then on the Low-end STB. Overall it does struggle to get a higher average FPS. The Lightning version is rock stable in the high 50 FPS zone and rarely dips and the overall experience on the Lightning app is buttery smooth.
In both tests the Lightning experience is much better. With less frequent dips the average FPS stays well above the threshold, keeping the FPS on both the low-end as well as the high-end as high as possible resulting in a smooth UI.
The impact of Lightning becomes more apparent on low-end devices where it prevents dips below 10-15 FPS, which is where users start noticing the frame drops and the experience comes across as "choppy" or "slow". On the high-end device the experience becomes really smooth and likely able to handle a lot more animations/effects for more added eye candy. However with every project you have to balance the target devices and the desired UX as a whole.
Overall the Lightning experience is a lot more pleasant and smooth, due to the higher average FPS and far less (to almost none) FPS dips during animations.