Introduction:

    The real technology sharing this time mainly introduces the definition and principle of PerfDog Jank and Stutter as well as effects, etc. The contents will be described in six parts: FrameTime, FPS, fluency, Jank, Stutter and effects. Analyze the significance of these six parts in performance test at a deeper level.

Part I: FrameTime

    Definition of FrameTime: frame interval (can also be simply considered as single frame rendering time).

    Whether FrameTime is related with Jank? Please see the case graphical representation below:

    From the figure, it can be found that the rendering time of Frame B in the GPU (frame generation time) is longer than the display refresh interval and consumes twice the display refresh time. That is to say, the screen does not refresh once. If the screen does not refresh many times (i.e. the screen remains unchanged), it may be one Jank.

In can be concluded here that: what a player user saw is the new image fresh frame rate rather than the eglSwapbuffers-GPU rendering completion (not submitted to screen to be displayed) interval. Thus, the Frametimes mentioned hereinafter all refer to the screen Display-Frametime.

Advantages of PerfDog tools: The statistics of FPS and Frametime performed by PerfDog are the real refresh FPS and frametime of screen Display new image. Thus, you can directly use Frametime to judge whether Jank occurs during testing.

Part II: FPS

    Definition of FPS: Frames per Second (average number of screen refreshes in one second).

    Average FPS: FPS referred conventionally, average number of screen refresh in one second.

    Instantaneous FPS: Real-time FPS calculated from single frame time – FrameTime, the real-time FPS converted from each image refresh time.

    The image rendering flowchart is as follows. FrameTime of each frame.

    Statistical FPS and FrameTime by PerfDog are as follows:

    Whether FPS is related to Jank? We further learn it from two scenarios of Android and iOS.

    iOS terminal

        Apple WDDC18

        1.     FramePacing

        For example, two game screens as follows. The view on the left runs at 60FPS, but can actually only reach 40FPS. The view on the right can run at 30FPS continuously and stably.

        The left in the picture above has a higher FPS, but shows a Jank phenomenon on the contrary. It is just caused by Micro Stuttering. The FrameTime in the left picture is as follows:

        It can be seen from FrameTime that one FrameTim>=117ms occurs in a higher FPS=40 on the left and the theoretical average FrameTime = 25ms. Therefore, in spite of an FPS as high as 40, the non-uniform rendering still makes a feeling of Jank very much.

        The FrameTime that runs at low FPS of 30fps on the right is as follows:

        It can be seen from the FrameTime above that the FPS = 30, the average FrameTime is 33ms and the image is very smooth.

        Summary: high FPS may not produce smooth image.

    Android terminal

        Test system version: Android4.4

        ①     Fluency mechanism – Project Butter (Jank)

        Google Jank calculation idea: in consideration of visual inertia and using the hardware vsync interval, no new image refresh for one vsync continuously is regarded as one Jank. That is to say, if the screen does not refresh at the next vsync time point, it will be regarded as one Jank.

Part III: Fluency

    The relation between fluency and Jank can be represented roughly using the following flowchart:

    Fluency influences Jank. It can be simply understood from two aspects as visual inertia and movie frame:

    1. Visual inertia

        It is the visually expected FPS. The user will consciously feel that the next frame should also be refreshed at current FPS. For example, if 60FPS is always used, the user will consciously feel the next frame will be also at 60FPS. If the refresh is always at 25FPS, the user will consciously feel the next frame uses 25FPS too. However, if the refresh FPS jumps suddenly from 60 to 25, it will disturb the user’s visual inertia. Then, the Jank feeling experienced by the user will occur.

    2. Movie frame

        Movie FPS (18–24) is usually 24. Movie FrameTime per frame: 1000ms/24≈41.67ms. The movie FPS is a critical point. If lower than this FPS, human eyes can substantially feel the screen discontinuity, i.e. a Jank.

Part IV: PerfDog-Jank

    PerfDog Jank calculation idea: in consideration of visual inertia and assuming that the average FrameTime of the previous three frames is referred as the vsync interval, no new image refresh for two vsync continuously will be regarded as one potential Jank. That is to say, if the FrameTime of the next frame is longer than twice the average FrameTime of previous three frames, it will be regarded as one potential Jank. Meanwhile, if the FrameTime of a single frame is longer than twice the movie FrameTime 1000ms/24*2 (because human eyes can only distinguish a screen discontinuity when the FPS is lower than 24), it will be regarded as a real Jank. Also, if the FrameTime of a single frame is longer than three times of the movie FrameTime, it will be regarded as a Big Jank.

    Notes: why it is two vsync? A GPU usually has three buffers. The current frame has occupied one buffer, so there are two remaining buffers. Human eyes can usually tolerate a latency of two frames.

    Why it is FrameTime of two movie frames? If FPS is lower than 24, human eyes can feel the screen discontinuity. Thus, a movie usually uses 24FPS. That is, the movie FrameTime is 1000ms/24≈41.67ms, the FrameTime of two movie frames is 41.67ms*2 and the FrameTime of three movie frames is 41.67ms*3.

    PerfDog Jank calculation method:

        If two conditions are both met, it will be regarded as one Jank.

        1. Display FrameTime > twice of average FrameTime of previous three frames.

        2. Display FrameTime > FrameTime of two movie frames (1000ms/24*2≈83.33ms).

        If two conditions are both met, it will be regarded as a Big Jank.

        1. Display FrameTime > twice of average FrameTime of previous three frames.

        2. Display FrameTime >FrameTime of three movie frames (1000ms/24*3≈125ms).

    Google-Jank as shown in figure:

    PerfDog-Jank as shown in figure:

    Really test FrameTime to calculate Jank as show in below figure:

    PerfDog-Jank as shown in figure:

    Remark: it is low FPS and not Jank. If FPS is only single-digit level, then what Jank will occur?

    Currently, the mainstream game Janks in comparison are as follows:

    PerfDog-web platform

Part V: PerfDog-Stutter

    PerfDog Stutter Definition: the proportion of Jank time during testing. That is, Stutter = Jank time / total time

    PerfDog Stutter calculation idea: based on PerfDog Jank, one Jank will have a Jank time. Multiple Janks may occur during testing, i.e. there are multiple Jank times. Total test duration is Time.

        Stutter = ∑Jank time / Time

    Notes: Jank denotes Jank times and Stutter denotes Jank rate. The tendency of Jank and Stutter are consistent but not completely linear because the severity of Jank is different at different times. Also, it demonstrates that the Stutter will be certainly 0 if no Jank occurs.

    Currently, the mainstream games are compared as follows:

Part VI: Effects

    After learning about the close relation between FrameTime, FPS, fluency, Jank and Stutter from the previous description, we can naturally draw our own conclusion: whether the APPs and games need to be concerned about Jank and Stutter? The answer is yes and it should be focused on! The reasons are as follows:

        Concerning games

        The game fluency has the greatest influence on the user experience. Thus, FPS, Jank and Stutter need to be focused on.

        Remark: for example, the predicted Jank in the game such as new UI pop-up leading to Jank can be regarded as interference, needs to be culled and should not be calculated as Jank. The interference data can be selected by checking the checkbox and right-clicking to delete them through the web cloud.

        Concerning APP

            APP also needs to focus on FPS, Jank and Stutter. But it needs to differentiate the usage scenario. For example:

            1)     Static page window

                It needs only to focus on FPS and the theoretical FPS should be 0. Otherwise, it indicates there is redundant refreshes which tends to cause mobile phones to get hot and consume power.

            2)     Window containing scrolling animation page

                It needs only to focus on FPS which only needs to be at an appropriate value and refresh at high frequency is unnecessary.

            3)     Quickly sliding page window

                It needs to focus on FPS, Jank and Stutter. The mobile interaction sensitivity comes from them. So, Project Butter Jank is launched for Android system. In the general sliding state, FPS is better to be higher and Jank is better to be smaller.

            4)     Play video page window

                It needs to focus on FPS, Jank and Stutter. The video Jank has a direct influence on the users. The video usually has an FPS of 18–24 and Jank is 0. For example, the play video on WeChat and video player, etc.

    After the explanation of these six parts, I believe all of you can clearly understand the importance of Jank and Stutter on APPs & games as well as the emphasis to focus on. The PerfDog is devoted to improving the quality of games and APPs.