Evalvid

Evalvid overview

Contents • • • •

Introduction Framework and Design Functionalities Tools

Introduction • Publicly available tools for video quality evaluation often assume synchronized frames at the sender and the receiver side, which means they can’t calculate the video quality in the case of frame drops or frame decoding errors • Evalvid: evaluation of transmission-distorted video • Trace files

Framework and Design

Functionalities • Determination of Packet and Frame Loss • Determination of Delay and Jitter • Video Quality Evaluation

Packet loss • unique packet id -> cancel the effect of reordering. • which kind of data is in the packets

Frame loss • 1 frame can be paketized in many packets -> frame loss

Determination of Delay and Jitter • play-out buffer – This would eliminate any possible jitter at the cost of a additional delay of the entire transmission time. The other extreme would be a buffer capable of holding exactly one frame. In this case no jitter at all can be eliminated but no additional delay is introduced – Delay the Entire video -> big size possible

• The formal definition of jitter • variance of the inter-packet or inter-frame time. • The “frame time” : the time at which the last segment of a segmented frame is received.

Video Quality Evaluation • two approaches to measure digital video quality • subjective quality measures – impression of the user

• Objective quality measures – metric

PSNR: closed to subjective evaluation

PSNR -> MOS

Tools

Files and Data Structures • Raw (uncoded) video files • trace files

VS - Video Sender • generate a trace file from the encoded video file • two trace files containing information about every frame in the video file and every packet generated for transmission

ET - Evaluate Traces • calculation of packet and frame losses and delay/jitter takes place • three trace files

ET (cont) • ET can also take into account the possibility of the existence of certain time bounds. – Playout buffer

• generation of a corrupted (due of losses) video file

FV – Fix Video • Digital video quality assessment is performed frame by frame. Therefore, the total number of video frames at the receiver side, including the erroneous frames, must be the same as that of the original video at the sender side. If the codec cannot handle missing frames, the FV component is used to tackle this problem by inserting the last successfully decoded frame in the place of each lost frame as an error concealment technique

PSNR

MOS

3rd party tools • TCP dump: phan tich luu luong