To view all video thumbnails.

Click the playlist icon at the top left corner of the video window below