Besides what Tom said, there are apparently two distinct software code paths for the skimmer in the event browser vs skimmer within the timeline or playback via playhead. On some material and certain machines there can be a difference in viewer update rate between event browser vs timeline. This difference may be even greater if effects are on timeline clips. On my iMac Pro I can definitely see a difference on certain types of H264 material, even with no Fx applied to the timeline clips.
That is for un-rendered material. Render files (if present) are apparently not used for skimmer playback in the event browser. If the timeline clip is rendered, timeline playback is via cached ProRes 422 render files, so in that case it may be much smoother than skimming the same clip in the event browser.
Also, decode performance for H264 and HEVC on certain machines (esp. the iMac Pro) is apparently limited by the first-generation T2 chip, whereas on newer x86 machines Intel's Quick Sync is faster, and on Apple Silicon the hardware decode acceleration is really fast. Those may be fast enough to hide any differences in skimmer vs timeline performance for compressed codecs.
An obvious question is for a given machine, why not make timeline playback (or timeline skimming) as responsive as skimming in the event browser? There must be additional software code executing for the timeline which becomes apparent on those machines. The timeline playback code path may be handling (or prepared to handle) certain items which the event browser is not. E.g, scrolling the timeline left/right, or timeline index, etc.
However on my iMac Pro this difference goes away for playback of ProRes 422 material. That could imply an H264 decode optimization in the browser which is not possible or not implemented in the timeline.
Try making a snapshot duplicate of your timeline, select all clips with CMD+A, then remove all effects with Edit>Remove Effects. Then compare the playback performance of that vs the event browser.
Then export a test clip as ProRes 422, import that and compare the skimmer performance in the event browser vs timeline. That should eliminate any differences in H264/HEVC decode performance. If that makes the timeline playback fast, it implies a fully-rendered timeline would also be fast. However there are certain cases where rendered cache files may not be used, typically involving certain combinations of Fx.