Harddave's Theory on Framerate Perception

[Harddave]

We all like to argue the question, "How many framerates do we need to achieve in our video games to see the smoothest gameplay possible?" Well, here is my attempt to shed some light on this subject. I will not cite references, being that I will not be quoting or relying heavily on books or research material, but if you would like some scientific info as to how I got this info, let me know and I will gladly hook you up.

Most of us know that video framerates on our tv's hover right around 29 fps, and with computer monitors, 60+ frames (a refresh rate is how fast a monitor will redraw the image on the screen in Hz, or cycles per second) per second. But when we play video games, we fret over our framerates. How much is enough?

In my opinion, the only figure we should worry over is the minimum framerate. Obviously, 29 fps should be a minimum, as most will contest to the choppiness of anything lower than that. Thirty fps is tolerable for most gamers. Sixty is nice and smooth, but I think many of us became spoiled when the GeForce video cards came along as Quake II was on its way out. Many of us gamers sat up and took notice when we broke the 100 fps barrier in such a sweet game. We could play the game with silky smooth graphics, with seemingly effortless gameplay. However, the quest for the highest framerate was not satisfied.

It has been proven in many studies, that a human being can discern the differences of motion perception between a human and a computer generated figure when using point-light walkers as a model. Why is this?

A computer may be able to represent motion at say, 80 fps, but this isn't anywhere close to what a human being can percieve. Our visual part of our nervous system can synapse at a rate of 1,000 times per second before inhibitory responses "skip" incoming data. This infers that we can discern differences in motion up to (roughly) 1,000 times per second. Our system is biological, whereas a video games' system is not, it is mechanically reproduced and limited to that of what we make it, or how fast our computers can display it.

In summary, we could in theory, discern the differences of motion of up to 1,000 fps. This could be the reason why we see biological beings move in such a way as to believe they are alive, and not produced. So, this leads us to believe (at least me anyway) that trying to achieve maximum framerates is not nearly as important as trying to optimize a system to never fall below 30 or so fps.

Disclaimer: It's late, I'm tired, I prolly forgot some stuff, as well as intentionally left out a few things, so don't rag my arse over it plz. :) I just thought you might be interested. Suggestions and comments welcome.

[Vincent_HRD]

Cool stuff :)

I was talking about this, with my brother ManxMan.

It's really interesting. :)

[Sharkfood]

many of us became spoiled when the GeForce video cards came along as Quake II was on its way out. Many of us gamers sat up and took notice when we broke the 100 fps barrier in such a sweet game. We could play the game with silky smooth graphics, with seemingly effortless gameplay. However, the quest for the highest framerate was not satisfied.

An actual interesting side-story concernin Quake2 and 100+fps...

The big quest for higher fps really had little to do with game "smoothness" or the like, but alternatively a method of "cheating" or saturating servers for superhuman effects, which reads as an in-game advantage.

Quake2's network code was based on a 1:1 ration of framerate to server packet. This meant your framerate and packet size needed to be tuned to what your connection limit could handle. As cable modems and DSL were beginning to hit mainstream, every avid Q2 player wanted a maxfps of 100-120 (or higher) and adjusted their packet size accordingly to their bandwidth to maintain that level of updates.

As the Q2 ded/non-ded server had an upper limit of 90 refreshes per client connection per second, anything over this was truncated/lost. At about the 3.1x level, QuakeWorld "prediction" tweening was added and this allowed the server to compromise loss by tweening between the client's version of where/what the player did and what the server's idea was at a lower refresh rate.

What this meant to the player? It meant strafe jumps could get that extra long distance. It also meant railshot kills were given a higher priority on the client timescale timescale versus the real, in-game server timescale. Players with broadband + 120+ fps were in a totally different timescale- one of their own client prediction and the tweening made it easier to target, jump and kill. Having a minimum of 30%+ additional reaction time and aiming time was a big benefit for the more caffeinated gamer. :)

bwadmin and other server side "cheat" correctors helped curb this, but also dropped the games popularity down a tad. These enforced client side cvar settings so they would simply reset the clients fps/packet values regularly. You would still be amazed at how many thousands of players made macro scripts to repeatedly set maxfps 120 and lower packet size during games to keep that "edge" around these tools. :)

They fixed all this with Quake3 by breaking the correlation between fps and server updates- so the two are no longer synchronized and the server holds the key to all decisions (hence why you can run through a pickup without ever getting it, or be SURE you nailed that sucker right between the eyes with a railgun but they didnt take damage.. it was moments of lag and your predicted, local version of the game did it only).

Just an interesting aside.. hehe. I remember for a period of about a year out here (remember, all the big Quake tournaments were local here)- *every* avid, pro-Quake gamer MUST have 120+ fps and broadband, otherwise they were fish food. :)

[WedgieMan]

The most important aspects for smooth gameplay are constant framerates and no visible refreshes. I usually leave Q3A framerate capped at my refresh rate (which is uaully 75 or 85, high enough for me not to notice flicker) and have vsync on. My video card and CPU can handle this framerate most of the time when playing, so I hardly ever notice a jitter, unless I'm on the internet on a lagged server. If it isn't capped, I can reach 100+ fps in certain scenarios, but I notice jitters in my gameplay which is displeasing.

[BloodyL]

Indeed, smoothness also plays a part, imagine a game where the frames are all over the place, in one scene it 100+, in another it's 10, this breaks the perception of smoothness and so it's easier to see.

A little far fetched, but you get my idea. :)

[Emig5m]

I dunno about you guys, but when it comes to the old high framerate vs lower framerate in games debate, I don't just emphasize what the human eye can "see" as smooth but more important to me is I can literaly "feel" the difference in control input response. Some games are very playable for me at low framerate like the Sims that don't demand smooth precision control input on top of complex physics engines like you would have in a racing game with alot more complex physics engines like in Motocross Madness 2 for example. But mainly fast paced racing games and fast paced first person shooters is where I really need the higher framerates for a smoother and more consistent control response, especialy when there's alot of action going on the screen.

I first started noticing this back when 3D Virtua Fighter 2 came out on the console playing at a rock steady 60 FPS. I noticed that not only did it look smoother, but how overall the games control felt considerably smoother over the average 30 FPS game of the time. I noticed this even more when I became a PC gamer as I would upgrade my system, the control input/physics would get smoother/more consistent as framerate went up.

And a big LOL at what people would do to just to have an advantage in Quake II online. Sure glad I wasn't around during the era. ;) Personally, I'd rather play on equal grounds when playing online so it's actually skill vs skill. I actualy stopped playing online all together because I got tired of all the stupid cheating that you see in online games today. If you suck, you just suck! Just deal with it and don't cheat! :D

[Whiggles]

Frame rates as low as 24 fps (movie theater frame rate) should theoretical be WATCHABLE, but as Emig5m said, when it comes to actual interraction, players need much higher frame rates. Remember the old P/S2 mice that only ran at 40 Hz. Then a great program called PS2Rate surfaced, letting you increase this to as high as 200 Hz. The difference was immediately noticeable, and once you've used it you can't look back. To me, the "feel" is always going to be more important than the "look". For example, I can WATCH someone playing a game at about 40 fps no problems, but as soon as I start PLAYING, I notice that it's not quite as smooth as I would like.

[OMAC]

w00t!


can't wait to play DOOM6 on a Gee-Farce10 at 1000 FPS!




w00t!

[Sharkfood]

I don't just emphasize what the human eye can "see" as smooth but more important to me is I can literaly "feel" the difference in control input response.

That's the other BIG issue with game framerates... it has nothing to do with visual smoothness. I could record a Quake2 game in video form and someone watching a 60 fps and 120 fps version of the same video couldn't possibly see the difference...

The problem is with the way most games are written. They have one master loop which is like:

Iterate:
Draw scene and render frame
Update sound effects
Calculate AI
Read input/controls
Adjust all movable objects
Goto Iterate

So, since the control and object adjustment calculations are inside the main render loop, the faster an engine can iterate through this, the more accurately a player can control their character in the game.

It doesn't matter if the player can "see" the translation of their on-screen persona go from point A to point B, what is important is the latency it takes FOR their on-screen persona to go from point A to point B. At 50 fps, this has twice the latency for this distance compared to 100 fps given the above main control loop in a game... especially given fixed velocities in game objects.

Most console games, you write your control loop totally differently. You know up front that your target framerate is going to be 25-29 fps interlaced, so you normally write your control and object adjust loops in an outside thread, totally seperate from the rasterizer/sound thread. It's either this or artificially adjust velocity/acceleration of movable objects in a main control loop, but this winds up looking "choppy"- hence why so many direct PC ports to console are so hosed.

[Harddave]

Grrrr....I had this nice long reply typed out, and then I started winamp and that little upgrade box popped up and with the "snap to button" thing enabled, I accidentally hit it and brought up another window, or at least I thought so, and I closed it, and *POOF*...

Anyway, to make a long post short, wouldn't the "master loop" be machine dependent? Meaning that the resulting framerates are correlated to the speed at which this loop can be completed?

I could be wrong, but in my logic, they are. Also, the faster you can render motion sequences, the more detail you can achieve, providing that the motion sequences are dependent only upon the rendering device (i.e., the game code and how fast the machine can calculate it). I don't know if there's a name for this, but I'll dub it "dynamic animation," as opposed to "static animation," or what you would see when you watch a movie. The DVD player doesn't get faster...

An example would be the following:

50 fps: F S F S F S F S F
100 fps: F F F F F F F F F
F=frame S=skipped frame

As you can see, the machine doing 100 fps can render more frames of detail than the 50 fps machine, resulting in better visual quality and smoother gameplay.

Now, I wouldn't know how a game is coded to render the maximum number of image frames (detail of movement), but I would guess that there would be a point where the maximum number is reached, and excess framerates would make no difference. In a movie, if you were to speed it up, the movie only gets faster, as there is no information there to manipulate.

This is similar to how the bitrates in audio work. The more bits you have for the player to sample, the better or "smoother" the recording will sound (excluding materials and design). That's why DVD music is better than CD music (48k & 44.1k, respectively). However, there are many that would argue there is no difference. This is what I was trying to get at earlier when I said that "there would be a point where the maximum number (of fps) is reached."

The only difference between the two is that our hearing is the limiting factor, whereas in video, our own visual system far exceeds that of what computers can produce as far as detail is concerned. Hence the reference to and the term "biological."

Oh, well, I guess that's enough speculation for now. I'll try to follow up.

[Sharkfood]

Anyway, to make a long post short, wouldn't the "master loop" be machine dependent? Meaning that the resulting framerates are correlated to the speed at which this loop can be completed?

Yes, you got it. :)

The frame skipped/frame thing isn't quite right though, unless you are describing a system that has the input loop outside the rasterizer thread. In mundane process loops, the faster the PC, the higher the framerate, which means the faster the input and accuracy of in-game characters...

In the case of a system that can't scale to the ingame action, the result is a change in the speed of the game (i.e. lag or slowdown). Games aren't like movies where their run-time is fixed to a timescale, such as that a sequence of 1200 frames may take 10 seconds or 100 seconds, doesn't matter. Fixed timescale like a DVD though- would be 60 minutes regardless of the speed of the decoding.

This is still not a golden rule either. A coder has the flexibility to also add "buffering" to movements, which basically allow large input sets to queue up until the rasterizer portion can catch up. This balances out machine differences and is what keeps games playable on future hardware- rather than unplayably fast.

There are hundreds of different ways to implement the main logic loops of a game, but most designers pick a method that is simple and easy to maintain. Tracing bugs in a dozen different control threads, and handling timing/timescale, race conditions and synchronization is obviously more complex, so simplified control/ai/audio/rasterize loops are usually preferred.

[msan_msw]

It has been proven in many studies, that a human being can discern the differences of motion perception between a human and a computer generated figure when using point-light walkers as a model. Why is this?

This whole thread is an interesting discussion that I've been interested in for awhile. My take on the question above is that the eye isn't independent rather only one part of an instance of environmental-->sensory input over time. We've observed and learned how the human body moves. We don't just pic it up for no reason either... we can see which movement means different things from what a hot lady walks like to that of a super fast athlete... and whatever else has 'learned meaning' in the world. There really isn't a prototypical "walk" but that's what 3d'ers are trying to create/mimic/synthetically develop and spread it over a wide variety of models. Most real life activities don't have just one speed, sound, motion, etc. That's why 3d models look emotionless/zombie'ish... There is a lack of distinguishable complexity that we see in our natural environment.

The eye is only one part of a whole system of perception. How many fps they "eye" can see is meaningless. It's how many fps the entire "system" can perceive to include the brain, eye, and learning/memory/experience. To go kinda weird for a second (and a interest of mine).... I can walk in my house and tell if the TV is on (and muted) or not without seeing it. I don't believe in ESP either. I can't see the TV light when I walk in.. I can't hear the sound... "smell" the smell an appliance makes when hot... feel vibration... whatever.. but I know it's on. I think there is just enough sensory stimulation of each sense that triggers in sum the fact the tv is on (i.e. through sensory input, learing/repitition/memory...experience... and then walking in to a situation). I think game designers have a long way to go to create that kind of situation.

I have way more questions than answers. Are we talking fps in the peripheral vision as well.. or only on the point of focus. How about at different light levels or with a variation of contrasting colors? How much other stimulation (activity) is co-occurring with the point of focus i.e. a bunch of sparking wires to shock you at left/right and a monster running right for you(?). "How many fps the eye can see..." is really asking a wide variety of sub-questions around a whole perceptual experience. Then again, we could do it scientifically (and has been done) with moving white blocks on a black screen and different refresh rates but that wouldn't necessarily translate over into other things.

I haven't found an article that describes how the human eye see fps or in relation to gaming that I feel comfortable referencing to... but if anyone else has one then please pass it on.

http://amo.net/NT/02-21-01FPS.html
http://www.accessexcellence.org/AE/AEC/CC/vision_background.html
http://www.acm.org/crossroads/xrds3-4/ellen.html
http://www.cs.brown.edu/research/graphics/research/animation/animation.html
http://www.llnl.gov/str/Lu.html
http://www.ic.ac.uk/templates/icr_text_3.asp?P=3367

[Despoiler]

I mean sure we theoretically can see upto 1000fps but how exactly is that useful? We wanted to know some absolutes or close to absolutes. Here is a good article that is more concrete than the one posted above.

http://www.penstarsys.com/editor/30v60/30v60p1.htm

[Emig5m]

Originally posted by msan_ewu
The eye is only one part of a whole system of perception. How many fps they "eye" can see is meaningless. It's how many fps the entire "system" can perceive to include the brain, eye, and learning/memory/experience. To go kinda weird for a second (and a interest of mine).... I can walk in my house and tell if the TV is on (and muted) or not without seeing it. I don't believe in ESP either. I can't see the TV light when I walk in.. I can't hear the sound... "smell" the smell an appliance makes when hot... feel vibration... whatever.. but I know it's on. I think there is just enough sensory stimulation of each sense that triggers in sum the fact the tv is on (i.e. through sensory input, learing/repitition/memory...experience... and then walking in to a situation). I think game designers have a long way to go to create that kind of situation.

I'm scared, hehe. :O

http://www.penstarsys.com/editor/30v60/30v60p1.htm
Yea, ive had that in my favorites list for a couple years now.

[Harddave]

Originally posted by msan_ewu
I have way more questions than answers. Are we talking fps in the peripheral vision as well.. or only on the point of focus. How about at different light levels or with a variation of contrasting colors? How much other stimulation (activity) is co-occurring with the point of focus i.e. a bunch of sparking wires to shock you at left/right and a monster running right for you(?). "How many fps the eye can see..." is really asking a wide variety of sub-questions around a whole perceptual experience. Then again, we could do it scientifically (and has been done) with moving white blocks on a black screen and different refresh rates but that wouldn't necessarily translate over into other things.
[/B]

Lol! Dang psychologists.... ;)

msan, you are taking this way out of the original context. I used the statement you quoted as an example and as a basis for my idea, no more. To answer all those questions would require years of research and experiments. There hasn't been near as much research on perception as compared to learning or behavior, so you will be wasting a lot of time looking for it. That's why in the subject I put "Harddave's THEORY..." (more like an idea)

Yes, you are correct in saying that the fps the eye can see is meaningless, as the eye doesn't see in frames. However, it can percieve them (what is displayed that is), and that's what I'm getting at. My idea was to offer and explanation to the ideal framerates gamers should try to achieve. That's why I kept it very, very simple. I could sit here and write a research paper on this subject alone and fill 30 pages easily, but I don't have the time or effort for it. Besides, much of it would still be mostly theory, and theory isn't sound research.

The article you guys suggested was pretty sound, and a good article to read to get your feet wet on the subject, but there is a lot more to perception than that. As the article suggested, there is an illusion happening ("motion blur" is one, like that article suggested) when you watch tv or a movie at a theater, that's why 24 or so fps is smooth. However, if you know what to look for, you can tell that the fps is a tad low (most apparent in a cheaper movie theater). Why in a movie theater? Because there isn't phosphorus in the screen like there is in a tv or monitor. Simply put, phosphorus is stimulated by electricity and when the charge is taken away, there is a time of decay, providing you with a nice after-image, or "motion blur." Remember watching "Fight Club," where Brad Pitt inserts frames of his own? You can easily tell what the images are while the movie is playing at full speed if you were paying attention. This is proof that when you take out the motion blur (phosphorus), you can percieve single frames at 24 fps. Don't forget that when you watch a movie, you are watching people, not computer generated polygons, and that helps a lot.

To sum it all up, yes there are many questions to be answered and many experiments to be performed. I'm just trying to provide a little more insight as to what fps would be ideal to play video games at (better than 24 fps, 30 is a good round number).

As far as what is the maximum possible framerate you need? Well, that is user dependent.

[msan_msw]

Heh, sorry, my little blurb "was in addition to" not "at you" Harddave :) I (probably much like you) have an interest in this since we have somewhat of the same background. Obviously I tend to take the psych/bio side although I'm far from even being a novice at this.

Perception peaked my interest when we had to develop 'optimal foraging' experiments. The Rhesus Macacs perception of the best way to get food using a game (w/ joystick and pacman) and the way they used motion and perception of the game was really interesting. I look forward to reading more of your ideas :)

Emig5m, I looked for an hour for that penstar article last night and couldn't remember who wrote it. Thanks for posting it. :)