What is FLOPS in field of deep learning?

ladofa picture ladofa · Oct 22, 2019 · Viewed 14.2k times · Source

What is FLOPS in field of deep learning? Why we don't use the term just FLO?

We use the term FLOPS to measure the number of operations of a frozen deep learning network.

Following Wikipedia, FLOPS = floating point operations per second. When we test computing units, we should consider of the time. But in case of measuring deep learning network, how can I understand this concept of time? Shouldn't we use the term just FLO(floating point operations)?

Why do people use the term FLOPS? If there is anything I don't know, what is it?

==== attachment ===

Frozen deep learning networks that I mentioned is just a kind of software. It's not about hardware. In the field of deep learning, people use the term FLOPS to measure how many operations are needed to run the network model. In this case, in my opinion, we should use the term FLO. I thought people confused about the term FLOPS and I want to know if others think the same or if I'm wrong.

Please look at these cases:

how to calculate a net's FLOPs in CNN

https://iq.opengenus.org/floating-point-operations-per-second-flops-of-machine-learning-models/

Answer

vladievlad picture vladievlad · May 26, 2020

Confusingly both FLOPs, floating point operations, and FLOPS, floating point operations per second, are used in reference to machine learning. FLOPs are often used to describe how many operations are required to run a single instance of a given model, like VGG19. This is the usage of FLOPs in both of the links you posted, though unfortunately the opengenus link incorrectly mistakenly uses 'Floating point operations per second' to refer to FLOPs.

You will see FLOPS used to describe the computing power of given hardware like GPUs which is useful when thinking about how powerful a given piece of hardware is, or conversely, how long it may take to train a model on that hardware.

Sometimes people write FLOPS when they mean FLOPs. It is usually clear from the context which one they mean.