Is MATLAB OOP slow or am I doing something wrong?
I'm experimenting with MATLAB OOP, as a start I mimicked my C++'s Logger classes and I'm putting all my string helper functions in a String class, thinking it would be great to be able to do things like a + b, a == b, a.find( b ) instead
of strcat( a b ), strcmp( a, b ), retrieve first element of strfind( a, b ), etc.
The problem: slowdown
I put the above things to use and immediately noticed a drastic slowdown. Am I doing it wrong (which is certainly possible as I have rather limited MATLAB experience), or does MATLAB's OOP just introduce a lot of overhead?
My test case
Here's the simple test I did for string, basically just appending a string and removing the appended part again:
Note: Don't actually write a String class like this in real code! Matlab has a native
stringarray type now, and you should use that instead.
classdef String < handle
....
properties
stringobj = '';
end
function o = plus( o, b )
o.stringobj = [ o.stringobj b ];
end
function n = Length( o )
n = length( o.stringobj );
end
function o = SetLength( o, n )
o.stringobj = o.stringobj( 1 : n );
end
end
function atest( a, b ) %plain functions
n = length( a );
a = [ a b ];
a = a( 1 : n );
function btest( a, b ) %OOP
n = a.Length();
a = a + b;
a.SetLength( n );
function RunProfilerLoop( nLoop, fun, varargin )
profile on;
for i = 1 : nLoop
fun( varargin{ : } );
end
profile off;
profile report;
a = 'test';
aString = String( 'test' );
RunProfilerLoop( 1000, @(x,y)atest(x,y), a, 'appendme' );
RunProfilerLoop( 1000, @(x,y)btest(x,y), aString, 'appendme' );
The results
Total time in seconds, for 1000 iterations:
btest 0.550 (with String.SetLength 0.138, String.plus 0.065, String.Length 0.057)
atest 0.015
Results for the logger system are likewise: 0.1 seconds for 1000 calls
to frpintf( 1, 'test\n' ), 7 (!) seconds for 1000 calls to my system when using the String class internally (OK, it has a lot more logic in it, but to compare with C++: the overhead of my system that uses std::string( "blah" ) and std::cout at the output side vs plain std::cout << "blah" is on the order of 1 millisecond.)
Is it just overhead when looking up class/package functions?
Since MATLAB is interpreted, it has to look up the definition of a function/object at run time. So I was wondering that maybe much more overhead is involved in looking up class or package function vs functions that are in the path. I tried to test this, and it just gets stranger. To rule out the influence of classes/objects, I compared calling a function in the path vs a function in a package:
function n = atest( x, y )
n = ctest( x, y ); % ctest is in matlab path
function n = btest( x, y )
n = util.ctest( x, y ); % ctest is in +util directory, parent directory is in path
Results, gathered same way as above:
atest 0.004 sec, 0.001 sec in ctest
btest 0.060 sec, 0.014 sec in util.ctest
So, is all this overhead just coming from MATLAB spending time looking up definitions for its OOP implementation, whereas this overhead is not there for functions that are directly in the path?
Answer
I've been working with OO MATLAB for a while, and ended up looking at similar performance issues.
The short answer is: yes, MATLAB's OOP is kind of slow. There is substantial method call overhead, higher than mainstream OO languages, and there's not much you can do about it. Part of the reason may be that idiomatic MATLAB uses "vectorized" code to reduce the number of method calls, and per-call overhead is not a high priority.
I benchmarked the performance by writing do-nothing "nop" functions as the various types of functions and methods. Here are some typical results.
>> call_nops Computer: PCWIN Release: 2009b Calling each function/method 100000 times nop() function: 0.02261 sec 0.23 usec per call nop1-5() functions: 0.02182 sec 0.22 usec per call nop() subfunction: 0.02244 sec 0.22 usec per call @()[] anonymous function: 0.08461 sec 0.85 usec per call nop(obj) method: 0.24664 sec 2.47 usec per call nop1-5(obj) methods: 0.23469 sec 2.35 usec per call nop() private function: 0.02197 sec 0.22 usec per call classdef nop(obj): 0.90547 sec 9.05 usec per call classdef obj.nop(): 1.75522 sec 17.55 usec per call classdef private_nop(obj): 0.84738 sec 8.47 usec per call classdef nop(obj) (m-file): 0.90560 sec 9.06 usec per call classdef class.staticnop(): 1.16361 sec 11.64 usec per call Java nop(): 2.43035 sec 24.30 usec per call Java static_nop(): 0.87682 sec 8.77 usec per call Java nop() from Java: 0.00014 sec 0.00 usec per call MEX mexnop(): 0.11409 sec 1.14 usec per call C nop(): 0.00001 sec 0.00 usec per call
Similar results on R2008a through R2009b. This is on Windows XP x64 running 32-bit MATLAB.
The "Java nop()" is a do-nothing Java method called from within an M-code loop, and includes the MATLAB-to-Java dispatch overhead with each call. "Java nop() from Java" is the same thing called in a Java for() loop and doesn't incur that boundary penalty. Take the Java and C timings with a grain of salt; a clever compiler could optimize the calls away completely.
The package scoping mechanism is new, introduced at about the same time as the classdef classes. Its behavior may be related.
A few tentative conclusions:
- Methods are slower than functions.
- New style (classdef) methods are slower than old style methods.
- The new
obj.nop()syntax is slower than thenop(obj)syntax, even for the same method on a classdef object. Same for Java objects (not shown). If you want to go fast, callnop(obj). - Method call overhead is higher (about 2x) in 64-bit MATLAB on Windows. (Not shown.)
- MATLAB method dispatch is slower than some other languages.
Saying why this is so would just be speculation on my part. The MATLAB engine's OO internals aren't public. It's not an interpreted vs compiled issue per se - MATLAB has a JIT - but MATLAB's looser typing and syntax may mean more work at run time. (E.g. you can't tell from syntax alone whether "f(x)" is a function call or an index into an array; it depends on the state of the workspace at run time.) It may be because MATLAB's class definitions are tied to filesystem state in a way that many other languages' are not.
So, what to do?
An idiomatic MATLAB approach to this is to "vectorize" your code by structuring your class definitions such that an object instance wraps an array; that is, each of its fields hold parallel arrays (called "planar" organization in the MATLAB documentation). Rather than having an array of objects, each with fields holding scalar values, define objects which are themselves arrays, and have the methods take arrays as inputs, and make vectorized calls on the fields and inputs. This reduces the number of method calls made, hopefully enough that the dispatch overhead is not a bottleneck.
Mimicking a C++ or Java class in MATLAB probably won't be optimal. Java/C++ classes are typically built such that objects are the smallest building blocks, as specific as you can (that is, lots of different classes), and you compose them in arrays, collection objects, etc, and iterate over them with loops. To make fast MATLAB classes, turn that approach inside out. Have larger classes whose fields are arrays, and call vectorized methods on those arrays.
The point is to arrange your code to play to the strengths of the language - array handling, vectorized math - and avoid the weak spots.
EDIT: Since the original post, R2010b and R2011a have come out. The overall picture is the same, with MCOS calls getting a bit faster, and Java and old-style method calls getting slower.
EDIT: I used to have some notes here on "path sensitivity" with an additional table of function call timings, where function times were affected by how the Matlab path was configured, but that appears to have been an aberration of my particular network setup at the time. The chart above reflects the times typical of the preponderance of my tests over time.
Update: R2011b
EDIT (2/13/2012): R2011b is out, and the performance picture has changed enough to update this.
Arch: PCWIN Release: 2011b
Machine: R2011b, Windows XP, 8x Core i7-2600 @ 3.40GHz, 3 GB RAM, NVIDIA NVS 300
Doing each operation 100000 times
style total µsec per call
nop() function: 0.01578 0.16
nop(), 10x loop unroll: 0.01477 0.15
nop(), 100x loop unroll: 0.01518 0.15
nop() subfunction: 0.01559 0.16
@()[] anonymous function: 0.06400 0.64
nop(obj) method: 0.28482 2.85
nop() private function: 0.01505 0.15
classdef nop(obj): 0.43323 4.33
classdef obj.nop(): 0.81087 8.11
classdef private_nop(obj): 0.32272 3.23
classdef class.staticnop(): 0.88959 8.90
classdef constant: 1.51890 15.19
classdef property: 0.12992 1.30
classdef property with getter: 1.39912 13.99
+pkg.nop() function: 0.87345 8.73
+pkg.nop() from inside +pkg: 0.80501 8.05
Java obj.nop(): 1.86378 18.64
Java nop(obj): 0.22645 2.26
Java feval('nop',obj): 0.52544 5.25
Java Klass.static_nop(): 0.35357 3.54
Java obj.nop() from Java: 0.00010 0.00
MEX mexnop(): 0.08709 0.87
C nop(): 0.00001 0.00
j() (builtin): 0.00251 0.03
I think the upshot of this is that:
- MCOS/classdef methods are faster. Cost is now about on par with old style classes, as long as you use the
foo(obj)syntax. So method speed is no longer a reason to stick with old style classes in most cases. (Kudos, MathWorks!) - Putting functions in namespaces makes them slow. (Not new in R2011b, just new in my test.)
Update: R2014a
I've reconstructed the benchmarking code and run it on R2014a.
Matlab R2014a on PCWIN64
Matlab 8.3.0.532 (R2014a) / Java 1.7.0_11 on PCWIN64 Windows 7 6.1 (eilonwy-win7)
Machine: Core i7-3615QM CPU @ 2.30GHz, 4 GB RAM (VMware Virtual Platform)
nIters = 100000
Operation Time (µsec)
nop() function: 0.14
nop() subfunction: 0.14
@()[] anonymous function: 0.69
nop(obj) method: 3.28
nop() private fcn on @class: 0.14
classdef nop(obj): 5.30
classdef obj.nop(): 10.78
classdef pivate_nop(obj): 4.88
classdef class.static_nop(): 11.81
classdef constant: 4.18
classdef property: 1.18
classdef property with getter: 19.26
+pkg.nop() function: 4.03
+pkg.nop() from inside +pkg: 4.16
feval('nop'): 2.31
feval(@nop): 0.22
eval('nop'): 59.46
Java obj.nop(): 26.07
Java nop(obj): 3.72
Java feval('nop',obj): 9.25
Java Klass.staticNop(): 10.54
Java obj.nop() from Java: 0.01
MEX mexnop(): 0.91
builtin j(): 0.02
struct s.foo field access: 0.14
isempty(persistent): 0.00
Update: R2015b: Objects got faster!
Here's R2015b results, kindly provided by @Shaked. This is a big change: OOP is significantly faster, and now the obj.method() syntax is as fast as method(obj), and much faster than legacy OOP objects.
Matlab R2015b on PCWIN64
Matlab 8.6.0.267246 (R2015b) / Java 1.7.0_60 on PCWIN64 Windows 8 6.2 (nanit-shaked)
Machine: Core i7-4720HQ CPU @ 2.60GHz, 16 GB RAM (20378)
nIters = 100000
Operation Time (µsec)
nop() function: 0.04
nop() subfunction: 0.08
@()[] anonymous function: 1.83
nop(obj) method: 3.15
nop() private fcn on @class: 0.04
classdef nop(obj): 0.28
classdef obj.nop(): 0.31
classdef pivate_nop(obj): 0.34
classdef class.static_nop(): 0.05
classdef constant: 0.25
classdef property: 0.25
classdef property with getter: 0.64
+pkg.nop() function: 0.04
+pkg.nop() from inside +pkg: 0.04
feval('nop'): 8.26
feval(@nop): 0.63
eval('nop'): 21.22
Java obj.nop(): 14.15
Java nop(obj): 2.50
Java feval('nop',obj): 10.30
Java Klass.staticNop(): 24.48
Java obj.nop() from Java: 0.01
MEX mexnop(): 0.33
builtin j(): 0.15
struct s.foo field access: 0.25
isempty(persistent): 0.13
Update: R2018a
Here's R2018a results. It's not the huge jump that we saw when the new execution engine was introduced in R2015b, but it's still an appreciable year over year improvement. Notably, anonymous function handles got way faster.
Matlab R2018a on MACI64
Matlab 9.4.0.813654 (R2018a) / Java 1.8.0_144 on MACI64 Mac OS X 10.13.5 (eilonwy)
Machine: Core i7-3615QM CPU @ 2.30GHz, 16 GB RAM
nIters = 100000
Operation Time (µsec)
nop() function: 0.03
nop() subfunction: 0.04
@()[] anonymous function: 0.16
classdef nop(obj): 0.16
classdef obj.nop(): 0.17
classdef pivate_nop(obj): 0.16
classdef class.static_nop(): 0.03
classdef constant: 0.16
classdef property: 0.13
classdef property with getter: 0.39
+pkg.nop() function: 0.02
+pkg.nop() from inside +pkg: 0.02
feval('nop'): 15.62
feval(@nop): 0.43
eval('nop'): 32.08
Java obj.nop(): 28.77
Java nop(obj): 8.02
Java feval('nop',obj): 21.85
Java Klass.staticNop(): 45.49
Java obj.nop() from Java: 0.03
MEX mexnop(): 3.54
builtin j(): 0.10
struct s.foo field access: 0.16
isempty(persistent): 0.07
Update: R2018b and R2019a: No change
No significant changes. I'm not bothering to include the test results.
Source Code for Benchmarks
I've put the source code for these benchmarks up on GitHub, released under the MIT License. https://github.com/apjanke/matlab-bench