I am trying to understand assembly to be able to solve a puzzle. However I encountered the following instructions:
0x0000000000401136 <+44>: cmpl $0x7,0x14(%rsp)
0x000000000040113b <+49>: ja 0x401230 <phase_3+294>
What I think it's doing is:
The value of 0x14(%rsp)
is -7380. According to my understanding cmpl
compares unsigned. Also the jump is performed.
So can it be that (unsigned)-7380 > 7
--> jump
I actually don't want it to jump. But is this the correct explanation or not? Am I flipping arguments?
Also if you have any advice about how to manipulate this jump!
According to my understanding
cmpl
compares unsigned.
It does both, in a way.
The difference in signed vs. unsigned is here the usage of the jump instructions.
For >
, there is ja
for unsigned and jg
for signed (jump if above and jump if greater).
For <
, there is jb
for unsigned and jl
for signed (jump if below and jump if less).
To be exact, here is the meaning of several jump commands:
For unsigned comparisons:
JB/JNAE (CF = 1) : Jump if below/not above or equal JAE/JNB (CF = 0) : Jump if above or equal/not below JBE/JNA (CF = 1 or ZF = 1) : Jump if below or equal/not above JA/JNBE (CF = 0 and ZF = 0): Jump if above/not below or equal
For signed comparisons:
JL/JNGE (SF <> OF) : Jump if less/not greater or equal JGE/JNL (SF = OF) : Jump if greater or equal/not less JLE/JNG (ZF = 1 or SF <> OF): Jump if less or equal/not greater JG/JNLE (ZF = 0 and SF = OF): Jump if greater/not less or equal