Where in a computer does the abstract (letters on the screen) meet the real (electrical current passing through circuits in memory and processors). How did that evolution happen?
I'm talking deeper than assembler here, I think. Maybe a level deeper than processor instructions? At some point commands are actually interpreted by hardware, but I don't understand where/how that happens.
I didn't go to university, so didn't take a CS course or anything. Like many, I am a hobbyist turned "pro". As a result, there are many pieces of the jigsaw I think I'm missing. I know a little C++, and understand the concept of pointers etc (though I've not used them in anger much), however I've only ever worked at a high level. I'm not sure knowing this kind've stuff will help me or not, but it's sure interesting.
A processor operates what is known as a fetch-decode-execute cycle. Machine code instructions are fairly low-level (i.e. they don't do all that much in a single instruction). For example, adding two numbers would have a sequence of instructions with semantics like:
Within the processor is a special set of fast memory known as a 'Register File', which contains the memory that the processor uses to store data that it is working on at the time. The register file has several registers, which are uniquely identified. Instructions typically work on registers, especially on RISC architectures; while this is not always the case it is a good enough abstraction for the moment.
Typically a processor has to load or store data into a register to do anything with it. Operations such as arithmetic work on registers, taking the operands from two registers and placing the result into a third (for the benefit of the peanut gallery, I have used a 6502 - lets not confuse the issue ;-). The processor has special instructions for loading or storing data from registers into the machine's main memory.
A processor has a special register called the 'program counter' that stores the address of the next operation to execute. Thus, the sequence for executing an instruction goes roughly like:
Executing the instruction will change the values in various registers. For example, a 'load' instruction will copy a value into a register. An arithmetic or logical (And, Or, Xor) will take two values and compute a third. A jump or branch instruction will change the address at the program counter so the processor starts to fetch instructions from a different location.
The processor can have special registers. An example of such is the program counter described above. Another typical one is a condition flags register. This will have several bits with special meanings. For example it may have a flag that is set if the result of the last arithmetic operation was zero. This is useful for conditional operations. You can compare two numbers. If they are equal, the 'zero' flag is set. The processor can have a conditional instruction that is only executed if this flag is set.
In this case, you could decrement a counter in a register and if it was zero, a condition flag is set. A conditional (branch on zero) can be used for a loop where you decrement a counter and exit the loop if the result of the decrement instruction is zero. On some processors (e.g. the ARM family) all instructions are conditional, with a special 'do always' condition for non-conditional instructions.
Some examples of typical processor instructions are:
This stackoverflow post has an example of a small snippet of compiled C code and the assembly language output from that snippet. It should give you an example of the sort of relationship between a high-level language and the machine code output that it compiles to.
The best way to learn this is to get an assembler and try it out. This used to be much easier on older, simpler computers like 8-bit micros of the 1980s. The closest thing to this type of architecture available these days are embedded systems. You can get a development board for an embedded processor like a Microchip PIC fairly cheaply. As this type of architecture has less baggage than a modern operating system there is less i-dotting and t-crossing to use system calls. This will make it easier to bootstrap an assembly language program on this type of architecture; the simpler architecture is also easier to understand.
Another option is to get an emulator such as SPIM. This will emulate a CPU and let you assemble and run programs on it. The advantage of such an emulator is that they will also have facilities for single stepping programs (much like a debugger) and showing the contents of the register file. This may be helpful in gaining insight as to what's actually going on.