What happens when an ISR is running and another interrupt happens?

Question

What happens if an ISR is running, and another interrupt occurs? Does the first interrupt get interrupted? Will the second interrupt get ignored? Or will it fire when the first ISR is done?

EDIT I forgot to include it in the question (but I included it in the tags) that I meant to ask how this worked on Atmel AVR's.

Answer 1

Normally, an interrupt service routine proceeds until it is complete without being interrupted itself in most of the systems. However, If we have a larger system, where several devices may interrupt the microprocessor, a priority problem may arise.

If you set the interrupt enable flag within the current interrupt as well, then you can allow further interrupts that are higher priority than the one being executed. This "interrupt of an interrupt" is called a nested interrupt. It is handled by stopping execution of the original service routine and storing another sequence of registers on the stack. This is similar to nested subroutines. Because of the automatic decrementing of the stack pointer by each interrupt and subsequent incrementing by the RETURN instruction, the first interrupt service routine is resumed after the second interrupt is completed, and the interrupts are serviced in the proper order. Interrupts can be nested to any depth, limited only by the amount of memory available for the stack.

For example, In the following diagram, Thread A is running. Interrupt IRQx causes interrupt handler Intx to run, which is preempted by IRQy and its handler Inty. Inty returns an event causing Thread B to run; Intx returns an event causing Thread C to run.

Image Ref

For hardware interrupts, Priority Interrupt Controller Chips (PIC's) are hardware chips designed to make the task of a device presenting its own address to the CPU simple. The PIC also assesses the priority of the devices connected to it. Modern PIC's can also be programmed to prevent the generation of interrupts which are lower than a desired level.

UPDATE: How Nested Interrupt Works on Atmel AVRs

The AVR hardware clears the global interrupt flag in SREG before entering an interrupt vector. Therefore, normally interrupts remain disabled inside the handler until the handler exits, where the RETI instruction (that is emitted by the compiler as part of the normal function epilogue for an interrupt handler) will eventually re-enable further interrupts. For that reason, interrupt handlers normally do not nest. For most interrupt handlers, this is the desired behaviour, for some it is even required in order to prevent infinitely recursive interrupts (like UART interrupts, or level-triggered external interrupts).

In rare circumstances though nested interrupts might be desired by re-enabling the global interrupt flag as early as possible in the interrupt handler, in order to not defer any other interrupt more than absolutely needed. This could be done using an sei() instruction right at the beginning of the interrupt handler, but this still leaves few instructions inside the compiler-generated function prologue to run with global interrupts disabled. The compiler can be instructed to insert an SEI instruction right at the beginning of an interrupt handler by declaring the handler the following way:

ISR(XXX_vect, ISR_NOBLOCK)
{
  ...
}

where XXX_vect is the name of a valid interrupt vector for the MCU type.

Also, have a look at this Application Note for more info on interrupts on Atmel AVRs.