Is an application server something like an additional layer of application servicing above a web server?
Does an application server always have a web server as its core?
What is the difference?
No,Application server does not contain web server...
Read following articles...
http://www.answers.com/topic/application-server
http://download.oracle.com/docs/cd/E19159-01/819-3671/ablat/index.html
Basically:
After the Web exploded in the mid-1990s, application servers became Web based.
Also following shows the difference between web server, web container and application server...
Difference between a Web Server, Web Container, and an Application Server
A Web Server is a server capable of receiving HTTP requests, interpreting them, processing the corresponding HTTP Responses and sending them to the appropriate clients (Web Browsers). Example: Apache Web Server. Read more about Web Servers and their working>>
A Web Container is a J2EE compliant implementation which provides an environment for the Servlets and JSPs to run. Putting it differently we can say that a Web Container is combination of a Servlet Engine and a JSP Engine. If an HTTP Request refers to a Web Component (typically a Servlet or a JSP) then the request is forwarded to the Web Container and the result of the request is sent back to Web Server, which uses that result to prepare the HTTP Response for the particular HTTP Request. Example: Tomcat is a typical Web Container. A typical setup would be to have Apache HTTP Server as the Web Server and Tomcat as the Web Container.
An Application Server is a complete server which provides an environment for running the business components (EJBs, ADF BCs, etc.) in addition to providing the capabilities of a Web Container as well as of a Web Server. Example: Bea WebLogic, IBM WebSphere, Oracle Application Server, etc.