Webpack 2 - Code splitting top-level dependencies

Question

Final Edit

The tl;dr resolution of this is that it's impossible. Though the top answer below does have some good information.

---

Consider the code below, from contacts.js. This is a dynamically loaded module, loaded on demand with System.import elsewhere in the code.

If SharedUtil1 is also used in other modules which are also dynamically loaded with System.import, how would I go about having SharedUtility1 excluded from all of these modules, and only loaded on demand the first time it's needed?

A top-level System.import of SharedUtil1 won't work, since my export depends on it: exports can only be placed in the top level of a module's code, not in any sort of callback.

Is this possible with Webpack? I'm on version 2.0.7 beta.

import SharedUtil1 from '../../SharedUtilities/SharedUtility1';

class Contacts{
    constructor(data){
        this.data = data;
        this.sharedUtil1 = new SharedUtil1();
    }
}

export default Contacts;

---

UPDATE 1

I thought the bundle loader was what I wanted, but no, that turns your imported module into a different function that you call with a callback to get to the actual module, once it's done loading asynchronously. This means you can't transparently make module X load asynchronously without making breaking changes to your code, to say nothing of the fact that you're back to the problem originally described, that if your top-level module depends on the now-asynchronously loaded dependency, there's no way to export it, since exports must be at the top level.

Is there no way in Webpack to denote that dependency X is to be loaded on-demand, if needed, and have any imported modules which import it to transparently wait out the importation process? I would think this use case would be a sine qua non for any remotely large application, so I have to think I'm just missing something.

UPDATE 2

Per Peter's answer, I attempted to get deduplication working, since the commonChunk plugin relates to sharing code between end points, as he mentioned, and since require.ensure places the loaded code into a callback, thereby preventing you from ES6 exporting any code that depends on it.

As far as deduplication, contacts.js and tasks.js both load the same sharedUtil like so

import SharedUtil1 from '../../sharedUtilities/sharedUtility1';

I tried running webpack as

webpack --optimize-dedupe

and also by adding

plugins: [
    new webpack.optimize.DedupePlugin()
]

to webpack.config. In both cases though the sharedUtil code is still placed in both the contacts and tasks bundles.

Answer 1

After reading your blog post I finally understand what you intended. I got a bit confused by the word "Top-level dependencies".

You have two modules (async-a and async-b) which are loaded on-demand from anywhere (here a module main) and both have a reference on a shared module (shared).

- - -> on-demand-loading (i. e. System.import)
---> sync loading (i. e. import)

main - - -> async-a ---> shared
main - - -> async-b ---> shared

By default webpack creates a chunk tree like this:

---> chunk uses other chunk (child-parent-relationship)

entry chunk [main] ---> on-demand chunk 1 [async-a, shared]
entry chunk [main] ---> on-demand chunk 2 [async-b, shared]

This is fine when shared < async-a/b or the probability that async-a and async-b are used both by the same user is low. It's the default because it's the simplest behaviors and probably what you would expect: one System.import => one chunk. In my opinion it's also the most common case.

But if shared >= async-a/b and the probability that async-a and async-b is loaded by the user is high, there is a more efficient chunking option: (a bit difficult to visualize):

entry chunk [main] ---> on-demand chunk 1 [async-a]
entry chunk [main] ---> on-demand chunk 2 [async-b]
entry chunk [main] ---> on-demand chunk 3 [shared]

When main requests async-a: chunk 1 and 3 is loaded in parallel
When main requests async-b: chunk 2 and 3 is loaded in parallel
(chunks are only loaded if not already loaded)

This is not the default behavior, but there is a plugin to archive it: The CommonChunkPlugin in async mode. It find the common/shared modules in a bunch of chunks and creates a new chunks which includes the shared modules. In async mode it does load the new chunk in parallel to the original (but now smaller) chunks.

new CommonsChunkPlugin({
    async: true
})

// This does: (pseudo code)
foreach chunk in application.chunks
  var shared = getSharedModules(chunks: chunk.children, options)
  if shared.length > 0
    var commonsChunk = new Chunk(modules: shared, parent: chunk)
    foreach child in chunk.children where child.containsAny(shared)
      child.removeAll(shared)
      foreach dependency in chunk.getAsyncDepenendenciesTo(child)
        dependeny.addChunk(commonsChunk)

Keep in mind that the CommonsChunkPlugin has a minChunks option to define when a module is threaded as shared (feel free to provide a custom function to select the modules).

Here is an example which explains the setup and output in detail: https://github.com/webpack/webpack/tree/master/examples/extra-async-chunk

And another one with more configuration: https://github.com/webpack/webpack/tree/master/examples/extra-async-chunk-advanced