Update node-main to work in (webpacked) browser as well as node, and rename node-main to main

[dpvc]

Currently, importing or requiring @mathjax or @mathjax/src gets you an init() command that can be used to configuration MathJax. With the changes from feature/component-types, the init() function now can return a correctly typed MathJax variable with the configuration being type checked as well. The init() function, however, is only for node applications, and getting proper type checking for web applications is a bit harder.

This PR makes the init() function usable in both node a web applications, making it easier to manage typing for the MathJax object in both settings. In fact, now the exact same file can be used in both settings (when packed for the web).

The original init() implementation always used the startup component, so one had to load all the needed components explicitly in the loader.load array. That's OK for node, but inefficient for web usage, where it is desirable to use a combined component like tex-chtml.js in order to reduce the number of file transfers that must be performed. This PR extends init() to allow a second (optional) argument that gives the component to load, with it defaulting to startup so that it is backward compatible with current usage. This means that node applications now can also load combined components through init().

Example usage is

import { init } from '@mathjax/src';

const MathJax = await init<'tex-chtml', '[tex]/action'>({
  loader: {
    load: ['[tex]/action']
  },
  tex: {
    packages: {'[+]': ['action']},
    tagSide: 'left';
  }
}, 'tex-chtml');

console.log(MathJax.startup.adaptor.outerHTML(await MathJax.tex2chtmlPromise('x+y')));

---

The code here replaces the old node-main component, which was node-only, with a new main component that can be used in either setting, and that will detect which setting to use automatically, adjusting the configuration to use the proper adaptor and asynchronous file loading methods. There are also new node and browser components that can be used to specify which version you want (without the checking code). These each come in two forms, one that used the bundled files, and one that loads the needed components directly from the source. As in the past, @mathjax/src/source will load the source version, while @mathjax/src will load the bundled one. Similarly, @mathjax/src/node and @mathjax/src/browser load the bundled versions, while @mathjax/src/node/source and @mathjax/src/browser/source load directly from the source files.

There are also a number of improvements to the typing, including checking the loader.load array and the tex.packages arrays for consistency with the packages that have been specified for the init() function.

Because the usual usage for init() is to do

const MathJax = await init<...>({
  ... // configuration
});

as illustrated above, the MathJax variable is not available within the configuration itself. That has implications for the startup.ready() function, which must call MathJax.startup.defaultReady(). In order to handle this, the various functions (startup.ready(), startup.pageReady(), loader.ready(), loader.failed(), etc.) have been modified in two ways: first, this is set to the calling object (startup or loader), so that you can use this.defaultReady(), which is nicer anyway; and second, they are passed the MathJax object, so that you can use MathJax._ or any of the other MathJax properties within the ready() function. That is, you could do

import { init } from '@mathjax/src';
const MathJax = await init<...>({
  startup: {
    ready(MathJax) {
      MathJax.startup.defaultReady();
    }
  }
});

without problems. The MathJax variable is properly typed for the ready() and pageReady() functions, based on the components being loaded.

Finally, I discovered a timing issue with the loading of combined components while working on this, and there are changes to the configurations for the combined files like tex-chtml.js to resolve that. The issue was that those files include not just loading their code, but also running the startup sequence (loading the loader.load components, creating the startup document and input/output jax, calling the startup ready function, and so on). That sequence is promise based, but the loader did not have access to those promises, and was indicating that the file was loaded and complete before that sequence was finished. That means that when the loader.load files are loaded, they could be interleaved with the startup actions, which is not good.

So this PR includes changes to the combined configurations to properly synchronize the loader with the startup sequence. That is, if you Loader.load() a combined component, its promise will not resolve until the startup sequence completes. This gets the nesting of loads to be correct.

---

##Details

The change in components/core/core.js is to make the changes to mathjax.asyncLoad and mathjax.json conditional on them not already being specified. This allows you to import ts/util/asyncLoad/esm.js, for example, and not have it overwritten when the specified component (e.g., tex-chtml) is loaded.

The change to components/core/locale.js is to set the directory where the message for the Locale class are taken. This is because they are registered when the Locale object is imported, which is before when Locale.isComponent is set, and the directory to be used is determined by that setting when it is registered. All other locales will use the component paths (e.g., [mathjax]/input/tex/__locales__), but the messages in Locale itself would try to use the directory names themselves. That would be OK in node, but not in the browser, which doesn't have access to the file system directly. This patches that one directory that is in place before the setting has been made.

The components/mjs/main directory holds the replacement code for the older node-main component. This consists of three components: one in the main directory itself, and two more in main/browser and main/node. This is so they each can have their own config.json files in order to make the separate components. They each also have a .d.ts file that is similar to the one that was added to node-main in the recent feature/component-types branch. These .d.ts files are used to give the type checking for the init() function's parameters and its return value, and are the key pieces for using this with Typescript. They are almost identical across the three components, but the browser version uses the browser adaptor as its default adaptor, while the others use the liteDOM adaptor.

The init() function takes a generic parameter, T, that gives the components that will be loaded. This can be an intersection of the names, as in 'tex-chtml' | '[tex]/action', or an array of such names, such as ['tex-chtml' | '[tex]/action'], or a COMPONENT_DEF object (so that third-party extensions can be handled), or an intersection of any mixture of these. The component names are type checked against the ones available in the ts/types/Components.ts file, so spelling errors will be identified at compile time. Note that you do need to include in this list the combined component that you will be loading. I could not find a way to have that done automatically based on the second parameter to init(), unfortunately. The default value for T is startup.

There is also a second generic, A that allows you to specify the adaptor that you will be using, which has a default value of browser for the browser component, and liteDOM for the others. But you can specify jsdom or linkedom (or liteDOM or browser) if you are loading a different adaptor. The node component (and the main one when used in node) will use include the liteDOMadaptor automatically if another adaptor hasn't already been included in theloader.loadarray. TheAvalue will also add theadaptor/...component to theT` list automatically, so the adaptor doesn't have to be added by hand.

The init() function takes two arguments: the configuration (type-checked against the configuration options available in the components specified by T and A), and an optional component to load (that must be one of the ones you have specified in T, guaranteeing that the configuration is correct for the component you are loading). The default component is the startup component. This means that using just init({...}) can be used, as in the past, but because it is now type-checked, only the loader and startup options will be allowed. And since the loader.load array it checked against the components in the T generic, you would not be able to specify anything to load without adding the needed components to the T first. This is potentially a breaking change. The solution, of course, is to add the needed T generic. Alternatively, one could use init({...} as any) as a work-around.

Finally, the return value for init() is the MathJax object with the types available for the components specified in T.

The browser/browser.js file contains the main code that is needed to replace node-main; this is the makeInit() function, which is used to create the init() function that can be used either in the browser or in node. It accepts two parameters, one to do additional setup (needed for node) and one to specify the adaptor to use. These are both empty for the browser, since all the components are set up for browser use already, and it is node usage that needs to replace the adaptor and specify how files should be loaded, etc.

The browser.js file loads the Loader object so that it can load the specified component, and gets the global MathJax object from global.js, which will have moved any original value into the MathJax.config property. It sets the loader.failed function to throw the error (rather than just report it), so that it can be traded by init().catch(), though this can be overridden in the configuration pass to init(), if desired. Then for in-browser use, we set the mathjax path for loading components to be jsDelivr.net (since the code that calls init() will need to be webpacked, and MathJax won't be able to determine the location of the local MathJax files, which may not even be available where the webpacked file has been placed). Again, that can be overridden in the configuration. Then, if there is extra setup (for node applications), that is done.

Now we apply the configuration that was passed to init(), which is where the settings above can be replaced. If an adaptor was requested, we insert that into the loader.load array, but only if there isn't another adaptor already being loaded. Then we set the MathJax.loader value to be the Loader that we obtained (when a combined component like tex-chtml.js is loaded, it will already have a copy of Loader, and we need to be able to reconcile that when the component is loaded; we will describe the details later).

Next, if a component was specified, we load it (if the component was set to and empty string or null, we don't load a component, and are left with a MathJax that only has the loader, which might be useful if one wants to load components by hand later). Waiting for this component to load was where I noticed the problem with the timing of the startup sequence described above. This Loader.load() call now does not resolve until the startup sequence has actually completed.

Finally, we wait on the MathJax.startup.promise (if present), and then return the MathJax global object.

The initial windows.exports ??= {} is to support CommonJS usage, where some webpacked files end up trying to set exports values rather than module.exports (it has to do with the Typescript compiling of .cts files like the ones used to load the default en.json locale files). I could don't find a better work-around.

The browser.js file exports MathJax and the init() function, just like node-main did.

The browser's config.json copies the browser.d.ts file, and makes the webpacked version in bundle/main.

The browser's json.cjs file is used in the webpack.cjs file to replace the ts/components/mjs/json.cjs file with a simplified one that is browser-specific. The original is a dual-purpose one that works in both node and the browser, but includes an import() call that needs special handling if webpacked. It includes a comment to prevent that from being a problem, but that comment is lost when browser.js is webpacked, and so if the code that imports @mathjax/src or @mathjax/src/browser is itself webpacked (as it will need to be), that magic comment is lost, and you would get a warning about importing a file from a variable. This substitution prevents that.

The webpack.cjs file is used by the MathJax web packing work-flow to modify the standard webpack configuration that it uses. In this case, it sets things up to produce a commonjs module (so its exports are available), to make the substitution of the json.cjs file, and to make only a single chunk (otherwise it creates several chunk files).

---

The components/mjs/main and components/mjs/main/node components are set up similarly. The main.js file code from both the browser and node implementation and selects which to use based on the presence of the window global object (via the hasWindow context variable) and exports the resulting MathJax and init() values.

The node component has two additional support files: node.cjs and setup.js. The first separates out some file-loading commands that we don't want to have webpacked into a bundled file that would be used on the web, e.g., when the main component is used for a web application. Since main loads both the node and browser code, if node.cjs were not separated out, the fs, path, and import calls would be included in web applications, which would lead to errors. So these are separated out to a file that is only loaded when the node version is selected by main or by importing @mathjax/src/node (or @mathjax/node).

The setup.js file is the one that provides the nodeSetup() function that makeInit() uses to customize the init() function for use in node. (The node.js component file just loads nodeSetup and adaptor from setup.js and makes the init() command using them. The reason this is separated out from node.js is that it gets loaded by the main.js component, but we don't want to make the init() file for node, which would cause the node.cjs file to load, in the case that the browser version is being used, in order to prevent the node.cjs code from being included in browser-based applications.

The nodeSetup() function is used by the makeInit() function customize the init() function for node. In this case, it loads node.cjs file to get the require function and path library and checks if the source.js file is available and loads that. When the loader starts up, it tries to determine the location of the MathJax files and saves that in MathJax.config.loader.paths.mathjax. If it can't do it, it sets the path to /. We check for that (or file:/// in the case of windows), and when the path can't be determined (as will be the case if the node application is bundled), we look to see if @mathjax/src or @mathjax can be located, and set the path to that. This means the configuration won't have to set the mathjax path explicitly if one of those packages can be found.

Then we check if the source mapping is available and use that if so. This is how @mathjax/src/source manages to work.

Then if there is no asyncLoad configured yet, we use the REQUIRE command from node.cjs, and mark it as synchronous. We also use REQUIRE for the loader.require value so that the loader knows how to get files. The configuration passed to init() can override that, if desired.

Finally, we use the liteDOM adaptor, unless the configuration overrides that.

---

The next dozen or so files are modifications to the combined components to handle the synchronization issue described above. A new readyAfter() function is added to the components/mjs/startup/init.js file that handles the details of the synchronization, and the files like mml-chtml.js that used to call loadFont(startup)) now use

readyAfter(COMPONENT, () => loadFont(startup));

where COMPONENT is the name of the component (mml-chtml in this case). Since the component name appears twice, I put it in a variable so that there is no chance of them getting out of sync. Note that startup/startup.js itself uses the same readyAfter() as the other combined components.

The way readyAfter() works is as follows. The Loader has a mechanism for allowing the landing of a component to require additional components to load (for example, the output jax can cause a font component to load when an alternative font is specified), and the Loader.load() call is not resolved until those additional actions are taken. That is handled through the component's checkLoad() function in the loader's configuration for the component. So, for example, MathJax.config.loader['output/chtml'].checkLoad() is the function that is called (if it exists) to decide if more actions need to be taken before output/chtml is considered fully loaded.

We hook into this feature to make sure the combined components' startup actions are complete before their Loader.load() calls are satisfied. To do that, readyAfter() creates a promise (called load) and set's up the component's checkReady() function to wait for that promise to resolve. The readyAfter() function is passed a function, startup, that returns a promise that we said on, and resolve the load promise after startup resolves. That means that Loader.load() won't resolve until the checkLoad() resolves, which happens when load resolves, which happens only after startup completes.

Note, however, that checkReady() is user-configurable, so we save the old checkReady() (if any) and call that after the load promise resolves, and return its promise (so that checkReady() will not resolve until after the user's original checkReady() also resolves). Whew!

One other change to startup/init.js is to add .catch((error) => MathJax.startup.promiseReject(error)); to the startup sequence. That way, if the CONFIG.failed() function (which is the Loader's failed() function) throws its error, this second catch will pick that up and cause the MathJax.startup.promise to reject with the error. This is what allows init().catch() to catch the error, otherwise we get a warning about a rejected promise not getting caught.

Of course, there are also 8 deleted files from the old node-main directory. The completes the changes in components/mjs.

---

The package.json file's exports table is modified to remove the node-main references and add the new main, node and browser packages. It also swaps the compilation order for the cjs projects so that the magic comments added to the json.ts files (below) are properly preserved.

The testsuites/src/setupTex.js file is similarly changed to load the main.js file rather than node-main.js

I made a separate PR for the change to the a11y/explorer/Region.ts file, but included it here as well since it causes problems with checking webpacking of browser applications using the new packages.

The semantic enrichment in a11y/semantic-enrich.ts forces the sre.locale to the current locale as set in Locale, which may not be the case, since it is the menu component that initializes document.options.sre, and if the menu isn't included, sre's values may not get initialized.

The magic comment for webpack is added to the ts/components/cjs/json.ts file so that webpack will not complain about the require() call on a variable file name. Similarly for the ts/components/mjs/json.ts file later on.

Some of the types in ts/components/loader.ts are improved, so that the configuration passed to init() can be better type-checked. The call to the loader.ready() function is adjusted so that this is the Loader object, and it now gets the MathJax object passed to it, ad described above. Finally, when MathJax.loader is already set, we check that it is the same as Loader itself, and if not, we transfer our variables over to the original MathJax.loader. This is because the new main and other components load ts/components/loader.ts directly, and then may load a bundled component (rather than its source version). When that is the case, the Loader in main/main.js and the one in the bundled component will be different, and while they may operate the same, their data about the versions of the loaded components, the path filters that are set up by the other components that are loaded, and the data in the Package object about the components that have been loaded would not be shared. This makes sure the two versions both have access to the same data. This allows the main init() function to bootstrap the loader used to load the main component requested by the init() call.

The ts/components/package.ts file has similar changes supporting better type checking, and passing the MathJax variable to the ready() and other configurable functions. The same for ts/components/startup.js, which also includes some formatting changes that were made ugly by prettier. The extra ? for Startup.document?.menu is because the document might not be produced (e.g., when no handler gets defined).

The change to FontData.ts is because some errors are just strings, and so err.message doesn't exist.

---

A number of updates are made to the ts/type files to improve the typing of the configuration and global MathJax objects. First, in ts/types/Components.ts, a new COMBINED type is used to create the data for the combined component files so that they include the startup component (as all combined ones do) without having to specify it explicitly in the list of components it includes, and also to add the component property that is used to get the list of components loaded that is used to type-check the component argument to init(), as well as the loader.load array. The COMBINED type then is used for all the combined components defined in the COMPONENTS list below.

The TEX_PACKAGE type is changed in ts/types/Types.ts so that it can be used for all the tex packages, even the ones with no configuration options that used to use EMPTY_COMPONENT to define them. This change was made so that the tex.packages array can be checked so that it doesn't try to configure packages that were not loaded.

The locale option that sets the initial locale is added to the startup and loader components, since one of those is always included, and since those are the components that use it.

Finally, a new ADAPTORS object type is added that includes the data needed to process the various adaptors. In particular, the entries specify the DOM types used by each adaptor, and the component definition for it. This object lets us use just liteDOM or browser, etc., for the A generic type for init(), for example, rather than something more complicated. In particular, it means we can produce the proper DOM typing automatically based on the adaptor selected.

Most of the improved typing comes from the ts/types/Types.js file. Here the COMPONENT_DEF type is extended to include a new tex_package field that lists the package names of the TeX packages that are loaded. This is used to check the tex.packages configuration so that only loaded extensions can be included. (If custom extensions are created in the startup.ready() function, one would need to add a custom COMPONENT_DEF to the T generic for init(). You could use TEX_PACKAGE to do that.)

Several new type processors are added to handle checking for the loader.load and tex.packages arrays, and to add the proper typing to the startup.ready() and startup.pageReady() functions so that the MathJax argument will have the correct types within those functions.

As mentioned earlier, the TEX_PACKAGE type is modified to work for packages with no configuration options. It also now sets the tex_packages value needed for checking the tex.packages array.

The CONFIG type that produces the types for the MathJax.config object is modified to include the LOAD, PACKAGES and READY adjustments for checking loader.load, tex.packages, startup.ready(), and startup.pageReady(). An extra generic boolean R is used to decide whether the startup functions are adjusted. This is to avoid a recursion when producing the type for the MathJax argument of ready() and pageReady(), since the MathJax object includes a config sub-object. So we use TYPES2MJX_OBJECT<T, D, false> here (the false is passed on to CONFIG) so that we don't get an infinite recursion. The CONFIG type now also needs the D generic to pass on to TYPES2MJX_OBJECT, so it is added here an in the calls to CONFIG.

A new COMPONENTS_OF selector gets the components object from the definition (this is needed later to get the intersection consisting of the component names for checking the second parameter to init()).

The R generic is added to TYPES2MJX_OBEJCT so that it can be passed on to CONFIG to avoid the infinite loop.

The MATHJAX, MATHJAX_OBJECT and MATHJAX_CONFIG types are moved out of the ts/types/dom/html.ts and ts/types/dom/lite.ts files and into the ts/types/mjx.ts file, which is renamed to ts/types/mathjax.js, as a common version can now be used. These now have an optional second generic A that specifies the adaptor to use (defaulting to liteDOM), and we use the ADAPTORS object to get the proper DOM node types instead of having DOM-sepecific MATHJAX types. One can use MATHJAX_OBJECT<'tex-chtml'> to get the types for MathJax with the tex-chtml component and the liteDOM, or MATHJAX_OBJECT<'tex-chtml', 'browser'> to get it with browser node typings.

The new ADAPTOR_LIST type is the list of adaptor names that are available, while the MJX_COMPONENTS type gives the list of components that have been included in a type configuration, while MATHJAX_COMPONENTS produces that given a list of components, and an adaptor name. Finally, ADAPTOR_DOM<A> gives the DOM node types for the adaptor specified by A.

---

The last changes are to the tsconfig configurations. These are to include the json.ts files in the compilations that preserve the magic comments for webpack.

[codecov]

Codecov Report

❌ Patch coverage is 76.74419% with 50 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.10%. Comparing base (91b8038) to head (ec5e485).

Files with missing lines	Patch %	Lines
components/mjs/main/node/setup.js	70.90%	16 Missing ⚠️
components/mjs/core/core.js	7.69%	12 Missing ⚠️
ts/components/package.ts	52.63%	9 Missing ⚠️
ts/components/loader.ts	54.54%	5 Missing ⚠️
components/mjs/main/browser/browser.js	89.18%	4 Missing ⚠️
components/mjs/startup/init.js	90.90%	3 Missing ⚠️
ts/output/common/FontData.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1566      +/-   ##
===========================================
- Coverage    87.12%   87.10%   -0.03%     
===========================================
  Files          392      394       +2     
  Lines        89187    89232      +45     
  Branches      5063     5074      +11     
===========================================
+ Hits         77706    77725      +19     
- Misses       11481    11487       +6     
- Partials         0       20      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.