continue request #5015 by @halfnelson

this patch got larger than expected. let me know if i should must make it smaller
most noise is in refactoring of tests

done - major changes

• bugfix in fn merge_tables (see end of comment)
• optimize: skip unnecessary encode+decode steps
  • between res.get_sourcemap() and remapping(sourcemap_list) - make preprocess 20% faster in benchmark
  • between svelte.preprocess and svelte.compile - waiting for add print option sourceMapEncodeMappings Rich-Harris/code-red#51 and add option decodeMappings to remapping ampproject/remapping#88
• use the loader interface of remapping to support any number of source files = convergent transform chains with multiple bundle steps

done - small changes

• fix suggestions by benmccann in request 5015
• rename variables
• rephrase comments
• indent with tabs
• micro optimize: skip column-shifting if offset is zero
• verified edge case: when concatting code segments, and when the first segment ends with a newline (empty line), then offset (for column-shifting) is zero
• move functions replace_async and get_replacement to global scope, make pure fn, avoid using global vars
• refactor sourcemap tests, move _preprocess.js to _config.js, rename variables, etc
• fix test preprocess/filename. preprocessors.filename is ignored if options.filename is set
• handle sourcemap.names, dont just return empty array
• add test: sourcemap with names field, verify merging and remapping of map.names
  names is an array of original symbol names that were changed in the transform (sample: terser.minify)
• micro optimize: use mutable data, much faster than immutable
  • replace array.concat with array.push
  • avoid array1.push(...array2) which is limited by max call stack size of ~100K
• micro optimize: unswitch loops, javascript does not do this for us - cost: more code
• bugfix in test/setup.js - support default + named import
• show warning on unnecessary decode_mappings. svelte.compile prefers to consume decoded mappings - transcoding of mappings is expensive
• return css.map.version = 3 in svelte.compile. solved by workaround. magic-string type DecodedSourceMap should have property version
• rebase to master -> https://github.com/milahu/svelte/tree/add-preprocessor-sourcemaps-rebased-to-ebbbc0d
• optimize: use for loops on large arrays - much faster than functional loops
• fix css remap, this must be done earlier in render_dom and render_ssr, so map.toUrl() etc can use the combined map - add test sourcemaps/compile-option-dev

removed

• show warning on potential low-resolution sourcemaps. these can make older mappings unreachable, so every preprocessor in the chain should return high-resolution sourcemaps. problem: this can not be reliably detected

todo

• cleanup commits. one to refactor tests + one to add feature preprocessor sourcemaps
• optimize: make code-red return decoded mappings. currently, mappings are encoded by code-red and decoded by compile/Component.ts
• optimize: make remapping return decoded mappings. currently, mappings are encoded by remapping and decoded by string_with_sourcemap.ts
• optimize "remapping" (combining of sourcemaps). the current format of decoded mappings (dense arrays, indexed by line and segment) is not optimal for this purpose (many lookups). currently we need binarySearch for lookup by line and column. better would be a data-structure indexed by line and column. converting between line-segment format and line-column format would be overhead, we need the sourcemap-generator to directly generate the optimized line-column format. see decode to object - make lookup faster Rich-Harris/sourcemap-codec#82 for possible solutions

todo - document

• preprocessors must generate high-resolution sourcemaps, so all segments can be traced back to original source files. this condition (hires maps) can not be easily tested by svelte, we would have to tokenize source files and compare with sourcemap.mappings = probably out of scope, rather users should fix their tools
• recommend returning decoded sourcemaps in preprocessor, to avoid unnecessary encode+decode steps
• show how to do multiple bundle steps. the bundle file must always have the filename ThisComponent.svelte. limitation: included files must be 'original source files' = leaf nodes = can not have sourcemaps
• on next release, update section svelte.preprocess: "In current versions of Svelte it will be ignored, but future versions of Svelte may take account of preprocessor sourcemaps."

 function merge_tables<T>(original: T[], extended: T[]): { table: T[]; new_idx: number[] } {
   const table = original.slice();
   const new_idx = [];
-  for (let j = 0; j < original.length; j++) {
+  for (let j = 0; j < extended.length; j++) {
     const current = extended[j];
     const existing = table.indexOf(current);
     if (existing < 0) {
       table.push(current);
       new_idx[j] = table.length - 1;
     } else {
       new_idx[j] = existing;
     }
   }
   return { table, new_idx };
 }