Pitfall Chronicles - The Pain of the Windows Frontend Toolchain

Preface#

Yesterday I posted a tweet because I encountered compatibility issues related to Windows while working on the toolchain, which prompted me to vent a bit. However, it seems that in the atmosphere of Twitter, only such "outrageous" statements can attract inexplicable attention. This sparked dissatisfaction among some people and even led to a debate between Windows and macOS.

So let's take a look at Windows from the perspective of Unix-like systems and discuss the "pits" regarding Windows in the front-end toolchain (Node.js).

Path Troubles#

Strange Backslashes#

In Windows, a common path looks like D:\path\to\file.txt; whereas in Unix-like systems, it is /home/username/file.txt. It is evident that Windows uses backslashes \ as separators between path levels. In most programming languages, forward slashes usually do not require special handling, while backslashes are used to escape characters. For example:

console.log('\\') // Actually outputs a single backslash
console.log('\n') // Outputs a newline character
console.log('/') // As expected, outputs a forward slash

This means that if we only consider running on Unix-like systems, we can basically ignore special handling for paths.

`:` Separator#

Moreover, Windows has the concept of drive letters; while Linux uses mounting to manage different physical disks and partitions. This is another area that requires consideration for compatibility.

🌰 An Example#

Nothing is more fitting than an example. You can find my records of being tormented by Windows on GitHub.

test: try fix path for windows #

This commit aims to solve compatibility issues for unit tests on Windows. We need to pass a glob expression to the entry option. In Unix-like systems, we can directly pass the file path /path/to/file to match the exact file, which is great.

However, glob is a feature that originated in UNIX, and in Windows, using backslashes does not work normally in fast-glob as it does in Unix-like systems. This creates a consistency issue. Although fast-glob provides the convertPathToPattern function to help escape Windows backslashes, there are still some cases that remain unresolved.

In this commit, what's even more troublesome is that I used 4 backslashes in the code. Why? Because this code is double-wrapped with backticks ` and single quotes '. Therefore, 4 backslashes will ultimately represent just one actual backslash.

fix: watch `ignored` option for windows #

This is also a common example where we use a regular expression to check if a certain path is within the node_modules folder. To be compatible with Windows, we need to write /[\\/]node_modules[\\/]/, and similarly, the double backslashes will ultimately match just one backslash. But the more troublesome part is that for third-party dependencies, you seem unable to determine whether it will pass backslash paths on Windows or automatically replace them with forward slash paths. In this example, chokidar has quite strange handling rules; for passing functions, it will pass forward slash paths, while for static arrays, it will handle them specially. Hence this commit.

fix: support Node 22/23 strip types feature #

Remember the : separator I just mentioned? I previously thought it didn't seem to have compatibility issues. That was too naive! If we want to import an absolute path in Node.js, it's quite simple in Unix-like systems.

require('/path/to/test.cjs')

Nothing special. However, in Windows, it can be a big problem!

require('C:\\path\\test.cjs') // ✅
import('C:\\path\\test.cjs') // ❌
// Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'

This is because the require API seems to only accept paths as parameters, while import() supports both paths and File URLs. So Node.js treats C: as a URL, with its protocol being c:. This is essentially not much different from http://path/test.cjs. File URLs and paths can be confusing 🤷

import('file://C:/path/test.cjs') // ✅
console.log(new URL('C:\\path\\test.cjs')) // protocol = 'c:'

Import Statement#

import mod from 'a/b/n'
import mod from 'a\b\n'

In ESM, you must use forward slashes as separators. Directly using backslashes may lead to unexpected escapes. Extra caution is needed in code generation-related libraries.

🤨 What to Do?#

So how to solve this? Using pathe might be a good choice! Even in Windows environments, it will directly convert paths to forward slashes. Forward slashes are also treated as backslashes in Windows. So for most cases, it can directly help you solve consistency issues. But what about edge cases? There are still quite a few.

Open Source Community#

As far as I know, most of the people around me who develop and contribute to open source projects are using Unix-like systems. Meanwhile, most users are likely using Windows. I understand that this may have something to do with market share. But for authors and maintainers, it is also a helpless situation.

For example, my open source projects run unit tests on Windows using GitHub Actions. Every time I push code and receive an error notification from GitHub Action, seeing that it failed to run on Windows is quite frustrating. Because I don't have a Windows computer with a development testing environment set up. Out of desperation, I am now debugging bugs on Windows using a virtual machine.

Moreover, unit tests do not always cover every aspect; there will always be oversights. This requires actual users to discover and report issues!

Overview of Compatibility Issues#

Paths
- Backslashes
  - May need to be escaped
  - May need to be converted to forward slashes
- Drive letters :
  - May need to be converted to File URLs
Unable to set environment variables using KEY=VALUE command
- Use pnpm's shell-emulator
- Can use cross-env
No rm -fr command
- Use fs.rmSync(path, { recursive: true, force: true })
- Use rimraf

There are countless such situations. I cannot list all of them; these are just some of the pitfalls I remember encountering.

Postscript#

Encountering ten path issues does not mean there are only ten; there may be many more on the way 🛣️... I cannot accept the statement "can't even handle paths correctly."

If you have noticed, this article consistently uses the term Unix-like. This is because besides Windows, there are many other operating systems in the world.

This article and the tweet are not intended to spark a "debate" between Windows and Linux, macOS. Windows still plays an important role in many areas (such as gaming). However, in terms of cross-platform fields, it indeed causes many inconveniences.

My Library#

For my own open source library, I will still strive to be compatible with Windows and add Windows to CI runs. But for some edge cases, I may overlook them. If you encounter issues while using my library, feel free to raise issues and PRs, and I will do my best to resolve them.

Alternatives#

To cherish precious life, it is recommended to try using Unix-like systems for front-end development (including WSL and third-party virtual machines). Of course, you can continue using Windows, but when encountering compatibility issues, please complain less and invest extra time to actively report issues and contribute to the open source community.

Ramblings#

If after reading the article, you still think this is "arrogance" or "ignorance," feel free.