September 30, 2021 · Windows WSL2 Linux Docker

WSL2 Extremely slow file read fix (mutagen)

Current Env:
Windows 10
WSL2 unbuntu 20.04

This document is intended to solve the WSL2 slow file read issue via using a docker container bridge to construct a forward file sync from Windows host to docker container volume mirror mount to the WSL2 container.

The development of IDE env will be hosted on Windows session, not WSL2 session.
The hosting / script / test session can be on both Windows / WSL2 session.

The pros and cons of using this method:

Pros:

Cons:

Windows:

  1. Download mutagen to your windows, I am using mutagen v0.11.8 here , https://github.com/mutagen-io/mutagen/releases
  2. Config windows path for mutagen (copy to C:\.bin, than add C:\.bin to sys env path)
  3. Open Powershell session, test your if mutagen is working corectly,
    > mutagen
    
    PS C:\Users\Engineer> mutagen                                                           
    Mutagen is a remote development tool built on high-performance synchronization          
    
    Usage:                                                                                  
      mutagen [flags]                                                                       
      mutagen [command]                                                                     
    
    Available Commands:                                                                     
      sync        Create and manage synchronization sessions                                
      forward     Create and manage forwarding sessions [Experimental]                      
      project     Orchestrate sessions for a project [Experimental]                         
      tunnel      Create and manage tunnels [Experimental]                                  
      login       Log in to mutagen.io                                                      
      logout      Log out from mutagen.io                                                   
      daemon      Control the lifecycle of the Mutagen daemon                               
      version     Show version information                                                  
      legal       Show legal information                                                    
      help        Help about any command                                                    
    
    Flags:                                                                                  
      -h, --help   Show help information                                                    
    
    Use "mutagen [command] --help" for more information about a command.                    
    

On WSL2:

  1. goto ~/ , setup mirror path
    cd ~
    mkdir docker-mirrors
    cd docker-mirrors
    
    # remove UBUNTU_BRIDE container if exists
    docker rm UBUNTU_BRIDGE  
    # start the session
    docker run -it -v $PWD:/app --name UBUNTU_BRIDGE ubuntu:20.04 /bin/bash 
    
    # In the docker session
    # config a user with the same user id of WSL2 , default is 1000
    adduser ubuntu
    cd /app # here are the synchronized files
    

Setup development environment on Windows

  1. goto %HOMEPATH%, default is C:\Users\{YOUR_USERNAME}\
  2. create a default .mutagen.yml, (there is . before the mutagen.yml, make sure you have it), for ignoring node_modules and other files that should only be working on the host (for example linux based binaries)
    # write this file at %HOMEPATH%.mutagen.yml
    
    sync:
      defaults:
        ignore:
          paths:
          - "mutagen.yml"
          - "mutagen.yml.lock"
          - ".vagrant"
          - ".git"
          - ".idea"
          - "node_modules"
          vcs: true
    

Setup Project file sync on Windows (You only need to do step 7-10 for new projects)

  1. goto your project root path, and start the mutagen session:

    > mutagen sync create . docker://[email protected]_BRIDGE/app/project_name
    Created session sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
    
    > mutagen sync list sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
    Identifier: sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
    Labels: None
    Alpha:
            URL: C:\Users\Engineer\Projects\xxx\yyy
            Connection state: Connected
    Beta:
            URL: docker://[email protected]_BRIDGE/app/project_name
                    DOCKER_HOST=
                    DOCKER_TLS_VERIFY=
                    DOCKER_CERT_PATH=
                    DOCKER_API_VERSION=
            Connection state: Connected
    Status: Watching for changes
    
  2. On your docker mirror session, verify the sync

    [email protected]:/app# ls -la project_name/
    total 236
    drwx------   8 ubuntu ubuntu   4096 Sep 30 01:19 .
    drwxr-xr-x   4 ubuntu ubuntu   4096 Sep 30 01:19 ..
    -rw-------   1 ubuntu ubuntu    141 Sep 30 01:19 .editorconfig
    drwx------   7 ubuntu ubuntu   4096 Sep 30 01:19 .git
    -rw-------   1 ubuntu ubuntu     84 Sep 30 01:19 .gitignore
    drwx------   2 ubuntu ubuntu   4096 Sep 30 01:19 .idea
    -rw-------   1 ubuntu ubuntu    345 Sep 30 01:19 README.md
    -rw-------   1 ubuntu ubuntu    229 Sep 30 01:19 _config.yml
    -rw-------   1 ubuntu ubuntu    942 Sep 30 01:19 package.json
    drwx------   2 ubuntu ubuntu   4096 Sep 30 01:19 scaffolds
    drwx------   5 ubuntu ubuntu   4096 Sep 30 01:19 source
    drwx------   3 ubuntu ubuntu   4096 Sep 30 01:19 themes
    -rw-------   1 ubuntu ubuntu 171151 Sep 30 01:19 yarn.lock
    [email protected]aab60681:/app#
    
  3. On your WSL2, verify the sync

    > ~/projects/docker-mirrors# ls -la project_name  
    total 236
    drwx------   8 engineer engineer   4096 Sep 30 09:19 .
    drwxr-xr-x   4 engineer engineer   4096 Sep 30 09:19 ..
    -rw-------   1 engineer engineer    141 Sep 30 09:19 .editorconfig
    drwx------   7 engineer engineer   4096 Sep 30 09:19 .git
    -rw-------   1 engineer engineer     84 Sep 30 09:19 .gitignore
    drwx------   2 engineer engineer   4096 Sep 30 09:19 .idea
    -rw-------   1 engineer engineer    345 Sep 30 09:19 README.md
    -rw-------   1 engineer engineer    229 Sep 30 09:19 _config.yml
    -rw-------   1 engineer engineer    942 Sep 30 09:19 package.json
    drwx------   2 engineer engineer   4096 Sep 30 09:19 scaffolds
    drwx------   5 engineer engineer   4096 Sep 30 09:19 source
    drwx------   3 engineer engineer   4096 Sep 30 09:19 themes
    -rw-------   1 engineer engineer 171151 Sep 30 09:19 yarn.lock
    
    # you can now run all your dev scripts here:
    > yarn 
    > ls -la .
    total 232
    drwx------   6 engineer engineer   4096 Sep 30 09:28 .
    drwxr-xr-x   4 engineer engineer   4096 Sep 30 09:25 ..
    -rw-------   1 engineer engineer    141 Sep 30 09:25 .editorconfig
    -rw-------   1 engineer engineer     84 Sep 30 09:25 .gitignore
    -rw-------   1 engineer engineer    345 Sep 30 09:25 README.md
    -rw-------   1 engineer engineer    229 Sep 30 09:25 _config.yml
    drwxr-xr-x 484 engineer engineer  20480 Sep 30 09:29 node_modules
    -rw-------   1 engineer engineer    942 Sep 30 09:25 package.json
    drwx------   2 engineer engineer   4096 Sep 30 09:25 scaffolds
    drwx------   5 engineer engineer   4096 Sep 30 09:25 source
    drwx------   3 engineer engineer   4096 Sep 30 09:25 themes
    -rw-------   1 engineer engineer 171151 Sep 30 09:25 yarn.lock
    
    # you may have fun now
    # everything will work as normal, including browsersync etc.
    > yarn server
    
  4. Once completed the session, terminate it on Powershell

    mutagen sync terminate sync_Gpwdlf1Dor5PEOtkNBBzZYN1LXci6hkl7AZzJPVZavp
    

Bye , see you guys around.