Import a File System

Whether you are using the Enterprise Edition or the Community Edition, JuiceFS supports exporting and importing file system metadata using juicefs [dump|load]. Since JuiceFS employs a design that separates metadata from data, file data is independently stored in your chosen object storage service. Importing a file system essentially involves importing its metadata—there is no need to move the object storage data. If you are unfamiliar with this separation design, read our architecture overview.

tip

Unlike the Community Edition, you do not need to manually back up metadata in JuiceFS Enterprise Edition. Our operations team ensures all metadata services are backed up regularly. In short, Enterprise Edition users do not need to worry about backup and recovery tasks.

For Enterprise Edition users, the export and import features can be used for the following purposes:

• Migrating from the Enterprise Edition to the Community Edition. The specific steps for this process are not covered in this guide; please contact our engineers for assistance.
• Exporting metadata for analysis and troubleshooting.

The following example shows how to import a file system from the Community Edition to the Enterprise Edition.

Compatibility considerations

The file systems of the Community Edition and the Enterprise Edition are compatible and can be migrated between each other. However, note that JuiceFS Enterprise Edition supports and enables UID/GID auto map by default. In simple terms, JuiceFS maps users and groups with the same name to the same UID/GID.

Since this feature is not available in the Community Edition, it is automatically disabled for imported file systems to prevent permission issues.

Stop writing

Before backing up metadata, make sure that the source file system has stopped writing and no modification will occur. In practice, all clients (except read-only ones) must be unmounted to prevent any background task or compaction from tampering data integrity.

To be absolutely safe, use juicefs status to ensure that all clients have been taken down (the Sessions field should be empty) before proceeding with the subsequent steps.

Export a file system

Before exporting, record the file system's configuration, such as block size, compression, and encryption settings. These configurations must match between the source and target file systems for the import process to succeed. For example, if the source and target file systems use different block sizes or compression settings, the object storage data will be unreadable.

In an environment with access to the metadata engine, use the status command to print this information:

$ juicefs status META-URL
{
  "Setting": {
    "Name": "myjfs",
    "UUID": "6b0452fc-0502-404c-b163-c9ab577ec766",
    "Storage": "s3",
    "Bucket": "https://xxx.s3.amazonaws.com",
    "AccessKey": "xxx",
    "SecretKey": "removed",
    "BlockSize": 4096,
    "Compression": "none",
    "TrashDays": 1,
    "MetaVersion": 1
  },
  ...
}

Next, use the dump command to export the file system metadata in JSON format:

# For large-scale file systems, add a .gz suffix to the target file to enable compression.
juicefs dump META-URL /tmp/meta.json.gz

The exported result also includes the file system's configuration information. If you did not save the output of the juicefs status command in the previous step, you can retrieve it directly from the JSON file:

# Record other critical file system configurations, which are listed in the Setting field of the JSON file.
head -n 20 meta.json

With the metadata exported and the file system configuration recorded, the next step is to create the target file system.

Prepare the target file system

The workflow requires a brand new metadata cluster, which does not contain any existing file systems. Contact a Juicedata engineer to deploy a dedicated metadata region for you, and ensure the migration destination is a new metadata cluster.

Log in to the cloud service console and click Create File System in the top right corner. However, instead of filling in the information immediately, click Import File System in the top right corner of the dialog box. Carefully review the settings on the following page and fill them out according to the source file system's configuration.

Import metadata

To import metadata, you need to use the JuiceFS client. Therefore, you should first mount the file system. If you are not familiar with installing the JuiceFS client or mounting a file system, refer to the Quick Start guide.

# Mount the file system.
juicefs mount myjfs /jfs

# Import the metadata file.
juicefs load /jfs /tmp/meta.json.gz

If the critical settings of the two file systems are properly aligned, the import command will complete successfully (exit code 0). Next, use ls to access the mount point, and you will see that the file metadata has been successfully imported. Select a file with actual content to read and confirm that everything is functioning correctly.

Verify

After juicefs load is completed, follow these steps below to verify that the file system is operating correctly:

• Again, for the source file system, verify that all its clients have been completely unmounted and that there is no writing at all. This is very important, if the source file system is still modified after juicefs dump, it will cause inconsistencies and damage the imported file system.
• Read a file that was created or modified a long time ago (mtime as early as possible) and verify that it can be read normally.
• Read a file that was recently created or modified (with the mtime as recent as possible) and verify that it can be read normally.
• Use a command like date > delete.txt to verify that writing is normal. Do not use the touch command, as it is a pure metadata operation.

If all of the above verifications and tests are successfully passed, the migration is considered successful.

If you are unsure whether the source file system has completely stopped writing, or if there are indeed problems with the read tests, troubleshoot according to the following steps:

• If the source file system is JuiceFS Community Edition, first check the database log of the metadata engine to verify whether there are still modifications after the dump.

• Run juicefs fsck to scan for corruption:

  juicefs fsck myjfs

• According to the output, evaluate the scope and cause of the corruption, and determine whether it is necessary to start over. If it is indeed caused by unwanted source file system modification, then before making the backup again, you need to use the juicefs status command in advance to verify that all clients have been taken offline. If the cause of the corruption cannot be determined, contact Juicedata engineers to assist with the investigation.

Set the payment plan

After verification, please select an appropriate payment plan according to the current usage. Cloud Service users should go to this page to set things up. For on-prem deployments, please contact a Juicedata engineer for help.