Files

Stores allow you to exchange and save files. In this section, we'll take a closer look at file structures and how to manage them effectively. We'll also cover some best practices when working with Stores.

About Files

Along with a file's main content, additional metadata is stored to allow easy file management. Below is the structure of the metadata associated with each file:

Uploading a Simple File

To place a file in the Store, you need to create a file handle using the createFile(...) function. The function accepts a StoreId, public and private metadata,the total size of the file that will be uploaded, and set up a flag that tells whether the file has the random write feature enabled.

Then you need to call writeToFile(...) providing the created file handle and a portion of data to write. For small files, you can write the entire file with one request. For large files, it's more optimal to write them piece by piece, loading them in portions, for example from the disk, and then calling writeToFile(...) on each piece. To complete the entire file upload process - call closeFile(...) with the file handle as a parameter.

// for example, let's put some extra file info in the file's private meta
const sampleFileInfo = {
    name: file.name,
    mimetype: file.type
};

const sampleFileData = serializeObject("sample file data");

const fileHandle = await storeApi.createFile(storeId,
    serializeObject("file's public meta-data"),
    serializeObject(sampleFileInfo),
    sampleFileData.length,
    true
);

await storeApi.writeToFile(fileHandle, sampleFileData);
const fileId = await storeApi.closeFile(fileHandle);

Streaming

For larger files or scenarios where you need more control over the upload process (e.g. managing upload progress, pausing, or canceling uploads), we recommend using streaming. Streaming allows you to upload the file in chunks, enabling better interactivity, monitoring, and control.

Streaming is especially useful when:

• You need to upload large files and a single upload might take a long time;
• You need the ability to cancel the upload in the middle of the process.

Uploading large file - writing to Store chunk by chunks:

Let's say you have an Html input element in the page body:

<input type="file" id="inp"/>

Now, access that above input element from the JS code and get a file from there:

const fileInput = document.getElementById('inp');
if (!fileInput.files || fileInput.files.length === 0) {
    return;
}

// prepare a stream to read from the file
const stream = fileInput.files[0].stream();
const name = fileInput.files[0].name;

const fHandle = await storeApi.createFile(storeId, 
    this.strToUint8("file's public meta-data"), 
    this.strToUint8(JSON.stringify({name})), 
    fileSize
);

const reader = stream.getReader();
while( true ) {
    const result = await reader.read();
    if( !result || result.done ) { 
        break; 
    }
    console.log("Writing the file's chunk to the Store...");
    await storeApi.writeToFile(fHandle, result.value);
}
await storeApi.closeFile(fHandle);

Reading a file

To read a file, you can do it with a single request or - which is usually a better solution for large files - by reading it piece by piece. You must start by opening the file for reading using the openFile(fileId) function. As a result, you'll receive a file handle that you then use when calling readFromFile(...) to read portions of data from the file. At the same time, you have the ability to move through the file and indicate from which position you want to read it using the seekInFile(...) function.

// read file contents of the file added to the Store
const fileInfo = await storeApi.getFile(fileId);

const fileHandle = await storeApi.openFile(fileId);
const data = await storeApi.readFromFile(fileHandle, fileInfo.size);
await storeApi.closeFile(fileHandle);

console.log("File meta\n public: ", deserializeObject(fileInfo.publicMeta));
console.log("private: ", deserializeObject(fileInfo.privateMeta));
console.log("data: ", deserializeObject(data));

Downloading Files

The code example below shows how to download a file from a Store in PrivMX:

// read file contents of the file added to the Store
const fileInfo = await storeApi.getFile(fileId);

const fileHandle = await storeApi.openFile(fileId);
const data = await storeApi.readFromFile(fileHandle, fileInfo.size);
await storeApi.closeFile(fileHandle);

console.log("File meta\n public: ", this.uint8ToStr(fileInfo.publicMeta));
console.log("private: ", this.uint8ToStr(fileInfo.privateMeta));
console.log("data: ", this.uint8ToStr(data));

Listing Files

Listing files in a chosen Store:

const defaultListQuery = {skip: 0, limit: 100, sortOrder: "desc"};

const filesList = await storeApi.listStores(
    storeId,
    defaultListQuery
);

As a result you will receive an object:

// filesList:
{
    readItems: [<fileObject1>, <fileObject2>,..., <fileObjectN>],
    totalAvailable: <number_of_all_files_in_the_Store>
}

A detailed description of the File object fields can be found in API Reference.

Modifying Files

File modification can be considered in two categories:

1. Modifying file information.
2. Modifying data in the file.

// Example: let's update given file's private meta to reflect the file's name change.
const currentFile =  storeApi.getFile(fileId);
const currentFilePrivateMeta = deserializeObject(currentFile.privateMeta);
const newFilePrivateMeta = {
    name: "newFileName"
};
await storeApi.updateFileMeta(
    fileId,
    currentFile.publicMeta,
    serializeObject(newFilePrivateMeta)
);

Deleting a File

To delete a file, you need to use the deleteFile(...) method.

await storeApi.deleteFile(fileId);

Notes

• Metadata: helps you identify files later on by storing additional information about them. You can use both private metadata (which might contain sensitive or internal data) and public metadata (which can be shared publicly).
• Error Handling: During the streaming process, it's important to handle potential errors, such as network interruptions, by using try-catch blocks and the ability to abort the upload.
• Progress Monitoring: Streaming enables you to monitor the progress of the file upload, allowing you to display the current upload percentage to the user.