Sia Foundation Update – Feb 2024

Progress Report for the PBS Cloud Backup module – Feb 2024

What progress was made on your grant this month?

Progress has been slow due to some health issues with our developer and some problems faced in code understanding.

Phase 5. Build the cloud backup module:

1. What was done in the code:
   1. added /api2/json/cloud/backup endpoint
   1. copied tape/backup method inside cloud/backup without changes
   1. added src\cloud\pool_writer folder
   1. added src/api2/cloud/backup.rs and added ancillary structs
   1. added method bakcup and backup_worker in src/api2/cloud/backup.rs
   1. added pbs-api-types/src/cloud/mod.rs
   1. Added pbs-config/cloud_job.rs
   1. Implement the relevant cloud functionality and removed existing tape functionality, like MediaPool, used_tapes, etc.
   1.  
2. What was done in DevOps:
   1. Setup the renterd service again
   1. Worked on the renterd client to upload file through renterd API: https://api.sia.tech/renterd

Accomplishments for the Month:

• Not much has been achieved accept for fixing the error on adding cloud backup structs, similar to tape backup structs.
• One of the milestones which were discussed were to figure out how will the rust-S3 plugin be used to interact with the sia S3 cloud storage network:

Architectural discussion was done to ensure the right architecture is being implemented, the following is what will be implemented:

              The Following tasks were closed this month in openproject (project management tool) :

• Construct Traits for Client Functionality
• Implement Backup Functionality
• Implement List Functionality
• Reading and Management of Snapshots

Links to repos worked on this month:

https://github.com/AZComputerSolutions/PBSCloudBackupModule

What will you be working on this month?

During this month the backup module’s development would be continued, which will include:

During the next month(March) the backup module’s development would be continued, which will include, the unfinished tasks from last month:

Implementation of Backup functionality

• Following structs/traits need to be implemented in relation to the cloud backup functionality in api2/cloud/backup.rs:
  • TapeBackupJobStatus -> CloudBackupJobStatus
  • TapeBackupJobConfig -> CloudBackupJobConfig
  • TapeBackupJobSetup  -> CloudBackupJobSetup
  • TapeBackupJobStatus -> CloudBackupJobStatus
  • TapeBackupJobSummary -> CloudBackupJobSummary
  • use::crate::tape -> change to relevant cloud trait and implement the functionality
  • TAPE_BACKUP_JOB_ROUTER -> CLOUD_BACKUP_JOB_ROUTER
  • tape-backup-job -> cloud-backup-job
  • do_tape_backup_job -> do_cloud_bakcup_job
  • backup_job.setup -> implement relevant cloud functionality, to accept cloud jobs
  • let drive_lock = lock_tape_device(&drive_config, &setup.drive)?; -> tape locking and device locking functionality needs to be removed
  • let _ = set_tape_device_state(&setup.drive, “”); -> tape status needs to be removed
  • TAPE_STATUS_DIR -> needs to be removed
  • pool_writer.set_media_status_full(&uuid)?; -> This would be done along with the CatalogSet and NewChunksIterator as mentioned above
  •  bail!(“Tape backup finished with some errors. Please check the task log.”); -> Update the notifications according to the notifications from S3 network
  •  summary.used_tapes = match pool_writer.get_used_media_labels() -> after the classes, mentioned above are modified, this would be implemented according to the cloud

Currently, the development is being done in an isolated repository.  As mentioned in the risks initially, there are chances that this S3 cloud backup feature does not get accepted into the code base of proxmox, so in the short term we have elected to isolate the code until it does get accepted.  There was also an issue where the developer was using renterd api which needed to be corrected to use the rust-s3 create instead.  This was caught and corrected early (due to weekly reporting) and the developer is now back on track.