Add support for Amazon S3 cloud backed block devices to istgt / Elasto
an idea by dmdiss
Updated about 8 years ago. No love. Has no hacker: grab it!

Elasto is a cloud library and client utility for managing and manipulating cloud storage objects via REST.

Functionality was recently added to istgt so that it can expose an Azure storage blob for access via iSCSI, it does so using Elasto's file API.

Elasto's file API should be extended, so that it supports Amazon S3 object IO. This task would be difficult, as the S3 REST API does not currently support length@offset writes to objects.

Once complete, istgt could be extended to pass through Amazon S3 credentials to Elasto, and thus expose an iSCSI target backed by an S3 object. This would allow for cloud redundancy / failover by layers above (RAID, etc.).

No Hackers yet

Join this project

Looking for hackers with the skills:

iscsi scsi cloud rest amazon s3 azure

This project is part of:

Hack Week 10

Activity

  • over 11 years ago: dmdiss added keyword "iscsi" to this project.
  • over 11 years ago: dmdiss added keyword "scsi" to this project.
  • over 11 years ago: dmdiss added keyword "cloud" to this project.
  • over 11 years ago: dmdiss added keyword "rest" to this project.
  • over 11 years ago: dmdiss added keyword "amazon" to this project.
  • over 11 years ago: dmdiss added keyword "s3" to this project.
  • over 11 years ago: dmdiss added keyword "azure" to this project.
  • over 11 years ago: dmdiss originated this project.


    • Comments

    • bmwiedemann
      over 11 years ago by bmwiedemann | Reply

      I think, S3 is not designed for this. It is more like a filesystem, where keys are pathnames and values are file content. Is there an API for access to Amazon's Elastic Block Storage (EBS) ? What about owncloud?

    • bmwiedemann
      over 11 years ago by bmwiedemann | Reply

      ceph/rbd (used for volumes) and swift (S3 equivalent) from SUSE Cloud would also be a worthy target.

    • dmdiss
      over 11 years ago by dmdiss | Reply

      Indeed, S3's REST interface is not designed for this. Nevertheless, I'd still like to implement it, as this would allow for transparent encryption and compression on the client using existing tools such as dm-crypt and Btrfs. Failover and redundancy between Azure and Amazon S3 storage should also be possible. Amazon already offer a Storage Gateway with a similar purpose, but this promotes vendor lock-in.

    • dmdiss
      over 11 years ago by dmdiss | Reply

      Ceph's RADOS gateway purportedly offers the same REST protocol as Amazon S3. I agree that it and swift would both be worthy targets.

    Similar Projects

    This project is one of its kind!