OpenStack Summit Fall 2012

This session will be a progress report of an inter-cloud object storage project called colony which is based on swift. Inter-cloud object storage is a common storage which can be accessed from multiple clouds. It should be deployed geographically distributed for availability and good performance. Colony can be viewed as a network-aware version of swift, because it needs to perform without huge performance degradation even with high latency network connections.

The discussion points:

1. How to make Swift network-aware
In the PUT operation, all replicas are written in the same site as the proxy server, instead of writing them to the location the ring specifies.
The replications to the original positions are done by the object replicator asynchronously. After confirming the replication, the local copies corresponding to the replicas are deleted. In the GET operation, the ‘nearest’ replica is chosen by the mechanism described in the next section, instead of randomly chosen. The proxy server works with the cache mechanism as well.
2. How to measure network distance
We use the zone information in the ring for the network distance mesurement. The zone information is fixed decimal numbers and we can allocate them freely. Therefore, we can assign these decimal numbers to specify actual locations. We also add two config items (ring_zone_site_number and near_distance) to proxy server for detecting which account/container/object servers are nearby. When difference between zone information and ring_zone_site_number is within near_distance value, proxy server treats this account/container/object servers are located in the same sites. Let’s say the nodes in data center #1 are from zone-100 to zone-199, the nodes in data center #2 are from zone-200 to zone-299, and so on. If proxy server #1 has 100 as a ring_zone_site_number and 100 as a near_distance, object put through proxy #1 is stored in data center #1 at first, and replicated to proper location asynchronously, By using this sort of convention, the software can know the network distance without our modifying the ring structure or the code related to it.
3. Prototyping
We introduce a prototyped a network-aware OpenStack Swift based on swift-1.4.8. The modifications were made to the following code.
- Swift/proxy/server.py: proxy server code
- Swift/common/ring/ring.py: query process to the ring
4. Evaluations
We describe the evaluation result of the prototype in the following environment.
Two sites (Tokyo and Sapporo) are connected through high speed network called SINET-4. Both sites have proxy-server and storage servers, which consisted of account-servers, container-servers, and object-servers. There are three zones at each site.