tree ef662439ea9a0375e6bea9a73b127dfc37a70480
parent c28a0f10aa9f6e12fce2fef35a24d40206aaac68
author Mike Burrows <m3b@google.com> 1467851482 -0700
committer Mike Burrows <m3b@google.com> 1467851482 -0700

syncbase blobs: Make servers fetch blobs automatically.

This change adds a syncbase mechanism that fetches blobs
automatically when the syncbase is a server within a
syncgroup that mentions the blob's id.

The main code is in the new file
	v.io/x/ref/services/syncbase/vsync/server_blob_fetcher.go
It uses the Signpost database to discover blobrefs and whether it's a server,
and the blob database to determine whether the blobs are already present.
(Blobrefs are added to the Signpost database by processBlobRefs() routine in
services/syncbase/vsync/blob.go.)

Details:

v.io/x/ref/services/syncbase/server/interfaces/sync_types.vdl
        Add "FetchAttempts" count to the Signpost data so that a server can
        track how many times it has attempted to fetch a blob across reboots.
        The value from a remote syncbase is always ignored, because
        mergeSignposts() in blob.go does not transfer it.

v.io/x/ref/services/syncbase/server/service.go
	Start the new ServerBlobFetcher() process when a syncbase starts.

v.io/x/ref/services/syncbase/vsync/blob.go
	Fix a bug in syncService.FetchBlob().  I had added an element to a Go map
	without first using make() to create the map.

	Make these methods on syncDatabase become methods on syncService.
	None need to know the datbase, and we need to call them from contexts where no database is known.
		fetchBlobRemote()
		locateBlob()
		getMountTables()

v.io/x/ref/services/syncbase/vsync/parameters.go
	Parameters to control the automatic blob fetching on servers.

v.io/x/ref/services/syncbase/vsync/server_blob_fetcher.go
v.io/x/ref/services/syncbase/vsync/server_blob_fetcher_test.go
	The main blob fetcher code and unittest.
	The integration test is in  syncbase/featuretests/blob_v23_test.go

v.io/v23/syncbase/types.vdl
	Change the order of the BlobDevType* constants so that "BlobDevTypeNormal"
	has the zero value.   This causes devices to be "normal" by default.
	The ordering isn't used anywhere.

v.io/v23/syncbase/featuretests/blob_v23_test.go
	Add an integration test for the automatic server blob fetching code.

        Modify the fetchBlob() helper function so that the "skipIncStatus" is
        used to indicate instead whether the blob is expected to be present,
        and perform checks in either case.  This doesn't change the existing
        calls, because they were using this parameter to skip the checks
        exactly when the blob was expected to be already present.

        Pass an explicit SyncgroupMemberInfo struct to createSyncgroup() and
        joinSyncgroup() helpers, so the new test can specify that some syncbases
        are servers.

v.io/v23/syncbase/featuretests/test_util_test.go
        Make createSyncgroup() and joinSyncgroup() helpers take an explicit
        SyncgroupMemberInfo struct so some tests can specify that some
        syncbases are servers.

v.io/v23/syncbase/featuretests/cr_v23_test.go
v.io/v23/syncbase/featuretests/ping_pong_test.go
v.io/v23/syncbase/featuretests/sync_v23_test.go
v.io/v23/syncbase/featuretests/syncgroup_v23_test.go
v.io/v23/syncbase/featuretests/vclock_v23_test.go
        Pass an explicit SyncgroupMemberInfo struct to createSyncgroup() and
        joinSyncgroup() helpers.

MultiPart: 2/2
Change-Id: I25e90d63a0b454265522e15a548cc33208815bb8