syncbase blobs: Make servers fetch blobs automatically.

This change adds a syncbase mechanism that fetches blobs
automatically when the syncbase is a server within a
syncgroup that mentions the blob's id.

The main code is in the new file
It uses the Signpost database to discover blobrefs and whether it's a server,
and the blob database to determine whether the blobs are already present.
(Blobrefs are added to the Signpost database by processBlobRefs() routine in

        Add "FetchAttempts" count to the Signpost data so that a server can
        track how many times it has attempted to fetch a blob across reboots.
        The value from a remote syncbase is always ignored, because
        mergeSignposts() in blob.go does not transfer it.
	Start the new ServerBlobFetcher() process when a syncbase starts.
	Fix a bug in syncService.FetchBlob().  I had added an element to a Go map
	without first using make() to create the map.

	Make these methods on syncDatabase become methods on syncService.
	None need to know the datbase, and we need to call them from contexts where no database is known.
	Parameters to control the automatic blob fetching on servers.
	The main blob fetcher code and unittest.
	The integration test is in  syncbase/featuretests/blob_v23_test.go
	Change the order of the BlobDevType* constants so that "BlobDevTypeNormal"
	has the zero value.   This causes devices to be "normal" by default.
	The ordering isn't used anywhere.
	Add an integration test for the automatic server blob fetching code.

        Modify the fetchBlob() helper function so that the "skipIncStatus" is
        used to indicate instead whether the blob is expected to be present,
        and perform checks in either case.  This doesn't change the existing
        calls, because they were using this parameter to skip the checks
        exactly when the blob was expected to be already present.

        Pass an explicit SyncgroupMemberInfo struct to createSyncgroup() and
        joinSyncgroup() helpers, so the new test can specify that some syncbases
        are servers.
        Make createSyncgroup() and joinSyncgroup() helpers take an explicit
        SyncgroupMemberInfo struct so some tests can specify that some
        syncbases are servers.
        Pass an explicit SyncgroupMemberInfo struct to createSyncgroup() and
        joinSyncgroup() helpers.

MultiPart: 2/2
Change-Id: I25e90d63a0b454265522e15a548cc33208815bb8
7 files changed
tree: ef662439ea9a0375e6bea9a73b127dfc37a70480
  1. cmd/
  2. examples/
  3. internal/
  4. lib/
  5. runtime/
  6. services/
  7. test/
  8. .gitignore
  12. envvar.go
  13. envvar_test.go


This repository contains a reference implementation of the Vanadium APIs.

Unlike the APIs in, which promises to provide backward compatibility this repository makes no such promises.