syncbase: use "any" instead of []byte in Put/Get RPCs
This CL and its companion use the VDL type "any" instead of []byte in the
definitions of rows in the syncbase API. See
v.io/v23/services/syncbase/service.vdl
in the Row interface.
The intent is to avoid unnecessary trips through marshalling/unmarshalling when
communicating with syncbase. With this change, all data in syncbase will be
VOM encoded, and on a Get() will not be fully decoded by the syncbase server
before the VOM-encoded data is transmitted via RPC to the client. If the
client wishes to store byte vectors, he can, but they too will be VOM encoded
inside syncbase.
The change also affects the scan and conflict resolution interfaces by changing
the KeyValue and Value structs in
v.io/v23/services/syncbase/types.vdl
v.io/syncbase/types.vdl
The strategy is to use vom.RawBytes at levels of the system near the RPC
interfaces, and []byte in the watchable and store abstractions. The
converstion between the two ios done primarily in the "row" abstraction, and
the collection abstraction (for scans)
v.io/v23/syncbase/model.go
v.io/v23/syncbase/row.go
v.io/v23/syncbase/scan_stream.go
v.io/x/ref/services/syncbase/server/collection.go
v.io/x/ref/services/syncbase/server/row.go
Details:
v.io/v23/services/syncbase/service.vdl
Row.Put() and Row.Get() pass parameters of type "any" rather than type
"[]byte".
v.io/v23/services/syncbase/types.vdl
The structs KeyValue and Value now use the type "any" to represent the
VOM-encoded value.
v.io/v23/syncbase/cr_test.go
makeResolution() now takes a *vom.RawBytes parameter.
compareResult() now encodes vom.RawBytes values to []byte values before comparing.
encode() now converts to vom.RawBytes instead of []byte.
v.io/v23/syncbase/model.go
Get() now uses RawBytes.ToValue() instead of vom.Decode.
NewValue() nbow uses RawBytesFromValue() instead of vom.Encode().
v.io/v23/syncbase/row.go
Get() and Put() now use RawBytes instead of []byte interally to
communicate with the layer below.
v.io/v23/syncbase/scan_stream.go
Value() now uses RawBytes.ToValue() instead of vmo.Decode().
v.io/v23/syncbase/syncgroup_v23_test.go
This test used to have subtests called "*NonVomData", that tested
putting raw bytes directly into the store. We no longer do this: raw
bytes are now accommodated by VOM encoding them, so all values in the
store are expected to be VOM encoded.
v.io/v23/syncbase/types.vdl
Struct Value's Val field is now of type "any" rather than "[]byte".
--------------------
v.io/x/ref/services/syncbase/server/collection.go
Scan() now decodes each value retrieved from the store into a
vom.RawBytes for transmission to the client.
v.io/x/ref/services/syncbase/server/row.go
Get()/get() now decodes a value obtained from the store into a
vom.RawBytes for transmission to the client.
Put()/put() now encodes a vom.RawBytes from the client into a []byte to
give to the store.
v.io/x/ref/services/syncbase/server/watchlog_test.go
TestWatchLogPerms() uses the row interface directly, and so now uses
vom.RawBytes.
v.io/x/ref/services/syncbase/testutil/util.go
WatchChangeEq() now expects a vom.RawBytes in the WatchChange data
structure, rather than []byte.
v.io/x/ref/services/syncbase/vsync/cr_app_resolves.go
processResInfos() now converts the vom.RawBytes data it finds to []byte.
createValueObj() now converts the []byte it finds to a vom.RawBytes
v.io/x/ref/services/syncbase/vsync/cr_app_resolves_test.go
setResInfData() now gives vom.RawValue elements to addResInfo().
checkConflictRow() converts vom.RawValue elements to []byte for comparisons.
addResInfo() now takes vo.RawValue instead of []byte
saveValue() now saves a VOM-encoded value, rather than a byte vector.
--------------------
java/io/v/v23/syncbase/RowImpl.java
test/java/io/v/v23/syncbase/SyncbaseTest.java
In the Java world, there is as yet no equivalent of vom.RawBytes. The types
are mapped to VdlAny. Currently, I convert to and from the desired types by using
VomUtil.Decode(VomUtil.encode(...)...).
MultiPart: 3/3
Change-Id: I93f3b41b96dbab2f1acba1e66e34e3cd0c3d76f7
diff --git a/lib/src/main/java/io/v/v23/syncbase/RowImpl.java b/lib/src/main/java/io/v/v23/syncbase/RowImpl.java
index 92926c2..03ba47c 100644
--- a/lib/src/main/java/io/v/v23/syncbase/RowImpl.java
+++ b/lib/src/main/java/io/v/v23/syncbase/RowImpl.java
@@ -14,6 +14,7 @@
import io.v.v23.services.syncbase.RowClient;
import io.v.v23.services.syncbase.RowClientFactory;
import io.v.v23.syncbase.util.Util;
+import io.v.v23.vdl.VdlAny;
import io.v.v23.verror.VException;
import io.v.v23.vom.VomUtil;
@@ -57,10 +58,10 @@
@Override
public ListenableFuture<Object> get(VContext ctx, final Type type) {
return VFutures.withUserLandChecks(ctx, Futures.transform(client.get(ctx, this.batchHandle),
- new AsyncFunction<byte[], Object>() {
+ new AsyncFunction<VdlAny, Object>() {
@Override
- public ListenableFuture<Object> apply(byte[] data) throws Exception {
- return Futures.immediateFuture(VomUtil.decode(data, type));
+ public ListenableFuture<Object> apply(VdlAny vdlAny) throws Exception {
+ return Futures.immediateFuture(VomUtil.decode(VomUtil.encode(vdlAny), type));
}
}));
}
@@ -68,8 +69,7 @@
@Override
public ListenableFuture<Void> put(VContext ctx, Object value, Type type) {
try {
- byte[] data = VomUtil.encode(value, type);
- return client.put(ctx, this.batchHandle, data);
+ return client.put(ctx, this.batchHandle, (VdlAny) VomUtil.decode(VomUtil.encode(value, type), VdlAny.class));
} catch (VException e) {
return VFutures.withUserLandChecks(ctx, Futures.<Void>immediateFailedFuture(e));
}
diff --git a/lib/src/test/java/io/v/v23/syncbase/SyncbaseTest.java b/lib/src/test/java/io/v/v23/syncbase/SyncbaseTest.java
index ef35b74..55c4936 100644
--- a/lib/src/test/java/io/v/v23/syncbase/SyncbaseTest.java
+++ b/lib/src/test/java/io/v/v23/syncbase/SyncbaseTest.java
@@ -147,8 +147,8 @@
assertThat(sync(collection.get(ctx, "row2", String.class))).isEqualTo("value2");
assertThat(sync(InputChannels.asList(
collection.scan(ctx, RowRange.range("row1", "row3"))))).containsExactly(
- new KeyValue("row1", VomUtil.encode("value1", String.class)),
- new KeyValue("row2", VomUtil.encode("value2", String.class)));
+ new KeyValue("row1", (VdlAny) VomUtil.decode(VomUtil.encode("value1", String.class), VdlAny.class)),
+ new KeyValue("row2", (VdlAny) VomUtil.decode(VomUtil.encode("value2", String.class), VdlAny.class)));
sync(collection.deleteRange(ctx, RowRange.range("row1", "row3")));
assertThat(sync(collection.getRow("row1").exists(ctx))).isFalse();
assertThat(sync(collection.getRow("row2").exists(ctx))).isFalse();
@@ -369,8 +369,8 @@
// First try failed (wrote foo), second succeeded (bar commit succeeded).
assertThat(sync(InputChannels.asList(collection.scan(ctx, RowRange.prefix("")))))
.containsExactly(
- new KeyValue("bar-2", VomUtil.encode("bar", String.class)),
- new KeyValue("foo-1", VomUtil.encode("foo", String.class)));
+ new KeyValue("bar-2", (VdlAny) VomUtil.decode(VomUtil.encode("bar", String.class), VdlAny.class)),
+ new KeyValue("foo-1", (VdlAny) VomUtil.decode(VomUtil.encode("foo", String.class), VdlAny.class)));
}
public void testRunInBatchReadOnly() throws Exception {
@@ -409,7 +409,7 @@
// Single uncommitted iteration.
assertThat(sync(InputChannels.asList(collection.scan(ctx, RowRange.prefix("")))))
.containsExactly(
- new KeyValue("foo", VomUtil.encode("oof", String.class)));
+ new KeyValue("foo", (VdlAny) VomUtil.decode(VomUtil.encode("oof", String.class), VdlAny.class)));
}
public void testSyncgroup() throws Exception {
