tree 245ba6cd8b1e6a93ca0602401135c54044277c2a
parent e1057af1b627138fa6e659f57538c80f53f6d5b9
author Kaiyuan Wang <kaiyuanw@google.com> 1467844735 -0700
committer Kaiyuan Wang <kaiyuanw@google.com> 1467844735 -0700

feat(mdtest auto): introduce app-device coverage concept

Introduce app-device coverage concept.  We first cluster device specs
based on the app path, all specs with the same app path is clustered
together.  We also cluster devices based on model name, devices with
same model name is clustered together.  The clustering rule can be
easily changed.  In the current version, only the above clustering rule
is supported.  Some app-device coverage definition:

* Coverage Definition:
  If an app in an app cluster A is installed and run on a device in a
  device cluster D, then we say app cluster A covers device cluster D
  and the app-device path A-D is covered.

* Spec-Device Mapping Suite Definition:
  A spec-device mapping suite is defined as the union of all spec to
  device mappings for the same test script.  For example, a single
  application can be run on different types of devices.  In such case,
  the test script and spec never change but the spec-device mapping is
  changed so an application can be run on different devices.

* Full Coverage Definition:
  If a test script is repeatedly executed according to a spec-device
  mapping suite S, in all such runs, if every app cluster covers
  every device cluster, then we say the spec-device mapping suite S
  gives full app-device coverage.

* Coverage Score Definition:
  Suppose there are N app clusters, M device clusters, and a
  spec-device mapping suite S covers X app-device paths, then the
  coverage score is computed by C(S) = X / (N * M)

An algorithm is implemented to find the minimum set of spec-device
matches that gives the maximum app-device coverage.  A new command
"mdtest auto ..." is implemented to invoke the algorithm.

Change-Id: Idcc7e33daba4754d26e409d8701f8e3b857935de