aboutsummaryrefslogtreecommitdiff
path: root/doc/source/api/workload.rst
blob: 26ac4beca15f244c6db383fd70aea467ab696145 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
.. _workloads-api:

Workloads
^^^^^^^^^
.. _workload-api:

Workload
""""""""

The base :class:`Workload` interface is as follows, and is the base class for
all :ref:`workload types <workload-types>`. For more information about to
implement your own workload please see the
:ref:`Developer How Tos <adding-a-workload-example>`.

All instances of a workload will have the following attributes:

``name``
   This identifies the workload (e.g. it is used to specify the
   workload in the :ref:`agenda <agenda>`).

``phones_home``
    This can be set to True to mark that this workload poses a risk of
    exposing information to the outside world about the device it runs on.
    For example a benchmark application that sends scores and device data
    to a database owned by the maintainer.

``requires_network``
    Set this to ``True`` to mark the the workload will fail without a network
    connection, this enables it to fail early with a clear message.

``asset_directory``
    Set this to specify a custom directory for assets to be pushed to, if
    unset the working directory will be used.

``asset_files``
    This can be used to automatically deploy additional assets to
    the device. If required the attribute should contain a list of file
    names that are required by the workload which will be attempted to be
    found by the resource getters

methods
~~~~~~~

.. method:: Workload.init_resources(context)

    This method may be optionally overridden to implement dynamic
    resource discovery for the workload. This method executes
    early on, before the device has been initialized, so it
    should only be used to initialize resources that do not
    depend on the device to resolve. This method is executed
    once per run for each workload instance.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.validate(context)

    This method can be used to validate any assumptions your workload
    makes about the environment (e.g. that required files are
    present, environment variables are set, etc) and should raise a
    :class:`wa.WorkloadError <wa.framework.exception.WorkloadError>`
    if that is not the case. The base class implementation only makes
    sure sure that the name attribute has been set.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.initialize(context)

    This method is decorated with the ``@once_per_instance`` decorator,
    (for more information please see
    :ref:`Execution Decorators <execution-decorators>`)
    therefore it will be executed exactly once per run (no matter
    how many instances of the workload there are). It will run
    after the device has been initialized, so it may be used to
    perform device-dependent initialization that does not need to
    be repeated on each iteration (e.g. as installing executables
    required by the workload on the device).

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.setup(context)

    Everything that needs to be in place for workload execution should
    be done in this method. This includes copying files to the device,
    starting up an application, configuring communications channels,
    etc.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.setup_rerun(context)

    Everything that needs to be in place for workload execution should
    be done in this method. This includes copying files to the device,
    starting up an application, configuring communications channels,
    etc.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.run(context)

    This method should perform the actual task that is being measured.
    When this method exits, the task is assumed to be complete.

    :param context: The :ref:`Context <context>` for the current run.

    .. note:: Instruments are kicked off just before calling this
            method and disabled right after, so everything in this
            method is being measured. Therefore this method should
            contain the least code possible to perform the operations
            you are interested in measuring. Specifically, things like
            installing or starting applications, processing results, or
            copying files to/from the device should be done elsewhere if
            possible.



.. method:: Workload.extract_results(context)

    This method gets invoked after the task execution has finished and
    should be used to extract metrics from the target.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.update_output(context)

    This method should be used to update the output within the specified
    execution context with the metrics and artifacts from this
    workload iteration.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.teardown(context)

    This could be used to perform any cleanup you may wish to do, e.g.
    Uninstalling applications, deleting file on the device, etc.

    :param context: The :ref:`Context <context>` for the current run.


.. method:: Workload.finalize(context)

    This is the complement to ``initialize``. This will be executed
    exactly once at the end of the run. This should be used to
    perform any final clean up (e.g. uninstalling binaries installed
    in the ``initialize``)

    :param context: The :ref:`Context <context>` for the current run.

.. _apkworkload-api:

ApkWorkload
""""""""""""

The :class:`ApkWorkload` derives from the base :class:`Workload` class however
this associates the workload with a package allowing for an apk to be found for
the workload, setup and ran on the device before running the workload.

In addition to the attributes mentioned above ApkWorloads this class also
features the following attributes however this class does not present any new
methods.


``loading_time``
    This is the time in seconds that WA will wait for the application to load
    before continuing with the run. By default this will wait 10 second however
    if your application under test requires additional time this values should
    be increased.

``package_names``
    This attribute should be a list of Apk packages names that are
    suitable for this workload. Both the host (in the relevant resource
    locations) and device will be searched for an application with a matching
    package name.

``view``
    This is the "view" associated with the application. This is used by
    instruments like ``fps`` to monitor the current framerate being generated by
    the application.

``apk``
    The is a :class:`PackageHandler`` which is what is used to store
    information about the apk and  manage the application itself, the handler is
    used to call the associated methods to manipulate the application itself for
    example to launch/close it etc.

``package``
    This is a more convenient way to access the package name of the Apk
    that was found and being used for the run.


.. _apkuiautoworkload-api:

ApkUiautoWorkload
"""""""""""""""""

The :class:`ApkUiautoWorkload` derives from :class:`ApkUIWorkload` which is an
intermediate class which in turn inherits from
:class:`ApkWorkload`, however in addition to associating an apk with the
workload this class allows for automating the application with UiAutomator.

This class define these additional attributes:

``gui``
    This attribute will be an instance of a :class:`UiAutmatorGUI` which is
    used to control the automation, and is what is used to pass parameters to the
    java class for example ``gui.uiauto_params``.


.. _apkreventworkload-api:

ApkReventWorkload
"""""""""""""""""

The :class:`ApkReventWorkload` derives from :class:`ApkUIWorkload` which is an
intermediate class which in turn inherits from
:class:`ApkWorkload`, however in addition to associating an apk with the
workload this class allows for automating the application with
:ref:`Revent <revent_files_creation>`.

This class define these additional attributes:

``gui``
    This attribute will be an instance of a :class:`ReventGUI` which is
    used to control the automation

``setup_timeout``
    This is the time allowed for replaying a recording for the setup stage.

``run_timeout``
    This is the time allowed for replaying a recording for the run stage.

``extract_results_timeout``
    This is the time allowed for replaying a recording for the extract results stage.

``teardown_timeout``
    This is the time allowed for replaying a recording for the teardown stage.


.. _uiautoworkload-api:

UiautoWorkload
""""""""""""""

The :class:`UiautoWorkload` derives from :class:`UIWorkload` which is an
intermediate class which in turn inherits from
:class:`Workload`, however this allows for providing generic automation using
UiAutomator without associating a particular application with the workload.

This class define these additional attributes:

``gui``
    This attribute will be an instance of a :class:`UiAutmatorGUI` which is
    used to control the automation, and is what is used to pass parameters to the
    java class for example ``gui.uiauto_params``.


.. _reventworkload-api:

ReventWorkload
""""""""""""""

The :class:`ReventWorkload` derives from :class:`UIWorkload` which is an
intermediate class which in turn inherits from
:class:`Workload`, however this allows for providing generic automation
using :ref:`Revent <revent_files_creation>` without associating with the
workload.

This class define these additional attributes:

``gui``
    This attribute will be an instance of a :class:`ReventGUI` which is
    used to control the automation

``setup_timeout``
    This is the time allowed for replaying a recording for the setup stage.

``run_timeout``
    This is the time allowed for replaying a recording for the run stage.

``extract_results_timeout``
    This is the time allowed for replaying a recording for the extract results stage.

``teardown_timeout``
    This is the time allowed for replaying a recording for the teardown stage.