doc: Reorganise Developer Guide/Reference

5 files changed, 287 insertions, 287 deletions

diff --git a/doc/Makefile b/doc/Makefile
index 994c75a1..f22a1c08 100644
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -49,7 +49,7 @@ help:
 clean:
 	rm -rf $(BUILDDIR)/*
 	rm -rf source/plugins/*
-	rm -rf source/developer_reference/instrument_method_map.rst
+	rm -rf source/developer_guide/instrument_method_map.rst
 	rm -rf source/run_config/*
 
 coverage:
diff --git a/doc/source/api/workload.rst b/doc/source/api/workload.rst
index a9be30ff..26ac4bec 100644
--- a/doc/source/api/workload.rst
+++ b/doc/source/api/workload.rst
@@ -10,7 +10,7 @@ Workload
 The base :class:`Workload` interface is as follows, and is the base class for
 all :ref:`workload types <workload-types>`. For more information about to
 implement your own workload please see the
-:ref:`Developer How Tos <adding-a-workload>`.
+:ref:`Developer How Tos <adding-a-workload-example>`.
 
 All instances of a workload will have the following attributes:
 
diff --git a/doc/source/conf.py b/doc/source/conf.py
index 5cc42553..803f3a1d 100644
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -314,7 +314,7 @@ def setup(app):
     generate_target_documentation('plugins')
     generate_run_config_documentation('run_config')
     generate_meta_config_documentation('run_config')
-    generate_instrument_method_map(os.path.join('developer_information', 'developer_reference',
+    generate_instrument_method_map(os.path.join('developer_information', 'developer_guide',
                                                 'instrument_method_map.rst'))
     app.add_object_type('confval', 'confval',
                         objname='configuration value',
diff --git a/doc/source/developer_information/developer_guide/writing_plugins.rst b/doc/source/developer_information/developer_guide/writing_plugins.rst
index 3d33f947..57451b08 100644
--- a/doc/source/developer_information/developer_guide/writing_plugins.rst
+++ b/doc/source/developer_information/developer_guide/writing_plugins.rst
@@ -34,129 +34,6 @@ This sub-section covers things common to implementing plugins of all types. It
 is recommended you familiarize yourself with the information here before
 proceeding onto guidance for specific plugin types.
 
-.. _context:
-
-The Context
-^^^^^^^^^^^
-
-The majority of methods in plugins accept a context argument. This is an
-instance of :class:`wa.framework.execution.ExecutionContext`. It contains
-information about the current state of execution of WA and keeps track of things
-like which workload is currently running.
-
-Notable methods of the context are:
-
-:context.get_resource(resource, strict=True):
-       This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
-
-:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
-      Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
-      output directory for WA and associate them with a description and/or
-      :ref:`classifier <classifiers>`.
-
-:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
-        This method should be used to add :ref:`metrics <metrics>` that have been
-        generated from a workload, this will allow WA to process the results
-        accordingly depending on which output processors are enabled.
-
-Notable attributes of the context are:
-
-:context.workload:
-        :class:`wa.framework.workload` object that is currently being executed.
-
-:context.tm:
-        This is the target manager that can be used to access various information
-        about the target including initialization parameters.
-
-:context.current_job:
-        This is an instance of :class:`wa.framework.job.Job` and contains all
-        the information relevant to the workload job currently being executed.
-
-:context.current_job.spec:
-        The current workload specification being executed. This is an
-        instance of :class:`wa.framework.configuration.core.JobSpec`
-        and defines the workload and the parameters under which it is
-        being executed.
-
-:context.current_job.current_iteration:
-        The current iteration of the spec that is being executed. Note that this
-        is the iteration for that spec, i.e. the number of times that spec has
-        been run, *not* the total number of all iterations have been executed so
-        far.
-
-:context.job_output:
-        This is the output object for the current iteration which
-        is an instance of :class:`wa.framework.output.JobOutput`. It contains
-        the status of the iteration as well as the metrics and artifacts
-        generated by the workload.
-
-
-In addition to these, context also defines a few useful paths (see below).
-
-
-Paths
-^^^^^
-
-You should avoid using hard-coded absolute paths in your plugins whenever
-possible, as they make your code too dependent on a particular environment and
-may mean having to make adjustments when moving to new (host and/or device)
-platforms. To help avoid hard-coded absolute paths, WA defines a number of
-standard locations. You should strive to define your paths relative
-to one of these.
-
-On the host
-~~~~~~~~~~~
-
-Host paths are available through the context object, which is passed to most
-plugin methods.
-
-context.run_output_directory
-        This is the top-level output directory for all WA results (by default,
-        this will be "wa_output" in the directory in which WA was invoked.
-
-context.output_directory
-        This is the output directory for the current iteration. This will an
-        iteration-specific subdirectory under the main results location. If
-        there is no current iteration (e.g. when processing overall run results)
-        this will point to the same location as ``root_output_directory``.
-
-
-Additionally, the global ``wa.settings`` object exposes on other location:
-
-settings.dependency_directory
-        this is the root directory for all plugin dependencies (e.g. media
-        files, assets etc) that are not included within the plugin itself.
-
-As per Python best practice, it is recommended that methods and values in
-``os.path`` standard library module are used for host path manipulation.
-
-On the target
-~~~~~~~~~~~~~
-
-Workloads and instruments have a ``target`` attribute, which is an interface to
-the target used by WA. It defines the following location:
-
-target.working_directory
-        This is the directory for all WA-related files on the target. All files
-        deployed to the target should be pushed to somewhere under this location
-        (the only exception being executables installed with ``target.install``
-        method).
-
-Since there could be a mismatch between path notation used by the host and the
-target, the ``os.path`` modules should *not* be used for on-target path
-manipulation. Instead target has an equipment module exposed through
-``target.path`` attribute. This has all the same attributes and behaves the
-same way as ``os.path``, but is guaranteed to produce valid paths for the target,
-irrespective of the host's path notation. For example:
-
-.. code:: python
-
-    result_file = self.target.path.join(self.target.working_directory, "result.txt")
-    self.command = "{} -a -b -c {}".format(target_binary, result_file)
-
-.. note:: Output processors, unlike workloads and instruments, do not have their
-          own target attribute as they are designed to be able to be ran offline.
-
 .. _resource-resolution:
 
 Dynamic Resource Resolution
@@ -271,166 +148,6 @@ additional assets should have their on target paths added to the workload's
 ``deployed_assests`` attribute or the corresponding ``remove_assets`` method
 should also be implemented.
 
-.. _plugin-parmeters:
-
-Parameters
-----------
-
-All plugins can be parametrized. Parameters are specified using
-``parameters`` class attribute. This should be a list of
-:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
-specified on parameter creation:
-
-:name:
-        This is the only mandatory argument. The name will be used to create a
-        corresponding attribute in the plugin instance, so it must be a valid
-        Python identifier.
-
-:kind:
-        This is the type of the value of the parameter. This must be an
-        callable. Normally this should be a standard Python type, e.g. ``int``
-        or ``float``, or one the types defined in :mod:`wa.utils.types`.
-        If not explicitly specified, this will default to ``str``.
-
-        .. note:: Irrespective of the ``kind`` specified, ``None`` is always a
-                  valid value for a parameter. If you don't want to allow
-                  ``None``, then set ``mandatory`` (see below) to ``True``.
-
-:allowed_values:
-        A list of the only allowed values for this parameter.
-
-        .. note:: For composite types, such as ``list_of_strings`` or
-                  ``list_of_ints`` in :mod:`wa.utils.types`, each element of
-                  the value  will be checked against ``allowed_values`` rather
-                  than the composite value itself.
-
-:default:
-        The default value to be used for this parameter if one has not been
-        specified by the user. Defaults to ``None``.
-
-:mandatory:
-        A ``bool`` indicating whether this parameter is mandatory. Setting this
-        to ``True`` will make ``None`` an illegal value for the parameter.
-        Defaults to ``False``.
-
-        .. note:: Specifying a ``default`` will mean that this parameter will,
-                  effectively, be ignored (unless the user sets the param to ``None``).
-
-        .. note:: Mandatory parameters are *bad*. If at all possible, you should
-                  strive to provide a sensible ``default`` or to make do without
-                  the parameter. Only when the param is absolutely necessary,
-                  and there really is no sensible default that could be given
-                  (e.g. something like login credentials), should you consider
-                  making it mandatory.
-
-:constraint:
-        This is an additional constraint to be enforced on the parameter beyond
-        its type or fixed allowed values set. This should be a predicate (a function
-        that takes a single argument -- the user-supplied value -- and returns
-        a ``bool`` indicating whether the constraint has been satisfied).
-
-:override:
-        A parameter name must be unique not only within an plugin but also
-        with that plugin's class hierarchy. If you try to declare a parameter
-        with the same name as already exists, you will get an error. If you do
-        want to override a parameter from further up in the inheritance
-        hierarchy, you can indicate that by setting ``override`` attribute to
-        ``True``.
-
-        When overriding, you do not need to specify every other attribute of the
-        parameter, just the ones you what to override. Values for the rest will
-        be taken from the parameter in the base class.
-
-
-Validation and cross-parameter constraints
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-A plugin will get validated at some point after construction. When exactly
-this occurs depends on the plugin type, but it *will* be validated before it
-is used.
-
-You can implement ``validate`` method in your plugin (that takes no arguments
-beyond the ``self``) to perform any additional *internal* validation in your
-plugin. By "internal", I mean that you cannot make assumptions about the
-surrounding environment (e.g. that the device has been initialized).
-
-The contract for ``validate`` method is that it should raise an exception
-(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
-further on this page) if some validation condition has not, and cannot, been met.
-If the method returns without raising an exception, then the plugin is in a
-valid internal state.
-
-Note that ``validate`` can be used not only to verify, but also to impose a
-valid internal state. In particular, this where cross-parameter constraints can
-be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
-another parameter, there is no way to express that declaratively when specifying
-the parameters. In that case the dependent attribute should be left unspecified
-on creation and should instead be set inside ``validate``.
-
-Logging
--------
-
-Every plugin class has it's own logger that you can access through
-``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
-log everything it is doing, so you shouldn't need to add much additional logging
-for device actions. However you might what to log additional information,  e.g.
-what settings your plugin is using, what it is doing on the host, etc.
-(Operations on the host will not normally be logged, so your plugin should
-definitely log what it is doing on the host). One situation in particular where
-you should add logging is before doing something that might take a significant
-amount of time, such as downloading a file.
-
-
-Documenting
------------
-
-All plugins and their parameter should be documented. For plugins
-themselves, this is done through ``description`` class attribute. The convention
-for an plugin description is that the first paragraph should be a short
-summary description of what the plugin does and why one would want to use it
-(among other things, this will get extracted and used by ``wa list`` command).
-Subsequent paragraphs (separated by blank lines) can then provide  a more
-detailed description, including any limitations and setup instructions.
-
-For parameters, the description is passed as an argument on creation. Please
-note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
-parameter, they do not need to be explicitly mentioned in the description (wa
-documentation utilities will automatically pull those). If the ``default`` is set
-in ``validate`` or additional cross-parameter constraints exist, this *should*
-be documented in the parameter description.
-
-Both plugins and their parameters should be documented using reStructureText
-markup (standard markup for Python documentation). See:
-
-http://docutils.sourceforge.net/rst.html
-
-Aside from that, it is up to you how you document your plugin. You should try
-to provide enough information so that someone unfamiliar with your plugin is
-able to use it, e.g. you should document all settings and parameters your
-plugin expects (including what the valid values are).
-
-
-Error Notification
-------------------
-
-When you detect an error condition, you should raise an appropriate exception to
-notify the user. The exception would typically be :class:`ConfigError` or
-(depending the type of the plugin)
-:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
-All these errors are defined in :mod:`wa.framework.exception` module.
-
-A :class:`ConfigError` should be raised where there is a problem in configuration
-specified by the user (either through the agenda or config files). These errors
-are meant to be resolvable by simple adjustments to the configuration (and the
-error message should suggest what adjustments need to be made. For all other
-errors, such as missing dependencies, mis-configured environment, problems
-performing operations, etc., the plugin type-specific exceptions should be
-used.
-
-If the plugin itself is capable of recovering from the error and carrying
-on, it may make more sense to log an ERROR or WARNING level message using the
-plugin's logger and to continue operation.
-
 .. _instrument-reference:
 
 Adding an Instrument
@@ -595,7 +312,7 @@ Below is a simple instrument that measures the execution time of a workload::
             context.add_metric('execution_time', execution_time, 'seconds')
 
 
-.. include:: developer_information/developer_reference/instrument_method_map.rst
+.. include:: developer_information/developer_guide/instrument_method_map.rst
 
 .. _adding-an-output-processor:
 
diff --git a/doc/source/developer_information/developer_reference/plugins.rst b/doc/source/developer_information/developer_reference/plugins.rst
index bb1e11ac..7c342d0d 100644
--- a/doc/source/developer_information/developer_reference/plugins.rst
+++ b/doc/source/developer_information/developer_reference/plugins.rst
@@ -32,6 +32,289 @@ Plugin Basics
 
 This section contains reference information common to plugins of all types.
 
+.. _context:
+
+The Context
+^^^^^^^^^^^
+
+The majority of methods in plugins accept a context argument. This is an
+instance of :class:`wa.framework.execution.ExecutionContext`. It contains
+information about the current state of execution of WA and keeps track of things
+like which workload is currently running.
+
+Notable methods of the context are:
+
+:context.get_resource(resource, strict=True):
+       This method should be used to retrieve a resource using the resource getters rather than using the ResourceResolver directly as this method additionally record any found resources hash in the output metadata.
+
+:context.add_artifact(name, host_file_path, kind, description=None, classifier=None):
+      Plugins can add :ref:`artifacts <artifact>` of various kinds to the run
+      output directory for WA and associate them with a description and/or
+      :ref:`classifier <classifiers>`.
+
+:context.add_metric(name, value, units=None, lower_is_better=False, classifiers=None):
+        This method should be used to add :ref:`metrics <metrics>` that have been
+        generated from a workload, this will allow WA to process the results
+        accordingly depending on which output processors are enabled.
+
+Notable attributes of the context are:
+
+:context.workload:
+        :class:`wa.framework.workload` object that is currently being executed.
+
+:context.tm:
+        This is the target manager that can be used to access various information
+        about the target including initialization parameters.
+
+:context.current_job:
+        This is an instance of :class:`wa.framework.job.Job` and contains all
+        the information relevant to the workload job currently being executed.
+
+:context.current_job.spec:
+        The current workload specification being executed. This is an
+        instance of :class:`wa.framework.configuration.core.JobSpec`
+        and defines the workload and the parameters under which it is
+        being executed.
+
+:context.current_job.current_iteration:
+        The current iteration of the spec that is being executed. Note that this
+        is the iteration for that spec, i.e. the number of times that spec has
+        been run, *not* the total number of all iterations have been executed so
+        far.
+
+:context.job_output:
+        This is the output object for the current iteration which
+        is an instance of :class:`wa.framework.output.JobOutput`. It contains
+        the status of the iteration as well as the metrics and artifacts
+        generated by the workload.
+
+
+In addition to these, context also defines a few useful paths (see below).
+
+
+Paths
+^^^^^
+
+You should avoid using hard-coded absolute paths in your plugins whenever
+possible, as they make your code too dependent on a particular environment and
+may mean having to make adjustments when moving to new (host and/or device)
+platforms. To help avoid hard-coded absolute paths, WA defines a number of
+standard locations. You should strive to define your paths relative
+to one of these.
+
+On the host
+~~~~~~~~~~~
+
+Host paths are available through the context object, which is passed to most
+plugin methods.
+
+context.run_output_directory
+        This is the top-level output directory for all WA results (by default,
+        this will be "wa_output" in the directory in which WA was invoked.
+
+context.output_directory
+        This is the output directory for the current iteration. This will an
+        iteration-specific subdirectory under the main results location. If
+        there is no current iteration (e.g. when processing overall run results)
+        this will point to the same location as ``root_output_directory``.
+
+
+Additionally, the global ``wa.settings`` object exposes on other location:
+
+settings.dependency_directory
+        this is the root directory for all plugin dependencies (e.g. media
+        files, assets etc) that are not included within the plugin itself.
+
+As per Python best practice, it is recommended that methods and values in
+``os.path`` standard library module are used for host path manipulation.
+
+On the target
+~~~~~~~~~~~~~
+
+Workloads and instruments have a ``target`` attribute, which is an interface to
+the target used by WA. It defines the following location:
+
+target.working_directory
+        This is the directory for all WA-related files on the target. All files
+        deployed to the target should be pushed to somewhere under this location
+        (the only exception being executables installed with ``target.install``
+        method).
+
+Since there could be a mismatch between path notation used by the host and the
+target, the ``os.path`` modules should *not* be used for on-target path
+manipulation. Instead target has an equipment module exposed through
+``target.path`` attribute. This has all the same attributes and behaves the
+same way as ``os.path``, but is guaranteed to produce valid paths for the target,
+irrespective of the host's path notation. For example:
+
+.. code:: python
+
+    result_file = self.target.path.join(self.target.working_directory, "result.txt")
+    self.command = "{} -a -b -c {}".format(target_binary, result_file)
+
+.. note:: Output processors, unlike workloads and instruments, do not have their
+          own target attribute as they are designed to be able to be ran offline.
+
+.. _plugin-parmeters:
+
+Parameters
+^^^^^^^^^^^
+
+All plugins can be parametrized. Parameters are specified using
+``parameters`` class attribute. This should be a list of
+:class:`wa.framework.plugin.Parameter` instances. The following attributes can be
+specified on parameter creation:
+
+:name:
+        This is the only mandatory argument. The name will be used to create a
+        corresponding attribute in the plugin instance, so it must be a valid
+        Python identifier.
+
+:kind:
+        This is the type of the value of the parameter. This must be an
+        callable. Normally this should be a standard Python type, e.g. ``int``
+        or ``float``, or one the types defined in :mod:`wa.utils.types`.
+        If not explicitly specified, this will default to ``str``.
+
+        .. note:: Irrespective of the ``kind`` specified, ``None`` is always a
+                  valid value for a parameter. If you don't want to allow
+                  ``None``, then set ``mandatory`` (see below) to ``True``.
+
+:allowed_values:
+        A list of the only allowed values for this parameter.
+
+        .. note:: For composite types, such as ``list_of_strings`` or
+                  ``list_of_ints`` in :mod:`wa.utils.types`, each element of
+                  the value  will be checked against ``allowed_values`` rather
+                  than the composite value itself.
+
+:default:
+        The default value to be used for this parameter if one has not been
+        specified by the user. Defaults to ``None``.
+
+:mandatory:
+        A ``bool`` indicating whether this parameter is mandatory. Setting this
+        to ``True`` will make ``None`` an illegal value for the parameter.
+        Defaults to ``False``.
+
+        .. note:: Specifying a ``default`` will mean that this parameter will,
+                  effectively, be ignored (unless the user sets the param to ``None``).
+
+        .. note:: Mandatory parameters are *bad*. If at all possible, you should
+                  strive to provide a sensible ``default`` or to make do without
+                  the parameter. Only when the param is absolutely necessary,
+                  and there really is no sensible default that could be given
+                  (e.g. something like login credentials), should you consider
+                  making it mandatory.
+
+:constraint:
+        This is an additional constraint to be enforced on the parameter beyond
+        its type or fixed allowed values set. This should be a predicate (a function
+        that takes a single argument -- the user-supplied value -- and returns
+        a ``bool`` indicating whether the constraint has been satisfied).
+
+:override:
+        A parameter name must be unique not only within an plugin but also
+        with that plugin's class hierarchy. If you try to declare a parameter
+        with the same name as already exists, you will get an error. If you do
+        want to override a parameter from further up in the inheritance
+        hierarchy, you can indicate that by setting ``override`` attribute to
+        ``True``.
+
+        When overriding, you do not need to specify every other attribute of the
+        parameter, just the ones you what to override. Values for the rest will
+        be taken from the parameter in the base class.
+
+
+Validation and cross-parameter constraints
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A plugin will get validated at some point after construction. When exactly
+this occurs depends on the plugin type, but it *will* be validated before it
+is used.
+
+You can implement ``validate`` method in your plugin (that takes no arguments
+beyond the ``self``) to perform any additional *internal* validation in your
+plugin. By "internal", I mean that you cannot make assumptions about the
+surrounding environment (e.g. that the device has been initialized).
+
+The contract for ``validate`` method is that it should raise an exception
+(either ``wa.framework.exception.ConfigError`` or plugin-specific exception type -- see
+further on this page) if some validation condition has not, and cannot, been met.
+If the method returns without raising an exception, then the plugin is in a
+valid internal state.
+
+Note that ``validate`` can be used not only to verify, but also to impose a
+valid internal state. In particular, this where cross-parameter constraints can
+be resolved. If the ``default`` or ``allowed_values`` of one parameter depend on
+another parameter, there is no way to express that declaratively when specifying
+the parameters. In that case the dependent attribute should be left unspecified
+on creation and should instead be set inside ``validate``.
+
+Logging
+^^^^^^^
+
+Every plugin class has it's own logger that you can access through
+``self.logger`` inside the plugin's methods. Generally, a :class:`Target` will
+log everything it is doing, so you shouldn't need to add much additional logging
+for device actions. However you might what to log additional information,  e.g.
+what settings your plugin is using, what it is doing on the host, etc.
+(Operations on the host will not normally be logged, so your plugin should
+definitely log what it is doing on the host). One situation in particular where
+you should add logging is before doing something that might take a significant
+amount of time, such as downloading a file.
+
+
+Documenting
+^^^^^^^^^^^
+
+All plugins and their parameter should be documented. For plugins
+themselves, this is done through ``description`` class attribute. The convention
+for an plugin description is that the first paragraph should be a short
+summary description of what the plugin does and why one would want to use it
+(among other things, this will get extracted and used by ``wa list`` command).
+Subsequent paragraphs (separated by blank lines) can then provide  a more
+detailed description, including any limitations and setup instructions.
+
+For parameters, the description is passed as an argument on creation. Please
+note that if ``default``, ``allowed_values``, or ``constraint``, are set in the
+parameter, they do not need to be explicitly mentioned in the description (wa
+documentation utilities will automatically pull those). If the ``default`` is set
+in ``validate`` or additional cross-parameter constraints exist, this *should*
+be documented in the parameter description.
+
+Both plugins and their parameters should be documented using reStructureText
+markup (standard markup for Python documentation). See:
+
+http://docutils.sourceforge.net/rst.html
+
+Aside from that, it is up to you how you document your plugin. You should try
+to provide enough information so that someone unfamiliar with your plugin is
+able to use it, e.g. you should document all settings and parameters your
+plugin expects (including what the valid values are).
+
+
+Error Notification
+^^^^^^^^^^^^^^^^^^
+
+When you detect an error condition, you should raise an appropriate exception to
+notify the user. The exception would typically be :class:`ConfigError` or
+(depending the type of the plugin)
+:class:`WorkloadError`/:class:`DeviceError`/:class:`InstrumentError`/:class:`OutputProcessorError`.
+All these errors are defined in :mod:`wa.framework.exception` module.
+
+A :class:`ConfigError` should be raised where there is a problem in configuration
+specified by the user (either through the agenda or config files). These errors
+are meant to be resolvable by simple adjustments to the configuration (and the
+error message should suggest what adjustments need to be made. For all other
+errors, such as missing dependencies, mis-configured environment, problems
+performing operations, etc., the plugin type-specific exceptions should be
+used.
+
+If the plugin itself is capable of recovering from the error and carrying
+on, it may make more sense to log an ERROR or WARNING level message using the
+plugin's logger and to continue operation.
+
 .. _metrics:
 
 Metrics