=== modified file 'doc/index.rst'
@@ -4,25 +4,12 @@
LAVA Dispatcher Documentation
=============================
-LAVA Dispatcher is to dispatch test jobs from server(master node) to the target
+LAVA Dispatcher is used to dispatch test jobs from server(master node) to the target
boards in validation farm, and publish the test result back to dashboard. It is
scheduled by validation scheduler, and it could also run as standalone.
.. seealso:: To learn more about LAVA see https://launchpad.net/lava
-60 second example
-=================
-
-This example will run on Ubuntu Lucid and beyond::
-
- $ sudo add-apt-repository ppa:linaro-validation/ppa
- $ sudo apt-get update
- $ sudo apt-get install lava-dispatcher
- $ sudo lava-dispatch ./lava-ltp-job.json
- (lava-ltp-job.json can be found in lava-dispatcher/doc)
-
-.. seealso:: For detailed installation instructions see :ref:`installation`
-.. seealso:: For writing a new dispatcher job file see :ref:`jobfile`
Features
========
@@ -31,27 +18,24 @@
and test cases, then upload test result to LAVA Dashboard on an ARM target
system.
* Support ARM target boards including Beagle, Panda, i.MX51 EVK, i.MX53
- QuickStart and Snowball, more boards support is coming.
-* Support Android system on Beagle, Panda and i.MX53 QuickStart board, more
- boards support is coming.
+ QuickStart, Snowball, Origen, Versatile Express, Fast Models, and QEMU.
+* Support Android system on Panda, i.MX53 QuickStart board, Snowball, Origen,
+ Versatile Express, and Fast Models.
* Support for local user-defined configuration data for boards, device types.
* Extensible device types and boards configuration editing, can add new device
and new board.
* Make use of the output of LAVA test, which is Linaro Dashboard Bundle format,
upload test results to the LAVA Dashboard for result archiving and analysis.
-.. seealso:: See what's new in :ref:`version_0_6`
-
-.. todo::
-
- Add inline document to source code and open code reference in doc
-
-Latest documentation
-====================
-
-This documentation may be out of date, we try to make sure that all the latest
-and greatest releases are always documented on
-http://lava-dispatcher.readthedocs.org/
+Installation
+============
+
+The best way to install this is by doing a full deployment of LAVA. This is
+documented on our `main project page`_. However, you can also setup the
+dispatcher for `stand-alone development and testing`_.
+
+.. _main project page: http://lava.readthedocs.org/en/latest/
+.. _stand-alone development and testing: standalonesetup.html
Source code, bugs and patches
=============================
@@ -75,7 +59,7 @@
.. toctree::
:maxdepth: 2
- installation.rst
+ standalonesetup.rst
configuration.rst
proxy.rst
jobfile.rst
=== removed file 'doc/installation.rst'
@@ -1,236 +0,0 @@
-.. _installation:
-
-Installation
-============
-
-Prerequisites
-^^^^^^^^^^^^^
-
-The following debian packages are needed to use LAVA Dispatcher:
-
-* cu - for serial port control
-* conmux - for console management
-* python >= 2.6
-* python-testrepository - for running unit tests
-* python-sphinx - for building documentation
-
-The following python module are needed to use LAVA Dispatcher:
-
-* pexpect
-
-A SD/MMC card containing a 'master image'.
-
-Creating a SD/MMC card with master image
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-You will need to have a card containing a 'master image' for your board. The
-process of creating a master image is outlined on
-https://wiki.linaro.org/Platform/Validation/Specs/MasterBootImage.
-
-The master image is a stable, default image the system can boot into for
-deployment and recovery. It's a simple modification of an existing stable
-image, and does not take long to create. Once you've created one for your
-board, you may want to consider making a dd image of it for future use. We do
-not provide dd images for this already. Due to differences in size of SD cards,
-it's likely that doing so would either waste a lot of space if you have a
-larger card, or be competely useless if you have a smaller sd card.
-
-Setup the image and test partitions
------------------------------------
-
-* Start with a stable Linaro image from the current, or previous cycle. For
- instance, the nano, or headless images make a good starting place. Using
- linaro-media-create, create an sdcard with this image, and the hardware pack
- for your board. (minimum 8GB sd card is recommended, but 4GB can work also).
-* Using parted (or gparted, etc), shrink the root partition so that only
- 1-1.5GB or so is used (Make sure to leave at least 2G free on the card).
-* Create an additional primary partition (should normally be partition 3), of
- about 70MB. Format it as vfat and give it the label testboot.
-* Create partition 4 as an Extended partition, using the remainder of the space
- on the card.
-* Create partition 5 (logical partition) using 2GB. Format it as ext3, and give
- it the label testrootfs
-* Finally, create one more logical partition using the remainder of the space.
- It should be vfat formatted, and have the label sdcard (NB: This is only
- needed if you want to test android images. If you don't wish to test android,
- this can be skipped)
-
-Prepare the master image
-------------------------
-
-* Boot the image created above on your test board, and attach to the serial
- console
-* Ensure that networking is set up so that the default network interface is
- automatically started. Generally you will want to use dhcp for this. For
- example, if your network interface is eth0, make sure that
- /etc/network/interfaces contains something like this::
-
- auto usb0
- iface usb0 inet dhcp
-
-* Edit /etc/hostname and /etc/hosts to change the hostname to "master". This
- will allow easy detection of which image we are booted into.
-* Install the following packages:
- * wget
- * dosfstools
-* If you want to test your own build kernel by a deb package(see following
- example job file for detail), you need to add linaro-media-tools support in
- master image::
-
- # apt-get install bzr python-distutils-extra python-testtools python-parted command-not-found python-yaml python-beautifulsoup python-wxgtk2.6
- # bzr branch lp:linaro-media-tools
- # cd linaro-media-tools
- # ./setup.py install
-
-Setting up serial access
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-To talk to the target test systems, the dispatcher uses a tool called conmux.
-If you are running Ubuntu 11.04 or later, conmux is in the archives and can be
-easily installed [TODO: point to ppa for conmux installation -lt 11.04].
-
-Configuring cu and conmux
--------------------------
-
-For LAVA development and testing using only locally attached resources, you
-should be able to make use of most features, even without the use of special
-equipment such as a console server.
-
-First install conmux and cu::
-
- sudo add-apt-repository ppa:linaro-maintainers/tools
- sudo apt-get update
- sudo apt-get install conmux cu
-
-Configuring cu
---------------
-
-You should be able to type commands and interact with the shell inside
-conmux-console. If you cannot, run "sudo stop conmux" and try running
-'sg dialout "cu -l ttyUSB0 -s 115200"'. If that doesn't work, you
-probably need to add some files to /etc/uucp. Add ::
-
- port ttyUSB0
- type direct
- device /dev/ttyUSB0
- hardflow false
- speed 115200
-
-to /etc/uucp/port and append ::
-
- system panda01
- port ttyUSB0
- time any
-
-to /etc/uucp/sys.
-
-Configuring conmux
-------------------
-
-Connect a development board to a local serial device (e.g. ttyUSB0). You may
-have permission problem with cu running as root under conmux.
-
-Configuration files for conmux are stored under /etc/conmux. It can be
-configured for either local connections (via serial or usb), or remote
-configurations such as console servers. Configurations for each board you wish
-to connect to should be stored in it's own .cf file under /etc/conmux.
-
-Create a configuration file for your board under /etc/conmux which should look
-something like this::
-
- listener panda01
- application console 'panda01 console' 'cu -l /dev/ttyUSB0 -s 115200'
-
-Make sure to give the file a '.cf' extension (e.g. panda01.cf).
-
-If you see this permission problem when running cu, you can try
-adjusting your .cf file to call cu using sg, and the group name owning
-the device. For example::
-
- sg dialout "cu -l ttyUSB0 -s 115200"
-
-Finally restart conmux::
-
- sudo stop conmux
- sudo start conmux
-
-You can test the connection using::
-
- conmux-console panda01
- (use ~$quit to exit)
-
-Another example config, a remote console server on 10.1.1.1 port 7777 attached
-to a board we will call beagle01::
-
- listener beagle01
- socket console 'beagle01 console' '10.1.1.1:7777'
-
-.. seealso::
-
- If you are using a snowball with serial USB, then you'll need to follow
- `this guide <https://wiki.linaro.org/Platform/Validation/LAVA/Documentation/GettingSnowballWorking>`_
-
-Installation Options
-^^^^^^^^^^^^^^^^^^^^
-
-There are several installation options available:
-
-Using pip
----------
-
-To install from pip::
-
- pip install lava-dispatcher
-
-To upgrade from pip::
-
- pip install --upgrade lava-dispatcher
-
-Using lava-deployment-tool
---------------------------
-
-To install from lava-deployment-tool, first checkout lava-deployment-tool::
-
- bzr branch lp:lava-deployment-tool
-
-Refer to README in lava-deployment-tool, make sure in "./lava-deployment-tool
-bundle" commands, requirements.txt includes lava-dispatcher.
-
-lava-dispatcher can be found in /srv/lava/instances/$LAVA_INSTANCE/bin.
-
-To use lava-dispatcher, activate virtualenv::
-
- cd /srv/lava/instances/$LAVA_INSTANCE
- . bin/activate
-
-Using source tarball
---------------------
-
-To install from source you must first obtain a source tarball from bazzar
-branch or from `Launchpad <https://launchpad.net/lava-dispatcher/+download>`_::
-
- bzr branch lp:lava-dispatcher
-
-To install the package unpack the tarball and run::
-
- sudo python setup.py install
-
-
-Using Ubuntu PPAs
------------------
-
-For Ubuntu 10.04 onward there is a stable PPA (personal package archive):
-
-* ppa:linaro-validation/ppa
-
-To add a ppa to an Ubuntu system use the add-apt-repository command::
-
- sudo add-apt-repository ppa:linaro-validation/ppa
-
-After you add the PPA you need to update your package cache::
-
- sudo apt-get update
-
-Finally you can install the package, it is called `lava-dispatcher`::
-
- sudo apt-get install lava-dispatcher
=== added file 'doc/jobfile-android.rst'
@@ -0,0 +1,66 @@
+Deploy Android
+==============
+
+Here's an example of a job file that will deploy and boot an Android image::
+
+ {
+ "job_name": "android_test",
+ "target": "panda01",
+ "timeout": 18000,
+ "actions": [
+ {
+ "command": "deploy_linaro_android_image",
+ "parameters":
+ {
+ "boot": "http://releases.linaro.org/12.09/android/leb-panda/boot.tar.bz2",
+ "system": "http://releases.linaro.org/12.09/android/leb-panda/system.tar.bz2",
+ "data": "http://releases.linaro.org/12.09/android/leb-panda/userdata.tar.bz2"
+ }
+ },
+ {
+ "command": "boot_linaro_android_image"
+ },
+ {
+ "command": "lava_android_test_install",
+ "parameters":
+ {
+ "tests": ["0xbench"]
+ }
+ },
+ {
+ "command": "lava_android_test_run",
+ "parameters":
+ {
+ "test_name": "0xbench"
+ }
+ },
+ {
+ "command": "submit_results_on_host",
+ "parameters":
+ {
+ "server": "http://validation.linaro.org/lava-server/RPC2/",
+ "stream": "/anonymous/lava-android-leb-panda/"
+ }
+ }
+ ]
+ }
+
+Installing Panda Binary Blobs
+-----------------------------
+
+Some Android builds for Panda require a binary blob to be installed. This can
+be done by adding the ``android_install_binaries`` after the
+``deploy_linaro_android_image``::
+
+ {
+ "command": "deploy_linaro_android_image",
+ "parameters":
+ {
+ "boot": "http://releases.linaro.org/12.09/android/leb-panda/boot.tar.bz2",
+ "system": "http://releases.linaro.org/12.09/android/leb-panda/system.tar.bz2",
+ "data": "http://releases.linaro.org/12.09/android/leb-panda/userdata.tar.bz2"
+ }
+ },
+ {
+ "command": "android_install_binaries"
+ }
=== added file 'doc/jobfile-lmc.rst'
@@ -0,0 +1,33 @@
+Deploy Using linaro-media-create
+================================
+
+Here's an example of a job file that will deploy an image on a target based on
+a hardware pack and root filesystem from Linaro::
+
+ {
+ "job_name": "panda-lmc",
+ "target": "panda01",
+ "timeout": 18000,
+ "actions": [
+ {
+ "command": "deploy_linaro_image",
+ "parameters":
+ {
+ "rootfs": "http://releases.linaro.org/12.09/ubuntu/precise-images/nano/linaro-precise-nano-20120923-417.tar.gz",
+ "hwpack": "http://releases.linaro.org/12.09/ubuntu/leb-panda/hwpack_linaro-lt-panda-x11-base_20120924-329_armhf_supported.tar.gz"
+ }
+ },
+ {
+ "command": "boot_linaro_image"
+ },
+ {
+ "command": "submit_results",
+ "parameters":
+ {
+ "server": "http://localhost/lava-server/RPC2/",
+ "stream": "/anonymous/test/"
+ }
+ }
+ ]
+ }
+
=== added file 'doc/jobfile-prebuilt.rst'
@@ -0,0 +1,31 @@
+Deploy Pre-Built Image
+======================
+
+Here's a minimal job that will deploy and boot a pre-built image::
+
+ {
+ "job_name": "panda-prebuilt",
+ "target": "panda01",
+ "timeout": 18000,
+ "actions": [
+ {
+ "command": "deploy_linaro_image",
+ "parameters":
+ {
+ "image": "http://releases.linaro.org/12.09/ubuntu/leb-panda/lt-panda-x11-base-precise_ubuntu-desktop_20120924-329.img.gz"
+ }
+ },
+ {
+ "command": "boot_linaro_image"
+ },
+ {
+ "command": "submit_results",
+ "parameters":
+ {
+ "server": "http://localhost/lava-server/RPC2/",
+ "stream": "/anonymous/test/"
+ }
+ }
+ ]
+ }
+
=== modified file 'doc/jobfile.rst'
@@ -2,103 +2,111 @@
Writing a Dispatcher Job File
*****************************
-
-Linaro Ubuntu Images
-====================
-
-Here's an example of a job file to run the stream test on an Ubuntu based Linaro Image. Stream is a small, fast test, and great for testing that everything works OK::
-
- {
- "job_name": "foo",
- "target": "panda01",
- "timeout": 18000,
- "actions": [
- {
- "command": "deploy_linaro_image",
- "parameters":
- {
- "rootfs": "http://snapshots.linaro.org/11.05-daily/linaro-developer/20110208/0/images/tar/linaro-n-developer-tar-20110208-0.tar.gz",
- "hwpack": "http://snapshots.linaro.org/11.05-daily/linaro-hwpacks/panda/20110208/0/images/hwpack/hwpack_linaro-panda_20110208-0_armel_supported.tar.gz"
- }
- },
- {
- "command": "lava_test_install",
- "parameters":
- {
- "tests": ["stream"]
- }
- },
- {
- "command": "boot_linaro_image"
- },
- {
- "command": "lava_test_run",
- "parameters":
- {
- "test_name": "stream"
- }
- },
- {
- "command": "submit_results",
- "parameters":
- {
- "server": "http://localhost/lava-server/RPC2/",
- "stream": "/anonymous/test/"
- }
- }
- ]
- }
-
-
-Linaro Android Images
+There are dozens of permutations for creating job files in the dispatcher.
+This page goes through some common scenarios:
+
+The base skeleton job files should look like:
+
+ * `Deploy a Pre-Built Image <jobfile-prebuilt.html>`_
+ * `Deploy Using linaro-media-create <jobfile-lmc.html>`_
+ * `Deploy an Android Image <jobfile-android.html>`_
+
+**NOTE:** Each of the above jobs uses the ``target`` parameter to specify the
+exact target to run the job on. If submitting a job via the scheduler, you'll
+likely want to just choose the ``device_type`` and let the scheduler find an
+idle device for you. This is done by removing the target line and adding::
+
+ "device_type": "panda",
+
+Executing Tests on Ubuntu
+=========================
+
+Tests are executed on Ubuntu by adding a ``lava_test_install`` and
+``lava_test_run`` action to your base job file::
+
+ {
+ "command": "lava_test_install",
+ "parameters": {
+ "tests": ["stream"]
+ }
+ },
+ {
+ "command": "boot_linaro_image"
+ },
+ {
+ "command": "lava_test_run",
+ "parameters": {
+ "test_name": "stream"
+ }
+ },
+
+**NOTE:** The ``lava_test_install`` action should follow the
+``deploy_linaro_image`` action.
+
+Executing Tests on Android
+==========================
+
+Tests are executed on Android by adding a ``lava_android_test_install`` and
+``lava_android_test_run`` action to your base job file::
+
+ {
+ "command": "lava_android_test_install",
+ "parameters": {
+ "tests": ["busybox"]
+ }
+ },
+ {
+ "command": "boot_linaro_android_image"
+ },
+ {
+ "command": "lava_android_test_run",
+ "parameters": {
+ "test_name": "busybox"
+ }
+ },
+
+Using LAVA Test Shell
=====================
-
-Here's an example showing how to run 0xbench on a Linaro Android image::
-
- {
- "job_name": "android_monkey_test2",
- "target": "panda01",
- "timeout": 18000,
- "actions": [
- {
- "command": "deploy_linaro_android_image",
- "parameters":
- {
- "boot": "https://android-build.linaro.org/jenkins/job/gerrit-bot_pandaboard/12/artifact/build/out/target/product/pandaboard/boot.tar.bz2",
- "system": "https://android-build.linaro.org/jenkins/job/gerrit-bot_pandaboard/12/artifact/build/out/target/product/pandaboard/system.tar.bz2",
- "data": "https://android-build.linaro.org/jenkins/job/gerrit-bot_pandaboard/12/artifact/build/out/target/product/pandaboard/userdata.tar.bz2"
- },
- "metadata":
- {
- "rootfs.type": "android",
- "rootfs.build": "12"
- }
- },
- {
- "command": "boot_linaro_android_image"
- },
- {
- "command": "lava_android_test_install",
- "parameters":
- {
- "tests": ["0xbench"]
- }
- },
- {
- "command": "lava_android_test_run",
- "parameters":
- {
- "test_name": "0xbench"
- }
- },
- {
- "command": "submit_results_on_host",
- "parameters":
- {
- "server": "http://validation.linaro.org/lava-server/RPC2/",
- "stream": "/anonymous/lava-android-leb-panda/"
- }
- }
- ]
- }
-
+The ``lava_test_shell`` action provides a way to employ a more black-box style
+testing appoach with the target device. The action only requires that a
+deploy action (deploy_linaro_image/deploy_linaro_android_image) has been
+executed. Its format is::
+
+ {
+ "command": "lava_test_shell",
+ "parameters": {
+ "testdef_urls": [
+ "http://people.linaro.org/~doanac/lava/lava_test_shell/testdefs/lt_ti_lava.json"
+ ],
+ "timeout": 1800
+ }
+ }
+
+You can put multiple test definition URLs in the "testdef_urls" section. These
+will be run sequentially without reboot. Alternatively, you can specify each
+URL in a separate ``lava_test_shell`` action which will allow for a reboot
+between each test.
+
+.. seealso:: The test definition format for ``lava_test_shell`` actions here_
+
+.. _here: lava_test_shell.html
+
+Adding Meta-Data
+================
+
+Both deploy actions support an optional field, ``metadata``. The value of this
+option is a set of key-value pairs like::
+
+ {
+ "command": "deploy_linaro_image",
+ "parameters": {
+ "image": "http://releases.linaro.org/12.09/ubuntu/leb-panda/lt-panda-x11-base-precise_ubuntu-desktop_20120924-329.img.gz",
+ "metadata": {
+ "ubuntu.image_type": "ubuntu-desktop",
+ "ubuntu.build": "61"
+ }
+ }
+ }
+
+This data will be uploaded into the LAVA dashboard when the results are
+submitted and can then be used as filter criteria for finding data.
=== added file 'doc/lava_test_shell.rst'
@@ -0,0 +1,81 @@
+LAVA Test Shell
+***************
+
+The ``lava_test_shell`` action provides a way to employ a more black-box style
+testing appoach with the target device. The test definition format is quite
+flexible allows for some interesting things.
+
+Minimal Test Definition
+=======================
+
+::
+
+ {
+ "format": "Lava-Test Test Definition 1.0",
+ "test_id": "pass_fail"
+ "run": {
+ "steps": ["echo test-1: pass", "echo test-2: fail"]
+ "parse": {
+ "pattern": "(?P<test_case_id>.*-*):\\s+(?P<result>(pass|fail))"
+ }
+ }
+
+The main thing to note is that the parse pattern requires regex expressions
+like \\s to be escaped, so it must be \\\\s
+
+Handling Dependencies (Ubuntu)
+==============================
+
+If your test requires some packages to be installed before its run it can
+express that in the ``install`` section with::
+
+ "install": {
+ "deps": ["linux-libc-dev", "build-essential"]
+ },
+
+Adding Git/BZR Repositories
+===========================
+
+If your test needs code from a shared repository, the action can clone this
+data on your behalf with::
+
+ "install": {
+ "bzr-repos": ["lp:lava-test"],
+ "git-repos": ["git://git.linaro.org/people/davelong/lt_ti_lava.git"]
+ },
+ "run": {
+ "steps": ["cd lt_ti_lava", "echo now in the git cloned directory"]
+ }
+
+This repository information will also be added to resulting bundle's software
+context when the results are submitted to the LAVA dashboard.
+
+Install Steps
+=============
+
+Before the test shell code is executed, it will optionally do some install
+work if needed. For example if you needed to build some code from a git repo
+you could do::
+
+ "install": {
+ "git-repos": ["git://git.linaro.org/people/davelong/lt_ti_lava.git"],
+ "steps": ["cd lt_ti_lava", "make"]
+ },
+
+**NOTE:** The repo steps are done in the dispatcher itself. The install steps
+are run directly on the target.
+
+Advanced Parsing
+================
+
+You may need to incorporate an existing test that doesn't output results in
+in the required pass/fail/skip/unknown format required by LAVA. The parse
+section has a fixup mechanism that can help::
+
+ "parse": {
+ "pattern": "(?P<test_case_id>.*-*)\\s+:\\s+(?P<result>(PASS|FAIL))",
+ "fixupdict": {
+ "PASS": "pass",
+ "FAIL": "fail"
+ }
+ }
=== added file 'doc/standalonesetup.rst'
@@ -0,0 +1,48 @@
+Quick Developer Setup
+=====================
+*NOTE:* You should most likely follow the normal installation instructions
+defined at http://lava.readthedocs.org/en/latest. However, these steps can
+get you a quick setup for local development on just the dispatcher::
+
+ # get code
+ $ sudo apt-get install python-virtualenv
+ $ bzr branch lp:lava-dispatcher
+ $ cd lava-dispatcher
+ # set up virtual environment for development
+ $ virtualenv .venv
+ $ . .venv/bin/activate
+ $ pip install keyring
+ $ ./setup.py develop
+ # setup configuration
+ $ mkdir .venv/etc
+ $ cp -r ./lava_dispatcher/default-config/lava-dispatcher .venv/etc/
+ $ cat >.venv/etc/lava-dispatcher/devices/qemu01.conf
+ device_type = qemu
+ $ echo "LAVA_IMAGE_TMPDIR = /tmp" >> .venv/etc/lava-dispatcher/lava-dispatcher.conf
+
+The set up a minimal job like::
+
+ # /tmp/qemu.json
+ {
+ "timeout": 18000,
+ "job_name": "qemu-test",
+ "device_type": "qemu",
+ "target": "qemu01",
+ "actions": [
+ {
+ "command": "deploy_linaro_image",
+ "parameters": {
+ "image": "file:///tmp/beagle-nano.img.gz"
+ }
+ },
+ {
+ "command": "boot_linaro_image"
+ }
+ ]
+ }
+
+And execute the dispatcher with::
+
+ $ lava-dispatch /tmp/qemu.json
+
+.. seealso:: For writing a new dispatcher job file see :ref:`jobfile`