debci (1.0.2) unstable; urgency=medium [ Martin Pitt ] * Fix some excessive amd64 hardcoding in the test suite. Thanks Colin Watson! * test/test_blame.sh: Sleep for one second between process() calls, to ensure that timestamps in run IDs are different. (Closes: #809265) [ Brian Murray ] * Include extra information in HTML tags, such as package name, architecture, suite, etc (Closes: #811563) -- Antonio Terceiro <terceiro@debian.org> Mon, 25 Jan 2016 18:57:03 -0200 debci (1.0.1) unstable; urgency=medium * debian/debci-worker.cron.daily: exit immediately if debci is no longer installed (Closes: #807360) * Drop Debconf templates * bin/debci-batch: make sure package status directory exists before creating queue marker -- Antonio Terceiro <terceiro@debian.org> Mon, 14 Dec 2015 15:17:30 -0200 debci (1.0) unstable; urgency=medium * This is a major overhaul of the debci infrastructure. debci now works as a distributed system, with one or more workers that process test runs, and one central node that collects tests results and generates the HTML web UI. The nodes communicate using an AMQP server (usually rabbitmq-server), and test requests can be submitted from any host that can connect to the AMQP server. One will usually have one specific node, perhaps the one that runs the result collection/web UI, running `debci batch` to schedule tests. * New packages: + debci-worker: contains the systemd service units necessary to run the worker daemons; must be installed on worker nodes. + debci-collector: contains the systemd service unit necessary to run the debci collector daemon: will receive test results, and generate the HTML web interface; must be installed on a single node in the network * The `debci` package contains the debci core, and is still the package that needs to be installed for developers trying out debci locally as instructed the documentation. * Change default backend to `lxc` [ Martin Pitt ] * Initial implementation of the master/worker setup * Add support for importing data from test runs executed elsewhere [ Antonio Terceiro ] * Finished implementation of the master/worker setup * schroot/create-testbed: find usable union-type (Closes: #799760) * debian/debci.postrm: remove /var/lib/debci on purge (Closes: #769766) * added debian/tests/integration-test: full-system smoke test [ Chris Lamb ] * Document how to get started without a virtualisation regime (Closes: #777531) [ Brandon Fairchild ] * Several documentation and frontend updates + Including displaying correct failing architectures (Closes: #797198) -- Antonio Terceiro <terceiro@debian.org> Thu, 03 Dec 2015 22:15:30 -0200 debci (0.13) unstable; urgency=medium * debci hint: generate a britney hints file -- Antonio Terceiro <terceiro@debian.org> Tue, 18 Aug 2015 16:52:06 +0200 debci (0.12) unstable; urgency=medium [ Brandon Fairchild ] * Debci::Package#failures: Determine the test status that occurred before a tmpfail [ Antonio Terceiro ] * web UI: read head.html and footer.html from config dir -- Antonio Terceiro <terceiro@debian.org> Wed, 17 Dec 2014 08:45:20 -0200 debci (0.11) unstable; urgency=medium [ Antonio Terceiro ] * debci-migrate: added a data migration mechanism that is also automatically executed during postinst. * Switch data storage strategy to compress all autopkgtest logs. This will save huge amounts of storage space. - tools/server.sh adjusted to make lighttpd serve *.log.gz files as regular text/plain files, but with the propers headers to tell browsers that they are compressed so they will do the right tihng. * debci-batch: will now stop even in the middle of a iteration when told to go offline. * backends/schroot/create-testbed, debci-setup-chdist: add buildd suites to sources.list so we detect newly uploaded packages faster * Removed embedded copy of Bootstrap, now depending on libjs-bootstrap [ Brandon Fairchild ] * Several web UI improvements * Documentation update on hacking the web interface -- Antonio Terceiro <terceiro@debian.org> Sat, 15 Nov 2014 14:56:14 -0200 debci (0.10.3) unstable; urgency=medium [ Brandon Fairchild ] * handle old test run data that didn't have a run_id. [ Antonio Terceiro ] * debci-status: fix generation of packages status file which was incorrectly repeating the status for the very first package over and over. -- Antonio Terceiro <terceiro@debian.org> Tue, 14 Oct 2014 18:55:27 -0300 debci (0.10.2) unstable; urgency=medium [ Antonio Terceiro ] * lib/functions.sh: accept any value of Testsuite matching `autopkgtest` insted of only exact matches when looking for source packages with test suites. This will allow detecting packages with specialized values such as autopkgtest-pkg-ruby and autopkgtest-pkg-perl. [ Brandon Fairchild ] * Detect JSON parsing errors on history files instead of silently failing and leaving a blank history (Closes: #759569) * Avoid also listing lib* packages under l/. -- Antonio Terceiro <terceiro@debian.org> Wed, 08 Oct 2014 08:53:37 -0300 debci (0.10.1) unstable; urgency=medium * Fix documentation build -- Antonio Terceiro <terceiro@debian.org> Tue, 30 Sep 2014 17:29:48 -0300 debci (0.10) unstable; urgency=medium [ Martin Pitt ] * Fix Makefile to be idempotent, previously it failed on already existing symlinks. * Add missing dctrl-tools build dependency, Makefile is calling grep-dctrl. * Makefile: Don't call "checkdeps" on "make check", the test suite doesn't actually need autopkgtest or the libjs packages (which are for the UI only). * test/test_helper.sh: Don't hardcode "amd64", to fix test failure when building on other architectures. [ Antonio Terceiro ] * debci-generate-index: update packages.json atomically * web UI improvements: - remove limit of number of items to display in the search (Closes: #762213) - trigger search immediately when there is text in the search box (this will happen if you reload the page) - store search query in the URL, and honor that when opening any page that contains a search box -- Antonio Terceiro <terceiro@debian.org> Fri, 26 Sep 2014 18:22:06 -0300 debci (0.9) unstable; urgency=medium [ Antonio Terceiro ] * schroot/create-testbed: update existing chroots * debci-generate-feeds: add description to feeds * Incorporate results of the Google Summer of Code 2014 project by Brandon Fairchild, providing uncountable improvements to the web interface. http://lists.alioth.debian.org/pipermail/soc-coordination/2014-August/002295.html [ Brandon Fairchild ] * web UI was improved to support multiple suites and architectures, reduce the dependency the UI had on Javascript, and provide more information to package maintainers. The documentation was also worked on to provide information to people interested in working on debci (specifically the web UI) -- Antonio Terceiro <terceiro@debian.org> Sat, 23 Aug 2014 18:10:04 -0300 debci (0.8.1) unstable; urgency=medium * Documentation updates * Fix obtaining the finish date of tests. Parsing the adt-run log started producing bad dates when adt-run changed to displaying only the time without the date. debci now uses the timestamp of when adt-run finished instead. * debci-batch: store offline mark in persistent storage to avoid going back online after a reboot. * debci-test: revert to updating only package-specific data; it turns out that updating global data after every package is too expensive. * debci-batch: update global data every 1 hour or so * debci-generate-index: avoid crash when regenerating data for extenal consumption from scratch. This process is not completely idempotent yet, so it needs more work. -- Antonio Terceiro <terceiro@debian.org> Mon, 18 Aug 2014 08:25:23 -0300 debci (0.8) unstable; urgency=medium * debci-test: update global data files, not only package-specific data file, after each successfull test run. * debci-generate-index: ensure that no two instances can run at the same time. -- Antonio Terceiro <terceiro@debian.org> Thu, 31 Jul 2014 11:55:27 -0300 debci (0.7.3) unstable; urgency=medium * debci-generate-feeds: - always update the global feed. - Point main link of feed entries to the debci log instead of the package page. * debci-generate-index: - report duration of each test run together with the status * debci-test: - fix regression with new packages caused by not mkdir'ing the full path to the adt-run output directory. `fake` backend adapted to emulate the behaviour of adt-run so that the tests would now fail without the fix. -- Antonio Terceiro <terceiro@debian.org> Wed, 18 Jun 2014 14:29:46 +0200 debci (0.7.2) unstable; urgency=medium * Fix calculation of test elapsed time. We can't store the beginning timestamp at the adt-run output directory because that will be wiped out by adt-run before running the test. * public/index.html: fix source code URL to point to the collab-main repository which is the canonical one. -- Antonio Terceiro <terceiro@debian.org> Tue, 10 Jun 2014 17:30:41 -0300 debci (0.7.1) unstable; urgency=medium * Fix handling of blame with "unknown" as value. Will avoid crashing when generating indexes -- Antonio Terceiro <terceiro@debian.org> Mon, 09 Jun 2014 11:07:11 -0300 debci (0.7) unstable; urgency=medium [ Antonio Terceiro ] * debci-batch: put packages with fastest testsuites first in the queue * debci-generate-feeds: allow generating feeds for specific packages * debci-generate-index: update package feed right away when indexing individual packages * public/app.js: Adjust the "Pass percentage" Y axis labels so that chart is actually a percentage chart. * public/index.html: add link to global Atom feed in the home page. * debci-test, debci-generate-index: more robust way of calculating test run duration. Parsing the autopkgtest log is just not good enough. [ Martin Pitt ] * Fix report_status() to work for bash scripts. * debci-test: Add --print-output option to write output directory to stdout. With that the caller can easily check the result. * debci, tools/server.sh: Exec the target program instead of running in a subprocess, for efficiency and proper SIGTERM handling. * Drop tools/convert-data. We only needed it for version 0.6, and we're going to need a different conversion script after the next data/ dir reorganization. * Reorganize data directory to move the "categories" packages/, autopkgtest/, status/, and feeds/ to the top level; they contain subdirs for all releases and architectures. This makes autopkgtest/ fully rsyncable and also makes it easier to the web UI to browse/present data for multiple releases and architectures. Also split release-arch/ into release/arch/. * Add tools/convert-data-0.7: Convert data directory to changed format from above. * Fix debci-status -l if there is just one package. * Add tests for debci-batch, covering skipping/re-running of tests under appropriate conditions and showing the cause for re-run in the log. * Replace test/json_validity (in ruby) with test/test_json.sh (shell with just calling ruby for the actual JSON validation). With that we can re-use test_helper's test bed setup instead of having to replicate it all in ruby. * Make test/runall.sh executable for convenience, and make it only run test_* scripts. * debci-batch: remove reading configuration from /etc/default/debci * lib/environment.sh: read configuration from $debci_basedir/config/debci.conf * Add debci_mirror config variable to configure an archive mirror. Use it for chdist and schroot setup. * Drop unnecessary faketime build dependency. * Add LXC backend. -- Antonio Terceiro <terceiro@debian.org> Tue, 03 Jun 2014 22:36:44 -0300 debci (0.6.3) unstable; urgency=medium * debian-status: avoid generating invalid JSON in the presence of packages without status files -- Antonio Terceiro <terceiro@debian.org> Thu, 29 May 2014 12:48:48 -0300 debci (0.6.2) unstable; urgency=medium * debci-generate-index: don't try merging dependencies diff when no package has been blamed before. -- Antonio Terceiro <terceiro@debian.org> Wed, 28 May 2014 18:14:06 -0300 debci (0.6.1) unstable; urgency=medium * Fix chdist calls on stable (See #736504) -- Antonio Terceiro <terceiro@debian.org> Thu, 15 May 2014 12:51:25 -0300 debci (0.6) unstable; urgency=medium [ Martin Pitt ] * Move schroot creation from bin/debci-setup to backends/schroot/create-testbed, as it is schroot backend specific and other backends will need different setup. * process-package: Call backend test-package with second argument that specifies the adt-run output directory. Implement that in schroot backend with adt-run --output-dir, and fake backend when passing $DEBCI_FAKE_DEPS. This will be used to collect and present log files, artifacts, package/version lists, etc. * Eliminate list-base-system and use autopkgtest's testbed-packages output file instead. This will also work with remote workers and other virt servers. * Eliminate check_version() and take this information from autopkgtest's output. * Move report_status() into lib/functions.sh as we'll need it in more places than just process-package. * Move "run needed" policy from process-package to bin/debci. With this, debci-test can be called manually at any time to retry tests or get called by e. g. britney which already evaluates when to run tests. Remove the "run test at most once a day" check completely as package updates can happen several times a day. As a by-product this also fixes debci's -f option to force a test run. * backends/schroot/list-dependencies: Ignore individual apt-get failures as that gets called for binaries which don't exist on the selected architecture. Fixes operation for e. g. glib2.0. * Use chdist instead of the local schroot for package queries: - Add scripts/setup-chdist to create/update a chdist for the selected release. - Use that in list-dependencies instead of querying the schroot. Move that script to scripts/ as it is now not backend specific any more. - Update lib/functions.sh to use chdist instead of the schroot. - Add devscripts dependency for chdist, and demote schroot dependency to recommends. This allows us to use other backends, like QEMU or remote ones. * Reorganize data directory: - Split out autopkgtest output into ${debci_data_dir}/autopkgtest/: This now contains a full directory for each test run with all of adt-run's --output-dir files. This provides access to individual test stdout/err logs, the package version lists, and additional artifacts that tests produce. With this, autopkgtest/ becomes eligible for being on a remote file system and can be synced/rebuilt independently of the metadata. - Use run IDs with format YYYYMMDD_HHMMSS[machine or tags] instead of the old YYYYMMDD-n as the former works for a distributed file system and remote workers. - base.txt is now replaced with adt-run's testbed-packages output file. - test-packages.txt gets dropped as there is no need to keep it around permanently. Build it from the per-test -packages logs from adt-run. - Move status.json into status/ directory and remove the redundant latest.json. - Move packages.json into status/ directory. * Add tools/convert-data: Convert data/ directory to changed format from above. This can be dropped again after running it on the production debci instance. * app.js: Filter out query strings from URL for constructing the .json URLs. Fixes wrong URLs after using the package search field. * backends/schroot/test-package: Drop unused temporary dir. * backends/*/test-package: Write exit code into "exitcode" file in output dir, so that we retain it even when we switch to asynchronous/remote processing. * backends/fake/test-package: Write log file into output dir, like adt-run. * Move test result evaluation, log and meta data creation from debci-test to debci-generate-indexes. This prepares the way for running adt-run asynchronously or remotely, and picking up the results from the autopkgtest/ data directory. * Reduce the per-test run <timestamp>.log files to the debci metadata (triggers, package versions, package diffs, overall result), as the actual autopkgtest log is now kept in data/*/autopkgtest/. This avoids duplicating the log contents. * app.js: In the per-package test list, add link to latest autopkgtest log and artifacts (other output files). * debci-status --all: Output a list also if there is only one package. * debci-status --all: Sort package names so that they appear sorted in the web UI. * Avoid some error messages noise when calling generate-index without any existing test data. * Drop debci-test --reason option and make debci-batch put the reason into the status directory directly. * Fold debci-cleanup into debci-generate-index. [ Antonio Terceiro ] * schroot backend: pass --session-id to adt-virt-schroot. This makes it easier to debug test suite that leave rogue schroot sessions behind. * rename `debci` binary to `debci-batch`. `debci` stays reserved for providing a entry point binary like `git` that will invoke subcommands that are kept out of $PATH to avoid cluttering the namespace. * debci-batch: make batch locking suite- and architecture-specific * debci-batch: add --offline and --online options. --offline will prevent new test runs from being started, and --online will revert to normal operation. * Rename scripts/*: - scripts/process-package: absorbed by the previously almost empty bin/debci-test - scripts/list-dep8-packages: absorbed by the previously almost empty bin/debci-list-packages - scripts/list-dependencies → bin/debci-list-dependencies - scripts/setup-chdist → bin/debci-setup-chdist - scripts/blame → bin/debci-blame - scripts/cleanup → bin/debci-cleanup - scripts/generate-index → bin/debci-generate-index * add `debci` binary to not clutter the namespace. When you call `debci COMMAND` it will set PATH to include the debci bin/ and then call `debci-COMMAND` with the supplied arguments. * debci-blame: replace usage of deprecated IO#lines method with IO#each_line * debci-generate-index: accept duration as command line switch * backends/schroot/create-testbed: drop usage of eatmydata to avoid breaking test suites that actually depend on a real fsync(). To compensate the performance loss when installing packages, set dpkg `--force-unsafe-io` option in a configuration file (Closes: #747308). * Added Ruby bindings and the corresponding documentation * Dropped usage of faketime in the test suite (Closes: #747715) * backends/fake: output a timestamp like the ones adt-run outputs * Added new debci-config, a helper to output value of debci configuration values * debci-generate-feeds: added generation of per-package and global Atom feeds. Called from debci-batch * public: now using FontAwesome from fonts-font-awesome package * public/app.js: add a link to the per-package Atom feed * public/style.css: make the status indicators use FontAwesome icons -- Antonio Terceiro <terceiro@debian.org> Thu, 15 May 2014 12:34:33 -0300 debci (0.5.2) unstable; urgency=medium * scripts/process-package: skip blame check without a dependencies diff * lib/*.sh: guard `set -eu` to avoid doing that on interactive shells * lib/functions.sh: fix list_binaries to work against the sources index - This fixes listing binaries coming from packages that were binNMUed -- Antonio Terceiro <terceiro@debian.org> Sat, 05 Apr 2014 20:03:22 -0300 debci (0.5.1) unstable; urgency=medium * etc/schroot/debci/fstab: improve schroot isolation - do not share /tmp with host - mount tmpfs at /dev/shm as quite some software needs it * test_shell_best_practices.sh: skip when checkbashisms is not available. - You don't want to install devscripts and all its dependencies when running as-installed tests - checking for best practices is only useful in development environments. * Use unified diff in "Change in the base system since last run" * debian/source/options: fix to include publib/data in the source tarball, but not data/* -- Antonio Terceiro <terceiro@debian.org> Tue, 01 Apr 2014 12:23:15 -0300 debci (0.5.0) unstable; urgency=medium [ Martin Pitt ] * Add missing dctrl-tools dependency. * Use "dpkg --print-architecture" instead of dpkg-architecture, to avoid a dependency on dpkg-dev and the (undeclared) dependency on gcc. * Fix grep_packages() to actually use the configured architecture instead of the host architecture. * Add --help output to debci-{list-packages,setup,test}, instead of failing with an "usage: not found" error. [ Antonio Terceiro ] * Initial upload to Debian (Closes: #736416) The API is not set yet, but I am uploading anyway to allow for early testing, improve contribution opportunities, to get a BTS etc. * Updated documentation - instructions for setting up a development instance. - Point traffic at debian-qa * modularize behaviour of a backend - existing implementation extracted as the `schroot` backend - added a `fake` backend that helps with testing * schroot backend optimization: always run with eatmydata * Add blame support: when tests of a package start failing after some dependencies were updated, those dependencies will be "blamed" for the failure. * improvements on autopkgtest usage: - debci now uses the autopkgtest support for downloading the source package instead of downloading manually and them calling adt-run - debci now differentiates between the non-zero adt-run exit codes, so that skipped tests will not count as failures. Also added a `tmpfail` status, to be used when for some reason the tests could not be run for some external failure (e.g. mirror failures, broken dependencies etc). * debci now keeps track of the base system. Tests will be run again every 2 weeks even if no explicit dependency changes. Test logs will also display the changes in the base system since the last run. * Add DEP-8 support to debci itself \o/ * extra safety: add `set -u` to all shell scripts -- Antonio Terceiro <terceiro@debian.org> Mon, 24 Mar 2014 20:06:56 -0300 debci (0.4.0) unstable; urgency=medium * Package renamed to debci * Require a newer autopkgtest. - In general we should always be using a backport of the most recent version in unstable. * Visual facelift - embedding a copy of bootstrap 3 for now :-( * Add backend support for multiple architectures. For now unstable/amd64 is hardcoded, but reading them from the command line or from the environment will not require a large change. -- Antonio Terceiro <terceiro@debian.org> Tue, 28 Jan 2014 14:40:22 -0300 dep8 (0.3.1) unstable; urgency=medium * Explicitly set PATH in crontab -- Antonio Terceiro <terceiro@debian.org> Sun, 12 Jan 2014 22:52:00 -0300 dep8 (0.3.0) unstable; urgency=medium * record duration of each run, and record more tham one status entry per day. -- Antonio Terceiro <terceiro@debian.org> Sun, 12 Jan 2014 19:40:23 -0300 dep8 (0.2.3) unstable; urgency=medium * Makefile: fix creating symlinks for JS files -- Antonio Terceiro <terceiro@debian.org> Sat, 11 Jan 2014 15:47:00 -0300 dep8 (0.2.2) unstable; urgency=medium * check_version: handle package with more than 1 version -- Antonio Terceiro <terceiro@debian.org> Sat, 11 Jan 2014 15:41:00 -0300 dep8 (0.2.1) unstable; urgency=medium * environment.sh: print error when dep8_base_dir is not set * cron-wrapper: change to base directory before loading environment -- Antonio Terceiro <terceiro@debian.org> Mon, 06 Jan 2014 16:22:18 -0300 dep8 (0.2.0) unstable; urgency=medium * keep track of the archive-wide status * process-package: set directory for schroot invocation * generate-index: better naming for status entries * cron-wrapper: fix user detection * Move web UI to public/ subdirectory * add test to check for bashisms * major reorganization for reuse and consolidation * webui: display global status in time series chart * Use explicit directory for gnupg home * Also install a config directory in /etc * ignore new top-level directories -- Antonio Terceiro <terceiro@debian.org> Mon, 06 Jan 2014 15:44:29 -0300 dep8 (0.1.1) unstable; urgency=medium * Add basic test suite * add `set -e` to all shell scripts -- Antonio Terceiro <terceiro@debian.org> Sat, 04 Jan 2014 14:20:53 -0300 dep8 (0.1.0) unstable; urgency=low * Initial Release. -- Antonio Terceiro <terceiro@debian.org> Wed, 01 Jan 2014 01:35:14 -0300