Commit Graph

77 Commits

Author SHA1 Message Date
Chris Church
440f0539b0 Flake8 fixes. 2016-10-28 21:58:24 -04:00
Chris Church
c18b6c1352 Add support for capturing stdout associated with job events and ad hoc command events.
* New event types for stdout lines not associated with a callback event.
* New stdout, start_line, end_line and verbosity fields for job/ahc events.
* Callback plugins to wrap Ansible default/minimal stdout callbacks and embed callback event data using ANSI escape sequences.
* Callback plugin library to wrap ansible.display.Display class methods.
* Output filter to extract event data from stdout and create job/ahc events.
* Update stdout formats to strip new ANSI escape sequences.
2016-10-28 21:58:24 -04:00
AlanCoding
5d4cf9d4fc Add job artifacts and workflow artifact passing
artifacts redact from job when no_log is set
parent no_log artifacts treated as survey passwords
2016-10-12 17:45:50 -04:00
Matthew Jones
3de4aae548 Fixing up HA induced flake8 issues 2016-09-15 13:51:17 -04:00
Matthew Jones
ab395b0009 Integrate callback receiver refactoring
* Drop ZMQ as the communication mechanism between job_event_callback and
  callback_receiver
* Setup queue and exchange for callback broker communication
* Refactor event plugin and callback receiver to efficiently handle
  message submission and processing
* Integrate django caching for parent processing
2016-09-14 11:42:13 -04:00
Akita Noek
601bd0da05 Fixed up 2.4.5 -> 3.0 no_log merge
Addresses #1295
2016-04-25 15:52:38 -04:00
Matthew Jones
cf7b71ae2c Merge branch 'release_2.4.5' into devel
* release_2.4.5:
  Remove distribute from the setup virtualenv if installed (#1631)
  Update changelogs for 2.4.5 release
  Only export changed targets in reprepro
  Improve the efficiency of the stdout dump database migration
  Added logic to not show the loop summary events in the UI by looking at event_data.event_loop.
  Handle runner items from ansible v2
  bump boto
  fix case of Ansible v2 _result.cmd is list for release 2.4.5
  Handle both string and list hosts in our hosts->name conversion
  Default play names to the hosts the play was run against
  Use better isinstance(x) type checking
  Backporting test fixes from PR #1020: Fix error with ad hoc command events when running in check mode.
  Fix for tasks breaking when using 'yum' with ansible 1.9.4
  Bump 2.4.5 version, changelogs, and reprepo
2016-04-22 10:54:23 -04:00
Matthew Jones
547506a3bf Remove contextual statsd reference 2016-04-06 16:47:36 -04:00
Matthew Jones
c0afa3ca0f More commenting out of statsd 2016-04-06 16:42:30 -04:00
Matthew Jones
3ba591cea2 Remove statsd from callback
This doesn't work with our current virtualenv setup and ansible
dependencies.  Will revisit later
2016-04-06 16:29:50 -04:00
Matthew Jones
f2284a0098 Handle runner items from ansible v2
Also denote whether the trailing runner_on_ was a loop event
2016-03-23 11:38:11 -04:00
Matthew Jones
0e2184902e Handle runner items from ansible v2
Also denote whether the trailing runner_on_ was a loop event
2016-03-22 12:31:05 -04:00
AlanCoding
24f7194648 fix case of Ansible v2 _result.cmd is list for release 2.4.5 2016-02-26 09:23:41 -05:00
Chris Church
60c3eeaef3 Include @anoek's changes from #1029. 2016-02-25 15:01:29 -05:00
Akita Noek
5dc164bb76 Handle both string and list hosts in our hosts->name conversion
Base our play.name fix off of what Ansible will be doing in the next
release: e2d2798a42
2016-02-25 11:38:09 -05:00
Akita Noek
26d11b2fb7 Default play names to the hosts the play was run against
Fixes #1030
2016-02-25 11:23:00 -05:00
Akita Noek
519a1911ea Use better isinstance(x) type checking 2016-02-24 18:02:20 -05:00
Akita Noek
f08aeeda47 Backporting test fixes from PR #1020: Fix error with ad hoc command events when running in check mode.
Backport of the test fixes from PR #1020 to 2.4.5, this fixes the unit
tests around our fix for #1019 (hopefully)
2016-02-24 15:56:43 -05:00
Akita Noek
df3715a36f Fix for tasks breaking when using 'yum' with ansible 1.9.4
Fixes #1019
2016-02-24 11:35:24 -05:00
Chris Church
2f763d0ff5 Fix error with ad hoc command events when running in check mode. 2016-02-23 18:26:38 -05:00
Matthew Jones
c3121211c3 Merge branch 'release_2.4.4' into devel
* release_2.4.4: (35 commits)
  Update changelog for 2.4.4 release
  Make pycompile non-fatal during deb build
  Revert "Add virtualenv site-pagkages to Python path before system dist-packages, to get new setuptools"
  Roll back mock version due to packaging issues
  Add virtualenv site-pagkages to Python path before system dist-packages, to get new setuptools
  change to warning behavior
  Resolve bug when building with /bin/sh on Ubuntu
  Attempt to workaround pip install issue
  point at packages with source on pypi
  Mock requires a newer setuptools when building requirements
  requests needs openssl
  Properly set the shell during directory migration
  pyrax bumpb new python license
  Typo's are bad and should be vanquished
  Conditionally install 2.6 python requirements
  separate pip requirements file for python2.6
  Added missing 'skipped' field for no_log
  Obey no_log even more when using ansible 2.0
  bump shade from 0.5.0 to 1.4
  RHEL5 compatibility and handling of error scenarios
  ...
2016-02-22 10:09:47 -05:00
Matthew Jones
7ab839c17f Fix up another statsd formatting error 2016-02-02 14:26:17 -05:00
Matthew Jones
f9f10a4ed7 Add some more statsd dummy methods 2016-01-30 12:05:10 -05:00
Matthew Jones
96b754c5ec Switch to StatsClient from StatsClientBase
So as not to use a purely abstract class
2016-01-29 16:27:32 -05:00
Matthew Jones
c2875c65ca Fix stats client import 2016-01-29 15:49:51 -05:00
Matthew Jones
fd2339dbb8 Fix up flake8 line seperation issue 2016-01-29 14:52:02 -05:00
Joshua "jag" Ginsberg
d36a4cf50d Fix to NoStatsClient in callback plugin 2016-01-29 12:41:07 -07:00
Akita Noek
f8a7fb0ea1 Added missing 'skipped' field for no_log 2016-01-28 12:43:49 -05:00
Akita Noek
94e6d2a72a Obey no_log even more when using ansible 2.0
Hopefully fixes #645 this time. New function handles recursing down our
results array when it's present, also attempts to proactively protect
against future data leaks by only allowing white listed fields through.
2016-01-28 11:34:43 -05:00
Joshua "jag" Ginsberg
f25b76aee9 Integrate statsd metrics into ansible playbook execution.
* Add dependencies for pystatsd and django-statsd-mozilla
* Default turned off except for development environment
* Modify docker-compose to install statsd/graphite host
2016-01-26 16:54:24 -05:00
Akita Noek
de7be08856 Obey no_log with ansible 2.0
Fixes #645
2016-01-20 15:34:59 -05:00
Akita Noek
1e1a023d19 Obey no_log with ansible 2.0
Fixes #645
2016-01-20 15:11:49 -05:00
Matthew Jones
70d8935461 Fix v2 playbook_on_start method signature 2016-01-14 12:26:24 -05:00
Matthew Jones
573403a05c Fix v2 playbook_on_start method signature 2016-01-14 11:59:27 -05:00
Matthew Jones
b579b76099 Handle playbook_on_include events from v2 ansible 2015-10-20 11:18:07 -04:00
Matthew Jones
5de5b0209b Remove unused time import 2015-09-30 16:20:41 -04:00
Matthew Jones
e034c0b326 Fix proot controlpersist cleanup issue
* Also removed psutil version and feature check from tasks.  We now
  include the most recent version of psutil all around
* Old versions of psutil must have had these Process() items as
  properties, they are functions now so we weren't properly evaluating
  them.  With this change we are and things are back to working normally
2015-09-30 15:56:31 -04:00
John Mitchell
32d1c0e4db fixed copyright date 2015-06-11 16:10:23 -04:00
Matthew Jones
31d0342d41 More copyright headers for api side stuff 2015-05-29 12:10:40 -04:00
James Laska
7064c9bed4 Additional flake8 cleanup
The flake8 command was identifying several warnings and errors. This change
addresses the flake8 warnings and updates the setup.cfg with additional
exclusions. If accepted, jenkins will be updated to use the flake8 command,
rather than using the django_jenkins plugin. This will expedite jenkins
testing.
2015-05-20 20:33:22 -04:00
Chris Church
8beed7f409 Disable ControlPersist for ad hoc commands, should avoid any issues with proot or needing to clean up sockets afterwards. 2015-04-09 10:32:24 -04:00
Chris Church
18e38707c2 Add callback methods for v2 support. Closes #115. 2015-04-03 00:06:49 -04:00
Chris Church
f7b8d510dc Implement support for ad hoc commands. 2015-03-30 13:04:19 -04:00
Matthew Jones
0f5beca9ae Major fixes for job callback receiver processing
* Add logic to ansible callback plugin to prevent it from waiting
  forever to submit events to Tower
* Lower process recycle threshold for tower callback receiver
* Make recycle threshold configurable
* Properly exit the main callback receiver management process if
  the event receiver process is dead so we don't leave dead worker
  processes
* Set a configurable maximum number of messages that can be waiting
  in a worker process queue before it is skipped instead of filling
  up memory on a dead worker process
* Skip over a dead worker process if it's queue is full
* Force restart callback receiver if all queues are dead
* Roll back transaction.atomic with the thought that it is causing
  deadlocks in the worker process.  Use the old commit_on_success
  mechanism with retry logic
* Seperate queue nonblocking expected exception from any other type
  of exception that could be encountered on the queue fetch operation
2015-03-13 11:18:18 -04:00
Chris Church
ee42ae8571 Support older versions of psutil that don't implement as_dict(). 2015-01-16 12:30:31 -05:00
Chris Church
4dff25884f Re-enable proot by default, add python-psutil as a dependency, implement support for terminating SSH control master processes so that playbook runs exit cleanly. 2015-01-14 17:17:15 -05:00
Chris Church
3668b3c761 Fix callback consumer port to always be a URL, instead of sometimes an int. 2014-12-03 20:49:08 -05:00
Chris Church
36cf1e1e10 Fix callback consumer port used by job event callback. 2014-12-03 20:44:19 -05:00
Matthew Jones
fcc7d3d7b1 Rework the callback emitter a little more, purge tower internals from
the module and fallback to the older codebase's mechanism.   Make sure
we are passing the callback port as a string otherwise it seems to
corrupt the shell
2014-12-03 11:27:37 -05:00
Luke Sneeringer
52aca5a081 Theoretically working Socket implementation. 2014-11-21 09:24:49 -06:00