Commit Graph

689 Commits

Author SHA1 Message Date
Ryan Petrello
18f254fc28 Merge pull request #769 from rbywater/feature/cloudformssuffix
Add ability to append suffix to host names for Cloudforms Inventory
2018-01-17 11:43:10 -05:00
Ryan Petrello
1e8c89f536 implement support for per-playbook/project/org virtualenvs
see: https://github.com/ansible/awx/issues/34
2018-01-09 22:47:01 -05:00
Chris Meyers
d07a946183 Merge pull request #921 from chrismeyersfsu/fix-handle_work_error-689
Fix handle_work_error()
2018-01-05 07:54:03 -05:00
Chris Meyers
a0038276a4 do not use a custom task exception
* Celery + json pickling do not handle custom Exceptions (and may never
do so). Mentioning of, if handling custom Exceptions then the code would
be susceptible to same arbitrary code execution that python pickle is
vulnerable to.
* So don't use custom Exceptions.
2018-01-04 15:30:52 -05:00
Chris Meyers
f0ff6ecb0a handle_work_error signature to work
* celery error callback signature isn't well defined. Thus, our error
callback signature is made to handle just about any call signature and
depend on only 1 attribute, id, existing.

See https://github.com/celery/celery/issues/3709
2018-01-04 15:23:13 -05:00
Ryan Petrello
0b30e7907b change stdout composition to generate from job events on the fly
this approach totally removes the process of reading and writing stdout
files on the local file system at settings.JOBOUTPUT_ROOT when jobs are
run; now stdout content is only written on-demand as it's fetched for
the deprecated `stdout` endpoint

see: https://github.com/ansible/awx/issues/200
2018-01-03 09:09:43 -05:00
Ryan Petrello
fc94b3a943 generalize stdout event processing to emit events for *all* job types
* introduces three new models: `ProjectUpdateEvent`,
  `InventoryUpdateEvent`, and `SystemJobEvent`
* simplifies the stdout callback management in `tasks.py` - now _all_
  job run types capture and emit events to the callback receiver
* supports stdout reconstruction from events for stdout downloads for
  _all_ job types
* configures `ProjectUpdate` runs to configure the awx display callback
  (so we can capture real playbook events for `project_update.yml`)
* ProjectUpdate, InventoryUpdate, and SystemJob runs no longer write
  text blobs to the deprecated `main_unifiedjob.result_stdout_text` column

see: https://github.com/ansible/awx/issues/200
2018-01-03 09:09:42 -05:00
Ryan Petrello
a978d094b4 fix a number of failing unit tests related to the 3.2.2 merge 2017-12-13 13:03:17 -05:00
Matthew Jones
9dbcc5934e Merge remote-tracking branch 'tower/release_3.2.2' into devel 2017-12-13 12:25:47 -05:00
Ryan Petrello
202161f090 move legacy UnifiedJob stdout data to a separate unmanaged model
This data often (in the case of inventory updates) represents large data
blobs (5+MB per job run).  Storing it on the polymorphic base class
table, `main_unifiedjob`, causes it to be automatically fetched on every
query (and every polymorphic join) against that table, which can result
in _very_ poor performance for awx across the board.  Django offers
`defer()`, but it's quite complicated to sprinkle this everywhere (and
easy to get wrong/introduce side effects related to our RBAC and usage
of polymorphism).

This change moves the field definition to a separate unmanaged model
(which references the same underlying `main_unifiedjob` table) and adds
a proxy for fetching the data as needed

see https://github.com/ansible/awx/issues/200
2017-12-12 18:16:19 -05:00
Bill Nottingham
9be438a60a Merge pull request #788 from wenottingham/no-country-for-old-python
Don't bother checking for python 2.6 in the venv
2017-12-11 14:57:25 -05:00
Bill Nottingham
c62430c282 Drop python2.6 checks. 2017-12-11 13:59:00 -05:00
AlanCoding
4adfb9804e allow using SystemJobTemplates in workflows 2017-12-11 08:58:45 -05:00
AlanCoding
1f8cab4171 intentionally forget start_args when job is done 2017-12-10 12:08:54 -05:00
AlanCoding
1c8217936d Bug fixes from integration ran on launchtime branch
Make error message for muti-vault validation more
consistent with historical message
2017-12-08 13:46:38 -05:00
AlanCoding
34a8e0a9b6 Feature: saved launchtime configurations
Consolidate prompts accept/reject logic in unified models
Break out accept/reject logic for variables
Surface new promptable fields on WFJT nodes, schedules

Make schedules and workflows accurately reject variables
  that are not allowed by the prompting
  rules or the survey rules on the template

Validate against unallowed extra_data in system job schedules
Prevent schedule or WFJT node POST/PATCH with unprompted data
Move system job days validation to new mechanism
Add new psuedo-field for WFJT node credential
Add validation for node related credentials
Add related config model to unified job
Use JobLaunchConfig model for launch RBAC check

Support credential overwrite behavior with multi-creds
  change modern manual launch to use merge behavior
Refactor JobLaunchSerializer, self.instance=None
Modularize job launch view to create "modern" data
Auto-create config object with every job
Add create schedule endpoint for jobs
2017-12-08 13:38:54 -05:00
Richard Bywater
9b05a41eec Add ability to append suffix to host names for Cloudforms Inventory
Allows for use of a suffix that will be appended to host names returned
from Cloudforms API if that suffix is not present.

For example with a suffix of 'example.org', the following results would
be shown for a particular Cloudforms host name:
someexample -> someexample.example.org
someexample.example.org -> someexample.example.org

The main use-case for this is, when one Inventory Source is returning
names that have a FQDN name whilst others are returning a shortname, to
ensure that the hosts in an inventory aren't effectively duplicated.
2017-12-05 14:47:33 +13:00
AlanCoding
eecf997856 add AWX meta extra_vars: WFJT + schedule 2017-12-04 15:33:05 -05:00
Aaron Tan
58ad214dcf Follow up fix #542
Relates
https://github.com/ansible/ansible-tower/issues/7737#issuecomment-348566452

Signed-off-by: Aaron Tan <jangsutsr@gmail.com>
2017-12-01 15:58:33 -05:00
AlanCoding
67867cf0c8 flake8: comply with new E722 rule 2017-12-01 12:16:44 -05:00
Ryan Petrello
a1f8f65add support specifying multiple vault IDs for a playbook run
see: https://github.com/ansible/awx/issues/352
2017-11-30 16:55:17 -05:00
Wayne Witzel III
ed28faa3db Use TMP instead of TMPDIR and only set it in RunProjectUpdate 2017-11-30 16:10:12 +00:00
Chris Meyers
6c89935521 correctly cascade job cancel
* Check the reason for a dependent project update failure. If it's
because of a cancel, then let the normal cancel mechanisms update the
jobs status and explanation. Do not update the dependent job's status
for a project update that was canceled, in the run code.
2017-11-27 12:34:55 -05:00
AlanCoding
fb29f68efc update smart inventory computed fields 2017-11-16 11:57:30 -05:00
Matthew Jones
03eca250d9 Fix an openshift issue writing the inventory file
Openshift was throwing an error here, though I'm not sure why it makes
a whole lot of difference to call fdopen() vs open(). This was
introduced when this method was unified under the new
ansible-inventory system. This fixes it for all cases. mkstemp(),
while not necessary, is a useful addition to keep from leaking
inventory details unnecessarily.
2017-11-15 13:12:54 -05:00
Matthew Jones
65d01d508b Fix an issue with handler tasks after celery upgrade
There's a bug in celery 4.X when using bound tasks as error
handlers. We don't actually need it to be bound especially since the
request object is now available in the function signature
2017-11-15 13:12:06 -05:00
Alan Rominger
dbe135991b Merge pull request #575 from AlanCoding/single_cancel
do not propogate cancel of inventory sync back up to project
2017-11-15 09:46:41 -05:00
Bill Nottingham
aaaae87aa7 Remove some environment variables the callback plugin doesn't actually use. 2017-11-14 15:57:49 -05:00
Ryan Petrello
28ce9b700e replace all Job/JT relations with a single M2M credentials relation
Includes backwards compatibility for now-deprecated .credential,
.vault_credential, and .extra_credentials

This is a building block for multi-vault implementation and Alan's saved
launch configurations (both coming soon)

see: https://github.com/ansible/awx/issues/352
see: https://github.com/ansible/awx/issues/169
2017-11-14 12:49:12 -05:00
AlanCoding
33574d70c8 do not propogate cancel of inventory sync back up to project 2017-11-13 08:44:00 -05:00
Wayne Witzel III
14c5123fda Update celery environ and tasks 2017-11-09 17:21:19 -05:00
Matthew Jones
5e37d6ea7e Remove unused TOWER_HOST and AWX_HOST env vars
These were conflicting with the new Tower credential
2017-11-02 10:44:06 -04:00
Chris Church
863b5e2e8e Output repr() of JSON in temporary inventory script to prevent Python from devouring escape sequences. 2017-11-01 12:59:49 -04:00
AlanCoding
d7ae95684c fix bug with dependent SCM inv updates
This change causes all SCM inventory updates to run a local
project sync unless they were specifically marked as a
dependency of an already-existing project update, as
opposed to just doing so on manual launch types.

This should be a more robust criteria.
2017-10-30 11:59:33 -04:00
Matthew Jones
f019452207 Merge pull request #551 from ansible/tower_inventory_source
Tower inventory source
2017-10-27 08:41:24 -04:00
Matthew Jones
5f3ebc26e0 Adding license checks for Tower inventory source
* For Tower the license must match between the source and destination
* For AWX the check is disabled
* Hosts imported from another Tower don't count against your license
  in the local Tower
* Fix up some issues with enablement
* Prevent slashes from being used in the instance filter
* Add &all=1 filter to make sure we pick up all hosts
2017-10-27 08:12:14 -04:00
Matthew Jones
fdc7f58bb4 Support passing instance filters to tower inventory src
* Switch ignore ssl errors to default on
* Application inventory source defaults for Tower src
2017-10-26 13:51:05 -04:00
Aaron Tan
e24e1fc1f0 Merge pull request #542 from jangsutsr/fix-7737
[3.2.2]support `AZURE_CLOUD_ENVIRONMENT`
2017-10-26 10:20:14 -04:00
Aaron Tan
f28b48a473 support AZURE_CLOUD_ENVIRONMENT
Relates #7737 of ansible-tower.

Signed-off-by: Aaron Tan <jangsutsr@gmail.com>
2017-10-26 10:11:21 -04:00
Ryan Petrello
c95d7d465a work around an ansible bug that can cause project syncs to fail
https://github.com/ansible/ansible-tower/issues/7746
https://github.com/ansible/ansible/issues/30064
2017-10-25 11:09:39 -04:00
AlanCoding
90f63774f4 flake8: comply with new E722 rule 2017-10-23 14:36:48 -04:00
Ryan Petrello
7b36630f47 Merge pull request #460 from ryanpetrello/cloudforms-cache-path
store cloudforms inventory cache files in the proper location on disk
2017-10-20 09:40:02 -04:00
AlanCoding
e660879a00 fix test fallout from 321 merge 2017-10-19 15:47:51 -04:00
Matthew Jones
5635f5fb49 Merge branch 'release_3.2.1' into devel
* release_3.2.1:
  fallback to empty dict when processing extra_data
  fix migration problem from 3.1.1
  move 0005a migration to 0005b
  feedback on ad hoc prohibited vars error msg
  Fix the way we include i18n files in sdist
  Fix migrations to support 3.1.2 -> 3.2.1+ upgrade path
  fix missing parameter to update_capacity method
  fix WARNING log when launching ad hoc command
  Validate against ansible variables on ad hoc launch
  do not allow ansible connection type of local for ad_hoc
  work around an ansible 2.4 inventory caching bug
  fix scan job migration unicode issue
  Assert isolated nodes have capacity set to 0 and restored based on version
  Set capacity to zero if the isolated node has an old version
2017-10-19 13:30:26 -04:00
Ryan Petrello
764356bf47 Merge pull request #459 from ryanpetrello/simplified-inventory-building
remove support for job-scoped auth tokens
2017-10-18 17:35:37 -04:00
Ryan Petrello
ea683344f5 remove support for job-scoped auth tokens
When Jobs and Adhoc Commands are launched, awx uses a job-scoped auth
token to dynamically fetch inventory via the awx REST API; this process
is complicated, hard to debug, and likely won't work going forward with
oauth2-based tokens in awx

see: https://github.com/ansible/awx/issues/21
2017-10-18 17:11:47 -04:00
Ryan Petrello
5895654538 store cloudforms inventory cache files in the proper location on disk
with process isolation enabled (which is the awx default), cloudforms
caches inventory script results on disk; awx should direct cloudforms to
store these cache files in a location that's exposed to the isolated
environment

see: ansible/ansible#31760
2017-10-17 17:07:21 -04:00
Ryan Petrello
59f9967dba store cloudforms inventory cache files in the proper location on disk
with process isolation enabled (which is the awx default), cloudforms
caches inventory script results on disk; awx should direct cloudforms to
store these cache files in a location that's exposed to the isolated
environment

see: ansible/ansible#31760
2017-10-17 17:06:48 -04:00
Bill Nottingham
058475c131 Set ANSIBLE_BECOME_ASK_PASS to avoid deprecation warning. 2017-10-17 16:00:16 -04:00
Matthew Jones
6fa283fc98 Merge branch 'release_3.2.0' into devel
* release_3.2.0: (66 commits)
  fix workflow maker lookup issues
  adding extra logic check for ansible_facts in smart search
  adding "admin_role" as a default query param for insights cred lookup
  changing insights cred lookup to not use hard coded cred type
  fix rounding of capacity percentage
  Catch potential unicode errors when looking up addrinfo
  fixing typo with adding query params for instance groups modal
  move percentage capacitty to variable
  Add unit test for inventory_sources_already_updated
  Check for inventory sources already updated from start args
  Fixed inventory completed jobs pagination bug by setting default page size
  Remove the logic blocking dependent inventory updates on callbacks
  fix instance group percentage
  Remove host-filter-modal import
  Fix partial hover highlight of host filter modal row
  Removed leading slash on basePath
  Fixed host nested groups pagination
  Added trailing slash to basePath
  Fixed nested groups pagination
  Fixed host_filter searching related fields
  ...
2017-10-10 16:09:43 -04:00