Commit Graph

446 Commits

Author SHA1 Message Date
Ryan Petrello 45bdd9f747 improve readability of the honcho console logs in the dev environment
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
  are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
2017-06-22 09:19:32 -04:00
Chris Meyers ec2e537f63 remove fache cache receiver 2017-06-21 13:45:27 -04:00
AlanCoding dd1a261bc3 setup playbook and heartbeat for isolated deployments
* Allow isolated_group_ use in setup playbook
* Tweaks to host/queue registration commands complementing setup
* Create isolated heartbeat task and check capacity
* Add content about isolated instances to acceptance docs
2017-06-19 12:13:36 -04:00
Ryan Petrello 1ea03aa4c9 more isolated task execution tweaking
* set a more reasonable default `AWX_ISOLATED_CHECK_INTERVAL`
* make manual cancellation work for high values of
  `AWX_ISOLATED_CHECK_INTERVAL`
* remove the `/tmp/ansible_tower/jobs/` symlink directory

see: #6616
2017-06-16 15:37:07 -04:00
Ryan Petrello 422950f45d Support for executing job and adhoc commands on isolated Tower nodes (#6524) 2017-06-14 11:47:30 -04:00
Chris Meyers aeb68f02f6 change settings comments 2017-06-07 15:03:53 -04:00
Chris Meyers ba5f553f0c fix up scm inventory updates
* implicit project updates inherit inv update instance group
* added needed settings for SCM group filter and others
2017-06-07 14:51:56 -04:00
Aaron Tan a993025e0d Merge pull request #6053 from jangsutsr/named_url
Named URL
2017-05-31 17:02:42 -04:00
Aaron Tan f25391fe86 Implement named URL feature. 2017-05-31 12:09:54 -04:00
Wayne Witzel III e28cd97ffb Add smart_inventories endpoint to Host 2017-05-26 10:16:12 -04:00
Bill Nottingham e93a61ac53 Allow setting the enabled/disabled flag for more than just cloud inventory sources.
Leverage the same _ENABLED_VAR (and other) variables that exist for cloud sources, and set some defaults for custom scripts.
2017-05-25 16:51:09 -04:00
Matthew Jones 1a4a6273a4 Merge branch 'rampart_groups_setup_playbook' into devel
* rampart_groups_setup_playbook:
  Updating changelog for Instance Groups
  Fix an incorrect reference on instance group jobs list
  Purge remaining references to rampart groups
  Simplify can_access for instance groups on job templates
  Adding Instance Group permissions and tests
  Increase test coverage for task scheduler inventory updates
  Exit logic fixes for instance group tools
  View Fixes for instance groups
  new view to allow associations but no creations
  Updating acceptance documentation and system docs
  Updating unit tests for task manager refactoring
  Update views and serializers to support instance group (ramparts)
  Implementing models for instance groups, updating task manager
  Updating the setup playbook to support instance group installation
  Add nginx to server start and switch back to first tmux win
  Fix an issue where the local queue wouldn't use the rabbitmq name
2017-05-12 13:40:30 -04:00
Matthew Jones 705f8af440 Update views and serializers to support instance group (ramparts)
* includes top level views for instances and instance groups and
  extending those views to be able to view running jobs
* Associative endpoints on Organizations, Inventories, and Job
  Templates
* Related and summary field entries where appropriate
* Adding job model references to executing instance group
* Fix up default queue properties for clustering from the settings file
* Update production and default settings for instance queues in settings
2017-05-10 12:33:03 -04:00
Aaron Tan cfb633e8a6 Dependency Updates
* Dynamic Inventory Source
Template against ansible 2.3 dynamic inventory sources.
The major change is removal of `rax.py`. Most upstream scripts except
`foreman.py` has quite trivial coding style changes, or minor functional
extensions  that does not affect Tower inventory update runs.
`foreman.py`, on the other hand, went through quite a major refactoring,
but functionalities stay the same.

Major python dependency updates include apache-libcloud (1.3.0 -->
2.0.0), boto (2.45.0 --> 2.46.1) and shade (1.19.0 --> 1.20.0). Minor
python dependency updates include indirect updates via `pip-compile`,
which are determined by base dependencies.

Some minor `task.py` extensions:
 - `.ini` file for ec2 has one more field `stack_filter=False`, which
   reveals changes in `ec2.py`.
 - `.ini` file for cloudforms will catch these four options from
   `source_vars_dict` of inventory update: `'version', 'purge_actions',
   'clean_group_keys', 'nest_tags'`. These four options have always been
   available in `cloudforms.py` but `cloudforms.ini.example` has not
   mentioned them until the latest version. For consistency with upstream
   docs, we should make these fields available for tower user to customize.
 - YAML file of openstack will catch ansible options `use_hostnames`,
   `expand_hostvars` and `fail_on_errors` from `source_vars_dict` of
   inventory update as a response to issue #6075.

* Remove Rackspace support
Supports of Rackspace as both a dynamic inventory source and a cloud
credential are fully removed. Data migrations have been added to support
arbitrary credential types feature and delete rackspace inventory
sources.

Note also requirement `jsonschema` has been moved from
`requirements.txt` to `requirements.in` as a primary dependency to
reflect it's usage in `/main/fields.py`.

Connected issue: #6080.

* `pexpect` major update
`pexpect` stands at the very core of our task system and underwent a
major update from 3.1 to 4.2.1. Although verified during devel, please
still be mindful of any suspicious issues on celery side even after this
PR gets merged.

* Miscellaneous
 - requests now explicitly declared in `requirements.in` at version 2.11.1
   in response to upstream issue
 - celery: 3.1.17 -> 3.1.25
 - django-extensions: 1.7.4 -> 1.7.8
 - django-polymorphic: 0.7.2 -> 1.2
 - django-split-settings: 0.2.2 -> 0.2.5
 - django-taggit: 0.21.3 -> 0.22.1
 - irc: 15.0.4 -> 15.1.1
 - pygerduty: 0.35.1 -> 0.35.2
 - pyOpenSSL: 16.2.0 -> 17.0.0
 - python-saml: 2.2.0 -> 2.2.1
 - redbaron: 0.6.2 -> 0.6.3
 - slackclient: 1.0.2 -> 1.0.5
 - tacacs_plus: 0.1 -> 0.2
 - xmltodict: 0.10.2 -> 0.11.0
 - pip: 8.1.2 -> 9.0.1
 - setuptools: 23.0.0 -> 35.0.2
 - (requirements_ansible.in only)kombu: 3.0.35 -> 3.0.37
2017-05-08 12:03:02 -04:00
AlanCoding 8e6020436c modularization of inventory_import command
This separates file parsing logic that was mixed
in with other important code inside of the
inventory import command.

The logic around MemObject data structures was
moved to utils, and the file parsing was moved
to a legacy module. As of this commit, that
module can operate within the Tower environment
but it will be removed.

Also refactor the loggers to fix old bug and
work inside of the different contexts - the
Loader classes, mem objects, and hopefully
the inventory modules eventually.
2017-05-01 14:37:42 -04:00
Matthew Jones e1e83598e9 Merge branch 'release_3.1.3' into devel
* release_3.1.3: (52 commits)
  ack fact scan messages
  making ldap user/group search fields into codemirror instances
  removing UI parsing for LDAP User and Group Search fields
  Allow exception view to accept all valid HTTP methods.
  Restore ability of parsing extra_vars string for provisioning callback.
  Fix up backup/restore role broken in f7a8e45809758322d9ee41c5305850dd70ed5faf
  Stop / start ansible-tower-service during restores
  value_to_python should encode lookup fields as ascii
  fix brace interpolation on standard out pane
  Adjust some hardcoded usages of 'awx' to use 'aw_user' and 'aw_group'.
  Pull Spanish updates from Zanata
  Temporarily grant awx user createdb role
  Stop giving ownership of backups to postgres
  don't display chunked lines'
  Add dropdown li truncation with ellipsis
  CTiT -> adhoc modules should allow the user to add new modules
  Remove task that was replacing the supervisor systemd tmp file
  Fix failing supervisorctl commands on RH-based distros
  Give ownership of the supervisor socket to awx
  Setting for external log emissions cert verification
  ...
2017-04-28 13:57:04 -04:00
Aaron Tan 8d2ee8c30f Refactor log handler and support TCP/UDP communications 2017-04-25 11:07:57 -04:00
Ryan Petrello ba259e0ad4 Introduce a new CredentialTemplate model
Credentials now have a required CredentialType, which defines inputs
(i.e., username, password) and injectors (i.e., assign the username to
SOME_ENV_VARIABLE at job runtime)

This commit only implements the model changes necessary to support the
new inputs model, and includes code for the credential serializer that
allows backwards-compatible support for /api/v1/credentials/; tasks.py
still needs to be updated to actually respect CredentialType injectors.

This change *will* break the UI for credentials (because it needs to be
updated to use the new v2 endpoint).

see: #5877
see: #5876
see: #5805
2017-04-21 15:42:26 -04:00
AlanCoding 3af0f505d8 Setting for external log emissions cert verification
Stopgap solution for issue for 3.1.3 where Splunk servers with a
self-signed certificate could not receive logs from Tower.
Users should set `LOG_AGGREGATOR_VERIFY_CERT = False` in settings
to use this.
2017-04-17 14:50:01 -04:00
Aaron Tan fd4aaa6ca9 Implement tacacs+ auth backend. 2017-04-12 16:19:42 -04:00
Aaron Tan f2c99eeaf5 Basic tacacs+ CTiT. 2017-04-12 16:19:42 -04:00
Ryan Petrello b3733d233e always use async http logging - even in celery workers
additionally, add a timeout to requests to avoid thread starvation

see: #5718
see: #5729
2017-03-15 13:17:01 -04:00
Matthew Jones 2656e3f3fa Insights project api side integration
* Extend scm credential type to support username and password for
  redhat portal accounts
* Update project update playbook to work with remote insights server
2017-03-07 16:48:44 -05:00
Ryan Petrello 45819f6b9a requeue websocket messages that don't (yet) have an established user
There's a race between our `ws_connect` and `ws_receive` methods;
it's possible to fall into a scenario where we're handling a legitimate
message *before* django-channels is able to persist the `user_id` into
the channel session.  This results in a scenario where a user can open
a browser tab and never receive new websocket messages.  In this
scenario, we should just toss the message back into the queue and try
again later (up to a reasonable limit of retries).
2017-02-28 11:12:59 -05:00
Wayne Witzel III c919f1ef30 Merge pull request #5487 from wwitzel3/issue-5341
explicit ordering in workflow_nodes views
2017-02-27 10:06:58 -05:00
Ryan Petrello 5dfdef6e6b retain the default interstitial value for TOWER_URL_BASE
to support #5470, check for the presence of this special interstitial
value and replace it with the request hostname for settings OPTIONS
requests
2017-02-23 16:38:16 -05:00
Ryan Petrello 94ad12ab0d set the default in OPTIONS for TOWER_URL_BASE to the request hostname
see: #5470
2017-02-22 11:16:08 -05:00
AlanCoding 7c5856cee6 set expiration time for the disposable system tasks 2017-02-21 15:54:18 -05:00
Wayne Witzel III 87e0c9020f explicit ordering in workflow_nodes views 2017-02-21 02:33:45 -05:00
Wayne Witzel III a55e79e94d make OrderingFilter part of our default filters 2017-02-17 16:16:28 -05:00
AlanCoding 1c6f037076 stop using Django NullHandler which is being removed in future version 2017-02-09 11:53:51 -05:00
Matthew Jones 4ed054f568 Implement optional message persistence on callback events 2017-02-08 11:11:14 -05:00
Ryan Petrello f4d55659f0 enforce a sane default OPT_NETWORK_TIMEOUT for LDAP connections
see: #5208
2017-02-07 12:37:05 -05:00
Matthew Jones e389084298 Add a periodic celery task to clean up local stdout files
This works in tandem with the on-demand stdout file generator and
decouples stdout locally from the persistent database representation
ob the job itself
2017-01-31 15:52:45 -05:00
Matthew Jones df249eda2e Raise default job event worker queue size 2017-01-23 10:47:47 -05:00
Matthew Jones 7ca496b6ef Merge branch 'release_3.1.0' into multi_worker_callbacks 2017-01-18 20:39:15 -05:00
Bill Nottingham fb6d3ad547 Merge pull request #4702 from wenottingham/csrf-fu
Set CSRF_COOKIE_AGE to 'None'.
2017-01-13 12:47:26 -05:00
Bill Nottingham ef27d771fb Set CSRF_COOKIE_AGE to 'None'.
This causes them to be sent as session cookies, not 1-year cookies.
2017-01-12 16:25:56 -05:00
jlmitch5 7c8c42bfc1 Merge branch 'release_3.1.0' into jobResultsPerf 2017-01-12 16:15:09 -05:00
Bill Nottingham 946a5bde9a Merge pull request #4635 from wenottingham/time-after-time
Don't hardcode a timezone
2017-01-11 15:20:44 -05:00
Matthew Jones 8796af7e28 Make session and csrf cookies secure by default 2017-01-11 10:39:46 -05:00
Matthew Jones e594a3ee3e Implement a worker-based callback receiver system 2017-01-10 14:25:41 -05:00
John Mitchell b8ce360227 update api max events header based on performance findings 2017-01-10 11:30:56 -05:00
Bill Nottingham efe0f86705 Set TIME_ZONE to None.
This causes Tower to use the system timezone.
2017-01-10 10:51:23 -05:00
Alan Rominger 6921cbfdd0 Merge pull request #4538 from AlanCoding/celery_log_reset
Celery log reset on CTiT change / task start
2017-01-06 15:36:45 -05:00
Matthew Jones 3006e357f6 Add a hint header for max job events for the UI 2017-01-05 15:16:36 -05:00
AlanCoding cc4aa49556 pool restart with thread-based external log config
for CTiT enablement of celery task Tower logs
2017-01-04 14:53:49 -05:00
Matthew Jones 60fa34e8cc Fix a redis reference issue
We are no longer using redis for Celery
2017-01-04 14:40:40 -05:00
Bill Nottingham acaa8a75b2 Update boto to 2.45.0, add eu-west-2 and ca-central-1 2016-12-20 16:24:57 -05:00
Chris Meyers d985f07838 use vmware uuid that is most unique 2016-12-19 17:12:52 -05:00