Commit Graph

381 Commits

Author SHA1 Message Date
AlanCoding
f3427d1359 Logging Integration, ELK docker-compose as update file 2016-11-29 13:09:47 -05:00
Chris Church
2e8faeaf0d AzureAD auth updates. Add auth backend to settings, add microsoft icon to fontcustom, add azuread support to login modal. 2016-11-28 22:12:50 -05:00
Chris Church
0b76546ea6 Merge pull request #4080 from cchurch/event-stdout-max-bytes-display
Add configurable limit to size of stdout shown for a single job event
2016-11-22 14:50:38 -05:00
Matthew Jones
ddaf4ff2d0 Merge branch 'fix_fact_caching' into devel
* fix_fact_caching:
  Convert the fact data structure to the right type
  Fixing up fact cachine related unit tests
  Port fact caching system to rabbitmq
2016-11-21 23:15:13 -05:00
Matthew Jones
c6ebdb9b26 Switch to using callback queue by default
* Fix an issue where we grab the wrong setting for the callback
  connection
* Remove old ansible.cfg that we shipped with the setup playbook for
  centos 6
2016-11-21 13:26:32 -05:00
Chris Church
b9eb619697 Add configurable limit to size of stdout shown for a single job event in list view. 2016-11-21 11:30:11 -05:00
Matthew Jones
3b07d9745d Port fact caching system to rabbitmq
* Purge all references to zmq also
* New setting to control the queue it works on
2016-11-18 09:44:17 -05:00
Aaron Tan
9e4655419e Fix flake8 E302 errors. 2016-11-15 20:59:39 -05:00
Aaron Tan
473ed016de Fix flake8 E305 errors. 2016-11-15 16:32:27 -05:00
Matthew Jones
78b8876ed9 Support expiring of capacity if a node is down
For a certain amount of time
2016-11-10 09:52:04 -05:00
Chris Meyers
5d205f1e1b remove deprecated mongo fact code and deps
* We have the requirements that users with Tower < 3.0.x must upgrade to
3.0.x before > 3.0.x. Thus, we are save to delete all mongo migration
code and deps.
2016-11-09 16:25:27 -05:00
Chris Meyers
b66256eb98 fixes test_settings unit tests 2016-11-08 11:10:52 -05:00
sundeep-co-in
688b5fef68 resolve conflict: settings/defaults.py 2016-11-02 15:19:57 +05:30
Chris Meyers
13c89ab78c HAify job schedules and more task_manager renaming 2016-11-01 13:50:42 -05:00
Chris Meyers
555f0bb90f project and jobs running correctly 2016-11-01 13:50:42 -05:00
Matthew Jones
c6cf02a602 Merge branch 'release_3.0.3' into devel
* release_3.0.3: (55 commits)
  Revert "Revert "Add needed types for selinux change""
  Revert "Add needed types for selinux change"
  interpret backslash escapes when displaying url in welcome message
  Bump the SELinux policy version
  Add needed types for selinux change
  Update SELinux policy to allow httpd_t to execute files in lib_t and var_lib_t
  Bumping changelog for 3.0.3
  Update rax.py inventory
  Revert "filter internal User.admin_roles from the /roles API list view"
  fix spelling of disassociated
  Resolves 404 when assigning resources/users to organizations in card view. Sidesteps a bug in the Refresh() utility, where pagination calculations are not made against filtered results.
  Sync azure changes to Tower virtual environment
  Add regions here as well.
  Also bump boto for new regions, per ryansb.
  More regions!
  Revert "bump shade version"
  bump shade version
  Hack copying of job_template.related.survey_spec into ui job copy flow, resolves #3737
  Revert "bump shade version"
  bump shade version
  ...
2016-11-01 11:49:28 -04:00
sundeep-co-in
4237b2e207 resolve conflict: index.html 2016-10-31 23:40:56 +05:30
Bill Nottingham
83cd055f10 Merge pull request #3569 from wenottingham/forever_blowing_bubbles
Use bubblewrap (https://github.com/projectatomic/bubblewrap) instead of proot.
2016-10-28 11:55:14 -04:00
sundeep-co-in
d65e121b19 flake8 compliance 2016-10-28 14:07:08 +05:30
AlanCoding
1fec29dcb3 remove awx.compat, since it looks like it is not necessary 2016-10-26 17:22:03 -04:00
sundeep-co-in
576db5120f resolve conflict 2016-10-25 16:18:14 +05:30
Matthew Jones
73b5b47e94 flake8 cleanup 2016-10-20 15:27:59 -04:00
Matthew Jones
6e22460f1e Add scm revision to the job model
This also cleans up flake8 issues
2016-10-20 15:18:05 -04:00
Matthew Jones
e6bcc039f2 Rearchitect project update strategy
* Instead of using a fanout for project updates initial project
  updates will sync the latest commit hash
* Prior to a node running a job it will ensure that the latest project
  is synced
2016-10-20 13:30:44 -04:00
Bill Nottingham
5c69ed2e9f More regions! 2016-10-19 16:15:09 -04:00
sundeep-co-in
78a8ce9479 django i18n 2016-10-18 18:10:39 +05:30
AlanCoding
8d89d68d5e flake8 fixes for Jenkins 2016-10-14 09:46:41 -04:00
Chris Church
8883738a7c Fix issue when string list settings field is null. Resolves #3683. 2016-10-12 14:47:22 -04:00
James Laska
77a56b3dfd Properly recognize foreman.enabled
The ENABLED_VALUE must always be a string.  Thanks to @cchurch for the
suggestion.

Resolves #3632
2016-10-07 09:50:18 -04:00
Wayne Witzel III
5d8e7c17c2 Merge branch 'devel' into jtabor-sockets 2016-10-07 09:50:03 -04:00
James Laska
5881736769 Properly detect enabled foreman inventory
Related #3632
2016-10-07 09:49:31 -04:00
Matthew Jones
babe29ebfa Implement cluster health checks
* Add a local node queue to execute targeted jobs
* Add a setting for active cluster node id (per-node)
* Base the heartbeat time on the `modified` time on the Instance table
* Add periodic task that calls save() on the instance to update the
  heartbeat time if services are up
* Purge/update any ha/instance management commands
* Fix up CELERY_ROUTES settings data structure
2016-10-06 16:05:39 -04:00
Wayne Witzel III
1a9843d5d1 adding initial channels integration 2016-10-04 14:04:52 -07:00
Wayne Witzel III
530a5c0c88 merging devel 2016-10-03 10:35:50 -04:00
Chris Meyers
203df91a5d more robust test mode checking 2016-10-03 09:28:44 -04:00
Bill Nottingham
8265934c2d Use bubblewrap (https://github.com/projectatomic/bubblewrap) instead of proot. 2016-09-29 16:23:30 -04:00
Chris Meyers
dd3c3c9f29 Merge branch 'feature-ha_task_manager' into devel 2016-09-29 10:27:17 -04:00
Wayne Witzel III
258ddba333 add channels to reqs and defaults 2016-09-27 18:39:19 -04:00
Chris Meyers
d65120538d scheduler messages need not be durable 2016-09-27 16:05:30 -04:00
Chris Meyers
04f69727f2 fully message driven job execution
TODO:
* Need a distributed lock (leverage postgres)
* Less memory-intensive graph representation
* Maybe serializer/deserializer graph to database
  * Iterative graph building instead of full rebuild.
2016-09-27 14:16:18 -04:00
Chris Meyers
cdb65ccac9 replace task manager with event driven scheduler 2016-09-27 14:16:18 -04:00
Chris Meyers
cc90204b0f task manager using messages
* First pass, adapt singleton task manager to process messages and run
jobs based on events instead of a busy loop.
* Still need to make message handing run in celery, not in a consumption
loop
2016-09-27 14:05:29 -04:00
Chris Church
6ebe45b1bd Configure Tower in Tower:
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.

f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
2016-09-26 22:14:47 -04:00
Matthew Jones
f3a8eb9daf Merge pull request #3509 from ansible/ha_installer
Improvements to the setup/installer to support new HA workflows
2016-09-16 15:53:38 -04:00
Matthew Jones
d8ea0920c5 Update configuration and playbook for rabbitmq
* Purge remaining zeromq settings
* Make sure we have CALLBACK_QUEUE defined where needed
* Update setup playbook to not be tied to old ha primary/secondary concept
2016-09-16 11:35:46 -04:00
Matthew Jones
3de4aae548 Fixing up HA induced flake8 issues 2016-09-15 13:51:17 -04:00
Chris Meyers
bf3c68d8b4 Merge pull request #3497 from chrismeyersfsu/fix-flake8_deps
Fix flake8 deps
2016-09-15 13:33:41 -04:00
Chris Meyers
f06b212266 flake8 fixes 2016-09-15 10:25:36 -04:00
Matthew Jones
f5d2c5c18a Merge branch 'ramparts_and_ha' into devel
* ramparts_and_ha: (21 commits)
  Rename database migrations for devel integration
  Integrate callback receiver refactoring
  Fix an issue running jobs in the cluster
  Implement a more dynamic celery queue system
  Purge old munin monitors and tools
  Refactor Tower HA Instance logic and models
  Docker compose improvements
  Initial Docker Compose workflow for Tower cluster
  Add memcached role for setup playbook
  Removing qpid from deb packaging
  Refactor rabbitmq role
  Integrate memcached into setup playbook
  Remove mongodb shutdown task
  Remove dependency on erlang_sd_notify
  Add initial rabbitmq role
  Initial rabbitmq setup playbook integration
  Update development environment for rabbit
  Replace qpid with rabbitmq
  Remove redis role from setup playbook
  Update qpid packaging, remove migrations
  ...
2016-09-15 10:19:57 -04:00
Matthew Jones
56d4210047 Implement Performance improvements shown in #3492
* Make callback workers tunable
* Disable callback worker recycling - doesn't appear to be needed
  anymore
* Tweak pexpect behavior by limiting its read buffer size and search
  window
* Use copy instead of deepcopy for job event callback emitter censor
2016-09-14 16:08:47 -04:00