Commit Graph

77 Commits

Author SHA1 Message Date
Ryan Petrello
d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Ryan Petrello
57c22c20b2 add support for building swagger/OpenAPI JSON
to build, run `make swagger`; a file named `swagger.json` will be
written to the current working directory
2018-02-06 10:12:57 -05:00
Chris Meyers
c9ff3e99b8 celeryd attach to queues dynamically
* Based on the tower topology (Instance and InstanceGroup
relationships), have celery dyamically listen to queues on boot
* Add celery task capable of "refreshing" what queues each celeryd
worker listens to. This will be used to support changes in the topology.
* Cleaned up some celery task definitions.
* Converged wrongly targeted job launch/finish messages to 'tower'
queue, rather than a 1-off queue.
* Dynamically route celery tasks destined for the local node
* separate beat process

add support for separate beat process
2018-02-01 16:37:33 -05:00
Ryan Petrello
1e8c89f536 implement support for per-playbook/project/org virtualenvs
see: https://github.com/ansible/awx/issues/34
2018-01-09 22:47:01 -05:00
Matthew Jones
da0b686369 Adding jupyter notebook support to the AWX development environment
* Jupyter starts alongside the other awx services and is available on
  0.0.0.0:8888
* make target: make jupyter
* default settings in settings/development.py
* Added jupyter, matplotlib, numpy to dev dependencies
2017-12-05 23:46:18 -05:00
Wayne Witzel III
8faf588775 Update package versions, settings, and tooling 2017-11-09 17:17:30 -05:00
Alan Rominger
d7f869f566 Merge pull request #155 from AlanCoding/dev_ig_supervisor
functionality for dev + IG + supervisor configuration
2017-08-08 09:22:39 -04:00
Ryan Petrello
7db9b48e9c add a configurable for disabling the auto-generated isolated RSA key
some users won't want to utilize the RSA key we auto-generate for
isolated node SSH access, but will instead want to manage SSH
authentication by hand outside of Tower

see: https://github.com/ansible/ansible-tower/issues/7380
2017-08-03 17:16:28 -04:00
AlanCoding
4d5bedce74 functionality for dev + IG + supervisor configuration 2017-08-02 14:32:35 -04:00
Matthew Jones
b3b4a515e2 Refactor some tower periodic tasks to label as awx 2017-07-26 13:35:30 -04:00
AlanCoding
eaeff7e290 rename setting TOWER_VENV_PATH to AWX_VENV_PATH 2017-07-24 12:23:54 -04:00
AlanCoding
6e3978fe66 update TOWER_VENV_PATH setting to new awx location 2017-07-24 12:13:46 -04:00
AlanCoding
70b1b9c81d isolated connection timeout and log file for playbook out 2017-07-05 08:48:01 -04:00
Ryan Petrello
bd3bfbe1d1 Merge pull request #6640 from ryanpetrello/remove_use_venv
remove ANSIBLE_USE_VENV and TOWER_USE_VENV
2017-06-28 10:22:09 -04:00
Chris Meyers
fee16fe391 Merge pull request #6588 from chrismeyersfsu/feature-fact_cache
initial tower fact cache implementation
2017-06-22 09:58:28 -04:00
Ryan Petrello
45bdd9f747 improve readability of the honcho console logs in the dev environment
* colorize uwsgi and celery logs; DEBUG lines are green, WARN lines
  are yellow, ERROR lines (and tracebacks) are red
* pretty-print fact callback receiver JSON
* simplify the uwsgi log format so it's more legible
2017-06-22 09:19:32 -04:00
Chris Meyers
ec2e537f63 remove fache cache receiver 2017-06-21 13:45:27 -04:00
Ryan Petrello
2e608f4eac remove ANSIBLE_USE_VENV and TOWER_USE_VENV
modern Tower deployments (both production and dev container-based)
always build ansible and Tower in separate, distinct virtualenvs, not
the global site-packages
2017-06-19 16:11:15 -04:00
AlanCoding
dd1a261bc3 setup playbook and heartbeat for isolated deployments
* Allow isolated_group_ use in setup playbook
* Tweaks to host/queue registration commands complementing setup
* Create isolated heartbeat task and check capacity
* Add content about isolated instances to acceptance docs
2017-06-19 12:13:36 -04:00
Ryan Petrello
1ea03aa4c9 more isolated task execution tweaking
* set a more reasonable default `AWX_ISOLATED_CHECK_INTERVAL`
* make manual cancellation work for high values of
  `AWX_ISOLATED_CHECK_INTERVAL`
* remove the `/tmp/ansible_tower/jobs/` symlink directory

see: #6616
2017-06-16 15:37:07 -04:00
Ryan Petrello
422950f45d Support for executing job and adhoc commands on isolated Tower nodes (#6524) 2017-06-14 11:47:30 -04:00
AlanCoding
7e3a5fd2c2 move reload functionality to its own file 2017-02-15 13:42:49 -05:00
AlanCoding
186b672e4f move service definition into settings 2017-02-15 12:07:30 -05:00
Matthew Jones
8796af7e28 Make session and csrf cookies secure by default 2017-01-11 10:39:46 -05:00
Chris Church
092c2be0e7 Make any settings read-only that have been modified in custom Python config files. 2016-12-14 16:57:32 -05:00
Chris Meyers
5d205f1e1b remove deprecated mongo fact code and deps
* We have the requirements that users with Tower < 3.0.x must upgrade to
3.0.x before > 3.0.x. Thus, we are save to delete all mongo migration
code and deps.
2016-11-09 16:25:27 -05:00
Chris Meyers
5c71f1d6f7 allow all hosts during development 2016-11-02 10:45:42 -05:00
Matthew Jones
babe29ebfa Implement cluster health checks
* Add a local node queue to execute targeted jobs
* Add a setting for active cluster node id (per-node)
* Base the heartbeat time on the `modified` time on the Instance table
* Add periodic task that calls save() on the instance to update the
  heartbeat time if services are up
* Purge/update any ha/instance management commands
* Fix up CELERY_ROUTES settings data structure
2016-10-06 16:05:39 -04:00
Chris Church
6ebe45b1bd Configure Tower in Tower:
* Add separate Django app for configuration: awx.conf.
* Migrate from existing main.TowerSettings model to conf.Setting.
* Add settings wrapper to allow get/set/del via django.conf.settings.
* Update existing references to tower_settings to use django.conf.settings.
* Add a settings registry to allow for each Django app to register configurable settings.
* Support setting validation and conversion using Django REST Framework fields.
* Add /api/v1/settings/ to display a list of setting categories.
* Add /api/v1/settings/<slug>/ to display all settings in a category as a single object.
* Allow PUT/PATCH to update setting singleton, DELETE to reset to defaults.
* Add "all" category to display all settings across categories.
* Add "changed" category to display only settings configured in the database.
* Support per-user settings via "user" category (/api/v1/settings/user/).
* Support defaults for user settings via "user-defaults" category (/api/v1/settings/user-defaults/).
* Update serializer metadata to support category, category_slug and placeholder on OPTIONS responses.
* Update serializer metadata to handle child fields of a list/dict.
* Hide raw data form in browsable API for OPTIONS and DELETE.
* Combine existing licensing code into single "TaskEnhancer" class.
* Move license helper functions from awx.api.license into awx.conf.license.
* Update /api/v1/config/ to read/verify/update license using TaskEnhancer and settings wrapper.
* Add support for caching settings accessed via settings wrapper.
* Invalidate cached settings when Setting model changes or is deleted.
* Preload all database settings into cache on first access via settings wrapper.
* Add support for read-only settings than can update their value depending on other settings.
* Use setting_changed signal whenever a setting changes.
* Register configurable authentication, jobs, system and ui settings.
* Register configurable LDAP, RADIUS and social auth settings.
* Add custom fields and validators for URL, LDAP, RADIUS and social auth settings.
* Rewrite existing validator for Credential ssh_private_key to support validating private keys, certs or combinations of both.
* Get all unit/functional tests working with above changes.
* Add "migrate_to_database_settings" command to determine settings to be migrated into the database and comment them out when set in Python settings files.
* Add support for migrating license key from file to database.
* Remove database-configuable settings from local_settings.py example files.
* Update setup role to no longer install files for database-configurable settings.

f 94ff6ee More settings work.
f af4c4e0 Even more db settings stuff.
f 96ea9c0 More settings, attempt at singleton serializer for settings.
f 937c760 More work on singleton/category views in API, add code to comment out settings in Python files, work on command to migrate settings to database.
f 425b0d3 Minor fixes for sprint demo.
f ea402a4 Add support for read-only settings, cleanup license engine, get license support working with DB settings.
f ec289e4 Rename migration, minor fixmes, update setup role.
f 603640b Rewrite key/cert validator, finish adding social auth fields, hook up signals for setting_changed, use None to imply a setting is not set.
f 67d1b5a Get functional/unit tests passing.
f 2919b62 Flake8 fixes.
f e62f421 Add redbaron to requirements, get file to database migration working (except for license).
f c564508 Add support for migrating license file.
f 982f767 Add support for regex in social map fields.
2016-09-26 22:14:47 -04:00
Matthew Jones
d8ea0920c5 Update configuration and playbook for rabbitmq
* Purge remaining zeromq settings
* Make sure we have CALLBACK_QUEUE defined where needed
* Update setup playbook to not be tied to old ha primary/secondary concept
2016-09-16 11:35:46 -04:00
Matthew Jones
f5d2c5c18a Merge branch 'ramparts_and_ha' into devel
* ramparts_and_ha: (21 commits)
  Rename database migrations for devel integration
  Integrate callback receiver refactoring
  Fix an issue running jobs in the cluster
  Implement a more dynamic celery queue system
  Purge old munin monitors and tools
  Refactor Tower HA Instance logic and models
  Docker compose improvements
  Initial Docker Compose workflow for Tower cluster
  Add memcached role for setup playbook
  Removing qpid from deb packaging
  Refactor rabbitmq role
  Integrate memcached into setup playbook
  Remove mongodb shutdown task
  Remove dependency on erlang_sd_notify
  Add initial rabbitmq role
  Initial rabbitmq setup playbook integration
  Update development environment for rabbit
  Replace qpid with rabbitmq
  Remove redis role from setup playbook
  Update qpid packaging, remove migrations
  ...
2016-09-15 10:19:57 -04:00
Matthew Jones
ab395b0009 Integrate callback receiver refactoring
* Drop ZMQ as the communication mechanism between job_event_callback and
  callback_receiver
* Setup queue and exchange for callback broker communication
* Refactor event plugin and callback receiver to efficiently handle
  message submission and processing
* Integrate django caching for parent processing
2016-09-14 11:42:13 -04:00
Matthew Jones
799f321760 Fix an issue running jobs in the cluster
The old VENV_PATH settings were still in place
2016-09-12 10:43:40 -04:00
Leigh Johnson
6d643ca339 Add uncached template loaders to development settings 2016-09-07 15:47:50 -04:00
Alan Rominger
2fada7a6ad fix flake8 2016-09-01 15:34:06 -04:00
Chris Meyers
ac2f0bdf57 handle mime svg in devel 2016-09-01 12:53:26 -04:00
Matthew Jones
116e40dbe7 Fix some venv issues related to the new dev env 2016-08-30 14:21:11 -04:00
Chris Meyers
ae73be12f1 remove statsd 2016-05-10 11:56:04 -04:00
Matthew Jones
6f79ec8830 Revert "Merge pull request #1534 from chrismeyersfsu/fix-config_performance"
This reverts commit 7cc144fa59f157a5e1789a913e0d9a31720141ec, reversing
changes made to 86af9950e754b06f40cb293a36ab365f589273db.
2016-04-14 11:40:07 -04:00
Chris Meyers
043f97fe35 fixes config endpoint performance issue 2016-04-13 17:28:38 -04:00
Matthew Jones
46bbc83d0d Changes for Tower virtualenv support
* Break requirements down into ansible and tower reqs
* Generate separate Ansible and Tower virtual environments
* Install appropriate requirements files into each one
* Modify development tools to use these venvs instead of our old
  site-packages
* Modify settings to indicate venv enablement and location of venvs
* Modify tasks to use the proper virtual environment for its purpose
2016-03-21 12:11:19 -04:00
Chris Meyers
7ffe46fc74 add postgres Fact model, update views, tests
* awx.main.models Fact added
* view host fact and timeline updated to use new Postgres Fact model instead of Mongo
* Removed license set start Mongo logic
* added View tests
* added Model tests
* Removed mongo fact unit tests
* point at modified jsonbfield that supports sqlite storage driver
* postgresify fact cache receiver
* test OPTIONS endpoint
* Note: single fact view not implemented yet.
2016-02-24 14:38:54 -05:00
Chris Church
60224cdbe4 Update Django to 1.8 and DRF to 3.3, add new Django migrations, update serializers/pagination/metadata, update browsable API styling. 2016-02-02 17:48:04 -05:00
Joshua "jag" Ginsberg
f25b76aee9 Integrate statsd metrics into ansible playbook execution.
* Add dependencies for pystatsd and django-statsd-mozilla
* Default turned off except for development environment
* Modify docker-compose to install statsd/graphite host
2016-01-26 16:54:24 -05:00
Chris Church
84e89ab6f8 Fix auth-related errors breaking unit tests. 2015-10-02 16:16:30 -04:00
Chris Church
2ba5e06e2c Add support for single-sign on using python-social-auth (with Google/Github OAuth2 and SAML support). Add support for RADIUS as another authentication backend. 2015-10-02 14:57:27 -04:00
Matthew Jones
3d7f25218a Add api side support for pendo
Also adds setup playbook support for pendo.  This change anticipates
that the default for new installations will be 'detailed'
2015-09-25 15:52:42 -04:00
Matthew Jones
5df23b95f6 Revert "Improve stdout behavior by implementing a downloadable stdout." 2015-07-24 14:54:13 -04:00
Matthew Jones
d9f5dee773 Change default stdout temp directory
* Use /var/lib/awx on production configurations
* Use /tmp for development configurations
2015-07-22 16:31:43 -04:00
John Mitchell
32d1c0e4db fixed copyright date 2015-06-11 16:10:23 -04:00