Commit Graph

616 Commits

Author SHA1 Message Date
chris meyers
7433aab258 switch memcached from tcp to unix domain socket 2020-04-06 08:35:12 -04:00
Ryan Petrello
9fe2211f82 get rid of the activity stream middleware
it has bugs and is very confusing

see: https://github.com/ansible/tower/issues/4037
2020-04-01 16:02:42 -04:00
Ryan Petrello
42336355bb bump django-extensions version to address a bug in shell_plus
see: https://github.com/ansible/awx/pull/6441
see: e8d5daa06e
2020-03-31 13:39:13 -04:00
Ryan Petrello
301d6ff616 make the job event bigint migration chunk size configurable 2020-03-27 09:28:10 -04:00
Ryan Petrello
61a0d1f77b fix busted shell_plus in the development environment
for some reason (unsure why), django-extensions has begun noticing
ipython importability and treating "shell_plus" as "start an IPython
notebook by default

it could be that this is a bug in django-extensions that will be fixed
soon, but for now, this fixes the issue
2020-03-26 13:37:13 -04:00
chris meyers
770b457430 redis socket support 2020-03-18 16:10:19 -04:00
chris meyers
d6594ab602 add broadcast websocket metrics
* Gather brroadcast websocket metrics and push them into redis every
configurable seconds.
* Pop metrics from redis in web view layer to display via the api on
demand
2020-03-18 16:10:18 -04:00
chris meyers
b6b9802f9e increase per-channel capacity
* 100 is the default capacity for a channel. If the client doesn't read
the socket fast enough, websocket messages can and will be lost. This
increases the default to 10,000
2020-03-18 16:10:18 -04:00
chris meyers
3b9e67ed1b remove channel group model
* Websocket user session <-> group subscription membership now resides
in Redis rather than the database.
2020-03-18 16:10:18 -04:00
chris meyers
3c5c9c6fde move broadcast websocket out into its own process 2020-03-18 16:10:18 -04:00
chris meyers
03b73027e8 websockets aware of Instance changes
* New tower nodes that are (de)registered in the Instance table are seen
by the websocket layer and connected to or disconnected from by the
websocket broadcast backplane using a polling mechanism.
* This is especially useful for openshift and kubernetes. This will be
useful for standalone Tower in the future when the restarting of Tower
services is not required.
2020-03-18 16:10:17 -04:00
chris meyers
be58906aed remove kombu 2020-03-18 16:10:17 -04:00
chris meyers
dc6c353ecd remove support for multi-reader dispatch queue
* Under the new postgres backed notify/listen message queue, this never
actually worked. Without using the database to store state, we can not
provide a at-most-once delivery mechanism w/ multi-readers.
* With this change, work is done ONLY on the node that requested for the
work to be done. Under rabbitmq, the node that was first to get the
message off the queue would do the work; presumably the least busy node.
2020-03-18 16:10:16 -04:00
chris meyers
2a2c34f567 combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
chris meyers
c8eeacacca POC channels 2 2020-03-18 16:10:12 -04:00
Ryan Petrello
b5e5fea117 respect home directory symlinks for BASE_VENV_PATH 2020-03-03 16:47:38 -05:00
Ryan Petrello
38a08d163c get rid of celery/celerybeat
alternative to https://github.com/ansible/awx/pull/2530 which makes use
of https://pypi.org/project/schedule/

this doesn't have support for any persistence (like how celery beat uses
a shelve file), because all of our periodic jobs run at most every few
minutes
2020-02-10 17:32:02 -05:00
Ryan Petrello
3c31e0ed16 some more minor callback cleanup and development tweaks 2020-01-27 17:18:09 -05:00
Ryan Petrello
78b00652bd add the ability to enable profiling for the callback receiver workers 2020-01-27 12:03:53 -05:00
softwarefactory-project-zuul[bot]
23b2b136d6 Merge pull request #5707 from AlanCoding/bulk_create_logs
Allow CTiT log level to log bulk_create lines

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-01-22 15:04:17 +00:00
Bill Nottingham
44e176dde8 Change how analytics is gathered to only gather once per interval 2020-01-21 11:40:51 -05:00
Ben Thomasson
652a428438 Fixes crontab for gather_analytics to run once every 4 hours 2020-01-20 13:30:10 -05:00
AlanCoding
ceed6f8d9b Allow CTiT log level to log bulk_create lines 2020-01-20 12:41:10 -05:00
Ryan Petrello
306f504fb7 optimize the callback receiver to buffer writes on high throughput
additionaly, optimize away several per-event host lookups and
changed/failed propagation lookups

we've always performed these (fairly expensive) queries *on every event
save* - if you're processing tens of thousands of events in short
bursts, this is way too slow

this commit also introduces a new command for profiling the insertion
rate of events, `awx-manage callback_stats`

see: https://github.com/ansible/awx/issues/5514
2020-01-14 12:04:26 -05:00
Seth Foster
7873d08311 Update pip and setuptools in requirements txt
Versions selected to be pre-19 pip
due to unresolved issues with the build systems

Upgrade everything, party on

document new process

rotate license files

fix Swagger schema generation target

Remove --ignore-installed flag
2020-01-07 17:14:32 -06:00
Ryan Petrello
4a6147d4c2 add the ability to generate dot graphs for per-request profiling 2020-01-04 07:09:42 -05:00
Graham Mainwaring
055c02072f Default LOGIN_REDIRECT_URL should be blank, not null 2019-12-18 15:13:46 -05:00
Graham Mainwaring
7700050d10 Add default for LOGIN_REDIRECT_OVERRIDE 2019-12-11 17:21:02 -05:00
Alan Rominger
8e7d607a47 Only turn off Galaxy cert verification via toggle (#3933) 2019-11-19 08:54:40 -05:00
Ryan Petrello
bd81fda05c get rid of a loud celery error in the dev environment 2019-11-01 09:25:41 -04:00
Bill Nottingham
5cdf2f88da Remove admin alerts, there are better mechanisms for this 2019-10-30 19:35:45 -04:00
Ryan Petrello
171f0d6340 Merge branch 'downstream' into devel 2019-10-29 13:02:17 -04:00
Graham Mainwaring
b2b33605cc Add UI toggle to disable public Galaxy (#3867) 2019-10-29 11:24:11 -04:00
AlanCoding
a6404bdd0d Move development PROJECTS_ROOT 2019-10-25 15:48:07 -04:00
Ryan Petrello
16812542f8 implement a simple periodic pod reaper for container groups
see: https://github.com/ansible/awx/issues/4911
2019-10-17 17:06:36 -04:00
Shane McDonald
8f75382b81 Implement retry logic for container group pod launches 2019-10-10 15:53:56 -04:00
Christian Adams
844b8a803f update frequency of collection for automation analytics 2019-10-09 21:10:48 -04:00
Ryan Petrello
a076e84a33 add a settings flag for writing all external logs to disk 2019-10-09 13:54:54 -04:00
AlanCoding
c09039e963 Add setting for auth_url
Also adjust public galaxy URL setting to
allow using only the primary Galaxy server

Include auth_url in token exclusivity validation
2019-10-07 14:02:43 -04:00
AlanCoding
922e779a86 Rename private to primary in galaxy settings
use a setting for the public galaxy URL
2019-10-07 14:02:43 -04:00
AlanCoding
093bf6877b Finish adding settings to UI 2019-10-07 14:02:42 -04:00
AlanCoding
d59d8562db Avoid redacting Galaxy URLs 2019-10-07 14:02:42 -04:00
AlanCoding
c566c332f9 Initial env var implementation of private galaxy server 2019-10-07 14:02:42 -04:00
Ryan Petrello
b45b9333e1 Merge pull request #4716 from jladdjr/perf_stats
Enable collection of performance stats
2019-10-04 17:09:30 -04:00
Jim Ladd
cf1c9a0559 Add awx settings for resource profiling 2019-10-04 12:48:29 -07:00
Shane McDonald
bd5003ca98 Task manager / scheduler Kubernetes integration 2019-10-04 13:21:21 -04:00
Seth Foster
db1ad2de95 Set REFRESH_TOKEN_EXPIRE_SECONDS
- Set OAUTH2 REFRESH_TOKEN_EXPIRE_SECONDS to 1 month
  (2628000 seconds)
- If not set, awx-manage cleartokens, or cleanup_tokens,
  will not work properly
- Once cleartokens is run, this setting is the amount of
  time after an access token expires that we keep its
  refresh token in the database
2019-10-02 13:51:46 -04:00
Ryan Petrello
27c4e35ee4 warn loudly if celerybeat encounters AMQP connection issues
related: https://github.com/ansible/awx/pull/4857
2019-10-01 09:32:22 -04:00
Ryan Petrello
b80eafe4a1 make kombu DNS failures louder in the logs 2019-09-30 16:48:09 -04:00
Ryan Petrello
82be87566f improve host key checking configurability
see: https://github.com/ansible/tower/issues/3737
2019-09-30 14:13:07 -04:00