Commit Graph

39 Commits

Author SHA1 Message Date
Ryan Petrello 0ad78874ce remove TCP ports for redis (it only listens on a unix domain socket) 2020-06-09 12:29:33 -04:00
Rebeccah d7f9e66710 added changelog entry 2020-06-09 11:09:04 -04:00
Rebeccah d2bbe7aa1a remove memcache from everywhere and add djagno-redis to cover it 2020-06-09 10:24:23 -04:00
Bill Nottingham 131f5ff018 Remove dev env futzing of supervisord.conf permissions
If we just link it into the dev env, we don't need to copy it at
startup, and we don't need write permissions on it.
2020-05-28 13:12:57 -04:00
chris meyers 72248db76d fix copy paste error in docker compose cluster 2020-04-16 14:12:30 -04:00
chris meyers c4d704bee1 fix memcached in dev env
* create memcached dir via git so that the current user owns it.
Otherwise, docker will create the dir as root at runtime
2020-04-06 16:35:52 -04:00
chris meyers 7433aab258 switch memcached from tcp to unix domain socket 2020-04-06 08:35:12 -04:00
Ryan Petrello 8f1db173c1 remove a bunch of RabbitMQ references 2020-03-24 18:46:58 -04:00
chris meyers 5ba90b629e fixup dev cluster bringup
* change from bootstrap script to launch_awx.sh script
2020-03-23 07:33:35 -04:00
chris meyers 87de0cf0b3 flake8, pytest, license fixes 2020-03-18 16:10:20 -04:00
chris meyers 770b457430 redis socket support 2020-03-18 16:10:19 -04:00
chris meyers 2a2c34f567 combine all the broker replacement pieces
* local redis for event processing
* postgres for message broker
* redis for websockets
2020-03-18 16:10:15 -04:00
chris meyers c8eeacacca POC channels 2 2020-03-18 16:10:12 -04:00
Yanis Guenane ca247182df yamllint: Make all files in awx pass yamllint
This commit updates all files that weren't passing yamllint for them to
pass.

A new yamllint target has been added. One can run `tox -e yamllint` or
`yamllint -s .` locally to ensure yaml files are still passing.

This check will be enabled in the CI so it can get on every new
contributions, and prevent merging non-compliant code.

Signed-off-by: Yanis Guenane <yguenane@redhat.com>
2019-12-02 15:12:51 +01:00
Ryan Petrello 4b62f4845a fix broken docker-compose-cluster config 2019-09-27 00:37:29 -04:00
Shane McDonald 22441d280e Fix pg password in cluster dev env 2019-09-17 08:45:19 -04:00
Ryan Petrello 5ed97e0f65 change the default port range for the sdb debugging tool
the current range conflicts w/ a port used by the pycharm editor
2019-09-13 11:55:43 -04:00
Christian Adams ec1e93cc69 Upgrade to postgres 10.6
- use awx-python in shebang in dev env
  - scl enable where needed for rhel7 & container installs
  - use scram-sha-256 pg user hashing by default
  - ensure psycopg2 is using the correct PG_CONFIG at build time for the right libpq version
2019-09-12 12:52:43 -04:00
Jose OrPa 04ab736f09 #3778 Upgrading postgresql to v10 2019-09-12 12:52:42 -04:00
Shane McDonald b29a9cd86e Fix dev environment when running as root on the host
Without this, CURRENT_UID isnt actually passed in from the host, and wipes out /etc/passwd even when we’re actually running as root.

I tested this as a non-root user on Linux, and on Docker for Mac
2019-03-06 17:08:56 -05:00
AlanCoding 0b32733dc8 set fixed container names 2018-11-26 08:26:57 -05:00
AlanCoding 19030b9d5f apply docker-compose fix to cluster target too 2018-10-26 09:36:11 -04:00
Ryan Petrello ff1e8cc356 replace celery task decorators with a kombu-based publisher
this commit implements the bulk of `awx-manage run_dispatcher`, a new
command that binds to RabbitMQ via kombu and balances messages across
a pool of workers that are similar to celeryd workers in spirit.
Specifically, this includes:

- a new decorator, `awx.main.dispatch.task`, which can be used to
  decorate functions or classes so that they can be designated as
  "Tasks"
- support for fanout/broadcast tasks (at this point in time, only
  `conf.Setting` memcached flushes use this functionality)
- support for job reaping
- support for success/failure hooks for job runs (i.e.,
  `handle_work_success` and `handle_work_error`)
- support for auto scaling worker pool that scale processes up and down
  on demand
- minimal support for RPC, such as status checks and pool recycle/reload
2018-10-11 10:53:30 -04:00
Ryan Petrello 1733a20094 make sdb-listen work for docker-compose-cluster
use a different port range for each container, because docker can't map
them all to the same port range
2018-06-12 13:57:28 -04:00
Matthew Jones 22dd6ddfea Remove the logstash container from the base dev docker compose
Now with less java running while you code!
2018-06-05 09:42:38 -04:00
Ryan Petrello d743b77353 replace our rdb tooling w/ the sdb PyPI package 2018-02-26 19:05:50 -05:00
Alan Rominger dcfcfb6c7b Merge pull request #245 from AlanCoding/fix_supervisor
specify all group queues, get supervisor working
2017-08-14 11:31:00 -04:00
AlanCoding 2385f62311 specify all group queues, get supervisor working 2017-08-10 14:57:25 -04:00
Matthew Jones 67474c8de1 Updating development tooling to be more generic
* Not assuming GCR is being used for image hosting
* Breaking out the dev environment bootstrapping from service starting
2017-08-09 15:53:25 -04:00
Matthew Jones eb64d6e808 Update cluster compose for repo migration 2017-07-24 21:03:55 -04:00
AlanCoding 3cedcf22a9 add instance groups and queues to cluster tooling 2017-06-22 10:40:58 -04:00
adamscmRH e0581af9ff Missing Procfile docker-compose fix 2017-06-21 14:29:49 -04:00
Matthew Jones eb2e7c38b0 Bring up logstash as part of the development cluster 2017-03-21 10:42:32 -04:00
Wayne Witzel III 50f187b78c run clustered as privleged to avoid namespace error 2017-02-24 11:12:39 -05:00
Matthew Jones 9f777f6153 Add some provisions for https into haproxy and the cluster 2017-01-03 13:59:29 -05:00
Matthew Jones 2872409271 Integrate rabbitmqctl interface to cluster and standalone containers 2016-09-15 12:07:16 -04:00
Matthew Jones 807cced571 Implement a more dynamic celery queue system
* Meant to be a starting point to more efficiently manage work routing
  and to balance work across all tower nodes
* Integrate flower as a dev tool that starts alongside other nodes.
  Helpful for observing and monitoring the queues/exchanges
* For the moment, force the task manager to only run on one node (not
  sure if this is needed)
* Define queues and routes for all task work
* Bump celery version to 3.1.23
* Expose flower through haproxy
2016-09-09 15:18:18 -04:00
Matthew Jones eafb6c92b5 Docker compose improvements
* Make sure we explicitly set a hostname for tower nodes
* Switch rabbit vhost to use the root
2016-09-08 11:02:51 -04:00
Matthew Jones ebf103f345 Initial Docker Compose workflow for Tower cluster
The goal is to share a common pattern with the existing development work
2016-09-08 10:18:14 -04:00