Commit Graph

341 Commits

Author SHA1 Message Date
Bill Nottingham 131f5ff018 Remove dev env futzing of supervisord.conf permissions
If we just link it into the dev env, we don't need to copy it at
startup, and we don't need write permissions on it.
2020-05-28 13:12:57 -04:00
Christian Adams 7e0ef6dd7b Remove unneeded dockerfile-sync 2020-05-15 17:43:07 -04:00
Ryan Petrello 54178a1982 stop pre-generating awx.egg-info 2020-05-15 11:47:53 -04:00
Bill Nottingham d5817fd87a Remove repo files no longer used in build process 2020-05-14 15:02:32 -04:00
Bill Nottingham 8b51e8eb82 Stage rsyslog.conf from existing version so we don't have two copies. 2020-05-14 15:02:32 -04:00
Bill Nottingham 66245d3094 Reorganize both dev and prod container builds
Use only one (templated) docker file for both builds.

Move to two-stage container builds in order to save space in the
final image.
2020-05-14 15:02:32 -04:00
Seth Foster 4da0e0dd80 Vendor collections for isolated jobs to work in ansible 2.10
kubectl and synchronize are now part of community.kubernetes
and ansible.posix collections, respectively. This change installs
these collections to a local directory to be used in inventory and
isolated management playbooks.

awx issue #6930
2020-05-13 10:41:01 -04:00
Bill Nottingham caa4e90fa0 Remove extraneous re-install of rsyslog 2020-05-11 22:24:16 -04:00
Christian Adams c565130b35 TERM rsyslogd instead of KILL for more graceful shutdown 2020-05-05 15:38:57 -04:00
Shane McDonald b12c0def7d Fix permissions for vendored collections in dev env 2020-05-04 12:44:04 -04:00
Christian Adams c07b6285da Increase stopwait time for rsyslogd service 2020-04-27 14:33:46 -04:00
Yanis Guenane 541b9607f5 Collections: Adding a requirements.yml file 2020-04-16 20:55:59 -04:00
softwarefactory-project-zuul[bot] e19194b883 Merge pull request #6721 from shanemcd/dockerfile-cleanup
Dockerfile organization

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-16 14:48:58 +00:00
Shane McDonald 29a582f869 Dockerfile organization 2020-04-15 14:43:59 -04:00
Christian Adams c8ceb62269 Rename awx rsyslog socket and PID dir 2020-04-15 14:11:15 -04:00
Ryan Petrello 0f74a05fea rsyslogd: ignore /dev/log when we load imuxsock 2020-04-14 11:34:58 -04:00
Christian Adams 5d54877183 Add action to default rsyslog.conf so supervisor starts correctly the first time 2020-04-13 11:44:00 -04:00
Christian Adams b942fde59a Ensure log messages have valid json
- Fix messages getting contatenated at 8k
 - Fix rsyslog cutting off the opening brace of log messages
 - Make valid default conf and emit logs based on prescence of .sock and
 settings
2020-04-13 11:44:00 -04:00
Christian Adams 70391f96ae Revert rsyslog valid config to one that fails intentionally 2020-04-13 11:43:59 -04:00
Christian Adams 2329c1b797 Add rsyslog config to container from file for consistency 2020-04-13 11:43:59 -04:00
Christian Adams 470159b4d7 Enable innocuous but valid config for rsyslog if disabled 2020-04-13 11:43:59 -04:00
Christian Adams 4d5507d344 Add default rsyslog.conf without including /etc/rsyslog.conf 2020-04-13 11:43:59 -04:00
Christian Adams d350551547 Tweaks to Test Button logic and cleans up flake8 and test failures 2020-04-13 11:43:59 -04:00
Christian Adams fb047b1267 Add unit tests for reconfiguring rsyslog & for test endpoint 2020-04-13 11:43:59 -04:00
Christian Adams d31c528257 Fix Logging settings "Test" button functionality 2020-04-13 11:43:59 -04:00
Christian Adams 996d7ce054 Move supervisor and rsyslog sock files to their own dirs under /var/run 2020-04-13 11:43:59 -04:00
Christian Adams 7040fcfd88 Fix container rsyslog dir permissions 2020-04-13 11:43:59 -04:00
Shane McDonald c0af3c537b Configure rsyslog to listen over a unix domain socket instead of a port
- Add a placeholder rsyslog.conf so it doesn't fail on start
 - Create access restricted directory for unix socket to be created in
 - Create RSyslogHandler to exit early when logging socket doesn't exist
 - Write updated logging settings when dispatcher comes up and restart rsyslog so they  take effect
 - Move rsyslogd to the web container and create rpc supervisor.sock
 - Add env var for supervisor.conf path
2020-04-13 11:43:59 -04:00
Christian Adams f8afae308a Add rsyslog to supervisor for the task container
- Add proper paths for rsyslog's supervisor logs
 - Do not enable debug mode for rsyslogd
 - Include system rsyslog.conf, and specify tower logging conf when
   starting rsyslog.
2020-04-13 11:43:59 -04:00
Ryan Petrello 589d27c88c POC: replace our external log aggregation feature with rsyslog
- this change adds rsyslog (https://github.com/rsyslog/rsyslog) as
  a new service that runs on every AWX node (managed by supervisord)
  in particular, this feature requires a recent version (v8.38+) of
  rsyslog that supports the omhttp module
  (https://github.com/rsyslog/rsyslog-doc/pull/750)
- the "external_logger" handler in AWX is now a SysLogHandler that ships
  logs to the local UDP port where rsyslog is configured to listen (by
  default, 51414)
- every time a LOG_AGGREGATOR_* setting is changed, every AWX node
  reconfigures and restarts its local instance of rsyslog so that its
  fowarding settings match what has been configured in AWX
- unlike the prior implementation, if the external logging aggregator
  (splunk/logstash) goes temporarily offline, rsyslog will retain the
  messages and ship them when the log aggregator is back online
- 4xx or 5xx level errors are recorded at /var/log/tower/external.err
2020-04-13 11:43:59 -04:00
softwarefactory-project-zuul[bot] 606c3c3595 Merge pull request #6338 from rooftopcellist/update_logstash_docs
Update logstash docs

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-04-06 21:10:44 +00:00
Shane McDonald bb319136e4 Merge pull request #6585 from shanemcd/cleanup-cleanup
Tidy up the dev environment a bit
2020-04-06 13:09:39 -04:00
Shane McDonald 6fc815937b Tidy up the dev environment a bit 2020-04-06 11:13:51 -04:00
chris meyers 7433aab258 switch memcached from tcp to unix domain socket 2020-04-06 08:35:12 -04:00
Christian Adams 6335004c94 Add common debugging tools to the dev container
- nmap-ncat
 - sdb
 - tcpdump
 - strace
 - vim
2020-03-25 08:03:32 -04:00
Christian Adams 9a38971d47 Update ELK Stack container files 2020-03-19 09:35:08 -04:00
Christian Adams c4e697879d Improve docs for using the logstash container 2020-03-18 18:32:45 -04:00
chris meyers 770b457430 redis socket support 2020-03-18 16:10:19 -04:00
chris meyers 0da94ada2b add missing service name to dev env
* Dev env was bringing the wsbroadcast service up but not under the
tower-process dependency. This is cleaner.
2020-03-18 16:10:18 -04:00
chris meyers 3c5c9c6fde move broadcast websocket out into its own process 2020-03-18 16:10:18 -04:00
chris meyers c8eeacacca POC channels 2 2020-03-18 16:10:12 -04:00
Christian Adams 3755151cc5 Update dev container to be consistent with other installation methods
- rename start_development.sh script to `launch_awx.sh`, like it is in k8 installations
2020-03-11 16:23:31 -04:00
Philip Douglass 7ae1c7c3d2 Set setuid bit on bwrap in development Dockerfile
Related: #5224

Signed-off-by: Philip Douglass <philip.douglass@amadeus.com>
2020-02-25 15:20:01 -05:00
Ryan Petrello ad5d0b92db pin virtualenv < 20 for ansible venv builds
virtualenv version 20 just got released and broken a bunch of stuff
(like the --system-site-packages flag)
2020-02-10 16:31:34 -05:00
Shane McDonald 3f57061509 Add packages missing from base images
Related:

- https://github.com/ansible/awx/issues/5770
- https://github.com/ansible/awx/issues/5724
2020-02-07 13:06:42 -05:00
AlanCoding d2289fe9c6 add pycurl to container images 2020-02-04 14:41:51 -05:00
Christian Adams d41322c63c Remove outdated Zanata translation script as part of migration to Memsource
- adds requirement for gettext to dev container for use by Memsource automation
2019-12-05 09:29:33 -05:00
Shane McDonald c439a1ec8f Fix permissions on projects directory by pre-creating it
This broke after
https://github.com/ansible/awx/commit/f78c9f357dc9cce2d446fb2eddfc05d6cc3c0fff
but people seem to like this functionality so instead of reverting it we can do this.
2019-12-04 17:39:38 -05:00
chris meyers 129374a1c2 keep *-devel package in the dev container
* requirements/updater.sh does pip magic. In doing this magic, devel
system packages are required to download/install/build. This change
ensures those dev packages are available.
2019-12-03 14:31:09 -05:00
Shane McDonald fa61aef194 Install missing locales in dev container image. 2019-11-01 08:29:11 -04:00