Commit Graph

335 Commits

Author SHA1 Message Date
Christian Adams
2329c1b797 Add rsyslog config to container from file for consistency 2020-04-13 11:43:59 -04:00
Christian Adams
470159b4d7 Enable innocuous but valid config for rsyslog if disabled 2020-04-13 11:43:59 -04:00
Christian Adams
4d5507d344 Add default rsyslog.conf without including /etc/rsyslog.conf 2020-04-13 11:43:59 -04:00
Christian Adams
d350551547 Tweaks to Test Button logic and cleans up flake8 and test failures 2020-04-13 11:43:59 -04:00
Christian Adams
7fd79b8e54 Remove unneeded logging sock variable 2020-04-13 11:43:59 -04:00
Christian Adams
fb047b1267 Add unit tests for reconfiguring rsyslog & for test endpoint 2020-04-13 11:43:59 -04:00
Christian Adams
d31c528257 Fix Logging settings "Test" button functionality 2020-04-13 11:43:59 -04:00
Christian Adams
996d7ce054 Move supervisor and rsyslog sock files to their own dirs under /var/run 2020-04-13 11:43:59 -04:00
Shane McDonald
c0af3c537b Configure rsyslog to listen over a unix domain socket instead of a port
- Add a placeholder rsyslog.conf so it doesn't fail on start
 - Create access restricted directory for unix socket to be created in
 - Create RSyslogHandler to exit early when logging socket doesn't exist
 - Write updated logging settings when dispatcher comes up and restart rsyslog so they  take effect
 - Move rsyslogd to the web container and create rpc supervisor.sock
 - Add env var for supervisor.conf path
2020-04-13 11:43:59 -04:00
Christian Adams
f8afae308a Add rsyslog to supervisor for the task container
- Add proper paths for rsyslog's supervisor logs
 - Do not enable debug mode for rsyslogd
 - Include system rsyslog.conf, and specify tower logging conf when
   starting rsyslog.
2020-04-13 11:43:59 -04:00
Christian Adams
4cd0d60711 Properly handle logger paths and https/http configuration
- log aggregator url paths were not being passed to rsyslog
 - http log services like loggly will now truly use http and port 80
 - add rsyslog.pid to .gitignore
2020-04-13 11:43:59 -04:00
Ryan Petrello
589d27c88c POC: replace our external log aggregation feature with rsyslog
- this change adds rsyslog (https://github.com/rsyslog/rsyslog) as
  a new service that runs on every AWX node (managed by supervisord)
  in particular, this feature requires a recent version (v8.38+) of
  rsyslog that supports the omhttp module
  (https://github.com/rsyslog/rsyslog-doc/pull/750)
- the "external_logger" handler in AWX is now a SysLogHandler that ships
  logs to the local UDP port where rsyslog is configured to listen (by
  default, 51414)
- every time a LOG_AGGREGATOR_* setting is changed, every AWX node
  reconfigures and restarts its local instance of rsyslog so that its
  fowarding settings match what has been configured in AWX
- unlike the prior implementation, if the external logging aggregator
  (splunk/logstash) goes temporarily offline, rsyslog will retain the
  messages and ship them when the log aggregator is back online
- 4xx or 5xx level errors are recorded at /var/log/tower/external.err
2020-04-13 11:43:59 -04:00
Ryan Petrello
2cefba6f96 properly serialize external logs that contain decimal.Decimal objects 2020-04-06 14:24:24 -04:00
AlanCoding
5e595caf5e Add workflow node identifier
Generate new modules WFJT and WFJT node
Touch up generated syntax, test new modules

Add utility method in awxkit

Fix some issues with non-name identifier in
  AWX collection module_utils

Update workflow docs for workflow node identifier

Test and fix WFJT modules survey_spec
Plug in survey spec for the new module
Handle survey spec idempotency and test

add associations for node connections
Handle node credential prompts as well

Add indexes for new identifier field

Test with unicode dragon in name
2020-03-23 22:00:00 -04:00
AlanCoding
7d0b207571 Organization on JT as read-only field
Set JT.organization with value from its project

Remove validation requiring JT.organization

Undo some of the additional org definitions in tests

Revert some tests no longer needed for feature

exclude workflow approvals from unified organization field

revert awxkit changes for providing organization

Roll back additional JT creation permission requirement

Fix up more issues by persisting organization field when project is removed

Restrict project org editing, logging, and testing

Grant removed inventory org admin permissions in migration

Add special validate_unique for job templates
  this deals with enforcing name-organization uniqueness

Add back in special message where config is unknown
  when receiving 403 on job relaunch

Fix logical and performance bugs with data migration

within JT.inventory.organization make-permission-explicit migration

remove nested loops so we do .iterator() on JT queryset

in reverse migration, carefully remove execute role on JT
  held by org admins of inventory organization,
  as well as the execute_role holders

Use current state of Role model in logic, with 1 notable exception
  that is used to filter on ancestors
  the ancestor and descentent relationship in the migration model
    is not reliable
  output of this is saved as an integer list to avoid future
    compatibility errors

make the parents rebuilding logic skip over irrelevant models
  this is the largest performance gain for small resource numbers
2020-03-12 15:45:46 -04:00
gamuniz
350046d495 changed syntax as insert isn't required 2020-02-26 14:28:22 +00:00
root
58c06d5aea fixed precedence on ansible.cfg 2020-02-23 14:58:47 +00:00
Ryan Petrello
3c31e0ed16 some more minor callback cleanup and development tweaks 2020-01-27 17:18:09 -05:00
Ryan Petrello
e18639b26b further optimize conf.settings access when logging is enabled
the callback receiver is still fairly slow when logging is enabled due
to constant setting lookups; this speeds things up considerably

related: https://github.com/ansible/awx/pull/5618
2020-01-22 16:17:33 -05:00
softwarefactory-project-zuul[bot]
0809c27bd1 Merge pull request #5581 from AlanCoding/hush_managed_types
Remove incorrect activity stream entries related to managed types

Reviewed-by: https://github.com/apps/softwarefactory-project-zuul
2020-01-02 18:31:49 +00:00
AlanCoding
807f4ea757 Remove incorrect activity stream entries related to managed types 2020-01-02 11:37:33 -05:00
AlanCoding
1f46878652 Revert "Apply migration flag check to task manager"
This reverts commit a0910eb6de.
2020-01-02 09:08:17 -05:00
Shane McDonald
c1abc56753 Merge remote-tracking branch 'downstream/release_3.6.2' into downstream-changes 2019-12-16 11:13:10 -05:00
AlanCoding
a0910eb6de Apply migration flag check to task manager 2019-12-15 22:56:57 -05:00
Ryan Petrello
7396e2e7ac add an awx-manage command for re-generating SECRET_KEY 2019-12-12 16:19:20 -05:00
Ryan Petrello
208e36f83b add an awx-manage command for re-generating SECRET_KEY 2019-12-02 18:07:22 -05:00
Christian Adams
05e6f4ab3c Add User-Agent to analytics upload POST 2019-11-25 22:22:05 -05:00
Rebeccah
4bbdce3478 removed policy_instance variables from container groups default values in the API put/patch view 2019-10-29 11:24:15 -04:00
Rebeccah
d25e6249fd Added in validation for each of the 3 fields that should not be changed if the instance is a container group, defaults in the textarea persist with these 3 options 2019-10-29 11:24:15 -04:00
Ryan Petrello
8bc6367e1e fix a bug introduced upstream with settings.LOG_AGGREGATOR_AUDIT 2019-10-29 11:24:12 -04:00
Martin Juhl
321aa3b01d Update handlers.py
The setFormatter tries to create the external.log file.. So we should check if LOG_AGGREGATOR_AUDIT is active here as well
2019-10-29 11:24:11 -04:00
Ryan Petrello
8701f83922 fix a bug introduced upstream with settings.LOG_AGGREGATOR_AUDIT 2019-10-22 12:17:20 -04:00
Martin Juhl
facec0fe76 Update handlers.py
The setFormatter tries to create the external.log file.. So we should check if LOG_AGGREGATOR_AUDIT is active here as well
2019-10-22 07:25:01 -04:00
Ryan Petrello
b559860c78 Merge pull request #3804 from jbradberry/cli-no-truncate
Do not truncate job event list stdout when called from the CLI
2019-10-16 10:36:29 -04:00
Jake McDermott
cd18ec408c Remove unused variable 2019-10-16 09:30:43 -04:00
Bill Nottingham
60ca843b71 Use logger.exception instead of logger.warning. 2019-10-15 14:20:09 -04:00
Jeff Bradberry
a803cedd7c Break out a new reusable truncate_stdout utility function 2019-10-10 16:07:08 -04:00
Bill Nottingham
31bdde00c9 Check the user's ansible.cfg for role/collection paths.
There's no other way to add our new paths reliably without breaking things.
2019-10-10 15:09:26 -04:00
Ryan Petrello
a076e84a33 add a settings flag for writing all external logs to disk 2019-10-09 13:54:54 -04:00
Jake McDermott
9697e1befb comment fixup 2019-09-16 14:51:56 -04:00
Jake McDermott
84a8559ea0 psuedo -> pseudo 2019-09-16 14:51:56 -04:00
AlanCoding
e8581f6892 Implement WFJT prompting for limit & scm_branch
add feature to UI and awxkit

restructure some details of create_unified_job
  for workflows to allow use of char_prompts
  hidden field
avoid conflict with sliced jobs in char_prompts copy logic

update developer docs

update migration reference

bump migration
2019-09-16 14:51:53 -04:00
Ryan Petrello
b5225bd80d record the parent WFJ ID for job events that originate from a WFJT 2019-09-13 08:28:32 -04:00
Jeff Bradberry
bcbb768dd3 Remove the use of the deprecated Field.rel attribute
Also, rename a number of variables named `rel` for ease of searching.
2019-07-12 15:11:21 -04:00
olia-dev
522dcf5ed3 related #4274 - moved function 'create_temporary_fifo' to 'awx/main/utils/common.py' and referenced it in other plugins (fixed errors) 2019-07-10 12:40:26 +02:00
olia-dev
7b390fa2fc related #4274 - moved function 'create_temporary_fifo' to 'awx/main/utils/common.py' and referenced it in other plugins
Signed-off-by: olia-dev <olia-dev@ktah.net>
2019-07-10 11:41:21 +02:00
Ryan Petrello
dfa8d44eb8 don't ship external logs from the main thread of the dispatcher
this is a fairly esoteric change that attempts to work around a bug
we've discovered in cpython itself

context: https://github.com/ansible/awx/issues/4181
2019-06-27 16:24:36 -04:00
AlanCoding
b11995e638 Remove duplicated type methods and old Django logic 2019-05-23 08:24:24 -04:00
AlanCoding
d3f047d731 Move dynamic log level logic to filter 2019-05-22 14:00:39 -04:00
Ryan Petrello
b1d75327e3 add the ability to toggle DEBUG logging on dynamically 2019-05-16 07:58:31 -04:00