add a new awx-manage command `custom_venvs`
add an awx-manage command that gets pip freeze data from custom_venv and outputs to command line stdout
SUMMARY
part of #7062 - this command is a glorified pip freeze + some extra stuff, people could navigate to each of their custom virtual environments themselves and run a pip freeze, but this allows them to not, and everyone likes their life to be easier. The extra stuff allows users to see the connections that their existing virtual envs have in awx to things like organizations, jobs, inventory updates, and projects.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
awx: 19.1.0
ADDITIONAL INFORMATION
This is built off of existing code and there is a line that gets custom venv paths from the settings module, that line does not seem to be working. I have written around that but want to make a note of it.
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Rebeccah Hunter <rhunter@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Shane McDonald <me@shanemcd.com>
Reviewed-by: Elijah DeLee <kdelee@redhat.com>
* Before, we would get the min and max pk of the set we are to gather.
This changeset removes that.
* Before, we would, basically, know the size of the set we are to gather
and would query 100,000 of those job event records at a time. That logic
is now gone.
* Now, for unpartitioned job events we gather 4 hours at a time by
created time.
* Now, for partitioned job events we gather 4 hours at a time by
modified time.
* The order by results in an in-memory sort that COULD blow out the
worker mem buffer and result in sorting having to take place on disk.
* This WILL happen with a default postgres 4MB mem buffer. We saw as
much as 20MB used. Note that AWX defaults postgres mem worker buffer to
3% of the DB memory on external installs and 1% on same-node installs.
So for a 16GB remote DB this would not be a problem.
* We are going to avoid this problem all together by NOT doing a sort
when gathering. Instead, we will sort remotely, in analytics.
* Old, _unpartitioned_main_jobevent table does not have the job_created
column
* New, main_jobevent does.
* Always in clude the job_created column. NULL if old, job_created if
new
* Bump events_table schema version from 1.2 to 1.3 because of the
job_created field
* We found that having multiple `::json` casts in a query slows down
queries more and more by =~> 33%.
* This change coerces postgres into only casting once. Micro
benchmarking shows =~ 2-3x performance boost
* pre-migration jobevents live in unpartitioned table
where only created field has index
* post-migration jobevents live in partitions
where modified field has index
(and should be used to ensure no events are missing)
* when collecting job events by creation time
it is possible to miss events that were created
at one point, but actually committed to the db
much later.
* since events' modified time is set when they are
committed to the db, we shouldn't miss any job events
* selecting job events by modified time wasn't possible
beforehand because we didn't have an index for
jobevent's modified field
Isolated removal
SUMMARY
Removal of the isolated nodes feature.
ISSUE TYPE
Feature Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Alan Rominger <arominge@redhat.com>
Reviewed-by: Jeff Bradberry <None>
Reviewed-by: Elyézer Rezende <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
[devel] add clarifying comment re: AUTOMATION_ANALYTICS_LAST_GATHER being set regardless
(the comment in the changeset pretty much says it all)
Reviewed-by: Chris Meyers <None>
Adjust datetimes to be aware when using awx-manage gather_analytics
SUMMARY
Adjust datetimes to be aware when using awx-manage gather_analytics
Also, make sure that an explicit since parameter will win over
default until=now() when calculating the 4-week data limit.
ISSUE TYPE
Bugfix Pull Request
COMPONENT NAME
API
AWX VERSION
Reviewed-by: Ryan Petrello <None>
- `since` should not be after `until`
- neither `since` nor `until` should be in the future
- `since`, `AUTOMATION_ANALYTICS_LAST_GATHER`, and `AUTOMATION_ANALYTICS_LAST_ENTRIES[key]` should be truncated to 4 weeks prior to `until`
- an explicit `since` parameter should always take precedence over the settings values
- stop trying to ship csv slices when one breaks
- only update LAST_ENTRIES if all of the files in a time/pk slice succeed
- don't allow an explicit --until parameter to set the GATHER/ENTRIES values backwards
Previously, invoking the command with neither of the --ship or
--dry-run flags would result in effectively doing a dry run. With the
stricter checks now in place in analytics.core.gather, let's make sure
that we pass the 'dry-run' parameter in to gather() in the no-flags
case.
With the change to use pk-based interval slicing for the job events
table, we need analytics.gather to be the code that manages all of the
"expensive" collector slicing. While we are at it, let's ship each
chunked tarball file as we produce it.
remove ansible_version from the API config and metrics endpoints
AWX no longer includes Ansible on the control plane and there is no
"global" version of Ansible aside from what's configured at the
Execution Environment level
see: #9472
Reviewed-by: Ryan Petrello <None>
Reviewed-by: Ladislav Smola <lsmola@redhat.com>
Reviewed-by: Yanis Guenane <None>
Reviewed-by: Bianca Henderson <beeankha@gmail.com>
Reviewed-by: Jake McDermott <yo@jakemcdermott.me>
- Adds a Metrics() class that can track data such as number of
events the callback receiver inserted into database
- Exposes this metric data at the api/v2/metrics/ endpoint.
This data is prometheus-friendly
- Metric data is stored in memory, then periodically saved to Redis.
- Metric data is periodically broadcast to other nodes in the cluster,
so that each node has a copy of the most recent metric data collected.
AWX no longer includes Ansible on the control plane and there is no
"default" version of Ansible aside from what's configured at the
Execution Environment level
see: https://github.com/ansible/awx/issues/9472
